Baseline
Compare your CI to a baseline you name
Two sets of timings, matched job by job. The baseline is written out above every number.
Baselinename it
Candidatename it
Baseline: GitHub-hosted ubuntu-latest. Compared with Larger runner, same commit, job by job by name, on 3 matched workloads, from the timings pasted. Not a prediction for other workloads.
GitHub-hosted ubuntu-latest
19m 27s
Larger runner, same commit
10m 26s
Difference
−46%
| Workload | Phases, baseline over candidate | GitHub-hosted ubuntu-latest | Larger runner, same commit | Difference |
|---|---|---|---|---|
| build | 5m 51s | 3m 04s | −2m 47s −48% | |
| test | 10m 55s | 5m 43s | −5m 12s −48% | |
| lint | 2m 41s | 1m 39s | −1m 02s −39% |
| Phase, all matched jobs | GitHub-hosted ubuntu-latest | Larger runner, same commit | Difference |
|---|---|---|---|
| Queue | 1m 03s | 28s | −35s |
| Setup | 6s | 4s | −2s |
| Cache restore | 1m 38s | 44s | −54s |
| Install | 4m 30s | 2m 37s | −1m 53s |
| Build | 3m 10s | 1m 40s | −1m 30s |
| Test | 8m 05s | 4m 12s | −3m 53s |
| Other | 55s | 41s | −14s |
Runs in your browser. Nothing is uploaded.
Make it like for like
A comparison is only as fair as its inputs. Keep everything but the runner the same.
| Hold constant | Why | How to check |
|---|---|---|
| The commit | Different code is a different workload. | Same SHA in both runs' headers. |
| Cache state | A warm cache on one side decides the result. | Both runs log a hit, or both a miss. |
| The verdict | A faster run that tests less isn't faster. | Same conclusion and test count. |
| Several runs | One run includes noise from a busy network or queue. | Three runs each; compare medians. |
Jobs are matched by name, case-insensitive. Jobs on one side only are listed and left out of the totals. See the method.