mutmut used 5.5 times less runner time than cosmic-ray (56 s against 309 s) and killed 71.2 % of the mutants against 77.1 %.
Estimated gCO2eq of one mutation run, by grid intensity
us-east1 (579 g/kWh)us-east5 (343 g/kWh)
Table view
Item
us-east1 (579 g/kWh) (gCO2eq per run)
us-east5 (343 g/kWh) (gCO2eq per run)
cosmic-ray
0.272
0.161
mutmut
0.049
0.029
Source: CI job seconds from docs/submission/mutation-tools-measurement.md; coefficients and grid data from docs/adr/0009-measure-sustainability-with-sci.md
Runner time of one mutation run
Table view
Item
Job seconds (seconds)
cosmic-ray
309
mutmut
56
Source: CI job seconds measured on the runner, docs/submission/mutation-tools-measurement.md
Method and limits
SCI = (O + M) per R, with O = E x I, as the Green Software Foundation specification defines it. The numbers come from scripts/sci_estimate.py, written by the Duo Developer Flow, and the inputs are printed here.
Energy per run
Tool
Job seconds
Energy (Wh)
Mutants killed (%)
cosmic-ray
309
0.469
77.1
mutmut
56
0.085
71.2
Assumptions behind every number
Assumption
Value
vCPUs of the runner
2
CPU utilization
0.5
Watts per vCPU, minimum and maximum
0.71 and 4.26
PUE
1.1
Grid intensity, gCO2eq per kWh
us-east1 579, us-east5 343
Embodied emissions (M)
0 g, not estimated
Functional unit (R)
1 mutation run
Power is estimated from published per-vCPU coefficients, not measured on the runner.
Embodied emissions are not estimated: this is an SCI without M.
Only one comparison was measured: two mutation tools on the same code.