Unnecessary CI/CD pipeline runs and misconfigured rules can significantly increase costs.

The bill for my CI runner fleet came to $270, which doesn't sound like a lot of money until it's money you haven't got. The two easy answers were spend more or test less. I wasn't having the second one. I'd rather pay in waiting time than drop a single test. So I went through the receipt line by line: three weeks, 3,373 merge-request pipelines, 31,247 jobs. My hunch about where the money went was wrong. The scary heavy jobs (mutation, fuzz, cross-arch) came to 5.7% of merge-request minutes. Renovate's branches were nearly half. Release merge requests were another 20.4%, entirely self-inflicted: I'd put CHANGELOG.md in the test jobs' change rules, so a file of markdown woke the whole suite. Then main was rerunning the heavy jobs on the exact commit the merge request had already tested. 1,075 runner minutes to produce the same answer. The fixes add up to something like 40% fewer merge-request minutes, and not one of them is a test I stopped running. Most of that receipt was things I'd bought twice, which stings a bit more than the $270 did. #CICD #GitLab #DevOps #Testing https://phpboyscout.uk/which-tests-would-you-cut-when-the-bill-arrives/

The bill for my CI runner fleet came to $270, which doesn't sound like a lot of money until it's money you haven't got. The two easy answers were spend more or test less. I wasn't having the second one. I'd rather pay in waiting time than drop a single test. So I went through the receipt line by line: three weeks, 3,373 merge-request pipelines, 31,247 jobs. My hunch about where the money went was wrong. The scary heavy jobs (mutation, fuzz, cross-arch) came to 5.7% of merge-request minutes. Renovate's branches were nearly half. Release merge requests were another 20.4%, entirely self-inflicted: I'd put CHANGELOG.md in the test jobs' change rules, so a file of markdown woke the whole suite. Then main was rerunning the heavy jobs on the exact commit the merge request had already tested. 1,075 runner minutes to produce the same answer. The fixes add up to something like 40% fewer merge-request minutes, and not one of them is a test I stopped running. Most of that receipt was things I'd bought twice, which stings a bit more than the $270 did. #CICD #GitLab #DevOps #Testing https://phpboyscout.uk/which-tests-would-you-cut-when-the-bill-arrives/