Free estimate
Menu

Case studies

One test coverage report for a whole product, on every build

One report shows which code the tests exercise across a customer’s whole simulation product, on every build. We designed, built, and run it for plugin and library repositories written by many developers.

A customer's multi-team simulation software. We designed, built, and run the coverage analysis system, as a subcontractor.

Every buildone report across the customer's plugins and libraries

Starting point: one product, built from many repositories

The customer extends a large simulation framework with plugins and libraries that many developers write in separate repositories. The customer’s release build, which we built and run, already brings them together into one product. The customer wanted one view of what the whole product’s tests exercise, across its teams’ code. We built it from the coverage pipeline we had already built for another simulation core.

  • The report had to show the customer’s own plugin and library code, not the framework or third-party libraries.
  • A test that runs through several plugins had to count toward each of them.
  • It had to run in the customer’s existing pipeline, on its Linux build image.
  • Developers had to see it where they already work: the pipeline and the repository page.

Approach: instrument the whole product in one build

  1. Instrumented the whole product in one build. One switch builds the framework with the customer’s plugins and libraries as a single debug build with coverage counters, on the Linux build image.
  2. Ran the tests inside that build. The unit and integration tests run in the instrumented build. An integration test starts the framework and runs through several plugins and a shared library, so it counts toward each one.
  3. Kept the report to the customer’s own code. Framework, third-party, and test code are filtered out, so the report shows the plugin and library code the customer’s teams write.
  4. Published it where developers look. Every build reports it, and the main branch publishes a browsable report and a badge on the repository page.
One build of the whole product: a single integration test runs through the framework core into two plugins and a shared library, so the lines it runs count toward each of them; the framework core is not in the report.one integrationtestone build of the productframework corenot in the reportpluginpluginpluginshared librarylines this test ranrest of the codeOne build of the whole product: a single integration test runs through the framework core into two plugins and a shared library, so the lines it runs count toward each of them; the framework core is not in the report.one integrationtestone build of the productframework corenot in the reportpluginpluginpluginshared librarylines this test ranrest of the code
Fig. 1 Built together, one integration test counts toward every plugin it runs through.

Results: one coverage report, on every build

Every build of the product now reports which lines of the customer’s plugins and libraries the tests run, in one browsable report.

Buy from us or team with us: Contracting

Send us the systems your software must run on.

Estimates are free. Send the scope through the contact form; we reply within one business day and send a written quote within five business days.

Get a free estimate