Technology1 min read

I Cut My Test Suite Time in Half by Slicing Up a CSV File

By · Published by Everything Blog

In short

Holiday API's continuous integration process faced heavy slowdowns because uncached tests had to constantly parse a single massive CSV file containing 557 columns and over 45,000 cells. The developer resolved this by dividing the data into 70 smaller files, immediately cutting test execution from over 10 minutes to under five. This architectural shift allowed the developer to append roughly 300,000 more regression tests without suffering any performance penalties. The optimization also helped stabilize a flaky Redis caching test, dramatically shortening the path from code push to production.

Key points

  • Total deployment times dropped from up to 40 minutes down to roughly 10 minutes from push…: Total deployment times dropped from up to 40 minutes down to roughly 10 minutes from push to production.
  • The streamlined testing architecture accommodated an additional 300,000 regression tests…: The streamlined testing architecture accommodated an additional 300,000 regression tests without increasing the overall run duration.
  • Faster test runs resolved a flaky test issue linked to a Redis caching container in the G…: Faster test runs resolved a flaky test issue linked to a Redis caching container in the GitLab pipeline.

Sources checked by our analysis pipeline: 1

I made a confession recently on LinkedIn:

I've been running Holiday API for the last decade on spreadsheets.

Not one of those fictional accounts that so many people post, I actually did run everything in a spreadsheet. It allowed me to have an admin interface without building one, and the friction was minimal but not zero. For the most part, CSV files are easy enough to parse and extract data from, even when they are massive.

Easy to work with isn't the same as fast though. I'm a caching wizard on the live site, but in my test code, I like to skip the cache so tests are run from a cold start. In this scenario, no one test can influence the results of another, as it should be.

My test suite was already plenty fast, clocking in just north of 10 minutes. Years ago it was taking 15+ minutes so I did some performance tuning and always felt like it was in a sufficient place. Keep in mind we're talking about over 1.2 million tests, mostly related to data…

Technology