Writing automated tests is now a well-accepted best practice of application development. But over time, slow test runs can become an albatross around the neck of developer productivity. At Salesforce/Pardot, our large-scale PHP application has thousands of automated tests that could take well over an hour to run in serial. But with the help of a test parallelization tool we built called Privet (that will be open sourced in early 2018), we can run the entire test suite in five minutes, enabling our developers to deploy many times per day and run the tests on every Git branch. In this talk, I share our journey from a 30-minute build, to a manually balanced build, through ultimately automating the process. I also share some challenges we had to overcome along the way, and some we're still figuring out.