The Ten-Minute Builds

  • Automation as documentation
  • Fast feedback when something becomes broken
    • Clear source of error
    • Lower context switching costs
  • Research of non-deterministic tests

Strategy #1: scope

An engine Unit tests
A car Functional and end-to-end tests

Unit tests do not depend on:

  • A database, whether relational or NoSQL
  • Web services or other machines
  • The filesystem
  • Application configuration

...but sometimes the risk is best tackled with end-to-end tests

Strategy #2: profiling

phpunit --log-junit suite.xml tests/
    <testcase name="testCellEvolution"

Strategy #3: hardware

Dell XPS 15
  • i5 is slow? Go to i7
  • Consider only SSD, even small space
  • Future-proof RAM
  • More than halved running times (from 9-10 minutes to 3-4)
  • May not solve CI

Strategy #4: parallelism

Before the cure (serial execution)

  • Unit: 14s
  • Integration: 1m36s
  • Acceptance: 1m46s

Total: 3m36s

After the cure (parallelized on 4 processes)

  • Unit: 15s
  • Integration serial: 9s
  • Integration parallel: 34s
  • Acceptance serial: 16s
  • Acceptance parallel: 31s

Total: 1m35s

How to parallelize?

        paratest --processes 4 --runner WrapperRunner tests/acceptance-parallel
switch (getenv('TEST_TOKEN')) {
    case 1:
        $onebipDatabase = 'onebip';
        $onebipDatabase = 'onebip' . getenv('TEST_TOKEN');
        $tmpdir = '/tmp/files' . uniqid();