The Ten-Minute Builds

  • Automation as documentation
  • Fast feedback when something becomes broken
    • Clear source of error
    • Lower context switching costs
  • Research of non-deterministic tests
http://www.jamesshore.com/Agile-Book/ten_minute_build.html

Strategy #1: scope

An engine Unit tests
A car Functional and end-to-end tests

Unit tests do not depend on:

  • A database, whether relational or NoSQL
  • Web services or other machines
  • The filesystem
  • Application configuration

...but sometimes the risk is best tackled with end-to-end tests

Strategy #2: profiling

phpunit --log-junit suite.xml tests/

    <testcase name="testCellEvolution"
              class="GameOfLifeTest"
              file="/home/giorgio/code/project/GameOfLifeTest.php"
              line="6"
              time="10.011456">
    ...

xdebug.profiler_enable=1

;zend_extension=/usr/lib/php5/20100525/xdebug.so

Strategy #3: hardware

Dell XPS 15
  • i5 is slow? Go to i7
  • Consider only SSD, even small space
  • Future-proof RAM
  • More than halved running times (from 9-10 minutes to 3-4)
  • May not solve CI

Strategy #4: parallelism

Before the cure (serial execution)

  • Unit: 14s
  • Integration: 1m36s
  • Acceptance: 1m46s

Total: 3m36s

After the cure (parallelized on 4 processes)

  • Unit: 15s
  • Integration serial: 9s
  • Integration parallel: 34s
  • Acceptance serial: 16s
  • Acceptance parallel: 31s

Total: 1m35s

How to parallelize?

        paratest --processes 4 --runner WrapperRunner tests/acceptance-parallel
      
switch (getenv('TEST_TOKEN')) {
    case 1:
        $onebipDatabase = 'onebip';
        break;
    default:
        $onebipDatabase = 'onebip' . getenv('TEST_TOKEN');
        break;
}
      
        $tmpdir = '/tmp/files' . uniqid();