33 lines | 1529 chars
| 1 | == Running the testsuite |
| 2 | |
| 3 | == Creating a new test |
| 4 | |
| 5 | == Useful Makefile targets |
| 6 | |
| 7 | `make parallel`:: |
| 8 | runs the tests in parallel using the |
| 9 | link:https://www.gnu.org/software/parallel/[GNU parallel] tool: tests run |
| 10 | twice as fast with no difference in output order. |
| 11 | |
| 12 | `make all-foo`, `make parallel-foo`:: |
| 13 | runs only the tests in the directories whose name starts with `foo`: |
| 14 | `parallel-typing`, `all-lib`, etc. |
| 15 | |
| 16 | `make one DIR=tests/foo`:: |
| 17 | runs only the tests in the directory `tests/foo`. This is often equivalent to |
| 18 | `cd tests/foo && make`, but sometimes the latter breaks the test makefile if |
| 19 | it contains fragile relative filesystem paths. Such errors should be fixed if |
| 20 | you find them, but `make one DIR=...` is the more reliable option as it runs |
| 21 | exactly as `make all` which is heavily tested. |
| 22 | |
| 23 | `make promote DIR=tests/foo`:: |
| 24 | Most tests run a program and compare the result of the program, store in a file |
| 25 | `foo.result`, with a reference output stored in `foo.reference` -- the test |
| 26 | fails if the two output differ. Sometimes a change in result is innocuous, it |
| 27 | comes from an intended change in output instead of a regression. |
| 28 | `make promote` copies the new result file into the reference file, making the |
| 29 | test pass again. Whenever you use this rule please check carefully, using |
| 30 | `git diff`, that the change really corresponds to an intended output |
| 31 | difference, and not to a regression. You then need to commit the change to |
| 32 | reference file, and your commit message should explain why the output changed. |
| 33 |