1 == Running the testsuite
3 == Creating a new test
5 == Useful Makefile targets
7 `make parallel`::
8 runs the tests in parallel using the
9 link:https://www.gnu.org/software/parallel/[GNU parallel] tool: tests run
10 twice as fast with no difference in output order.
12 `make all-foo`, `make parallel-foo`::
13 runs only the tests in the directories whose name starts with `foo`:
14 `parallel-typing`, `all-lib`, etc.
16 `make one DIR=tests/foo`::
17 runs only the tests in the directory `tests/foo`. This is often equivalent to
18 `cd tests/foo && make`, but sometimes the latter breaks the test makefile if
19 it contains fragile relative filesystem paths. Such errors should be fixed if
20 you find them, but `make one DIR=...` is the more reliable option as it runs
21 exactly as `make all` which is heavily tested.
23 `make promote DIR=tests/foo`::
24 Most tests run a program and compare the result of the program, store in a file
25 `foo.result`, with a reference output stored in `foo.reference` -- the test
26 fails if the two output differ. Sometimes a change in result is innocuous, it
27 comes from an intended change in output instead of a regression.
28 `make promote` copies the new result file into the reference file, making the
29 test pass again. Whenever you use this rule please check carefully, using
30 `git diff`, that the change really corresponds to an intended output
31 difference, and not to a regression. You then need to commit the change to
32 reference file, and your commit message should explain why the output changed.