4.0 KiB
This test module is part of matlab2tikz.
Its use is mainly of interest to the matlab2tikz developers to assert that the produced output is good. Ideally, the tests should be run on every supported environment, i.e.:
- MATLAB R2014a/8.3 (or an older version)
- MATLAB R2014b/8.4 (or a newer version)
- Octave 3.8
Preparing your environment
Before you can run the tests, you need to make sure that you have all relevant
functions available on your path. From within the /test directory run the
following code in your MATLAB/Octave console:
addpath(pwd); % for the test harness
addpath(fullfile(pwd,'..','src')); % for matlab2tikz
addpath(fullfile(pwd,'suites')); % for the test suites
Running the tests
We have two kinds of tests runners available that each serve a slightly different purpose.
- "Graphical" tests produce an output report that graphically shows test figures as generated by MATLAB/Octave and our TikZ output.
- "Headless" tests do not produce graphical output, but instead check the MD5 hash of the generated TikZ files to make sure that the same output as before is generated.
It is recommended to run the headless tests first and check the problems in the graphical tests afterwards.
Headless tests
These tests check that the TikZ output file produced by matlab2tikz matches
a reference output. The actual checking is done by hashing the file and the
corresponding hashes are stored in .md5 files next to the test suites.
For each environment, different reference hashes can be stored.
The headless tests can be invoked using
testHeadless;
, or, equivalently,
makeTravisReport(testHeadless)
There are some caveats for this method of testing:
- The MD5 hash is extremely brittle to small details in the output: e.g. extra whitespace or some other characters will change the hash.
- This automated test does NOT test whether the output is desirable or not. It only checks whether the previous output is not altered!
- Hence, when structural changes are made, the reference hash should be changed. This SHOULD be motivated in the pull request (e.g. with a picture)!
Graphical tests
These tests allow easy comparison of a native PDF print output and the
output produced by matlab2tikz. For the large amount of cases, however,
this comparison has become somewhat unwieldy.
You can execute the tests using
testGraphical;
or, equivalently,
makeLatexReport(testGraphical)
This generates a LaTeX report in test/output/current/acid.tex which can then be compiled.
Compilation of this file can be done using the Makefile test/output/current/Makefile if you are on a Unix-like system (OS X, Linux) or have Cygwin installed on Windows.
If all goes well, the result will be the file test/output/current/acid.pdf that contains
a list of the test figures, exported as PDF and right next to it the matlab2tikz generated plot.
Advanced Use
Both testHeadless and testGraphical can take multiple arguments, those are documented in the raw test runner testMatlab2tikz that is used behind the scenes. Note that this file sits in a private directory, so help testMatlab2tikz will not work!
Also, both can be called with a single output argument, for programmatical access to the test results as
status = testHeadless()
These test results in status can be passed to saveHashTable for updating the hash tables.
Obviously, this should be done with the due diligence!
Automated Tests
The automated tests run on Travis-CI for Octave and on a personal Jenkins server for MATLAB.
These are effectively the "headless" tests that get called by the runMatlab2TikzTests function.
Without verification of those automated tests, a pull request is unlikely to get merged.