You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
102 lines
4.0 KiB
102 lines
4.0 KiB
3 years ago
|
This test module is part of matlab2tikz.
|
||
|
|
|
||
|
|
Its use is mainly of interest to the matlab2tikz developers to assert that the produced output is good.
|
||
|
|
Ideally, the tests should be run on every supported environment, i.e.:
|
||
|
|
|
||
|
|
* MATLAB R2014a/8.3 (or an older version)
|
||
|
|
* MATLAB R2014b/8.4 (or a newer version)
|
||
|
|
* Octave 3.8
|
||
|
|
|
||
|
|
Preparing your environment
|
||
|
|
==========================
|
||
|
|
|
||
|
|
Before you can run the tests, you need to make sure that you have all relevant
|
||
|
|
functions available on your path. From within the `/test` directory run the
|
||
|
|
following code in your MATLAB/Octave console:
|
||
|
|
|
||
|
|
```matlab
|
||
|
|
addpath(pwd); % for the test harness
|
||
|
|
addpath(fullfile(pwd,'..','src')); % for matlab2tikz
|
||
|
|
addpath(fullfile(pwd,'suites')); % for the test suites
|
||
|
|
```
|
||
|
|
|
||
|
|
Running the tests
|
||
|
|
=================
|
||
|
|
|
||
|
|
We have two kinds of tests runners available that each serve a slightly different
|
||
|
|
purpose.
|
||
|
|
|
||
|
|
* "Graphical" tests produce an output report that graphically shows test
|
||
|
|
figures as generated by MATLAB/Octave and our TikZ output.
|
||
|
|
* "Headless" tests do not produce graphical output, but instead check the MD5
|
||
|
|
hash of the generated TikZ files to make sure that the same output
|
||
|
|
as before is generated.
|
||
|
|
|
||
|
|
It is recommended to run the headless tests first and check the problems in
|
||
|
|
the graphical tests afterwards.
|
||
|
|
|
||
|
|
Headless tests
|
||
|
|
--------------
|
||
|
|
These tests check that the TikZ output file produced by `matlab2tikz` matches
|
||
|
|
a reference output. The actual checking is done by hashing the file and the
|
||
|
|
corresponding hashes are stored in `.md5` files next to the test suites.
|
||
|
|
For each environment, different reference hashes can be stored.
|
||
|
|
|
||
|
|
The headless tests can be invoked using
|
||
|
|
```matlab
|
||
|
|
testHeadless;
|
||
|
|
```
|
||
|
|
, or, equivalently,
|
||
|
|
```matlab
|
||
|
|
makeTravisReport(testHeadless)
|
||
|
|
```
|
||
|
|
|
||
|
|
There are some caveats for this method of testing:
|
||
|
|
|
||
|
|
* The MD5 hash is extremely brittle to small details in the output: e.g.
|
||
|
|
extra whitespace or some other characters will change the hash.
|
||
|
|
* This automated test does NOT test whether the output is desirable or not.
|
||
|
|
It only checks whether the previous output is not altered!
|
||
|
|
* Hence, when structural changes are made, the reference hash should be changed.
|
||
|
|
This SHOULD be motivated in the pull request (e.g. with a picture)!
|
||
|
|
|
||
|
|
Graphical tests
|
||
|
|
---------------
|
||
|
|
These tests allow easy comparison of a native PDF `print` output and the
|
||
|
|
output produced by `matlab2tikz`. For the large amount of cases, however,
|
||
|
|
this comparison has become somewhat unwieldy.
|
||
|
|
|
||
|
|
You can execute the tests using
|
||
|
|
```matlab
|
||
|
|
testGraphical;
|
||
|
|
```
|
||
|
|
or, equivalently,
|
||
|
|
```matlab
|
||
|
|
makeLatexReport(testGraphical)
|
||
|
|
```
|
||
|
|
This generates a LaTeX report in `test/output/current/acid.tex` which can then be compiled.
|
||
|
|
Compilation of this file can be done using the Makefile `test/output/current/Makefile` if you are on a Unix-like system (OS X, Linux) or have [Cygwin](https://www.cygwin.com) installed on Windows.
|
||
|
|
|
||
|
|
If all goes well, the result will be the file `test/output/current/acid.pdf` that contains
|
||
|
|
a list of the test figures, exported as PDF and right next to it the matlab2tikz generated plot.
|
||
|
|
|
||
|
|
Advanced Use
|
||
|
|
------------
|
||
|
|
|
||
|
|
Both `testHeadless` and `testGraphical` can take multiple arguments, those are documented in the raw test runner `testMatlab2tikz` that is used behind the scenes. Note that this file sits in a private directory, so `help testMatlab2tikz` will not work!
|
||
|
|
|
||
|
|
Also, both can be called with a single output argument, for programmatical
|
||
|
|
access to the test results as
|
||
|
|
```matlab
|
||
|
|
status = testHeadless()
|
||
|
|
```
|
||
|
|
These test results in `status` can be passed to `saveHashTable` for updating the hash tables.
|
||
|
|
Obviously, this should be done with the due diligence!
|
||
|
|
|
||
|
|
Automated Tests
|
||
|
|
===============
|
||
|
|
|
||
|
|
The automated tests run on [Travis-CI](https://travis-ci.org) for Octave and on a [personal Jenkins server](https://github.com/matlab2tikz/matlab2tikz/wiki/Jenkins) for MATLAB.
|
||
|
|
These are effectively the "headless" tests that get called by the `runMatlab2TikzTests` function.
|
||
|
|
Without verification of those automated tests, a pull request is unlikely to get merged.
|