This test module is part of matlab2tikz.

Its use is mainly of interest to the matlab2tikz developers to assert that the produced output is good.
Ideally, the tests should be run on every supported environment, i.e.:

 * MATLAB R2014a/8.3 (or an older version)
 * MATLAB R2014b/8.4 (or a newer version)
 * Octave 3.8

Preparing your environment
==========================

Before you can run the tests, you need to make sure that you have all relevant
functions available on your path. From within the `/test` directory run the 
following code in your MATLAB/Octave console:

```matlab
addpath(pwd);                       % for the test harness
addpath(fullfile(pwd,'..','src'));  % for matlab2tikz
addpath(fullfile(pwd,'suites'));    % for the test suites
```

Running the tests
=================

We have two kinds of tests runners available that each serve a slightly different
purpose.

  *  "Graphical" tests produce an output report that graphically shows test
     figures as generated by MATLAB/Octave and our TikZ output.
  *  "Headless" tests do not produce graphical output, but instead check the MD5
     hash of the generated TikZ files to make sure that the same output
     as before is generated.

It is recommended to run the headless tests first and check the problems in
the graphical tests afterwards.

Headless tests
--------------
These tests check that the TikZ output file produced by `matlab2tikz` matches
a reference output. The actual checking is done by hashing the file and the
corresponding hashes are stored in `.md5` files next to the test suites.
For each environment, different reference hashes can be stored.

The headless tests can be invoked using
```matlab
testHeadless;
```
, or, equivalently,
```matlab
makeTravisReport(testHeadless)
```

There are some caveats for this method of testing:

 * The MD5 hash is extremely brittle to small details in the output: e.g.
   extra whitespace or some other characters will change the hash.
 * This automated test does NOT test whether the output is desirable or not.
   It only checks whether the previous output is not altered! 
 * Hence, when structural changes are made, the reference hash should be changed.
   This SHOULD be motivated in the pull request (e.g. with a picture)!

Graphical tests
---------------
These tests allow easy comparison of a native PDF `print` output and the
output produced by `matlab2tikz`. For the large amount of cases, however,
this comparison has become somewhat unwieldy.

You can execute the tests using
```matlab
testGraphical;
```
or, equivalently,
```matlab
makeLatexReport(testGraphical)
```
This generates a LaTeX report in `test/output/current/acid.tex` which can then be compiled.
Compilation of this file can be done using the Makefile `test/output/current/Makefile` if you are on a Unix-like system (OS X, Linux) or have [Cygwin](https://www.cygwin.com) installed on Windows.

If all goes well, the result will be the file `test/output/current/acid.pdf` that contains
a list of the test figures, exported as PDF and right next to it the matlab2tikz generated plot.

Advanced Use
------------

Both `testHeadless` and `testGraphical` can take multiple arguments, those are documented in the raw test runner `testMatlab2tikz` that is used behind the scenes. Note that this file sits in a private directory, so `help testMatlab2tikz` will not work!

Also, both can be called with a single output argument, for programmatical
access to the test results as
```matlab
status = testHeadless()
```
These test results in `status` can be passed to `saveHashTable` for updating the hash tables.
Obviously, this should be done with the due diligence!

Automated Tests
===============

The automated tests run on [Travis-CI](https://travis-ci.org) for Octave and on a [personal Jenkins server](https://github.com/matlab2tikz/matlab2tikz/wiki/Jenkins) for MATLAB.
These are effectively the "headless" tests that get called by the  `runMatlab2TikzTests` function.
Without verification of those automated tests, a pull request is unlikely to get merged.