TestU01

<h2 id="history">History</h2>
<p>An initial battery of randomness tests for RNGs was suggested in the 1969 first edition of <i><a href="/facts/The_Art_of_Computer_Programming/MR1bMD9e">The Art of Computer Programming</a></i> by <a href="/facts/Donald_Knuth/GozNzDM6">Donald Knuth</a>. Knuth's tests were then supplanted by <a href="/facts/George_Marsaglia/Jlu1Defc">George Marsaglia</a>'s <a href="/facts/Diehard_tests/6jj2Kylt">Diehard tests</a> (1996) consisting of fifteen different tests.  The inability to modify the test parameters or add new tests led to the development of the TestU01 library.
</p>
<h2 id="features">Features</h2>
<p>TestU01 offers four groups of modules for analyzing RNGs: 
</p>
<ol><li>Implementing (pre-programmed) RNGs;</li>
<li>Implementing specific statistical tests;</li>
<li>Implementing batteries of statistical tests;</li>
<li>Applying tests to entire families of RNGs.</li></ol>
<p>When a specific test is applied to a sample of size <i>n</i> produced by an RNG, the <a href="/facts/P-value/InnYd1bz"><i>p</i>-value</a> of the test usually will remain reasonable as the sample size increases until the sample size hits <i>n</i>0, say. After that, the <i>p</i>-value diverges to 0 or 1 with exponential speed. Module 4 allows the researcher to study the interaction between a specific test and the structure of the point sets produced by a given family of RNGs. This technique can be used to determine how large the sample size should be, as a function of the generator's period length, before the generator starts to fail the test systematically.
</p><p>TESTU01 offers several batteries of tests including "Small Crush" (which consists of 10 tests), "Crush" (96 tests), and "Big Crush" (106 tests). The specific tests applied by each battery are detailed in the user's guide.<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a> On a 1.7 GHz <a href="/facts/Pentium_4/6tIFuHIF">Pentium 4</a> running <a href="/facts/Red_Hat_Linux/kIU5PueU">Red Hat Linux</a> 9.0, for a simple RNG, Small Crush takes about 2 minutes. Crush takes about 1.7 hours. Big Crush takes about 4 hours. For a more complex RNG, all these times
increase by a factor of two or more. For comparison, the Diehard tests take about 15 seconds to run.
</p>
<h2 id="limitations">Limitations</h2>
<p>TestU01 only accepts 32-bit inputs, and interprets them as values in the range [0, 1].  This causes it to be more sensitive to flaws in the most-significant bits than the least significant bits.  It is important to test general-purpose generators in bit-reversed form, to verify their suitability for applications which use the low-order bits.<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a>: 4 
</p><p>Generators which produce 64 bits of output additionally require separate tests for their high and low halves.<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a>: 51 
</p>
<h2 id="see-also">See also</h2>
<ul><li><a href="/facts/Randomness_tests/My9XgtxB">Randomness tests</a></li>
<li><a href="/facts/Diehard_tests/6jj2Kylt">Diehard tests</a></li>
<li>PractRand</li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1"><p>The TestU01 web site. <a href="http://simul.iro.umontreal.ca/testu01/tu01.html" target="_blank">http://simul.iro.umontreal.ca/testu01/tu01.html</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li>
<li id="fn:2"><p>Pierre L’Ecuyer & Richard Simard (2007), "TestU01: A Software Library in ANSI C for Empirical Testing of Random Number Generators", ACM Transactions on Mathematical Software, 33: 22. <a href="http://dl.acm.org/citation.cfm?doid=1268776.1268777" target="_blank">http://dl.acm.org/citation.cfm?doid=1268776.1268777</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li>
<li id="fn:3"><p>TestU01 User's Guide. <a href="http://www.iro.umontreal.ca/~simardr/testu01/guideshorttestu01.pdf" target="_blank">http://www.iro.umontreal.ca/~simardr/testu01/guideshorttestu01.pdf</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li>
<li id="fn:4"><p>Vigna, Sebastiano (July 2016). "An experimental exploration of Marsaglia's xorshift generators, scrambled" (PDF). ACM Transactions on Mathematical Software. 42 (4): 30. arXiv:1402.6246. doi:10.1145/2845077. S2CID 13936073. <a href="http://vigna.di.unimi.it/ftp/papers/xorshift.pdf" target="_blank">http://vigna.di.unimi.it/ftp/papers/xorshift.pdf</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li>
<li id="fn:5"><p>O'Neill, Melissa E. (5 September 2014). PCG: A Family of Simple Fast Space-Efficient Statistically Good Algorithms for Random Number Generation (PDF) (Technical report). Harvey Mudd College. HMC-CS-2014-0905. <a href="http://www.pcg-random.org/pdf/hmc-cs-2014-0905.pdf#page=53" target="_blank">http://www.pcg-random.org/pdf/hmc-cs-2014-0905.pdf#page=53</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li>
</ol>

TestU01 open-in-new

TestU01