running_nspr_tests.rst (3036B)
1 Running NSPR tests 2 ================== 3 4 NSPR has a test suite in the ``pr/tests`` directory. 5 6 By default, we don't build the test programs. Running ``make`` in the 7 top-level directory only builds the NSPR libraries. 8 To build the test programs, you need to change directory to 9 ``pr/tests`` and run ``make``. Refer to :ref:`NSPR build 10 instructions` for details. 11 12 To run the test suite, run the shell script 13 ``pr/tests/runtests.sh`` in the directory where the test 14 program binaries reside, for example, 15 16 .. code:: 17 18 hg clone https://hg.mozilla.org/projects/nspr 19 cd nspr/ 20 ./configure 21 make 22 cd pr/tests 23 make runtests 24 25 The output of the test suite looks like this: 26 27 .. code:: 28 29 NSPR Test Results - tests 30 31 BEGIN Mon Mar 12 11:44:41 PDT 2007 32 NSPR_TEST_LOGFILE /dev/null 33 34 Test Result 35 36 accept Passed 37 acceptread Passed 38 acceptreademu Passed 39 affinity Passed 40 alarm Passed 41 anonfm Passed 42 atomic Passed 43 attach Passed 44 bigfile Passed 45 cleanup Passed 46 cltsrv Passed 47 concur Passed 48 cvar Passed 49 cvar2 Passed 50 ... 51 sprintf FAILED 52 ... 53 timetest Passed 54 tpd Passed 55 udpsrv Passed 56 vercheck Passed 57 version Passed 58 writev Passed 59 xnotify Passed 60 zerolen Passed 61 END Mon Mar 12 11:55:47 PDT 2007 62 63 .. _How_to_determine_if_the_test_suite_passed: 64 65 How to determine if the test suite passed 66 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 67 68 If all the tests reported **Passed** as the results, the test suite 69 passed. 70 71 What if some of the tests crashed or reported **FAILED** as the results? 72 It doesn't necessarily mean the test suite failed because some of the 73 test programs are known to fail. Until the test failures are fixed, you 74 should run NSPR tests against **a known good version of NSPR on the same 75 platform**, and save the test results as the benchmark. Then you can 76 detect regressions of the new version by comparing its test results with 77 the benchmark. 78 79 .. _Known_issues: 80 81 Known issues 82 ~~~~~~~~~~~~ 83 84 Other issues with the NSPR test suite are: 85 86 #. Some of the test programs test the accuracy of the timeout of NSPR 87 functions. Since none of our operating systems is a real-time OS, 88 such test programs may fail when the test machine is heavily loaded. 89 #. Some tests, such as ``pipepong`` and ``sockpong``, should not be run 90 directly. They will be invoked by their companion test programs 91 (e.g., ``pipeping`` and ``sockping``). This is not an issue if you 92 run ``runtests.sh`` because ``runtests.sh`` knows not to run such 93 test programs directly.