README.txt (4181B)
1 JS Test Suite Readme 2 ==================== 3 4 The JS test suite is a fairly extensive collection of correctness and regression 5 tests for the Spidermonkey engine. Two harnesses run these tests: the shell test 6 harness in this directory and the "reftest" harness built into the browser, used 7 in continuous integration. The browser reftests require additional manifest files; these are 8 generated automatically by the build phase 'package-tests' using the 9 '--make-manifests' option to jstests.py. 10 11 Running tests 12 ------------- 13 See 14 https://firefox-source-docs.mozilla.org/js/build.html#testing 15 16 Creating a test 17 --------------- 18 For general information, see 19 https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Creating_JavaScript_tests/jsreftests 20 21 22 Adding a test 23 ------------- 24 Drop your test file in an appropriate directory under js/src/tests/non262/. 25 26 <fineprint> Some names are forbidden. Do not name your test browser.js, 27 shell.js, template.js, user.js, js-test-driver-begin.js, or 28 js-test-driver-end.js. 29 </fineprint> 30 31 32 Test262, ECMAScript conformance tests 33 ------------------------------------- 34 35 js/src/tests/test262/ contains a snapshot of the ECMAScript 36 conformance test suite: https://github.com/tc39/test262. 37 38 We periodically update the snapshot, but it's a manual process: 39 40 1. File a bug, modeled after bug 1560206, to announce your intent to 41 update the test suite. 42 43 2. Run `test262-update.py`. 44 45 3. Run the tests. 46 47 4. Look at each failing test and either add an entry to the 48 `UNSUPPORTED_FEATURES` list in `test262-update.py` 49 or add a `skip script` entry to jstests.list, filing a bug if needed. 50 (Every skipped test should be associated with an open bug.) 51 52 5. Repeat steps 2-4 until there are no failing tests. 53 54 6. Post patches (as in bug 1560206). 55 56 When implementing a new feature, it is useful to enable tests for a 57 previously unsupported feature without going through all the steps 58 above. Here's how to do that: 59 60 1. Delete an entry from `UNSUPPORTED_FEATURES` in `test262-update.py`. 61 62 2. Run `./test262-update.py --revision $(head -n 1 test262/GIT-INFO | awk '{print $2}')`. 63 64 (This re-runs the script using the revision of test262 we most recently used, 65 instead of the latest revision.) 66 67 3. Run the tests. 68 69 To export tests to the test262 format, for potential contribution to 70 test262, see `test262-export.py`. 71 72 73 Adjusting when and how a test runs 74 ---------------------------------- 75 If the test is a test262 test, do this by editing jstests.list. 76 (The procedure below would be a bad idea because the test files are 77 autogenerated from the upstream test262 repository.) 78 79 Otherwise, put a comment at the top of the header matching the format: 80 // |reftest| <failure-type> -- <comment> 81 82 Where <failure-type> is a standard reftest <failure-type> string, as documented by: 83 http://searchfox.org/mozilla-central/source/layout/tools/reftest/README.txt 84 85 Example: 86 // |reftest| skip-if(!xulRuntime.shell) -- does not always dismiss alert 87 88 <fineprint> Either // or /* */ style comments may be used. The entire 89 comment must appear in the first 512 bytes of the file. The control 90 string must be in its own comment block. </fineprint> 91 92 When adding such comments to individual files is not feasible (e.g., for 93 imported tests), reftest manifest entries can be added to jstests.list 94 instead. Combining in-file comments with entries in this manifest file for 95 the same files is not supported (the one from the manifest file will be 96 used). Only the following two forms are supported: 97 <failure-type> include <relative_path> 98 <failure-type> script <relative_path> 99 include <relative_path> 100 The <type> "include" indicates that <failure-type> should apply to all test 101 cases within a directory. A statement for a nested directory or script 102 overrides one for an enclosing directory. The <type> "include" without a 103 <failure-type> recursively loads another jstests.list file for further 104 processing.