Mn Python tests

Marionette is the codename of a [remote protocol] built in to Firefox as well as the name of a functional test framework for automating user interface tests.

The in-tree test framework supports tests written in Python, using Python’s [unittest] library. Test cases are written as a subclass of MarionetteTestCase, with child tests belonging to instance methods that have a name starting with test_.

You can additionally define [setUp] and [tearDown] instance methods to execute code before and after child tests, and [setUpClass]/[tearDownClass] for the parent test. When you use these, it is important to remember calling the MarionetteTestCase superclass’ own [setUp]/[tearDown] methods since they handle setup/cleanup of the session.

The test structure is illustrated here:

from marionette_harness import MarionetteTestCase

class TestSomething(MarionetteTestCase):
    def setUp(self):
        # code to execute before any tests are run
        MarionetteTestCase.setUp(self)

    def test_foo(self):
        # run test for 'foo'

    def test_bar(self):
        # run test for 'bar'

    def tearDown(self):
        # code to execute after all tests are run
        MarionetteTestCase.tearDown(self)

[remote protocol]: Protocol.md [unittest]: https://docs.python.org/3/library/unittest.html [setUp]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUp [setUpClass]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.setUpClass [tearDown]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.tearDown [tearDownClass]: https://docs.python.org/3/library/unittest.html#unittest.TestCase.tearDownClass

Test assertions

Assertions are provided courtesy of [unittest]. For example:

from marionette_harness import MarionetteTestCase

class TestSomething(MarionetteTestCase):
    def test_foo(self):
        self.assertEqual(9, 3 * 3, '3 x 3 should be 9')
        self.assertTrue(type(2) == int, '2 should be an integer')

The API

The full API documentation is found [here], but the key objects are:

• MarionetteTestCase: a subclass for unittest.TestCase

used as a base class for all tests to run.

• {class}Marionette <marionette_driver.marionette.Marionette>: client that speaks to Firefox

[here]: /python/marionette_driver.rst

Registering Test Manifests

To run Marionette Python tests locally via mach or as part of the Mn tests jobs in CI they need to be registered. This happens by adding a manifest file to the tree, which includes a reference to the test files and expectations for results.

Such a manifest file can look like the following and is stored with the extension .toml:

[DEFAULT]

["test_expected_fail.py"]
expected = "fail"

["test_not_on_windows.py"]
skip-if = ["os == 'win'"]

The registration of such a manifest file is done in two different ways:

1. {class}Marionette <marionette_driver.marionette.Marionette>: client that speaks to Firefox

created Marionette manifest file needs to be referenced in the folder's related moz.build file by adding it to the MARIONETTE_MANIFESTS variable like:

MARIONETTE_MANIFESTS += ["test/marionette/manifest.toml"]

1. {class}Marionette <marionette_driver.marionette.Marionette>: client that speaks to Firefox

Marionette's own [master manifest file]. This ensures that the test packaging step will find the tests and include them as well in the test package.

[master manifest file]: https://searchfox.org/mozilla-central/source/testing/marionette/harness/marionette_harness/tests/unit-tests.toml