1 .. highlight:: javascript
6 Javascript Unit Testing
7 -----------------------
9 OpenERP Web 7.0 includes means to unit-test both the core code of
10 OpenERP Web and your own javascript modules. On the javascript side,
11 unit-testing is based on QUnit_ with a number of helpers and
12 extensions for better integration with OpenERP.
14 To see what the runner looks like, find (or start) an OpenERP server
15 with the web client enabled, and navigate to ``/web/tests`` e.g. `on
16 OpenERP's CI <http://trunk.runbot.openerp.com/web/tests>`_. This will
17 show the runner selector, which lists all modules with javascript unit
18 tests, and allows starting any of them (or all javascript tests in all
21 .. image:: ./images/runner.png
24 Clicking any runner button will launch the corresponding tests in the
25 bundled QUnit_ runner:
27 .. image:: ./images/tests.png
33 The first step is to list the test file(s). This is done through the
34 ``test`` key of the openerp manifest, by adding javascript files to it
35 (next to the usual YAML files, if any):
37 .. code-block:: python
40 'name': "Demonstration of web/javascript tests",
43 'test': ['static/test/demo.js'],
46 and to create the corresponding test file(s)
50 Test files which do not exist will be ignored, if all test files
51 of a module are ignored (can not be found), the test runner will
52 consider that the module has no javascript tests.
54 After that, refreshing the runner selector will display the new module
55 and allow running all of its (0 so far) tests:
57 .. image:: ./images/runner2.png
60 The next step is to create a test case::
62 openerp.testing.section('basic section', function (test) {
63 test('my first test', function () {
64 ok(false, "this test has run");
68 All testing helpers and structures live in the ``openerp.testing``
69 module. OpenERP tests live in a :js:func:`~openerp.testing.section`,
70 which is itself part of a module. The first argument to a section is
71 the name of the section, the second one is the section body.
73 :js:func:`test <openerp.testing.case>`, provided by the
74 :js:func:`~openerp.testing.section` to the callback, is used to
75 register a given test case which will be run whenever the test runner
76 actually does its job. OpenERP Web test case use standard `QUnit
77 assertions`_ within them.
79 Launching the test runner at this point will run the test and display
80 the corresponding assertion message, with red colors indicating the
83 .. image:: ./images/tests2.png
86 Fixing the test (by replacing ``false`` to ``true`` in the assertion)
89 .. image:: ./images/tests3.png
95 As noted above, OpenERP Web's tests use `qunit assertions`_. They are
96 available globally (so they can just be called without references to
97 anything). The following list is available:
99 .. js:function:: ok(state[, message])
101 checks that ``state`` is truthy (in the javascript sense)
103 .. js:function:: strictEqual(actual, expected[, message])
105 checks that the actual (produced by a method being tested) and
106 expected values are identical (roughly equivalent to ``ok(actual
107 === expected, message)``)
109 .. js:function:: notStrictEqual(actual, expected[, message])
111 checks that the actual and expected values are *not* identical
112 (roughly equivalent to ``ok(actual !== expected, message)``)
114 .. js:function:: deepEqual(actual, expected[, message])
116 deep comparison between actual and expected: recurse into
117 containers (objects and arrays) to ensure that they have the same
118 keys/number of elements, and the values match.
120 .. js:function:: notDeepEqual(actual, expected[, message])
122 inverse operation to :js:func:`deepEqual`
124 .. js:function:: throws(block[, expected][, message])
126 checks that, when called, the ``block`` throws an
127 error. Optionally validates that error against ``expected``.
129 :param Function block:
130 :param expected: if a regexp, checks that the thrown error's
131 message matches the regular expression. If an
132 error type, checks that the thrown error is of
134 :type expected: Error | RegExp
136 .. js:function:: equal(actual, expected[, message])
138 checks that ``actual`` and ``expected`` are loosely equal, using
139 the ``==`` operator and its coercion rules.
141 .. js:function:: notEqual(actual, expected[, message])
143 inverse operation to :js:func:`equal`
145 Getting an OpenERP instance
146 ---------------------------
148 The OpenERP instance is the base through which most OpenERP Web
149 modules behaviors (functions, objects, …) are accessed. As a result,
150 the test framework automatically builds one, and loads the module
151 being tested and all of its dependencies inside it. This new instance
152 is provided as the first positional parameter to your test
153 cases. Let's observe by adding javascript code (not test code) to the
156 .. code-block:: python
159 'name': "Demonstration of web/javascript tests",
160 'category': 'Hidden',
162 'js': ['static/src/js/demo.js'],
163 'test': ['static/test/demo.js'],
169 openerp.web_tests_demo = function (instance) {
170 instance.web_tests_demo = {
172 SomeType: instance.web.Class.extend({
173 init: function (value) {
180 and then adding a new test case, which simply checks that the
181 ``instance`` contains all the expected stuff we created in the
185 test('module content', function (instance) {
186 ok(instance.web_tests_demo.value_true, "should have a true value");
187 var type_instance = new instance.web_tests_demo.SomeType(42);
188 strictEqual(type_instance.value, 42, "should have provided value");
194 As in the wider client, arbitrarily accessing document content is
195 strongly discouraged during tests. But DOM access is still needed to
196 e.g. fully initialize :js:class:`widgets <~openerp.web.Widget>` before
199 Thus, a test case gets a DOM scratchpad as its second positional
200 parameter, in a jQuery instance. That scratchpad is fully cleaned up
201 before each test, and as long as it doesn't do anything outside the
202 scratchpad your code can do whatever it wants::
205 test('DOM content', function (instance, $scratchpad) {
206 $scratchpad.html('<div><span class="foo bar">ok</span></div>');
207 ok($scratchpad.find('span').hasClass('foo'),
208 "should have provided class");
210 test('clean scratchpad', function (instance, $scratchpad) {
211 ok(!$scratchpad.children().length, "should have no content");
212 ok(!$scratchpad.text(), "should have no text");
217 The top-level element of the scratchpad is not cleaned up, test
218 cases can add text or DOM children but shoud not alter
219 ``$scratchpad`` itself.
224 To avoid the corresponding processing costs, by default templates are
225 not loaded into QWeb. If you need to render e.g. widgets making use of
226 QWeb templates, you can request their loading through the
227 :js:attr:`~TestOptions.templates` option to the :js:func:`test case
228 function <openerp.testing.case>`.
230 This will automatically load all relevant templates in the instance's
231 qweb before running the test case:
233 .. code-block:: python
236 'name': "Demonstration of web/javascript tests",
237 'category': 'Hidden',
239 'js': ['static/src/js/demo.js'],
240 'test': ['static/test/demo.js'],
241 'qweb': ['static/src/xml/demo.xml'],
246 <!-- src/xml/demo.xml -->
247 <templates id="template" xml:space="preserve">
248 <t t-name="DemoTemplate">
249 <t t-foreach="5" t-as="value">
250 <p><t t-esc="value"/></p>
258 test('templates', {templates: true}, function (instance) {
259 var s = instance.web.qweb.render('DemoTemplate');
260 var texts = $(s).find('p').map(function () {
261 return $(this).text();
264 deepEqual(texts, ['0', '1', '2', '3', '4']);
270 The test case examples so far are all synchronous, they execute from
271 the first to the last line and once the last line has executed the
272 test is done. But the web client is full of :doc:`asynchronous code
273 </async>`, and thus test cases need to be async-aware.
275 This is done by returning a :js:class:`deferred <Deferred>` from the
279 test('asynchronous', {
282 var d = $.Deferred();
283 setTimeout(function () {
290 This example also uses the :js:class:`options parameter <TestOptions>`
291 to specify the number of assertions the case should expect, if less or
292 more assertions are specified the case will count as failed.
294 Asynchronous test cases *must* specify the number of assertions they
295 will run. This allows more easily catching situations where e.g. the
296 test architecture was not warned about asynchronous operations.
300 Asynchronous test cases also have a 2 seconds timeout: if the test
301 does not finish within 2 seconds, it will be considered
302 failed. This pretty much always means the test will not
303 resolve. This timeout *only* applies to the test itself, not to
304 the setup and teardown processes.
308 If the returned deferred is rejected, the test will be failed
309 unless :js:attr:`~TestOptions.fail_on_rejection` is set to
315 An important subset of asynchronous test cases is test cases which
316 need to perform (and chain, to an extent) RPC calls.
320 Because they are a subset of asynchronous cases, RPC cases must
321 also provide a valid :js:attr:`assertions count
322 <TestOptions.asserts>`.
324 By default, test cases will fail when trying to perform an RPC
325 call. The ability to perform RPC calls must be explicitly requested by
326 a test case (or its containing test suite) through
327 :js:attr:`~TestOptions.rpc`, and can be one of two modes: ``mock`` or
333 The preferred (and fastest from a setup and execution time point of
334 view) way to do RPC during tests is to mock the RPC calls: while
335 setting up the test case, provide what the RPC responses "should" be,
336 and only test the code between the "user" (the test itself) and the
337 RPC call, before the call is effectively done.
339 To do this, set the :js:attr:`rpc option <TestOptions.rpc>` to
340 ``mock``. This will add a third parameter to the test case callback:
342 .. js:function:: mock(rpc_spec, handler)
344 Can be used in two different ways depending on the shape of the
347 * If it matches the pattern ``model:method`` (if it contains a
348 colon, essentially) the call will set up the mocking of an RPC
349 call straight to the OpenERP server (through XMLRPC) as
350 performed via e.g. :js:func:`openerp.web.Model.call`.
352 In that case, ``handler`` should be a function taking two
353 arguments ``args`` and ``kwargs``, matching the corresponding
354 arguments on the server side and should simply return the value
355 as if it were returned by the Python XMLRPC handler::
357 test('XML-RPC', {rpc: 'mock', asserts: 3}, function (instance, $s, mock) {
359 mock('people.famous:name_search', function (args, kwargs) {
360 strictEqual(kwargs.name, 'bob');
362 [1, "Microsoft Bob"],
363 [2, "Bob the Builder"],
369 return new instance.web.Model('people.famous')
370 .call('name_search', {name: 'bob'}).then(function (result) {
371 strictEqual(result.length, 3, "shoud return 3 people");
372 strictEqual(result[0][1], "Microsoft Bob",
373 "the most famous bob should be Microsoft Bob");
377 * Otherwise, if it matches an absolute path (e.g. ``/a/b/c``) it
378 will mock a JSON-RPC call to a web client controller, such as
379 ``/web/webclient/translations``. In that case, the handler takes
380 a single ``params`` argument holding all of the parameters
381 provided over JSON-RPC.
383 As previously, the handler should simply return the result value
384 as if returned by the original JSON-RPC handler::
386 test('JSON-RPC', {rpc: 'mock', asserts: 3, templates: true}, function (instance, $s, mock) {
387 var fetched_dbs = false, fetched_langs = false;
388 mock('/web/database/get_list', function () {
390 return ['foo', 'bar', 'baz'];
392 mock('/web/session/get_lang_list', function () {
393 fetched_langs = true;
394 return [['vo_IS', 'Hopelandic / Vonlenska']];
397 // widget needs that or it blows up
398 instance.webclient = {toggle_bars: openerp.testing.noop};
399 var dbm = new instance.web.DatabaseManager({});
400 return dbm.appendTo($s).then(function () {
401 ok(fetched_dbs, "should have fetched databases");
402 ok(fetched_langs, "should have fetched languages");
403 deepEqual(dbm.db_list, ['foo', 'bar', 'baz']);
409 Mock handlers can contain assertions, these assertions should be
410 part of the assertions count (and if multiple calls are made to a
411 handler containing assertions, it multiplies the effective number
419 A more realistic (but significantly slower and more expensive) way to
420 perform RPC calls is to perform actual calls to an actually running
421 OpenERP server. To do this, set the :js:attr:`rpc option
422 <~TestOptions.rpc>` to ``rpc``, it will not provide any new parameter
423 but will enable actual RPC, and the automatic creation and destruction
424 of databases (from a specified source) around tests.
426 First, create a basic model we can test stuff with:
428 .. code-block:: javascript
430 from openerp.osv import orm, fields
432 class TestObject(orm.Model):
433 _name = 'web_tests_demo.model'
436 'name': fields.char("Name", required=True),
437 'thing': fields.char("Thing"),
438 'other': fields.char("Other", required=True)
444 then the actual test::
446 test('actual RPC', {rpc: 'rpc', asserts: 4}, function (instance) {
447 var Model = new instance.web.Model('web_tests_demo.model');
448 return Model.call('create', [{name: "Bob"}])
449 .then(function (id) {
450 return Model.call('read', [[id]]);
451 }).then(function (records) {
452 strictEqual(records.length, 1);
453 var record = records[0];
454 strictEqual(record.name, "Bob");
455 strictEqual(record.thing, false);
457 strictEqual(record.other, 'bob');
461 This test looks like a "mock" RPC test but for the lack of mock
462 response (and the different ``rpc`` type), however it has further
463 ranging consequences in that it will copy an existing database to a
464 new one, run the test in full on that temporary database and destroy
465 the database, to simulate an isolated and transactional context and
466 avoid affecting other tests. One of the consequences is that it takes
467 a *long* time to run (5~10s, most of that time being spent waiting for
468 a database duplication).
470 Furthermore, as the test needs to clone a database, it also has to ask
471 which database to clone, the database/super-admin password and the
472 password of the ``admin`` user (in order to authenticate as said
473 user). As a result, the first time the test runner encounters an
474 ``rpc: "rpc"`` test configuration it will produce the following
477 .. image:: ./images/db-query.png
480 and stop the testing process until the necessary information has been
483 The prompt will only appear once per test run, all tests will use the
484 same "source" database.
488 The handling of that information is currently rather brittle and
489 unchecked, incorrect values will likely crash the runner.
493 The runner does not currently store this information (for any
494 longer than a test run that is), the prompt will have to be filled
500 .. js:function:: openerp.testing.section(name[, options], body)
502 A test section, serves as shared namespace for related tests (for
503 constants or values to only set up once). The ``body`` function
504 should contain the tests themselves.
506 Note that the order in which tests are run is essentially
507 undefined, do *not* rely on it.
510 :param TestOptions options:
512 :type body: Function<:js:func:`~openerp.testing.case`, void>
514 .. js:function:: openerp.testing.case(name[, options], callback)
516 Registers a test case callback in the test runner, the callback
517 will only be run once the runner is started (or maybe not at all,
518 if the test is filtered out).
521 :param TestOptions options:
523 :type callback: Function<instance, $, Function<String, Function, void>>
525 .. js:class:: TestOptions
527 the various options which can be passed to
528 :js:func:`~openerp.testing.section` or
529 :js:func:`~openerp.testing.case`. Except for
530 :js:attr:`~TestOptions.setup` and
531 :js:attr:`~TestOptions.teardown`, an option on
532 :js:func:`~openerp.testing.case` will overwrite the corresponding
533 option on :js:func:`~openerp.testing.section` so
534 e.g. :js:attr:`~TestOptions.rpc` can be set for a
535 :js:func:`~openerp.testing.section` and then differently set for
536 some :js:func:`~openerp.testing.case` of that
537 :js:func:`~openerp.testing.section`
539 .. js:attribute:: TestOptions.asserts
541 An integer, the number of assertions which should run during a
542 normal execution of the test. Mandatory for asynchronous tests.
544 .. js:attribute:: TestOptions.setup
546 Test case setup, run right before each test case. A section's
547 :js:func:`~TestOptions.setup` is run before the case's own, if
550 .. js:attribute:: TestOptions.teardown
552 Test case teardown, a case's :js:func:`~TestOptions.teardown`
553 is run before the corresponding section if both are present.
555 .. js:attribute:: TestOptions.fail_on_rejection
557 If the test is asynchronous and its resulting promise is
558 rejected, fail the test. Defaults to ``true``, set to
559 ``false`` to not fail the test in case of rejection::
562 test('unfail rejection', {
564 fail_on_rejection: false
566 var d = $.Deferred();
567 setTimeout(function () {
574 .. js:attribute:: TestOptions.rpc
576 RPC method to use during tests, one of ``"mock"`` or
577 ``"rpc"``. Any other value will disable RPC for the test (if
578 they were enabled by the suite for instance).
580 .. js:attribute:: TestOptions.templates
582 Whether the current module (and its dependencies)'s templates
583 should be loaded into QWeb before starting the test. A
584 boolean, ``false`` by default.
586 The test runner can also use two global configuration values set
587 directly on the ``window`` object:
589 * ``oe_all_dependencies`` is an ``Array`` of all modules with a web
590 component, ordered by dependency (for a module ``A`` with
591 dependencies ``A'``, any module of ``A'`` must come before ``A`` in
594 * ``oe_db_info`` is an object with 3 keys ``source``, ``supadmin`` and
595 ``password``. It is used to pre-configure :ref:`actual RPC
596 <testing-rpc-rpc>` tests, to avoid a prompt being displayed
597 (especially for headless situations).
599 Running through Python
600 ----------------------
602 The web client includes the means to run these tests on the
603 command-line (or in a CI system), but while actually running it is
604 pretty simple the setup of the pre-requisite parts has some
607 1. Install unittest2_ and QUnitSuite_ in your Python environment. Both
608 can trivially be installed via `pip <http://pip-installer.org>`_ or
610 <http://packages.python.org/distribute/easy_install.html>`_.
612 The former is the unit-testing framework used by OpenERP, the
613 latter is an adapter module to run qunit_ test suites and convert
614 their result into something unittest2_ can understand and report.
616 2. Install PhantomJS_. It is a headless
617 browser which allows automating running and testing web
618 pages. QUnitSuite_ uses it to actually run the qunit_ test suite.
620 The PhantomJS_ website provides pre-built binaries for some
621 platforms, and your OS's package management probably provides it as
624 If you're building PhantomJS_ from source, I recommend preparing
625 for some knitting time as it's not exactly fast (it needs to
626 compile both `Qt <http://qt-project.org/>`_ and `Webkit
627 <http://www.webkit.org/>`_, both being pretty big projects).
631 Because PhantomJS_ is webkit-based, it will not be able to test
632 if Firefox, Opera or Internet Explorer can correctly run the
633 test suite (and it is only an approximation for Safari and
634 Chrome). It is therefore recommended to *also* run the test
635 suites in actual browsers once in a while.
639 The version of PhantomJS_ this was build through is 1.7,
640 previous versions *should* work but are not actually supported
641 (and tend to just segfault when something goes wrong in
642 PhantomJS_ itself so they're a pain to debug).
644 3. Set up :ref:`OpenERP Command <openerpcommand:openerp-command>`,
645 which will be used to actually run the tests: running the qunit_
646 test suite requires a running server, so at this point OpenERP
647 Server isn't able to do it on its own during the building/testing
650 4. Install a new database with all relevant modules (all modules with
651 a web component at least), then restart the server
655 For some tests, a source database needs to be duplicated. This
656 operation requires that there be no connection to the database
657 being duplicated, but OpenERP doesn't currently break
658 existing/outstanding connections, so restarting the server is
659 the simplest way to ensure everything is in the right state.
661 5. Launch ``oe run-tests -d $DATABASE -mweb`` with the correct
662 addons-path specified (and replacing ``$DATABASE`` by the source
663 database you created above)
667 If you leave out ``-mweb``, the runner will attempt to run all
668 the tests in all the modules, which may or may not work.
670 If everything went correctly, you should now see a list of tests with
671 (hopefully) ``ok`` next to their names, closing with a report of the
672 number of tests run and the time it took:
674 .. literalinclude:: test-report.txt
677 Congratulation, you have just performed a successful "offline" run of
678 the OpenERP Web test suite.
682 Note that this runs all the Python tests for the ``web`` module,
683 but all the web tests for all of OpenERP. This can be surprising.
685 .. _qunit: http://qunitjs.com/
687 .. _qunit assertions: http://api.qunitjs.com/category/assert/
689 .. _unittest2: http://pypi.python.org/pypi/unittest2
691 .. _QUnitSuite: http://pypi.python.org/pypi/QUnitSuite/
693 .. _PhantomJS: http://phantomjs.org/