It can be time-consuming to test queries in large knowledge maps that require a lot of user input. Automated tests can be used to reduce testing time.
Automated tests run through different scenarios and different interactions without the need to manually answer questions every time you run a query. Once a test has been created, it can be ran as many times as required.
Automated testing enables any changes made to the knowledge map to be quickly tested to confirm whether new any logic added to the knowledge map has not changed the expected behaviour of the knowledge map.
Note: Automated tests will always test the draft version of a knowledge map.
Setting up an automated test
To create a new automated test:
- Click on ‘TEST’
- Navigate to the ‘Automated Tests’ page
- Click ‘New Test’
- Entering the query you want to run and provide either the subject or object instance (just as you would when setting up a query)
- Click ‘next’ and the test will ask to define the object for each relationship used to find an answer to the intitial query or present a result if Rainbird is expected to produce a result at this point in the query
- Repeat step five until all questions for a query have a defined object and Rainbird produces a result(s)
Rainbird will automatically populate each step involved in a query if you click next, but each step can be entered manually by selecting the ‘+’ button and telling Rainbird whether to expect a question or a result at that point in the query.
If Rainbird expects a question, you can type the subject, relationship, object and certainty factor to respond with (if a second form question), or the yes/no response in the event of a first form question being asked. If Rainbird expects a response you can type or refine the response that Rainbird is expected to return. You can also choose to provide no response to a question. Providing no response for a question will replace the step with ‘Skip response‘ when the test is saved.
Note: You should always finish an automated test with an ‘Expect Result’ step; even if no response is expected, you can specify ‘No result’.
Figure 1: Setting up an automated test
Running an automated test
Once a test has been set up, it can be run at a later date to verify that a query runs as expected. Clicking the play button ‘>’ in the header of each test box will cause the test to run through each step, stopping at the first fail point, if one exists.
If the test comes across an unexpected question or result, you will presented with an ‘Add to Test’ button. Clicking the button will swap to the test to edit mode where you can add in the question or response, if applicable. Alternatively, you can amend the knowledge map as required and then re-run the test.
Test failure
There are two possible causes for a unit test to fail:
- Unexpected question or result
If Rainbird returns a question that wasn’t expected, then a test will fail. Similarly, if Rainbird returns an unexpected result, or if an expected result has not been specified, the test will fail. It’s worth noting that questions are expected to be asked in the specified test order and the test will fail if this is not the case.
- API call – bad request
Rainbird rejects API calls that do not correspond to the structure of the knowledge map. For example, Rainbird if a response generates multiple answers to a question whose relationship is singular.
