How to create an IVR feature test using TestIVR

In this post we are going to develop an IVR feature test using TestIVR

After signing up the TestIVR web portal will provide you with an "API key" which can be used to call the TestIVR RESTful API. Note that at this moment you can only communicate with TestIVR feature testing tool through our API. Using the API key you can call the TestIVR feature testing tool via different tools such as "curl", python libraries such as "requests" or tools such as postman .

The TestIVR API end point is https://api.testivr.com/. Keep in mind that in all your API calls to TestIVR API endpoint, you should add your "API key" in a Header called "Authorization" and you need to add "Api-Key " at its begining. So for instance in "curl" you will have something like (replace XXXXX with your "API key"):

curl -X POST -H "Content-Type: application/json" -H "Authorization: Api-Key XXXXX" -d '{"description":"sample path"}' https://api.testivr.com/path/
                

Or in python:

import requests

headers = {
    'Content-Type': 'application/json',
    'Authorization': 'Api-Key XXXXX',
}

json_data = {
    'description': 'sample path',
}

response = requests.post('https://api.testivr.com/path/', headers=headers, json=json_data)
                

To create an IVR feature test using TestIVR, you should start with creating a "path". A "path" is basically a route for your test to follow. Later you will add "action"s (such as pressing a button or saying something to the IVR) to this path. To create a "path" you simply need to submit a POST request to /path/ endpoint and pass a description as part of the payload. For example:

curl --request POST \
  --url https://api.testivr.com/path/ \
  --header 'Authorization: Api-Key XXXXX' \
  --header 'Content-Type: application/json' \
  --data '{
  "description": "path to get the bank account balance"
}'
                

The API will response with something like this:

{
  "status": "OK",
  "data": {
    "id": 6,
    "description": "path to get the bank account balance",
    "user": 5,
    "created_at": "2022-04-19T20:23:37.874237Z"
  }
}
              

What is important from above is "id": 6 part which basically tells us what is the id of the new path we just created. You can read more about the API request for adding "path" in the Add Path section of our document.

Next step is adding "action"s to our new "path". Actions are the communications we define between the TestIVR and the IVR we are testing. For instance after TestIVR calling the IVR we want to test, if we need the TestIVR to press a button that is an "action" that we should define on the test "path". Or if the IVR is going to tell something to the caller and we want the TestIVR to evaluate what it said, that is another "action" we can add to the "path". To "create" an "action" you need to submit a POST request to /action/ endpoint and pass path, order, type, value and description as part of the payload. For instance:

curl --request POST \
  --url https://api.testivr.com/action/ \
  --header 'Authorization: Api-Key XXXXX' \
  --header 'Content-Type: application/json' \
  --data '{
  "description": "listening to the ivr intro",
  "path": 6,
  "order": 0,
  "type": "listen",
  "value": "Hi this is bank virtual assistant. If you want to get your account balance please press 1, for any other requests press 2",
  "asr_threshold": 0.9,
  "pause_before": 1.5,
  "pause_after": 2.5
}'
                

In this call, path should be set to the id of the "path" we want to add this action to. In our case we want to add it to the same path we created in our previous call to the API, which was "id": 6. So we set "path": 6.

The order defines the order of this action in our path. For instance if we want the test to first run this action then we can set the "order": 0 for this action and then use higher numbers (i.e. 1,2,3,...) for next orders.

The type defines what TestIVR should do as part of that action. type can be one of the followings:

listen: listentning to the IVR response, transcribing it and compare the transcription with provided text defined in the value field.
say: saying the expression defined in the value field to the IVR
press: pressing the number provided in the value field
command: running the command provide in the value field

At this moment the only available command is terminate, which ends the call.

The asr_threshold is only being used when the type is set to listen and it is being used as the "acceptance" threshold when the system compares the trsncribed response from IVR with the text provide in the value field.

The pause_before and pause_after are the amount of time (in seconds) tha the TestIVR should wait before triggering the next action. By default these values are set to zero, but if you have a special case that needs some delay between each trigger you can set that up using these fields.

Finally description is a field for providing a brief description of the action, which is provided only for documenting purposes.

The next is step is running the test on an IVR. You can do that by submitting a POST request to /test/ endpoint and pass the path you want to use for your test and the phone number of the IVR you want to run the test on as part of the payload. For example:

curl --request POST \
  --url https://api.testivr.com/test/ \
  --header 'Authorization: Api-Key XXXXX' \
  --header 'Content-Type: application/json' \
  --data '{
  "description": "running a test on bank IVR",
  "path": 6,
  "phone": "19713510653"
}'
                

The response you will get is going to be something like this:

{
  "status": "OK",
  "data": {
    "id": 61,
    "description": "path to get the bank account balance",
    "user": 5,
    "created_at": "2022-04-19T20:23:37.874237Z",
    "status": "running",
    "phone": "19713510653"
  }
}
                

What you need to keep from the response is the "id" value since you are going to need it to retrive the "outcome" of your test.

Finally to ge the result of your test you can submit a GET request to /outcome/ endpoint and pass the test parameter in the url (e.g. ?test=61). For example:

curl --request GET \
  --url 'https://api.testivr.com/outcome/?test=61' \
  --header 'Authorization: Api-Key XXXXX' \
  --header 'Content-Type: application/json'
              

The response is going to be something like this:

{
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
      {
        "id": 138,
        "outcome": "match",
        "user": 5,
        "created_at": "2022-04-19T20:23:37.874237Z",
        "test": 61,
        "action": 31,
        "value": "hi this is bank virtual assistant if you want to get your account balance please press one for any other requests press two",
        "expected_value": "Hi this is bank virtual assistant. If you want to get your account balance please press 1, for any other requests press 2",
        "match_ratio": 0.9085931558935361
      }
    ]
  }
              

In the response, each items of the resuls shows the outcome of one of the "actions" you had on the "path" that you have run a "test" for. You can see the "action" id in the action field of the response. The field outcome is important when your action had a "type" of "listen" and the outcome shows whether the response of the IVR is a "match" to the "value" you provided as the expected response.

The field match_ratio is also closely related to the outcome. Basically it shows the closeness of the (transcribed) IVR response, which is provided above as value to the expected result, provide as expected_value. If the calcuated match_ratio is more than the asr_threshold you have provided for the "action", the outcome is going to be "match".

This article shows how you can create a simple feature test for IVR using TestIVR. Please let us know if you have any question through our email: support@testivr.com

We also provide a good document on our API which provides more detailed information on all the calls you can make to TestIVR.