Skip to main content

Tasks

Every topic in a JSON configuration file must include at least one task for the agent to complete. The type parameter denotes the action the agent will take. Each task type includes additional parameters specific to the task that is being performed.

Voice Tasks​

Voice tasks use our integrated text-to-speech, speech-to-text, and LLM to orchestrate audio conversations. Currently agents built using the JSON configuration file can only complete voice tasks over Zoom meetings.

speak​

Have the agent speak over audio.

The agent can speak an exact script using the copy parameter:

Or you can specify instructions for the LLM to generate spoken words using the instructions array:

question​

Have the agent ask the end user a question over audio. After the question is asked, the agent will indicate it is awaiting a response by making a notification sound, and will begin listening for input. After the user finishes speaking, the agent will make another notification sound to acknowledge that the response has been recorded.

The agent can optionally respond to a user question using the optional_response parameter.

Questions must be included as the last task on a topic.

Browser Tasks​

note

Browser tasks require hardcoded values for Javascript selectors, input content and URLs. If you want to set these values dynamically, please use the API.

go_to_URL​

Have the agent navigate to the specified URL.

Additional parameters

  • url (required): the URL to navigate to
"type": "go_to_url",   
"url": "http://example.com/login"

click​

Instructs the agent to click on a specific element on the page.

Additional parameters

  • selector (required): the Javascript selector to click on
"type": "click",
"selector": "#submit"

scroll​

Have the agent scroll to an element on a webpage, specified by the CSS selector.

Additional parameters

  • selector (required): the Javascript selector to scroll to. This element will be positioned at the top of the Agent's browser.
  • speed (optional): how quickly the Agent scrolls down the page - can be slow, medium, or fast (if unspecified, defaults to medium)
"type": "scroll",
"selector": "#footer",
"speed": "slow"

hover​

Directs the agent to hover its mouse over an element on the page.

Additional parameters

  • selector (required): the Javascript selector
"type": "hover",
"selector": "#live-stream > h2"

input​

Have the agent input text into a text field on a webpage.

Additional parameters

  • selector (required): the Javascript selector of the field
  • copy (required): the text to input
  • speed (optional): how quickly the Agent fills in the field - can be slow, medium, or fast (if unspecified, defaults to medium)
"type": "input",
"selector": "#phoneNumber",
"speed": "medium",
"copy": "123-456-7890"

select_input​

Instructs the agent to select an option from a dropdown or multi-select field.

Additional parameters

  • selector (required): the Javascript selector for the dropdown
  • values (required): An array of options to select.
"type": "select_input",
"selector": "#toppings",
"values": ["cheese", "tomatoes", "lettuce"]

keyboard​

Have the agent press a key on its virtual keyboard within the browser.

Additional parameters

    "type": "keyboard",
    "selector": "press",
    "text": "ArrowRight"
wait​

Have the agent wait a number of sections before completing the next task.

Additional parameters

  • seconds (required): the number of seconds the agent should wait.
    "type": "wait",
    "seconds": 5

screenshot​

Take a screenshot of the current active tab, and save it to workstation files.

    "type": "screenshot"

Meeting Tasks​

share_screen​

Have the agent share its screen in the current meeting. Use the exact selector value shown below.

Additional parameters

  • selector (required): the Javascript selector to start screenshare. Use the exact selector shown below - do not change this value.
    "type": "share_screen",
    "selector": "button[aria-label=\"Share Screen\"]"

stop_share​

Have the agent stop sharing its screen in the current meeting. Use the exact selector value shown below.

Additional parameters

  • selector (required): the Javascript selector to stop screenshare. Use the exact selector shown below - do not change this value.
"type": "stop_share",
"selector": "button.sharer-button.sharer-button--stop"