Updated Nov 9, 2026

Agent versions and tests

Publishing gives an agent numbered versions, so you can change it without touching what your contacts talk to, test the change, put it live, and go back if you need to.

How publishing works

Until you first publish an agent, saving it changes what it does right away. After you publish, saving only changes its draft, and your contacts keep talking to the live version until you publish again. Only the agent's name and its Log level change as soon as you save.

To publish, save your changes, open the Releases tab on the agent and click Publish. The new version goes live on the next message. Publish without going live creates the version and leaves the current one running.

The Releases tab warns you when the saved draft has changes that aren't live, and when a header value changed. New header values aren't used until you publish.

Publishing always publishes the saved draft on your screen. If someone saved the agent after you opened it, for example in another browser tab or through the API, publishing stops and loads the new draft, so you can review it and publish again.

In the app, only company owners can see versions and test runs, publish, roll back or run tests. Through the AI Assistant, only owners can publish, roll back or run tests.

Going back

Every published version stays in the Releases tab. Older ones are under Show older versions. Click Make live on any of them, and new messages use it straight away.

This restores the agent and nothing else. Workflows, templates and variables stay as they are now, and that version's header values come back with it, so a token you've since revoked will stop working. AutoTalk first checks the credentials that version would use, such as its model key or a key in one of its steps. If one was revoked, it refuses and asks you to enter it again.

note

Versions keep their header values so going back works. They are hidden when you view or compare versions. If a secret ends up in a version, revoke it at the provider.

Testing

The Tests tab holds scripted conversations that run on the real agent. Tests are part of the draft, so save them like any other change, then click Run all tests. The tests must be valid JSON before you can save.

Only people can change tests, in the app, through MCP or through the API. Workflow steps and agent tools can't, even with a token they create with Generate JWT Token, so an agent can't rewrite the tests it is judged by. An API key you put into a step yourself can still change them, so keep such keys out of steps.

[
  {
    "name": "cancels after hours",
    "repeat": 2,
    "clock": "2026-10-02T17:00:00-03:00",
    "contact": {"name": "Test", "attributes": {"plan": "new"}},
    "mocks": [
      {"action": "actions/network/http/request/send", "tool": "register", "output": {"status": 202, "data": {"ok": true}}}
    ],
    "turns": [
      {
        "user": "I need to cancel",
        "expect": {
          "replies": {"min": 1, "max": 1},
          "reply_match": ["(?i)monday"],
          "tools_required": [{"name": "register", "args": {"reason": "^cancel$"}}],
          "effects": [{"action": "actions/messaging/contact/send", "min": 1, "match": ["(?i)cancellation"]}]
        },
        "expect_soft": {"reply_not_match": ["(?i)tomorrow"]}
      }
    ]
  }
]

Field	What it does
name	Unique name for the test
repeat	How many times to run it, up to 5. Every run must pass, because model replies vary
clock	The date and time the agent sees during this test
contact	The test contact's name and custom attributes
mocks	Answers for steps that can't really run in a test
turns	The conversation: each turn has user, or tap to tap an option on the last menu
expect	What must be true after that turn
expect_soft	What should be true. It's reported and never fails the test

What you can check

Check	Passes when
replies	The number of messages sent to the contact is within min and max
reply_match, reply_not_match	The replies match, or don't match, these regular expressions
tools_required	These tools were called, optionally with arguments matching a regular expression
tools_forbidden	These tools weren't called
menu	A menu was or wasn't sent
paused	The conversation was or wasn't paused
lost	The message was or wasn't ignored
effects	A step with an outside effect happened within min and max times. match, each_match and not_match check what it would have sent

Every turn must also reach the contact, end cleanly, send nothing after a menu and show no internal planning text. If silence is correct, use "replies": {"max": 0} or "lost": true.

A run refuses a test with an unknown check or a field of the wrong type, so a typo can't quietly check nothing.

What a test can do

Nothing reaches a real person or an outside system:

• Messages to anyone else are captured, not delivered.
• HTTP requests are never sent. Give them a mock. The results keep only part of the request: the address's scheme, host, a short first path part such as api or v1, and its query parameter names. The rest of the path, the query values and any password in the address are hidden, and so are values in the body under names like token, password or apiKey. This only applies to the stored results: the mock's when and effects both check the request as the agent sent it.
• Variables are never decrypted: the test sees a placeholder.
• A test only sees its own contacts, conversations and messages: searches find only those, and fetching anyone else's fails the test. A test can update its own contact and conversation, but a change that would move them to another channel or person needs a mock and is never saved.
• Anything else with an outside effect needs a mock: creating or deleting records, running workflows, code steps, media and monitors. A test with a missing mock fails and names the step.

Running and reading results

An agent runs one set of tests at a time. Runs from every company share the same workers and take turns, so a long run can pause between tests and continue later. Cancel stops a run within a few seconds, even in the middle of a turn. A run that hasn't started, or is waiting between tests, stops at once, and you can start a new one right away.

Each test shows Passed, Failed or Couldn't run. Couldn't run means AutoTalk never let the agent answer, for example because of a limit, maintenance, the model provider failing, a turn that took too long, or a company or channel setting that made AutoTalk ignore the message. It says nothing about the agent, so run the test again instead of changing the agent. Each turn has a time limit (for an LLM agent, its own run limit plus about two and a half minutes), and each run of a test gets at most 30 minutes.

Results are kept for 90 days. Very long replies and values are shortened in the stored results, and a very large run keeps only the verdict of its last tests. The checks always see everything.

Cost

Tests use real tokens, billed as usual. A run stops if it reaches its token limit or your company's budget, and the test it stopped at shows Out of budget with the reason. A run won't start if your company is out of budget or credit. Tokens on your own provider keys count toward the limit too, and so do tokens a test spent before it was interrupted and run again.

What can't be tested yet

Typebot agents, agents with MCP servers, and agents with a Connect MCP Server, AutoTalk MCP Connect or Generate JWT Token step can't be tested yet. The Tests tab tells you why. Without tests, these agents publish as untested. With tests, the tests can't run, so publishing needs Publish anyway.

Tests and publishing

An agent with tests can only be published after a full run of all its tests passes on that exact draft. Any change to the draft or its tests needs a new run. Changing a credential doesn't.

If you need to publish anyway, the refusal offers Publish anyway, which asks for a reason. The reason and who published it stay on that version. It skips the test requirement and nothing else: configuration errors, a missing or revoked credential, and a credential entered for a different endpoint still block publishing.

What tests don't cover

A passing run covers the agent itself. Mocked steps aren't tested, and workflows and templates can change after you publish, so test those separately.

Using the API or MCP

These are functions on agents, which you call with the MCP execute_function tool or the API's executeFunction.

• Validating, publishing, rolling back, and starting or canceling tests need automation:admin. forcePublish needs the same as publish.
• Reading versions and test runs needs data:read on the API. On MCP, execute_function itself needs automation:admin, so reading there needs automation:admin and data:read.

Function	What it does
validateDraft	Checks the draft and returns the contentHash needed to publish, whether it can be tested, and which steps need mocks
publish	Publishes, with expectedContentHash and optionally activate: false
forcePublish	Publishes without passing tests, with expectedContentHash and reason
setLiveRelease	Makes a version live, with releaseId and expectedLiveReleaseId
listReleases, getRelease	Reads versions. Header values are hidden
testDraft	Starts a run. only runs some tests, but only a full run counts for publishing
getTestRun, listTestRuns, cancelTestRun	Reads or stops runs

Workflow and agent steps can't call these, and a token a step created with Generate JWT Token can't publish, run tests or change tests. So an agent can't publish or test itself from one of its own tools. An API key you put into a step yourself still works with its own scopes, so keep keys with automation:admin out of steps.