Two AI voice agents get on a phone call with each other. Both of them run the full stack on every single turn: speech to text, think, text to speech. They're using a phone call to send each other words that would've fitted in a small JSON object, and paying for it three times over — codec latency, model latency, and per-minute telephony charges.
Tonecall lets them notice they're both machines and move the conversation onto a data channel without dropping the call. That call stays up, because a human might walk in at any moment. Only the transport changes.
Why you can't prompt your way out
Obvious first idea: solve it in the model. Tell each agent to be terse, or have it spot a robot from how the transcript reads. Both fail for the same structural reason — by the time there's a transcript to read, you've already paid the cost you were trying to avoid. STT ran. The model ran. TTS ran. Those four to six seconds are gone.
So detection has to happen below the language layer, in the audio path, before anything gets transcribed. That turns it from a prompting exercise into a signalling one (different job, different toolbox).
Just enough telephony
A phone call is two conversations at once. SIP is signalling — ring, answer, hang up, roughly HTTP for calls. RTP is the audio itself, usually 20 ms of compressed sound per packet (so a one-second gap is fifty of them). A CPaaS provider runs all of that and hands you a REST API on top, so this project never touches SIP or RTP directly.
When someone dials a number you own, the provider answers and asks your server what to do by POSTing to an answer URL you set up in advance. You reply with a small XML document of call-control verbs. Four matter here.
| Verb | What it does | Used here |
|---|---|---|
<Dial> | Ring another number and bridge this call to it | yes, the bridge |
<Stream> | Open a WebSocket and pump the call's audio through it | yes, the audio tap |
<Speak> | Say text to the caller | intro / outro |
<Hangup> | End the call | teardown |
<Stream> is the one that makes the rest possible. It turns
a live phone call into a WebSocket of audio frames you can inspect as they
arrive.
Asking "are you a machine?" in audio
That channel only carries sound, so the probe has to be sound. What I landed on is in-band DTMF — the touch-tone digits every phone system on earth already forwards without being asked. Each agent emits a short tone sequence acting as a pairing token, and anything that answers with a well-formed token is, by construction, not a human.
Reaching for something that old is deliberate. DTMF survives transcoding, works through carriers you've no agreement with, and asks nothing of the network (no agreement, no cooperation, no config). Closest comparison I've got is fax machines, but for modern AI agents: two machines screeching at each other for a moment to agree on a better way to talk.
Four phases
A completed agent-to-agent call runs dial, intro, data, outro. Both voice phases are there to be heard: they keep the call legible to a human who joins late, and they leave an audible record of what got agreed.
Middle number's the one that matters. Everything the two agents actually need to settle happens in a single round trip over a WebSocket, instead of n turns at four to six seconds each. Savings grow with how much they've got to say.
The human path is the real test
An optimisation that only works in the happy case isn't shippable here. If the peer never answers the DTMF probe, that peer is a person, and the system falls through to a genuine voice agent — STT, an LLM, TTS, in a loop — with no hint anything else was attempted.
Federated, not centralised
My first build put both agents behind one orchestrator. Easier, and also cheating — if a single process owns both sides of the call, it never has to discover anything and the protocol is doing no work at all.
Current design runs two independent middleware instances, each owning exactly one agent. They find each other through the in-band DTMF exchange, plus a broker that resolves which instance owns a given token.
You can run the whole topology on a laptop with no telephony account and no API keys, because the LLM, STT and TTS are all stubbable:
npm run dev:a # vegetable_vendor + broker on :3000
npm run dev:b # pizza_shop on :3001
curl -X POST http://localhost:3000/api/simulate \
-H 'Content-Type: application/json' \
-d '{"peerBase":"http://localhost:3001"}'Instance B forwards its events to instance A, so one dashboard shows both
legs. Being able to exercise a four-phase telephony protocol with
USE_STUBS=true and two terminals is probably the single thing that
made this tractable to build at all.
What I'd change
DTMF is a low-bandwidth probe. Perfect for "are you a machine, and what's your token", hopeless for anything richer. A capability exchange — which schema version, which codecs — belongs on the data channel right after pairing, not in tones.
My broker is a soft centralisation. Pairing gets discovered in-band, but token resolution still goes through a known endpoint, and fully peer-to-peer discovery is the honest end state. I'm not sure yet what that looks like without reinventing a directory.
Trust is the one I haven't resolved. Right now a well-formed token is enough to be believed. Anything real needs the side-channel authenticated, otherwise what you've built is a machine that'll happily negotiate with whoever tones at it first.
Design docs and a page-by-page walkthrough of the code are in the repo under
docs/, and the protocol writeup covering the <Dial>
bridge is docs/avip-1-dial.md.