Getting an AI agent to make phone calls sounds simple. Install the skill, set the API key, tell it to call someone. What could go wrong?
Quite a bit, it turns out. Not because the tools are bad, but because phone systems are complex and AI agents are still learning how to interact with them. After helping hundreds of developers debug their AgentLine setups, I've seen the same problems come up over and over.
Here are the seven most common ones and how to fix them fast.
1. The Call Goes Through But Nobody Says Anything
This is the number one issue we see. The call connects, the person on the other end says hello, and your AI agent sits there in silence.
What's usually happening: your agent isn't receiving the transcript. The call connected, but the text never reached your agent's brain.
Fix: Check your webhook configuration. Make sure the webhook URL is publicly accessible (localhost won't work unless you're using a tunnel). Test it with a simple curl request to make sure it responds with a 200 status. If you're using the AgentLine hosted mode, make sure your agent_id is correct in the call request.
2. The AI Agent Talks Over the Person
This one makes calls feel terrible. The person is mid-sentence and your agent starts talking like it didn't hear anything.
Fix: Add explicit turn-taking instructions to your system prompt. Something like "Wait until the person has clearly finished speaking before you respond. If you're not sure, give them an extra second. It's better to have a brief pause than to interrupt."
Also check if you're using a voice with very fast TTS. Some voices speak faster than others. Try female-2 for a more relaxed pace.
3. The Transcript Is Garbled or Misses Words
Nobody expects perfect transcription, but if your agent is consistently misunderstanding callers, something's off.
Fix: Check the caller's audio environment. Are they on speakerphone? In a noisy room? On a bad connection? Your agent can only work with what it hears. If you're testing, use headphones and a quiet room.
Also make sure your prompt gives the agent permission to ask for clarification. "If you didn't catch what the person said, don't guess. Say 'Sorry, I missed that last part. Could you say it again?'"
4. Calls Drop After a Few Seconds
The call connects, the greeting plays, and then silence. The call ends without warning.
Fix: This is almost always a webhook timeout. Your agent's webhook endpoint is taking too long to respond, so the call drops. Check that your endpoint responds within a few seconds. If your agent needs time to think, consider using the AgentLine hosted mode where the server handles call state for you.
5. The Agent Sounds Robotic or Unnatural
This isn't a technical bug, but it's probably the most impactful issue. Customers hear a robot and hang up.
Fix: Voice selection matters. AgentLine gives you female-1, female-2, male-1, and custom Cartesia voices. Test all of them. female-2 tends to sound the most natural for general use.
Prompt writing matters even more. See our guide on improving call quality for specific prompt techniques that make a huge difference.
6. Can't Provision a Phone Number
You're ready to go but the system won't give you a number.
Fix: Check your account balance. Numbers cost $2.00 (one-time). Make sure you have enough credit. Also check that you're requesting a US number (that's all we support right now). If you're still stuck, contact support at agentline.cloud.
7. SMS Isn't Working
You're receiving calls fine but texts aren't coming through.
Fix: First, make sure you're trying to receive SMS, not send it. AgentLine supports inbound SMS (reading texts people send to your number) but outbound SMS sending is not yet available.
If you're expecting inbound texts and they're not arriving, check your message endpoint. Make sure it's configured to handle the same webhook format as calls. SMS events arrive through the same events endpoint as call events.
When All Else Fails
If you've checked all seven and something's still not working:
- Read the logs. Every failed call or event has logs that tell you exactly what happened.
- Simplify. Strip your setup down to the absolute minimum: one agent, one number, one simple prompt, one test call to yourself.
- Get it working in that minimal state, then add complexity back one piece at a time.
Most issues come down to either the webhook configuration, the prompt, or the network environment. Start there.
Need more help? Check the docs at agentline.cloud or reach out to support.