From: gregisenberg
Voice AIs are capable of much more than people currently estimate, with the potential to automate work done over the phone within the next one to three years [00:00:54]. The process of building apps with AI tools and specifically voice AI agents involves understanding tools, workflows, and prompts [00:01:11].
Case Study: The “Lowballer 9000” AI
Tony, an automation expert, created a voice AI named “Lowballer 9000” (also called Alex) [00:01:28]. Its purpose was to call secondhand luxury watch dealers in the U.S. and lowball them for a Rolex Daytona, a $35,000 watch [00:00:10].
The AI’s job was to:
- Identify itself as looking for a Rolex Daytona [00:01:46].
- Recognize it was speaking to luxury watch dealers [00:01:49].
- Gather critical details, such as the watch’s condition, original documentation, and packaging, which significantly impact the quoted price [00:01:56].
- Lowball offers to dealers [00:00:10].
Key Components and Tools for Voice AI Agents
Voice Agent Platform: VAPI
VAPI is a flexible platform used for building voice agents [00:02:54]. It allows users to:
- Bring their own prompts [00:03:02].
- Select providers for different services [00:03:06].
- Run the agent directly [00:03:08].
Prompt Engineering
Crafting an effective prompt is crucial for voice AI performance [00:03:26].
- Iterative Process: Prompts are often refined through testing and observation of AI responses [00:03:45]. For example, if the AI asks too many questions at once, the prompt can be adjusted to ensure concise, one-to-two sentence responses [00:04:17].
- Simplicity: Using language at a “sixth-grade English level” can lead to longer call durations and more human-sounding interactions [00:05:30].
Model Selection
The choice of AI model can significantly impact performance [00:06:12].
- Models can be “flavor of the month,” with popularity affecting availability [00:06:18].
- API reliability is a concern; a model’s API being slammed can cause the bot to crash or not respond [00:08:09]. Selecting a model that is currently available is recommended [00:15:17].
- Temperature Setting: This parameter controls the creativity of the AI’s responses [00:08:24].
- A temperature closer to zero makes the system more deterministic and predictable [00:08:29].
- Higher temperatures allow for more creative responses, which can be useful when handling uncommon questions [00:08:41].
Voice Configuration
Voice AI agents allow for specific voice configurations [00:09:06]. For example, the “Lowballer 9000” used a voice labeled “Cartisia” or “New York man” [00:09:12].
Tool Calls and Integrations
The true power of a voice AI lies in its ability to interact with existing tools [00:10:12].
- Data Collection and Analysis: The Lowballer 9000 used a tool call to record and analyze all calls in a database (like an AirTable spreadsheet) [00:10:31]. After each conversation, the AI would process the transcript to determine:
- If an offer was received [00:11:07].
- If the bot engaged [00:11:11].
- The lowballed amount [00:11:16].
- Whether the lowball was accepted or if another price was given [00:11:18].
- This enabled easy sifting through hundreds of calls to find actionable outcomes [00:11:46].
- Lindy Integration: Lindy is particularly useful for flexibly extracting and organizing large amounts of text into spreadsheets [00:12:47]. It can automatically categorize and classify information based on column names, such as determining if a restaurant is “vegan friendly” from a menu scrape [00:13:54].
- Feedback Bots: A voice AI can be used to collect feedback by allowing users to “brain dump” verbally. This information can then be categorized, classified, summarized, and posted to platforms like Slack, providing instant, actionable insights for product improvement [00:20:27].
Telephony: Twilio and STIR/SHAKEN
A significant challenge in voice AI deployment is ensuring calls actually ring through [00:14:57].
- Twilio: This service is used for placing calls [00:14:53].
- STIR/SHAKEN: This FCC mechanism requires phone numbers to be registered in a database for KYC (Know Your Customer) [00:15:07]. Without proper registration, calls may go straight to voicemail [00:15:28]. It’s crucial to bring your own registered number and complete KYC before placing calls [00:15:32].
Use Cases and Business Opportunities
Building and deploying AI agents for business tasks offers numerous opportunities:
- Sales Automation: Voice AIs can prospect leads by asking specific questions to qualify them. For example, in real estate, an AI could call a prospect list to identify houses likely to go on the market soon, then dispatch human agents to pursue qualified leads [00:16:20].
- Data Arbitrage: Voice AIs are excellent for gathering data, providing an unfair advantage [00:17:32].
- The “Lowballer 9000” demonstrated this by identifying opportunities to buy watches at low prices and resell them for significant profit [00:17:42].
- This concept can be applied to any business where getting specific data allows for arbitrage [00:18:03].
- If knowledge is an unfair advantage, there’s likely a voice AI opportunity to collect that data [00:18:25].
- Customer Feedback Collection: Voice AIs can automate the collection of customer feedback, making it easier for users to provide detailed input without filling out forms, and instantly summarizing it for internal teams [00:20:27]. This could be offered as a service to other companies [00:22:05].
The ability to use AI to create voice character apps and then automate negotiation or data collection on a massive scale presents significant opportunities for building businesses around AI agents and achieving high-volume arbitrage [00:19:21].