OpenAI released its full playbook for prompting gpt-realtime, and it turns out voice AI needs totally different techniques than text models. Here are the game-changing tactics.
Essential Realtime prompting techniques
- Structure your prompt with clear sections: In this order: Role & Objective, Personality & Tone, Tools, Conversation Flow, Safety & Escalation.
- Bullets beat paragraphs: Short bullet points outperform long paragraphs for instruction following.
- Use ALL CAPS for key rules: Capitalizing important instructions makes them stand out
- Convert conditional logic to plain text: Instead of “IF x > 3 THEN ESCALATE,” write “IF MORE THAN THREE FAILURES, THEN ESCALATE.”
- Add tool call preambles: Have the model say “I’m checking that now” before calling functions.
- Control language drift: Pin responses to target languages to prevent unwanted switching.
- Handle unclear audio: Give explicit instructions for background noise and partial words.
- Use sample phrases: Provide examples that show style without being repetitive.
- Add variety rules: The model follows sample phrases so closely that you need explicit variety instructions, or it will sound like a broken record. For example: “Do not repeat the same sentence twice.”
- Include pronunciation guides: For example, “Pronounce ‘SQL’ as ‘sequel’“ (so adding the phonetic pronunciation) for brand terms.
- Read numbers character-by-character: For phone numbers, codes, etc.
- Use LLMs to review your prompts: Meta-prompt to identify conflicts and ambiguity in what you write.
- Iterate relentlessly: Be prepared to test many small variations — swapping individual words, trying different phrasings for the same instruction, adjusting sentence structure, etc.
Our favorite insight: That last point. Small word swaps like “inaudible” to “unintelligible” can dramatically improve performance. Like GPT-5, the model is incredibly sensitive to precise wording in ways that most text models aren’t.
For more specifics about these Realtime prompting tips, read OpenAI’s full playbook. For details about Realtime API’s new features and the gpt-realtime model, read eWeek’s news coverage.
Editor’s note: This content originally ran in a newsletter from our sister publication, The Neuron. Read all of the prompt tips featured in The Neuron newsletters sent in August. To read more from The Neuron, sign up for its newsletter.