Agent Platform Documentation

Comprehensive API reference and integration guide. Multi-provider WebRTC voice agent supporting OpenAI Realtime API and ElevenLabs Conversational AI.

🚀 Quick Start

1. Set up a session with OpenAI

curl -X POST http://localhost:3001/api/set_prompt \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $INTERNAL_AGENT_PLATFORM_API_KEY" \
  -d '{
    "uuid": "openai-session-123",
    "prompt": "You are a helpful voice assistant. Be conversational and friendly.",
    "provider": "openai",
    "voice": "alloy"
  }'

Then access: http://localhost:3001/voice/openai/openai-session-123

2. Set up a session with ElevenLabs

curl -X POST http://localhost:3001/api/set_prompt \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $INTERNAL_AGENT_PLATFORM_API_KEY" \
  -d '{
    "uuid": "elevenlabs-session-456",
    "prompt": "You are a friendly conversational AI assistant.",
    "provider": "elevenlabs",
    "voice": "ballad"
  }'

Then access: http://localhost:3001/voice/elevenlabs/elevenlabs-session-456

📡 API Reference

POST /api/set_prompt

Configure and start a voice session.

Headers:

  • Content-Type: application/json
  • Authorization: Bearer $INTERNAL_AGENT_PLATFORM_API_KEY

Request Body:

{
  "uuid": "string",              // Unique session identifier
  "prompt": "string",            // System prompt for the AI agent
  "provider": "openai|elevenlabs", // Voice provider (default: "elevenlabs")
  "voice": "string"              // Voice name (default: "ballad")
}

🎯 Supported Providers

OpenAI Realtime API

  • • Low-latency voice interactions
  • • Built-in speech recognition
  • • Natural conversation flow
  • • Multiple voice options

Route: /voice/openai/[sessionId]

ElevenLabs Conversational AI

  • • High-quality voice synthesis
  • • Advanced emotional expression
  • • Custom voice cloning
  • • Real-time processing

Route: /voice/elevenlabs/[sessionId]

🔧 Development & Debug

Debug Mode

Add ?debug=true to any voice session URL for detailed logging:

http://localhost:3001/voice/openai/session-123?debug=true
http://localhost:3001/voice/elevenlabs/session-456?debug=true

Required Environment Variables

INTERNAL_AGENT_PLATFORM_API_KEY=your-secure-token-here

🏗️ Architecture

  • WebRTC: Real-time peer-to-peer audio communication
  • Redis Long-Polling: Session configuration management
  • Provider Factory: Abstracted voice provider management
  • Multi-Provider Support: Seamless switching between voice services
  • Token Management: Secure ephemeral token generation
← Back to Home