> ## Documentation Index > Fetch the complete documentation index at: https://docs.hiperai.com/llms.txt > Use this file to discover all available pages before exploring further. # Initiate Speech-to-Speech WebRTC Session > Establishes a WebRTC connection for real-time speech-to-speech conversations using OpenAI Realtime API. This endpoint accepts a WebRTC SDP offer and returns an SDP answer that can be used to establish a peer connection with OpenAI's Realtime API. **Usage Flow:** 1. Client creates a WebRTC offer (RTCPeerConnection.createOffer) 2. Client sends the SDP offer to this endpoint 3. This endpoint proxies the offer to OpenAI Realtime API 4. Client receives SDP answer and establishes WebRTC connection 5. Client can then have real-time voice conversations with the AI **S2S Time Tracking:** - Each user has a monthly S2S time quota based on their license - Time is tracked in minutes and deducted when sessions are logged - Use `/speech/s2s/status` to check remaining time - Use `/speech/s2s/log-session` to log session duration and deduct time **SMLTP Integration:** - All requests are processed through SMLTP for security and compliance - Model validation is enforced based on the specified SMLTP policy - Requests are audited and logged for compliance tracking # Initiate Speech-to-Speech WebRTC Session Establish a WebRTC connection for real-time speech-to-speech conversations using OpenAI Realtime API. ## Endpoint ``` POST /speech/s2s/webrtc ``` ## Description Establishes a WebRTC connection for real-time speech-to-speech conversations using OpenAI Realtime API. This endpoint accepts a WebRTC SDP offer and returns an SDP answer that can be used to establish a peer connection with OpenAI's Realtime API. ### Usage Flow 1. Client creates a WebRTC offer (RTCPeerConnection.createOffer) 2. Client sends the SDP offer to this endpoint 3. This endpoint proxies the offer to OpenAI Realtime API 4. Client receives SDP answer and establishes WebRTC connection 5. Client can then have real-time voice conversations with the AI ### S2S Time Tracking * Each user has a monthly S2S time quota based on their license * Time is tracked in minutes and deducted when sessions are logged * Use `/speech/s2s/status` to check remaining time * Use `/speech/s2s/log-session` to log session duration and deduct time ### SMLTP Integration * All requests are processed through SMLTP for security and compliance * Model validation is enforced based on the specified SMLTP policy * Requests are audited and logged for compliance tracking ## Authentication Required: API Key ```bash theme={null} Authorization: Bearer sk-your-api-key-here ``` ## Request Body | Parameter | Type | Required | Description | | -------------- | ------- | -------- | --------------------------------------------------------------- | | `sdp` | string | Yes | WebRTC SDP offer from the client's RTCPeerConnection | | `model` | string | No | OpenAI Realtime model (default: "gpt-4o-mini-realtime-preview") | | `voice` | string | No | Voice to use for AI response (default: "alloy") | | `smltp_policy` | string | No | SMLTP policy (default: "internal") | | `output_audio` | boolean | No | Whether to enable audio output (default: true) | | `user_id` | string | No | User ID to bill this session to (defaults to API key owner) | | `instructions` | string | No | Optional system instructions for the AI assistant | ### Available Models * `gpt-4o-mini-realtime-preview` * `gpt-4o-realtime-preview` ### Available Voices * `alloy` (default) * `echo` * `fable` * `onyx` * `nova` * `shimmer` * `ash` * `ballad` * `coral` ### Available SMLTP Policies * `public` * `internal` (default) * `internal-strict` * `confidential` * `hipaa` * `gdpr` * `pci-dss` ## Request Example ```bash theme={null} curl -X POST "https://{customer.name}.hiperai.ai/api/external/speech/s2s/webrtc" \ -H "Authorization: Bearer sk-your-api-key-here" \ -H "Content-Type: application/json" \ -d '{ "sdp": "v=0\r\no=- 4611731400430051336 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\n...", "model": "gpt-4o-mini-realtime-preview", "voice": "alloy", "smltp_policy": "internal", "output_audio": true, "instructions": "You are a helpful customer service agent." }' ``` ### JavaScript/Node.js ```javascript theme={null} // Create WebRTC peer connection const pc = new RTCPeerConnection(); // Create offer const offer = await pc.createOffer(); await pc.setLocalDescription(offer); // Send SDP offer to SecureAI const response = await fetch('https://{customer.name}.hiperai.ai/api/external/speech/s2s/webrtc', { method: 'POST', headers: { 'Authorization': 'Bearer sk-your-api-key-here', 'Content-Type': 'application/json' }, body: JSON.stringify({ sdp: offer.sdp, model: 'gpt-4o-mini-realtime-preview', voice: 'alloy', smltp_policy: 'internal', output_audio: true, instructions: 'You are a helpful customer service agent.' }) }); // Get SDP answer const sdpAnswer = await response.text(); // Set remote description await pc.setRemoteDescription(new RTCSessionDescription({ type: 'answer', sdp: sdpAnswer })); // Now you can have real-time voice conversations ``` ### Python ```python theme={null} import requests url = "https://{customer.name}.hiperai.ai/api/external/speech/s2s/webrtc" headers = { "Authorization": "Bearer sk-your-api-key-here", "Content-Type": "application/json" } data = { "sdp": "v=0\r\no=- 4611731400430051336 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\n...", "model": "gpt-4o-mini-realtime-preview", "voice": "alloy", "smltp_policy": "internal", "output_audio": True, "instructions": "You are a helpful customer service agent." } response = requests.post(url, headers=headers, json=data) sdp_answer = response.text print('SDP Answer:', sdp_answer) ``` ## Response ### Success Response (200) **Content-Type**: `application/sdp` The response is an SDP answer string that can be used with `RTCPeerConnection.setRemoteDescription()`. ``` v=0 o=- 1234567890 2 IN IP4 127.0.0.1 s=- t=0 0 ... ``` ## Error Responses ### 400 Bad Request ```json theme={null} { "success": false, "error": "Invalid SDP offer", "message": "SDP offer is required and must be a string", "request_id": "96bba6a4-0e7a-4fd7-93a7-3b40428729fb" } ``` ### 403 Forbidden #### S2S Time Limit Reached ```json theme={null} { "success": false, "error": "S2S time limit reached", "message": "Insufficient S2S time remaining for this user", "remaining_minutes": 0, "next_renewal_date": "2024-12-01T12:55:35.721Z", "request_id": "96bba6a4-0e7a-4fd7-93a7-3b40428729fb" } ``` #### Model Validation Failed ```json theme={null} { "success": false, "error": "Model validation failed", "message": "Model gpt-4o-mini-realtime-preview is not allowed by SMLTP policy test-policy-active", "request_id": "96bba6a4-0e7a-4fd7-93a7-3b40428729fb" } ``` ### 500 Internal Server Error ```json theme={null} { "success": false, "error": "S2S WebRTC failed", "message": "An error occurred while processing your request", "request_id": "96bba6a4-0e7a-4fd7-93a7-3b40428729fb" } ``` ## Notes * The SDP offer must be a valid WebRTC SDP offer string * After receiving the SDP answer, use it to set the remote description on your RTCPeerConnection * Check S2S time status before initiating sessions using `/speech/s2s/status` * Log session duration after completion using `/speech/s2s/log-session` * All requests are processed through SMLTP for security and compliance * The `user_id` parameter allows billing to a different user account ## OpenAPI ````yaml POST /speech/s2s/webrtc openapi: 3.0.3 info: title: SecureAI External API description: > SecureAI External API provides AI chat completion and image generation capabilities with knowledge base retrieval, security policies, and comprehensive usage tracking. This API is designed for external developers and integrations using API key authentication. ## Key Features - **RAG (Retrieval-Augmented Generation)**: Automatically search knowledge bases for relevant context - **Multi-Model Support**: OpenAI, Anthropic, Google, Meta, and other AI models - **Model Redundancy & Failover**: Caller-defined failover chains (primary + fallbacks) with per-attempt timeouts - **OpenAI-Compatible Endpoint**: Point any OpenAI SDK at `/api/external/v1` — no code changes - **Image Generation**: Generate and edit images using Google Gemini 2.5 Flash Image - **Speech-to-Speech (S2S)**: Real-time voice conversations using OpenAI Realtime API with WebRTC - **Security Policies**: SMLTP policy enforcement, per-call Prompt Shield, and signed compliance receipts - **Usage Tracking**: Comprehensive usage monitoring, self-service quota, and rate limiting - **Knowledge Base Integration**: Access to personal and shared knowledge bases - **User Management**: Complete user, group, and role management capabilities - **Audit Logging**: Comprehensive activity and security audit logs ## Authentication All endpoints (except health check) require API key authentication using Bearer token: ``` Authorization: Bearer sk-your-api-key-here ``` ## Billing and Usage By default, API requests are billed to the user account that owns the API key. You can specify a different user to bill by including the `user_id` parameter in your request. This allows for: - Multi-tenant applications with per-user billing - Flexible completion limit management - Per-user "Usage by Model" settings ## Rate Limits - Default: 60 requests per minute, 1000 requests per hour - Daily limits: 100 requests (configurable) - Monthly limits: 10,000 requests (configurable) ## Base URL ``` https://secureai.hiperai.ai/api/external ``` version: 1.1.0 contact: name: SecureAI Support email: support@secureai.hiperai.ai license: name: Proprietary url: https://secureai.hiperai.ai/terms servers: - url: https://secureai.hiperai.ai/api/external description: Production server - url: http://localhost:3010/api/external description: Development server security: - ApiKeyAuth: [] tags: - name: System description: System health and status endpoints - name: Discovery description: Endpoints to discover available models, indexes, and policies - name: Chat description: AI chat completion endpoints - name: User Management description: Endpoints for managing user accounts and roles - name: Index Management description: Endpoints for managing knowledge base indexes - name: Group Management description: Endpoints for managing user groups - name: SMLTP Security description: Endpoints for managing SMLTP security policies and audit logs - name: Role Management description: Endpoints for managing user roles - name: Activity Logs description: Endpoints for retrieving activity logs externalDocs: description: SecureAI Documentation url: https://secureai.hiperai.ai/docs paths: /speech/s2s/webrtc: post: tags: - Speech summary: Initiate Speech-to-Speech WebRTC Session description: > Establishes a WebRTC connection for real-time speech-to-speech conversations using OpenAI Realtime API. This endpoint accepts a WebRTC SDP offer and returns an SDP answer that can be used to establish a peer connection with OpenAI's Realtime API. **Usage Flow:** 1. Client creates a WebRTC offer (RTCPeerConnection.createOffer) 2. Client sends the SDP offer to this endpoint 3. This endpoint proxies the offer to OpenAI Realtime API 4. Client receives SDP answer and establishes WebRTC connection 5. Client can then have real-time voice conversations with the AI **S2S Time Tracking:** - Each user has a monthly S2S time quota based on their license - Time is tracked in minutes and deducted when sessions are logged - Use `/speech/s2s/status` to check remaining time - Use `/speech/s2s/log-session` to log session duration and deduct time **SMLTP Integration:** - All requests are processed through SMLTP for security and compliance - Model validation is enforced based on the specified SMLTP policy - Requests are audited and logged for compliance tracking requestBody: required: true content: application/json: schema: type: object required: - sdp properties: sdp: type: string description: WebRTC SDP offer from the client's RTCPeerConnection example: "v=0\r\no=- 4611731400430051336 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\n..." model: type: string description: OpenAI Realtime model to use default: gpt-4o-mini-realtime-preview enum: - gpt-4o-mini-realtime-preview - gpt-4o-realtime-preview voice: type: string description: Voice to use for the AI response default: alloy enum: - alloy - echo - fable - onyx - nova - shimmer - ash - ballad - coral smltp_policy: type: string description: SMLTP policy to apply for this session default: internal enum: - public - internal - internal-strict - confidential - hipaa - gdpr - pci-dss output_audio: type: boolean description: 'Whether to enable audio output (default: true)' default: true user_id: type: string description: >- Optional user ID to bill this session to (defaults to API key owner) example: 60a7c8f5e8b4f5001f7a8c23 instructions: type: string description: Optional system instructions for the AI assistant example: You are a helpful customer service agent. responses: '200': description: SDP answer from OpenAI Realtime API content: application/sdp: schema: type: string description: >- SDP answer that can be used with RTCPeerConnection.setRemoteDescription() example: "v=0\r\no=- 1234567890 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\n..." '400': description: Invalid request content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' example: success: false error: Invalid SDP offer message: SDP offer is required and must be a string request_id: 96bba6a4-0e7a-4fd7-93a7-3b40428729fb '403': description: Access denied content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' examples: timeLimit: summary: S2S time limit reached value: success: false error: S2S time limit reached message: Insufficient S2S time remaining for this user remaining_minutes: 0 next_renewal_date: '2024-12-01T12:55:35.721Z' request_id: 96bba6a4-0e7a-4fd7-93a7-3b40428729fb modelValidation: summary: Model validation failed value: success: false error: Model validation failed message: >- Model gpt-4o-mini-realtime-preview is not allowed by SMLTP policy test-policy-active request_id: 96bba6a4-0e7a-4fd7-93a7-3b40428729fb '500': description: Internal server error content: application/json: schema: $ref: '#/components/schemas/ErrorResponse' example: success: false error: S2S WebRTC failed message: An error occurred while processing your request request_id: 96bba6a4-0e7a-4fd7-93a7-3b40428729fb security: - ApiKeyAuth: [] components: schemas: ErrorResponse: type: object properties: success: type: boolean example: false error: type: string example: Invalid API key message: type: string example: The provided API key is invalid or has been revoked request_id: type: string example: req-abc123 description: Request ID for tracking (if available) securitySchemes: ApiKeyAuth: type: http scheme: bearer description: | API key authentication using Bearer token format. Example: `Authorization: Bearer sk-your-api-key-here` ````