Initiate Speech-to-Speech WebRTC Session

curl --request POST \
  --url https://secureai.hiperai.ai/api/external/speech/s2s/webrtc \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "sdp": "v=0\r\no=- 4611731400430051336 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\n...",
  "model": "gpt-4o-mini-realtime-preview",
  "voice": "alloy",
  "smltp_policy": "internal",
  "output_audio": true,
  "user_id": "60a7c8f5e8b4f5001f7a8c23",
  "instructions": "You are a helpful customer service agent."
}
'

import requests

url = "https://secureai.hiperai.ai/api/external/speech/s2s/webrtc"

payload = {
    "sdp": "v=0
o=- 4611731400430051336 2 IN IP4 127.0.0.1
s=-
t=0 0
...",
    "model": "gpt-4o-mini-realtime-preview",
    "voice": "alloy",
    "smltp_policy": "internal",
    "output_audio": True,
    "user_id": "60a7c8f5e8b4f5001f7a8c23",
    "instructions": "You are a helpful customer service agent."
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    sdp: 'v=0\r\no=- 4611731400430051336 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\n...',
    model: 'gpt-4o-mini-realtime-preview',
    voice: 'alloy',
    smltp_policy: 'internal',
    output_audio: true,
    user_id: '60a7c8f5e8b4f5001f7a8c23',
    instructions: 'You are a helpful customer service agent.'
  })
};

fetch('https://secureai.hiperai.ai/api/external/speech/s2s/webrtc', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

"v=0\r\no=- 1234567890 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\n..."

Modelos & Generación AI

Initiate Speech-to-Speech WebRTC Session

Establishes a WebRTC connection for real-time speech-to-speech conversations using OpenAI Realtime API. This endpoint accepts a WebRTC SDP offer and returns an SDP answer that can be used to establish a peer connection with OpenAI’s Realtime API.

Usage Flow:

Client creates a WebRTC offer (RTCPeerConnection.createOffer)
Client sends the SDP offer to this endpoint
This endpoint proxies the offer to OpenAI Realtime API
Client receives SDP answer and establishes WebRTC connection
Client can then have real-time voice conversations with the AI

S2S Time Tracking:

Each user has a monthly S2S time quota based on their license
Time is tracked in minutes and deducted when sessions are logged
Use /speech/s2s/status to check remaining time
Use /speech/s2s/log-session to log session duration and deduct time

SMLTP Integration:

All requests are processed through SMLTP for security and compliance
Model validation is enforced based on the specified SMLTP policy
Requests are audited and logged for compliance tracking

POST

speech

s2s

webrtc

Initiate Speech-to-Speech WebRTC Session

curl --request POST \
  --url https://secureai.hiperai.ai/api/external/speech/s2s/webrtc \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "sdp": "v=0\r\no=- 4611731400430051336 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\n...",
  "model": "gpt-4o-mini-realtime-preview",
  "voice": "alloy",
  "smltp_policy": "internal",
  "output_audio": true,
  "user_id": "60a7c8f5e8b4f5001f7a8c23",
  "instructions": "You are a helpful customer service agent."
}
'

import requests

url = "https://secureai.hiperai.ai/api/external/speech/s2s/webrtc"

payload = {
    "sdp": "v=0
o=- 4611731400430051336 2 IN IP4 127.0.0.1
s=-
t=0 0
...",
    "model": "gpt-4o-mini-realtime-preview",
    "voice": "alloy",
    "smltp_policy": "internal",
    "output_audio": True,
    "user_id": "60a7c8f5e8b4f5001f7a8c23",
    "instructions": "You are a helpful customer service agent."
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    sdp: 'v=0\r\no=- 4611731400430051336 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\n...',
    model: 'gpt-4o-mini-realtime-preview',
    voice: 'alloy',
    smltp_policy: 'internal',
    output_audio: true,
    user_id: '60a7c8f5e8b4f5001f7a8c23',
    instructions: 'You are a helpful customer service agent.'
  })
};

fetch('https://secureai.hiperai.ai/api/external/speech/s2s/webrtc', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

"v=0\r\no=- 1234567890 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\n..."

Initiate Speech-to-Speech WebRTC Session

Establish a WebRTC connection for real-time speech-to-speech conversations using OpenAI Realtime API.

Endpoint

POST /speech/s2s/webrtc

Description

Usage Flow

Client creates a WebRTC offer (RTCPeerConnection.createOffer)
Client sends the SDP offer to this endpoint
This endpoint proxies the offer to OpenAI Realtime API
Client receives SDP answer and establishes WebRTC connection
Client can then have real-time voice conversations with the AI

S2S Time Tracking

Each user has a monthly S2S time quota based on their license
Time is tracked in minutes and deducted when sessions are logged
Use /speech/s2s/status to check remaining time
Use /speech/s2s/log-session to log session duration and deduct time

SMLTP Integration

All requests are processed through SMLTP for security and compliance
Model validation is enforced based on the specified SMLTP policy
Requests are audited and logged for compliance tracking

Authentication

Required: API Key

Authorization: Bearer sk-your-api-key-here

Request Body

Parameter	Type	Required	Description
`sdp`	string	Yes	WebRTC SDP offer from the client’s RTCPeerConnection
`model`	string	No	OpenAI Realtime model (default: “gpt-4o-mini-realtime-preview”)
`voice`	string	No	Voice to use for AI response (default: “alloy”)
`smltp_policy`	string	No	SMLTP policy (default: “internal”)
`output_audio`	boolean	No	Whether to enable audio output (default: true)
`user_id`	string	No	User ID to bill this session to (defaults to API key owner)
`instructions`	string	No	Optional system instructions for the AI assistant

Available Models

gpt-4o-mini-realtime-preview
gpt-4o-realtime-preview

Available Voices

alloy (default)
echo
fable
onyx
nova
shimmer
ash
ballad
coral

Available SMLTP Policies

public
internal (default)
internal-strict
confidential
hipaa
gdpr
pci-dss

Request Example

curl -X POST "https://{customer.name}.hiperai.ai/api/external/speech/s2s/webrtc" \
  -H "Authorization: Bearer sk-your-api-key-here" \
  -H "Content-Type: application/json" \
  -d '{
    "sdp": "v=0\r\no=- 4611731400430051336 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\n...",
    "model": "gpt-4o-mini-realtime-preview",
    "voice": "alloy",
    "smltp_policy": "internal",
    "output_audio": true,
    "instructions": "You are a helpful customer service agent."
  }'

JavaScript/Node.js

// Create WebRTC peer connection
const pc = new RTCPeerConnection();

// Create offer
const offer = await pc.createOffer();
await pc.setLocalDescription(offer);

// Send SDP offer to SecureAI
const response = await fetch('https://{customer.name}.hiperai.ai/api/external/speech/s2s/webrtc', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer sk-your-api-key-here',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    sdp: offer.sdp,
    model: 'gpt-4o-mini-realtime-preview',
    voice: 'alloy',
    smltp_policy: 'internal',
    output_audio: true,
    instructions: 'You are a helpful customer service agent.'
  })
});

// Get SDP answer
const sdpAnswer = await response.text();

// Set remote description
await pc.setRemoteDescription(new RTCSessionDescription({
  type: 'answer',
  sdp: sdpAnswer
}));

// Now you can have real-time voice conversations

Python

import requests

url = "https://{customer.name}.hiperai.ai/api/external/speech/s2s/webrtc"
headers = {
    "Authorization": "Bearer sk-your-api-key-here",
    "Content-Type": "application/json"
}
data = {
    "sdp": "v=0\r\no=- 4611731400430051336 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\n...",
    "model": "gpt-4o-mini-realtime-preview",
    "voice": "alloy",
    "smltp_policy": "internal",
    "output_audio": True,
    "instructions": "You are a helpful customer service agent."
}

response = requests.post(url, headers=headers, json=data)
sdp_answer = response.text
print('SDP Answer:', sdp_answer)

Response

Success Response (200)

Content-Type: application/sdp The response is an SDP answer string that can be used with RTCPeerConnection.setRemoteDescription().

v=0
o=- 1234567890 2 IN IP4 127.0.0.1
s=-
t=0 0
...

Error Responses

400 Bad Request

{
  "success": false,
  "error": "Invalid SDP offer",
  "message": "SDP offer is required and must be a string",
  "request_id": "96bba6a4-0e7a-4fd7-93a7-3b40428729fb"
}

403 Forbidden

S2S Time Limit Reached

{
  "success": false,
  "error": "S2S time limit reached",
  "message": "Insufficient S2S time remaining for this user",
  "remaining_minutes": 0,
  "next_renewal_date": "2024-12-01T12:55:35.721Z",
  "request_id": "96bba6a4-0e7a-4fd7-93a7-3b40428729fb"
}

Model Validation Failed

{
  "success": false,
  "error": "Model validation failed",
  "message": "Model gpt-4o-mini-realtime-preview is not allowed by SMLTP policy test-policy-active",
  "request_id": "96bba6a4-0e7a-4fd7-93a7-3b40428729fb"
}

500 Internal Server Error

{
  "success": false,
  "error": "S2S WebRTC failed",
  "message": "An error occurred while processing your request",
  "request_id": "96bba6a4-0e7a-4fd7-93a7-3b40428729fb"
}

Notes

The SDP offer must be a valid WebRTC SDP offer string
After receiving the SDP answer, use it to set the remote description on your RTCPeerConnection
Check S2S time status before initiating sessions using /speech/s2s/status
Log session duration after completion using /speech/s2s/log-session
All requests are processed through SMLTP for security and compliance
The user_id parameter allows billing to a different user account

Authorizations

Authorization

string

header

required

API key authentication using Bearer token format. Example: Authorization: Bearer sk-your-api-key-here

Body

application/json

sdp

string

required

WebRTC SDP offer from the client's RTCPeerConnection

Example:

"v=0\r\no=- 4611731400430051336 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\n..."

model

enum<string>

default:gpt-4o-mini-realtime-preview

OpenAI Realtime model to use

Available options:

gpt-4o-mini-realtime-preview,

gpt-4o-realtime-preview

voice

enum<string>

default:alloy

Voice to use for the AI response

Available options:

alloy,

echo,

fable,

onyx,

nova,

shimmer,

ash,

ballad,

coral

smltp_policy

enum<string>

default:internal

SMLTP policy to apply for this session

Available options:

public,

internal,

internal-strict,

confidential,

hipaa,

gdpr,

pci-dss

output_audio

boolean

default:true

Whether to enable audio output (default: true)

user_id

string

Optional user ID to bill this session to (defaults to API key owner)

Example:

"60a7c8f5e8b4f5001f7a8c23"

instructions

string

Optional system instructions for the AI assistant

Example:

"You are a helpful customer service agent."

Response

SDP answer from OpenAI Realtime API

SDP answer that can be used with RTCPeerConnection.setRemoteDescription()

Primeros Pasos & Setup

Modelos & Generación AI

Knowledge Base & RAG

Seguridad, Guardrails & Auditoría

Usuarios & Acceso (IAM)

Facturación, Cuotas & Webhooks

Initiate Speech-to-Speech WebRTC Session

Initiate Speech-to-Speech WebRTC Session

Endpoint

Description

Usage Flow

S2S Time Tracking

SMLTP Integration

Authentication

Request Body

Available Models

Available Voices

Available SMLTP Policies

Request Example

JavaScript/Node.js

Python

Response

Success Response (200)

Error Responses

400 Bad Request

403 Forbidden

S2S Time Limit Reached

Model Validation Failed

500 Internal Server Error

Notes

Authorizations

Body

Response

​Initiate Speech-to-Speech WebRTC Session

​Endpoint

​Description

​Usage Flow

​S2S Time Tracking

​SMLTP Integration

​Authentication

​Request Body

​Available Models

​Available Voices

​Available SMLTP Policies

​Request Example

​JavaScript/Node.js

​Python

​Response

​Success Response (200)

​Error Responses

​400 Bad Request

​403 Forbidden

​S2S Time Limit Reached

​Model Validation Failed

​500 Internal Server Error

​Notes

Authorizations

Body

Response

Initiate Speech-to-Speech WebRTC Session

Endpoint

Description

Usage Flow

S2S Time Tracking

SMLTP Integration

Authentication

Request Body

Available Models

Available Voices

Available SMLTP Policies

Request Example

JavaScript/Node.js

Python

Response

Success Response (200)

Error Responses

400 Bad Request

403 Forbidden

S2S Time Limit Reached

Model Validation Failed

500 Internal Server Error

Notes