Skip to main content
Normalized for Mintlify from knowledge-base/aiconnected-apps-and-modules/modules/aiConnected-voice/webrtc-bridge-technical-design.mdx.

Voice by aiConnected — WebRTC Bridge Technical Design \

Document Information \


Table of Contents \

Voice by aiConnected — WebRTC Bridge Technical Design Document Information Table of Contents 1. Introduction 1.1 Purpose 1.2 Scope 1.3 Design Goals 1.4 Key Terminology 2. Architecture Overview 2.1 High-Level Architecture 2.2 Component Responsibilities 2.3 Data Flow Summary 2.4 Threading Model 3. aiortc Fundamentals 3.1 aiortc Overview 3.2 Core Classes 3.3 RTCPeerConnection Lifecycle 3.4 Basic aiortc Setup 3.5 Custom Audio Tracks 4. SDP Exchange 4.1 SDP Overview 4.2 SDP Structure 4.3 SDP Offer/Answer Flow with GoToConnect 4.4 SDP Negotiator Implementation 4.5 SDP Examples 5. ICE Candidate Handling 5.1 ICE Overview 5.2 ICE Connection Process 5.3 ICE Manager Implementation 5.4 Trickle ICE Flow 6. Audio Frame Processing 6.1 Audio Frame Fundamentals 6.2 PyAV AudioFrame 6.3 Audio Resampling 6.4 Audio Format Conversion 7. Codec Management 7.1 Supported Codecs 7.2 Codec Handler Implementation 7.3 Codec Negotiation Strategy 7.4 Packet Loss Concealment 8. GoToConnect Integration 8.1 GoTo WebRTC Flow 8.2 GoTo Connection Handler 9. LiveKit Integration 9.1 LiveKit Overview 9.2 LiveKit Room Architecture 9.3 LiveKit Connection Handler 10. Bidirectional Bridge 10.1 Bridge Architecture 10.2 Audio Bridge Implementation 10.3 Audio Forking for STT 11. Connection Lifecycle Management 11.1 Complete Call Lifecycle 11.2 Lifecycle Manager Implementation 11.3 Graceful Shutdown 12. Error Handling and Recovery 12.1 Error Categories 12.2 Error Handler Implementation 12.3 ICE Restart Manager 13. Performance Optimization 13.1 Performance Targets 13.2 Audio Buffer Optimization 13.3 Connection Pooling 14. Monitoring and Debugging 14.1 Metrics Collection 14.2 Structured Logging 14.3 Debug Tools 15. Testing Strategy 15.1 Test Categories 15.2 Unit Test Examples 15.3 Integration Test Examples 15.4 Load Test Framework 16. Appendix 16.1 SDP Reference 16.2 Audio Format Reference Document Revision History

  1. Introduction \

1.1 Purpose \

This document provides the technical design for the WebRTC Bridge component in Voice by aiConnected. The WebRTC Bridge is the critical infrastructure that connects telephone calls from GoToConnect to our AI processing pipeline via LiveKit. The bridge must handle real-time audio with strict latency requirements while managing the complexity of two different WebRTC implementations, codec negotiation, and bidirectional audio streaming.

1.2 Scope \

This document covers:
  • aiortc library implementation for WebRTC
  • SDP offer/answer exchange with GoToConnect
  • ICE candidate handling and connectivity
  • Audio frame capture and injection
  • Codec negotiation and transcoding
  • LiveKit room integration
  • Bidirectional audio bridging
  • Error handling and recovery
This document does not cover:
  • GoToConnect API authentication (see ARCH-002)
  • Voice pipeline processing (see ARCH-003)
  • LiveKit server deployment

1.3 Design Goals \

1.4 Key Terminology \


  1. Architecture Overview \

2.1 High-Level Architecture \

2.2 Component Responsibilities \

2.3 Data Flow Summary \

2.4 Threading Model \


  1. aiortc Fundamentals \

3.1 aiortc Overview \

aiortc is a Python implementation of WebRTC and ORTC (Object Real-Time Communications). It provides:
  • Full WebRTC stack in pure Python
  • asyncio-based for non-blocking I/O
  • Support for audio/video/data channels
  • Built-in codecs (Opus, VP8, H.264)

3.2 Core Classes \

3.3 RTCPeerConnection Lifecycle \

3.4 Basic aiortc Setup \

3.5 Custom Audio Tracks \


  1. SDP Exchange \

4.1 SDP Overview \

Session Description Protocol (SDP) describes multimedia sessions. In WebRTC, SDP is used to negotiate:
  • Media types (audio, video, data)
  • Codecs and their parameters
  • Transport information
  • Security parameters (DTLS fingerprint)

4.2 SDP Structure \

4.3 SDP Offer/Answer Flow with GoToConnect \

4.4 SDP Negotiator Implementation \

4.5 SDP Examples \


  1. ICE Candidate Handling \

5.1 ICE Overview \

Interactive Connectivity Establishment (ICE) finds the best path for media between peers:

5.2 ICE Connection Process \

5.3 ICE Manager Implementation \

5.4 Trickle ICE Flow \


  1. Audio Frame Processing \

6.1 Audio Frame Fundamentals \

WebRTC audio is transmitted as discrete frames:

6.2 PyAV AudioFrame \

aiortc uses PyAV (FFmpeg bindings) for audio processing:

6.3 Audio Resampling \

6.4 Audio Format Conversion \


  1. Codec Management \

7.1 Supported Codecs \

7.2 Codec Handler Implementation \

7.3 Codec Negotiation Strategy \

7.4 Packet Loss Concealment \


  1. GoToConnect Integration \

8.1 GoTo WebRTC Flow \

8.2 GoTo Connection Handler \


  1. LiveKit Integration \

9.1 LiveKit Overview \

LiveKit is an open-source WebRTC SFU (Selective Forwarding Unit) that provides:
  • Scalable real-time audio/video
  • Room-based architecture
  • Server-side SDKs
  • Low-latency routing

9.2 LiveKit Room Architecture \

9.3 LiveKit Connection Handler \


  1. Bidirectional Bridge \

10.1 Bridge Architecture \

10.2 Audio Bridge Implementation \

10.3 Audio Forking for STT \


  1. Connection Lifecycle Management \

11.1 Complete Call Lifecycle \

11.2 Lifecycle Manager Implementation \

11.3 Graceful Shutdown \


  1. Error Handling and Recovery \

12.1 Error Categories \

12.2 Error Handler Implementation \

12.3 ICE Restart Manager \


  1. Performance Optimization \

13.1 Performance Targets \

13.2 Audio Buffer Optimization \

13.3 Connection Pooling \


  1. Monitoring and Debugging \

14.1 Metrics Collection \

14.2 Structured Logging \

14.3 Debug Tools \


  1. Testing Strategy \

15.1 Test Categories \

15.2 Unit Test Examples \

15.3 Integration Test Examples \

15.4 Load Test Framework \


  1. Appendix \

16.1 SDP Reference \

16.2 Audio Format Reference \


Document Revision History \