🎮 Custom Video SDK for Unity - Developer Documentation

📋 Overview

✨ Key Benefits

Self-hosted solution
Full source code control

🎮 Gaming Features

Mobile game optimized
No usage limits
Low-latency streaming

🛠️ Technology Stack

⚡

Backend

Node.js - Server runtime
Express.js - Web framework
Socket.IO - Real-time communication
WebRTC - Video/audio streaming
PM2 - Process management

🏗️

Infrastructure

Nginx - Reverse proxy
SSL/TLS - Security
VPS Hosting - Self-hosted
Auto-scaling - PM2 clusters
Load balancing - Built-in

🎮

Unity Client

C# SDK - Native integration
WebRTC Unity - Video handling
JSON API - Data exchange
Event-driven - Callbacks
Cross-platform - All devices

🎮 Unity Integration

🏗️ Basic Setup (Identical to Agora)

using UnityEngine;
                    using CustomVideoSDK;

                    public class VideoCallManager : MonoBehaviour
                    {
                    private IRtcEngine rtcEngine;
                    private string appId = "your-app-id";
                    private string serverUrl = "https://videosdk.genisisserver.space";

                    void Start()
                    {
                    // Initialize SDK (same as Agora)
                    rtcEngine = CustomVideoSDK.GetEngine(appId, serverUrl);

                    // Set up callbacks
                    rtcEngine.OnJoinChannelSuccess = OnJoinChannelSuccess;
                    rtcEngine.OnUserJoined = OnUserJoined;
                    rtcEngine.OnUserOffline = OnUserOffline;
                    rtcEngine.OnError = OnError;

                    // Enable video
                    rtcEngine.EnableVideo();
                    rtcEngine.EnableVideoObserver();
                    }
                    }

🧪 Testing Endpoints

Use these URLs to test server functionality:

🏥 Health Check

Server status and uptime monitoring

Test Health

📊 API Information

SDK version and available features

Test API

🎮 Host Dashboard

Stream hosting and management interface

Test Host

📺 Client Viewer

Stream viewing and interaction interface

Test Client

🔧 Debug Tools

Connection diagnostics and troubleshooting

Test Debug

⚙️ Configuration Examples

📹 Video-Only Mode

// Video streaming without audio
                        rtcEngine = CustomVideoSDK.GetEngine(appId, serverUrl);
                        rtcEngine.EnableVideo();
                        rtcEngine.DisableAudio(); // Disable audio

                        // Set video quality
                        rtcEngine.SetVideoEncoderConfiguration(new VideoEncoderConfiguration
                        {
                        dimensions = new VideoDimensions(1280, 720),
                        frameRate = VideoFrameRate.Fps30,
                        bitrate = 1500
                        });

                        rtcEngine.JoinChannelByKey("", channelName, 0);

🎤 Audio-Only Mode

// Voice chat without video
                        rtcEngine = CustomVideoSDK.GetEngine(appId, serverUrl);
                        rtcEngine.EnableAudio();
                        rtcEngine.DisableVideo(); // Disable video

                        // Set audio quality for gaming
                        rtcEngine.SetAudioProfile(
                        AudioProfile.MusicHighQuality,
                        AudioScenario.GameStreaming
                        );

                        rtcEngine.JoinChannelByKey("", channelName, 0);

🧪 Live Testing

📊 Endpoint Status

Loading endpoint status...

📚 Theory & Concepts

🌐 WebRTC Deep Dive

What is WebRTC?

WebRTC (Web Real-Time Communication) is an open-source project that enables real-time peer-to-peer communication of audio, video, and data in web browsers and mobile applications. It eliminates the need for plugins or additional software installations.

Key Components:

• MediaStream: Captures audio/video
• RTCPeerConnection: Handles connection
• RTCDataChannel: Transfers data
• getUserMedia: Accesses devices

Why WebRTC for Gaming?

WebRTC provides ultra-low latency communication essential for real-time gaming experiences. Traditional streaming protocols can have 5-30 second delays, while WebRTC achieves sub-second latency.

Gaming Benefits:

• <100ms latency: Real-time interaction
• P2P connections: Direct communication
• Adaptive bitrate: Quality optimization
• Cross-platform: Universal support

🔄 WebRTC Connection Flow

1️⃣

Signaling

Exchange connection info

2️⃣

STUN/TURN

NAT traversal

3️⃣

ICE Candidates

Find connection path

4️⃣

P2P Stream

Direct media flow

📹 Video Streaming Theory

🎥 Video Encoding

Video encoding compresses raw video data for efficient transmission. Modern codecs like H.264 and VP8 provide excellent quality-to-size ratios essential for real-time streaming.

H.264: Industry standard, widely supported

VP8/VP9: Open-source, WebRTC default

AV1: Next-gen efficiency

📊 Bitrate Management

Adaptive bitrate streaming adjusts video quality based on network conditions. This ensures smooth playback even with varying connection speeds.

High Quality: 2-4 Mbps (1080p)

Medium Quality: 1-2 Mbps (720p)

Low Quality: 0.5-1 Mbps (480p)

🎯 Quality Metrics

Video quality is measured by resolution, frame rate, and bitrate. Gaming applications typically prioritize low latency over maximum resolution.

Resolution: 720p-1080p optimal

Frame Rate: 30-60 FPS

Latency: <100ms target

🔧 Technical Parameters

// Unity Video Configuration
// Optimal settings for mobile gaming
rtcEngine.SetVideoEncoderConfiguration(new VideoEncoderConfiguration
{
    dimensions = new VideoDimensions(1280, 720), // 720p
    frameRate = VideoFrameRate.Fps30,           // 30 FPS
    bitrate = 1500,                             // 1.5 Mbps
    orientationMode = OrientationMode.FixedLandscape
});

// Network adaptation
rtcEngine.SetRemoteVideoStreamType(uid, VideoStreamType.Low);
rtcEngine.EnableDualStreamMode(true);

Resolution Impact:

4K (3840×2160): 15-25 Mbps

1080p (1920×1080): 3-6 Mbps

720p (1280×720): 1.5-3 Mbps

480p (854×480): 0.5-1 Mbps

Frame Rate Impact:

60 FPS: 2x bandwidth usage

30 FPS: Standard for most games

15 FPS: Minimum for smooth motion

🌐 Network Architecture

🏗️ Signaling Server Role

The signaling server acts as a matchmaker, helping clients discover each other and exchange connection information. Once peers connect directly, the server's job is done.

Server Responsibilities:

• Room/channel management
• User authentication
• SDP offer/answer relay
• ICE candidate exchange
• Connection state monitoring

// Server-side room management
socket.on('join-room', (data) => {
    const { roomId, userId } = data;
    socket.join(roomId);
    socket.to(roomId).emit('user-joined', userId);
    
    // Send existing users to new user
    const room = io.sockets.adapter.rooms.get(roomId);
    socket.emit('room-users', Array.from(room));
});

🔄 P2P Connection

Peer-to-peer connections allow direct communication between clients without routing through servers. This reduces latency and server load while improving privacy.

P2P Advantages:

• Ultra-low latency (<50ms)
• Reduced server bandwidth
• Direct media encryption
• Scalable architecture
• Private communication

// Client-side P2P setup
peerConnection = new RTCPeerConnection({
    iceServers: [
        { urls: 'stun:stun.l.google.com:19302' },
        { urls: 'turn:turn.example.com', 
          username: 'user', credential: 'pass' }
    ]
});

peerConnection.onicecandidate = (event) => {
    if (event.candidate) {
        socket.emit('ice-candidate', event.candidate);
    }
};

🛡️ NAT Traversal & STUN/TURN

🔍 STUN Servers

STUN (Session Traversal Utilities for NAT) helps clients discover their public IP address and the type of NAT they're behind.

Function: IP discovery

Cost: Free (Google, Cloudflare)

Usage: 80% of connections

🔄 TURN Servers

TURN (Traversal Using Relays around NAT) provides relay services when direct P2P connection fails due to restrictive NATs.

Function: Media relay

Cost: Bandwidth usage

Usage: 20% of connections

🧩 ICE Protocol

ICE (Interactive Connectivity Establishment) combines STUN and TURN to find the best connection path between peers.

Function: Path optimization

Fallback: STUN → TURN

Result: Best connection

🔒 Security & Privacy

🛡️ Encryption Standards

WebRTC uses mandatory encryption for all media streams. DTLS (Datagram Transport Layer Security) and SRTP (Secure Real-time Transport Protocol) protect data in transit.

Security Layers:

• DTLS: Handshake encryption
• SRTP: Media stream encryption
• ICE: Connection authentication
• Certificate: Identity verification

🔐 Privacy Features

Self-hosted solutions provide complete privacy control. Unlike cloud services, your data never leaves your infrastructure, ensuring GDPR compliance and data sovereignty.

Privacy Benefits:

• Data Control: Your servers only
• GDPR Compliant: EU data protection
• No Third-party: Direct P2P streams
• Audit Trail: Complete logging

🔧 Security Implementation

// Server-side security
const express = require('express');
const helmet = require('helmet');
const rateLimit = require('express-rate-limit');

app.use(helmet({
    contentSecurityPolicy: {
        directives: {
            defaultSrc: ["'self'"],
            connectSrc: ["'self'", "wss:", "https:"],
            mediaSrc: ["'self'", "blob:"]
        }
    }
}));

const limiter = rateLimit({
    windowMs: 15 * 60 * 1000, // 15 minutes
    max: 100 // limit each IP to 100 requests per windowMs
});

app.use('/api', limiter);

// Client-side security
// Verify server certificate
const rtcConfig = {
    iceServers: [
        { urls: 'stun:stun.l.google.com:19302' }
    ],
    iceCandidatePoolSize: 10,
    iceTransportPolicy: 'all', // Allow all transports
    bundlePolicy: 'max-bundle', // Bundle media
    rtcpMuxPolicy: 'require'    // Require RTCP mux
};

// Validate incoming connections
peerConnection.onconnectionstatechange = () => {
    if (peerConnection.connectionState === 'failed') {
        console.error('Connection failed - possible security issue');
        // Implement reconnection logic
    }
};

⚡ Performance Optimization

🚀 Latency Reduction

Minimizing latency is crucial for gaming applications. Every millisecond counts in competitive gaming scenarios.

Network: <20ms optimal

Encoding: 2-5ms typical

Decoding: 2-5ms typical

Display: 16ms (60Hz)

Total: <50ms target

📊 Quality Adaptation

Dynamic quality adjustment based on network conditions ensures smooth streaming experience.

Bandwidth: Monitor available

Packet Loss: Detect drops

RTT: Measure round-trip

Jitter: Smooth variations

Adaptation: Auto-adjust

🔧 Resource Management

Efficient resource utilization prevents bottlenecks and ensures stable performance.

CPU: Optimize encoding

Memory: Buffer management

GPU: Hardware acceleration

Network: Connection pooling

Storage: Caching strategy

🎯 Unity-Specific Optimizations

// Unity performance optimization
public class VideoOptimizer : MonoBehaviour
{
    private IRtcEngine rtcEngine;
    private float lastQualityCheck;
    
    void Update()
    {
        if (Time.time - lastQualityCheck > 1.0f)
        {
            MonitorNetworkQuality();
            lastQualityCheck = Time.time;
        }
    }
    
    void MonitorNetworkQuality()
    {
        var stats = rtcEngine.GetCallStats();
        
        if (stats.txPacketLossRate > 5) // 5% packet loss
        {
            // Reduce quality
            rtcEngine.SetVideoEncoderConfiguration(
                new VideoEncoderConfiguration
                {
                    dimensions = new VideoDimensions(640, 480),
                    frameRate = VideoFrameRate.Fps15,
                    bitrate = 500
                }
            );
        }
    }
}

Mobile Optimizations:

• Battery: Efficient encoding

• Heat: Thermal management

• Memory: Texture compression

• Network: Cellular awareness

Desktop Optimizations:

• CPU: Multi-threading

• GPU: Hardware encoding

• Network: Connection bonding

• Display: VSync optimization