Skip to main content

Advanced MediaSFU Capabilities by SDK

Build beyond a basic room without guessing which package owns each feature. This guide shows the public path for every shipping MediaSFU SDK and calls out the product or backend work that is not supplied by that client package.

Start with SDK setup and authority, then complete your SDK's room guide before adding the features below.

Use the focused guides for devices, backgrounds, and translation or large rooms, panelists, permissions, and breakouts.

Choose the result you want​

ResultBest starting point
Publish microphone or camera mediaYour SDK's audio/video controls
Add waiting, requests, or panelistsThe supplied room UI and moderation surfaces
Add a canvas or screenboardThe SDK-specific screenboard path below
Show subtitles or translationOnly the SDKs named below, in an enabled room
Build an app-owned operator interfaceUnity's programmatic client, or a supported custom UI path
Add WHIP, WHEP, or HLSMediaSFU protocol HTTP operations
Control an existing configured AI callServer-side AI call controls
Add file sharing, invitation delivery, or recording exportAn application/backend integration; not a client-SDK operation

ReactJS 4.3.0​

Use clickAudio and clickVideo with the complete current-room parameters to publish local media. Screenboard, ScreenboardModal, and captureCanvasStream provide the browser canvas path. For participant-facing applications, start with MediasfuGeneric, MediasfuConference, MediasfuWebinar, MediasfuBroadcast, or MediasfuChat and keep their room state mounted while a session is active.

Waiting, request, and panelist experiences are available through the supplied room components. They are not a general Q&A or hand-raise service; name the action in your product according to the request types your backend supports.

import {
  clickAudio,
  clickVideo,
  captureCanvasStream,
  Screenboard,
} from 'mediasfu-reactjs';

type AudioRequest = Parameters<typeof clickAudio>[0];
type VideoRequest = Parameters<typeof clickVideo>[0];

export const publishMicrophone = (request: AudioRequest) => clickAudio(request);
export const publishCamera = (request: VideoRequest) => clickVideo(request);

ReactJS 4.2.9 does not publish client operations for AI/operator takeover, agent handoff, file transfer, external-media ingress or egress, hybrid HLS delivery, invitation issuance, recording retrieval/export, SIP/PSTN calling, or an end-to-end teardown proof. Its subtitle implementation is part of the supplied room experience; a stable standalone transcription/translation service is not published from the package root.

Angular 2.3.1​

Use the exported ClickAudio, ClickVideo, and transport services from the active room. Screenboard, ScreenboardModal, and CaptureCanvasStream provide the canvas path. LiveSubtitleService, TranslationSettingsModal, and the translation receive services support subtitle presentation after the room has translation enabled.

For a custom participant card, read subtitles from Angular's supplied service instead of maintaining a second subtitle store:

import {Component, Input} from '@angular/core';
import {LiveSubtitleService} from 'mediasfu-angular';

@Component({
  selector: 'app-speaker-subtitle',
  standalone: true,
  template: `<p>{{ text }}</p>`,
})
export class SpeakerSubtitleComponent {
  @Input({required: true}) speakerId = '';
  @Input({required: true}) speakerName = '';

  constructor(private readonly subtitles: LiveSubtitleService) {}

  get text(): string {
    return this.subtitles
      .getSubtitleForSpeaker(this.speakerId, this.speakerName)?.text ?? '';
  }
}

Use the supplied waiting, requests, and panelist components for those exact workflows. Use the prebuilt room components when you want MediaSFU to own the participant workspace.

Angular 2.3.1 does not publish client operations for AI/operator takeover, agent handoff, file transfer, external-media ingress or egress, hybrid HLS, invitation issuance, recording retrieval/export, SIP/PSTN, a turnkey headless operator console, or an end-to-end teardown proof. Translation availability, language coverage, accuracy, and retention remain room/backend policy.

React Native 2.4.0​

Use clickAudio and clickVideo after native permissions and the room transport are ready. Screenboard, ScreenboardModal, Whiteboard, and captureCanvasStream are the advanced canvas path. TranslationSettingsModal and the translation receive controls are available only in rooms configured for translation.

The package supplies waiting, requests, panelists, and participant UI through MediasfuGeneric and its room variants. Treat request types as application meaning, not as a guaranteed Q&A or hand-raise protocol.

import {
  clickAudio,
  clickVideo,
  captureCanvasStream,
} from 'mediasfu-reactnative';

export type MicrophoneRequest = Parameters<typeof clickAudio>[0];
export type CameraRequest = Parameters<typeof clickVideo>[0];
export type CanvasRequest = Parameters<typeof captureCanvasStream>[0];

React Native 2.4.0 does not publish client operations for AI takeover, agent handoff, file sharing, external-media ingress/egress, hybrid HLS, invitation issuance, recording retrieval/export, SIP/PSTN, a turnkey operator console, or an end-to-end teardown proof. Test media and canvas capture on physical Android and iOS devices; do not promise translation accuracy or transcript retention.

Expo 2.5.0​

Use clickAudio and clickVideo after Expo/native permissions and room media readiness. Screenboard, ScreenboardModal, Whiteboard, and captureCanvasStream provide the canvas path. Translation uses the supplied translation settings and receive surfaces in a room whose backend configuration enables it.

Use MediasfuGeneric and its waiting, requests, panelist, and participant surfaces for the provided room experience. Request labels and permissions remain application/backend policy.

Expo 2.5.0 does not publish client operations for AI takeover, agent handoff, file sharing, external-media ingress/egress, hybrid HLS, invitation issuance, recording retrieval/export, SIP/PSTN, a turnkey operator console, or an end-to-end teardown proof. Test Android, iOS, and web separately; an exposed configuration field is not an operational HLS or telephony workflow.

Vue 1.1.1​

Use the root-exported clickAudio and clickVideo functions for active-room tracks. Use the supplied Screenboard component for the package-owned canvas experience. liveSubtitleService, createLiveSubtitleService, and the translation receive services support subtitle presentation in an enabled room.

The supplied room, waiting, requests, and panelist components are the supported participant-workspace path. Vue 1.1.1 does not export a standalone canvas capture function from its package root, so keep custom external-canvas capture inside your application unless you adopt a later public contract.

Vue 1.1.1 does not publish client operations for AI takeover, agent handoff, file transfer, external-media ingress/egress, hybrid HLS, invitation issuance, recording retrieval/export, SIP/PSTN, a turnkey operator console, or an end-to-end teardown proof. Translation availability and retention remain room/backend policy.

Flutter 2.3.0​

Use clickAudio and clickVideo from the package's public exports after room and permission readiness. Screenboard, ScreenboardModal, and captureCanvasStream provide the Flutter canvas path. Use MediasfuGeneric, MediasfuConference, MediasfuWebinar, MediasfuBroadcast, or MediasfuChat for a supplied participant workspace.

Waiting, requests, and panelist widgets provide those exact experiences. The root package does not publish its modern translation implementation as a stable application API, so do not build a custom subtitle integration against internal paths.

Flutter 2.3.0 does not publish client operations for AI takeover, agent handoff, file transfer, external-media ingress/egress, hybrid HLS, invitation issuance, recording retrieval/export, SIP/PSTN, a turnkey operator console, or an end-to-end teardown proof.

Android 1.0.5​

The Android room SDK provides Compose room shapes, participant workspaces, microphone and camera production, remote-media rendering, screen sharing, waiting and request panels, participant moderation, chat, polls, breakouts, whiteboard, and recording controls. Start with the supplied UI, then replace only the cards or workspace your product needs to own.

Android does not turn room configuration fields into WHIP/WHEP/HLS, SIP/PSTN, recording export, file transfer, or AI-agent lifecycle APIs. Use the relevant server-side MediaSFU API for those product workflows.

Kotlin Multiplatform 1.0.5​

Use clickAudio and clickVideo after permission and room readiness. captureCanvasStream and the supplied screenboard/whiteboard components are the Kotlin canvas path. MediasfuGeneric and the conference, webinar, broadcast, and chat variants provide Compose participant workspaces.

Kotlin exposes translation models and receive handlers for rooms configured for translation. They let the app present received translation state; they do not provide a transcription service or guarantee language availability, accuracy, or storage.

Use respondToWaiting, respondToRequests, and the panelist controls for those named workflows. They are not a generic Q&A or hand-raise API.

Kotlin Multiplatform 1.0.5 does not publish client operations for AI takeover, agent handoff, file transfer, external-media ingress/egress, hybrid HLS, invitation issuance, recording retrieval/export, SIP/PSTN, a turnkey operator console, or an end-to-end teardown proof.

Swift and Apple platforms 0.1.3​

The Apple release presents the hosted conference, webinar, broadcast, and chat room through MediaSFUIosHostBridge. Swift controls can toggle microphone, camera, and screen sharing or open supplied panels while the hosted controller owns participant and remote-media rendering.

The current Apple surface does not provide a credential-free Cloud create grant, standalone WHIP/WHEP/HLS controls, SIP/PSTN calling, recording retrieval, file transfer, or an AI-agent start/pause/stop contract. Use backend APIs for those workflows and pass only short-lived, app-approved room data to the device.

Unity 0.1.0-preview.2​

Unity's MediaSfuClient is a programmatic client for an application-owned UI. After the room and media backend are ready, use ProduceMediaTrackAsync for audio/video production and the explicit microphone/camera controls. Use RespondToWaitingParticipantAsync and participant-control methods for the waiting and moderation experiences your scene implements.

await client.RequestMediaPermissionAsync(MediaSfuTrackKind.Audio);
await client.RequestMediaPermissionAsync(MediaSfuTrackKind.Video);
await client.SetMicrophoneEnabledAsync(true);
await client.SetCameraEnabledAsync(true);

The Unity package does not render a participant widget or publish a canvas screenboard operation. It also does not publish client operations for AI takeover, agent handoff, file transfer, external-media ingress/egress, hybrid HLS, captions/transcription, invitation issuance, recording retrieval/export, SIP/PSTN, named Q&A/hand raise, or an end-to-end teardown proof. Room creation metadata such as HLS or SIP flags is not a playback, publishing, or calling API.

Shared core 1.1.0​

The shared core is a non-rendering TypeScript layer. It exports clickAudio, clickVideo, their send-transport helpers, and captureCanvasStream. It also exports subtitle state helpers such as createLiveSubtitle, updateLiveSubtitlesFromTranscript, and translationConsumerSwitch.

Use respondToWaiting, respondToRequests, and panelist helpers only with the complete current room/socket state. Your framework must render the interface; the shared package does not provide participant widgets.

import {
  clickAudio,
  clickVideo,
  captureCanvasStream,
  createLiveSubtitle,
} from 'mediasfu-shared';

type AudioRequest = Parameters<typeof clickAudio>[0];
type VideoRequest = Parameters<typeof clickVideo>[0];

Shared core 1.1.0 does not publish client operations for AI takeover, agent handoff, file transfer, external-media ingress/egress, hybrid HLS, invitation issuance, recording retrieval/export, SIP/PSTN, a turnkey operator console, or an end-to-end teardown proof. Subtitle helpers manage received/application state; they are not a transcription provider.

Product capabilities that require an explicit integration​

The following are not implied by a room shape, configuration field, marketing description, or similarly named UI component:

  • AI/operator takeover or an agent-session handoff
  • file/document upload and transfer
  • external-media ingest or egress and hybrid WebRTC/HLS delivery
  • invitation issuance, delivery, redemption, expiry, and reuse protection
  • recording catalogue, metadata retrieval, download, export, retention, and deletion
  • SIP/PSTN dialing, call control, and lifecycle
  • proof that every client and backend resource was removed

MediaSFU publishes separate server contracts for WHIP, WHEP, and HLS resources and for controlling an existing configured AI call. Keep their authority on the server. For the other items, do not infer a workflow from a client SDK field or import an internal file.

Release checklist​

  • The installed package and version match this guide.
  • Only public package entries are imported.
  • The backend enables the room feature before its control is shown.
  • Permission denial and unavailable-device states have a useful next step.
  • Audio, video, canvas, and translation state are tested on every target platform.
  • Requests are labelled according to your backend's actual meaning.
  • Translation UI explains availability without promising accuracy or retention.
  • Unsupported AI, file, HLS, invitation, recording-export, and telephony controls are hidden.
  • Final exit clears application-owned tracks, listeners, timers, drafts, and authority.