A proof of concept for a WebXR remote desktop environment utilizing WebRTC (with WHIP) and WebSockets.
This project is primarily created to provide a VR remote desktop environment for Linux environment.
It requires:
- NodeJS
- Docker / podman-compose
- Open Broadcaster Software (OBS)
docker compose up -d # starts nginx and OvenMediaEngine
npm start # start the NodeJS backend and frontend serverOvenMediaEngine will be started as part of the docker-compose services
docker compose logs -f omeWe use OvenMediaEngine's WHIP (WebRTC-HTTP Ingestion Protocol) for WebRTC ingest.
This will expose the following endpoints:
- Clear - http://127.0.0.1:10080/app/stream?direction=whip
- TLS - https://127.0.0.1:10081/app/stream?direction=whip
This guide uses OBS to capture the Desktop environment and stream it to the WHIP WebRTC ingest endpoint.
- OBS v30 or above
- WebRTC (WHIP) output support (>= v30)
- obs-websocket (>= v28)
You can configure any source you want here.
For remote desktop experience:
- Configure a Screen Capture source.
- Select the source, right click and choose: "Resize output (source size)"
Output size is currently limited to 4096x4096 due to the usage of H264 encoder. AV1 and H265 is likely unsupported when used with OvenMediaEngine WebRTC ingest.
Reference:
Open Settings > Stream:
- Select service: WHIP
- Server: http://127.0.0.1:10082/app/stream?direction=whip
- Bearer Token: leave blank
Configure your audio / video and video encoding as followed:
The following encoder are tested working on Arch Linux on Wayland with a NVIDIA RTX graphic card:
- x264 (CPU)
- NVIDIA NVENC H.264 (GPU)
Sample encoder settings
- Video Encoder: x264
- Encoder Settings:
- Keyframe Interval: 1s
- CPU Usage Preset: ultrafast
- Profile: baseline / high
- Tune: zerolatency
Sample encoder settings for NVIDIA graphic card
- Video Encoder: NVIDIA NVENC H.264
- Encoder Settings:
- Keyframe Interval: 1s
- Preset: P1: Fastest (Lowest Quality)
- Tuning: Ultra Low Latency
- Profile: baseline / high
- Confirm / Apply the settings,
- "Start Streaming"
- Access Control using Signed Policy - https://airensoft.gitbook.io/ovenmediaengine/access-control/signedpolicy
WRDXR_OBS_WEBSOCKET=http://localhost:4455
WRDXR_OBS_WEBSOCKET_PASSWORD=check-your-obs-settingUsage of WebXR Device API outside of localhost requires a secure context.
If you are loading this onto a standalone headset such as Oculus Quest, you will to:
- Option 1: provision SSL certs for the NodeJS frontend client and for the OvenMediaEngine
- Option 2: provide a local DNS server that resolve your application IP address to a
*.localhostURL.
The guide below shows usage example using option 1.
This project generate a self-signed TLS certificate when you run docker-compose up -d
The following ports are running in secure protocols:
| Type | URL |
|---|---|
| Frontend | https://127.0.0.1:10081 |
| OvenMediaEngine | https://127.0.0.1:10083 |
| OBS WebSocket | https://127.0.0.1:10085 |
Assuming both client and server are on the same network:
- server: 192.168.1.234 (This server)
- client: 192.168.1.100 (Your VR standalone headset)
You can find out your server ip using ip a | grep "inet "
Your SSL endpoints should look like this:
- Frontend -
https://192.168.1.234:10081 - OvenMediaEngine -
wss://192.168.1.234:10083/app/stream
You may create a .env.local with the following content:
VITE_WRDXR_DEFAULT_FILE=wss://192.168.1.234:10083/app/streamOn your standalone headset, open a WebXR capable web browser and go to:
https://192.168.1.234:10081/?file=wss://192.168.1.234:10083/app/stream
Accept the warning about the SSL self-signed certificate
You should see your broadcasted source on the video player.
Click "Open VR" to enter VR mode.
usage-quest-browser.mp4
Using a WebXR capable browser, browse to:
You should see your broadcasted source on the video player.
Click "Open VR" to enter VR mode.