Hello everyone!
I would like to share the results of an experiment with the Realtek AMB82 Mini (RTL8735B).
My goal was to stream Full HD H.264 video directly to a web browser over WebRTC, without AWS Kinesis, external signaling services, or a separate video transcoding server.
After adapting the official FreeRTOS SDK, I managed to establish a working local WebRTC connection between the AMB82 Mini and Firefox.
Confirmed result: 1920×1080 H.264 video, approximately 110–120 ms end-to-end latency in preliminary local tests.
1. Why I Started This Project
I initially used the Arduino SDK with H.264 streaming over TCP/WebSocket.
The video worked, but occasional freezes and delays, especially around large keyframes and network stalls, motivated me to investigate UDP-based real-time transport.
I decided to try the existing FreeRTOS WebRTC implementation, replacing its AWS-dependent signaling layer with a small local signaling server.
2. Hardware and Software
Hardware:
-
Realtek AMB82 Mini (RTL8735B)
-
Stock JXF37/JXF37P camera module
-
Wi-Fi network
-
Windows PC
Software:
-
Official Ameba Pro2 FreeRTOS SDK
-
GCC ARM toolchain 10.4.1
-
CMake with MSYS2/MinGW build environment
-
mbedTLS 2.28.1
-
Node.js signaling server
-
Mozilla Firefox
Official SDK: GitHub - Ameba-AIoT/ameba-rtos-pro2: Realtek Official IoT Software Development Kit for Ameba Series SoC: (Ameba Pro2) · GitHub
The base SDK example is kvs_webrtc_v2_mmf.
3. Architecture
The system uses two separate channels.
Signaling:
Firefox → WebSocket → Node.js → HTTP → AMB82 Mini
The browser connects to /signal using WebSocket. The AMB82 uses a small HTTP client to exchange SDP and ICE messages with the Node.js server.
Video:
AMB82 Mini → H.264 → RTP/SRTP → UDP → Firefox
Video is transmitted directly between the AMB82 and the browser.
Node.js does not receive, decode, transcode, or relay the video stream.
4. FreeRTOS SDK Modifications
I used the official kvs_webrtc_v2_mmf example as the starting point.
The main modifications were:
-
Introduced an
ASC_LOCAL_SIGNALINGCMake option to select a custom local signaling controller instead of the original AWS signaling implementation. -
Added a lightweight HTTP signaling adapter for the AMB82.
-
Implemented a small Node.js server to bridge WebSocket browser messages and HTTP device messages.
-
Adapted portions of the DTLS/mBedTLS integration for the SDK’s mbedTLS 2.28.1 configuration.
-
Applied IPv4-only compatibility fixes to ICE-related source files.
-
Disabled the unused SCTP library during this experiment.
-
Bypassed the blocking SNTP wait in the experimental local mode. This is a laboratory workaround, not a production-ready solution.
The WebRTC media implementation itself remains based on the Realtek SDK.
Important: These are real source modifications, not just CMake options. A clean upstream SDK requires the corresponding adapter and compatibility patches before it can build this configuration.
5. Important: Configure the Camera Sensor
This was the most significant hardware-related problem during the experiment.
Initially, the WebRTC connection was established successfully, but Firefox received no video RTP packets.
The UART output included:
VOE command 0x206 fail ret 0x0
I discovered that the FreeRTOS SDK was configured for the GC2053 sensor, while my AMB82 Mini uses the stock JXF37 camera.
The sensor configuration is located at:
project/realtek_amebapro2_v0_example/inc/sensor.h
For my board, I used:
-
SENSOR_MAX = 2 -
sen_id = { SENSOR_DUMMY, SENSOR_F37 } -
USE_SENSOR = SENSOR_F37 -
manual_iq = { "iq", "iq_f37" } -
MANUAL_SENSOR_IQ = 0xFF
The SDK already contained the necessary binary files:
-
sensor_f37.bin -
iq_f37.bin -
fcs_data_f37.bin
After modifying sensor.h, I performed a clean build and verified the generated amebapro2_sensor_set.json.
The selected SENSOR_SET1 correctly referenced all three F37 binary files.
After flashing the new F37 configuration, video streaming started working.
For anyone experiencing a successful WebRTC handshake but no video packets, I strongly recommend checking the actual sensor model and generated ISP configuration before debugging RTP.
6. Building the Firmware
I created a separate CMake build directory to preserve the original AWS example.
The important configuration options were:
-
EXAMPLE:STRING=kvs_webrtc_v2_mmf -
ASC_LOCAL_SIGNALING:BOOL=ON -
BUILD_USRSCTP_LIBRARY:BOOL=OFF -
CMAKE_C_USE_RESPONSE_FILE_FOR_INCLUDES=ON -
CMAKE_CXX_USE_RESPONSE_FILE_FOR_INCLUDES=ON
The build also uses the SDK’s toolchain.cmake.
For Windows, I recommend keeping the SDK path short. I used D:\rtk because very long compiler command lines caused problems in my environment.
After configuring the project, the final build command was:
cmake --build . --target flash --parallel 1
The generated firmware image was:
flash_ntz.bin
I flashed it using the official Realtek UART programming tool.
Make sure the programming tool, flash image, and boot mode match your board. Back up your working firmware before replacing it.
7. Connecting the Browser
The Node.js server listens on port 8080.
The AMB82 connects to the server over the local network. Once the board is online, the browser can open:
http://localhost:8080/
The browser sends a WebRTC SDP Offer. The AMB82 generates the SDP Answer, and both peers exchange ICE candidates.
For the initial Firefox LAN tests, I temporarily disabled mDNS host candidate obfuscation using:
about:config → media.peerconnection.ice.obfuscate_host_addresses → false
This was necessary because the embedded ICE implementation did not resolve Firefox’s .local host candidates.
This is only a testing workaround and should not be considered a production solution.
I also needed a compatibility fix in the Node.js server to recover an incorrectly formatted SDP Answer generated by the experimental signaling adapter.
The correct long-term solution is to fix JSON encoding in the embedded adapter.
8. Results
The following were confirmed on real hardware:
-
Wi-Fi connection established successfully.
-
Custom local signaling operational without AWS.
-
SDP Offer/Answer exchanged successfully.
-
ICE connection established.
-
WebRTC PeerConnection reached
connected. -
Firefox received and decoded H.264 video.
-
Full HD video displayed successfully.
In one diagnostic snapshot, Firefox reported:
| Metric | Value |
|---|---|
| RTP packets received | 4,589 |
| Frames received | 2,239 |
| Frames decoded | 2,238 |
| Keyframes decoded | 75 |
| ICE state | connected |
| Peer connection | connected |
Preliminary end-to-end latency measurements using a visible stopwatch were approximately 110–120 ms.
These measurements are indicative, not a controlled latency or packet-loss benchmark.
9. Current Limitations
This implementation is still an experimental proof of concept.
-
Tested successfully on a trusted local Wi-Fi network.
-
Currently supports one viewer.
-
Does not provide authentication or HTTPS/WSS.
-
Does not provide STUN/TURN NAT traversal.
-
Opening the signaling port from the internet is not sufficient for remote video playback.
-
Session teardown and reconnect behavior require further work.
-
SNTP handling and production security need to be restored and validated.
-
Long-term stability and packet-loss behavior have not yet been benchmarked.
I also tested the signaling server through an external internet connection. SDP exchange worked, but ICE failed because the peers did not have a usable UDP route through NAT.
This is expected with the current host-candidate-only configuration.
10. Source Code and Reproduction
The working experiment consists of a local signaling adapter, a Node.js signaling server, a browser viewer, and several compatibility patches for the official SDK.
I am preparing these modifications for sharing separately from the Realtek SDK, so that users can apply them to their own SDK checkout.
I intend to share the patch bundle and a clean build procedure after reviewing the dependencies and redistribution terms.
In the meantime, I hope these findings are useful to anyone experimenting with local WebRTC on AMB82 Mini, especially the JXF37 sensor configuration.
I would also be interested to hear whether Realtek has a recommended implementation for private-network WebRTC signaling without AWS, and whether there is a preferred way to support local ICE and future TURN integration.
Thanks to the Realtek developers and community for maintaining the SDK and making this kind of experimentation possible!
Thanks for posting!