[RTL8735B] 1080p MJPEG on CH0 — runtime hal_video_open failure, and whether 1080p30 is reachable at all
Date: 2026-08-27 (revision 2 of the 2026-08-24 enquiry) Hardware: AMB82-mini (RTL8735B) Arduino SDK: AmebaPro2 4.1.0 VOE: RTL8735B_VOE_1.6.8.0 / Sensor RTL8735B_VOE_1.5.7.0 To: Pammy (Realtek FAE)
0. What changed since the 2026-08-24 enquiry
The first version of this enquiry said twelve ONVIF mandatory tests were failing on this path. That is no longer the situation, and the question has become sharper rather than going away.
We have since moved the ONVIF JPEG tests onto a second media profile backed by a boot-time JPEG channel (CH1, 1280x720, measured 30.2 fps). Those tests now pass. What remains is the part no workaround reaches:
- CH0 cannot produce JPEG at runtime at all (§2), and
- CH0’s boot-time 1080p JPEG runs at ~6.8 fps (§5), which is below what we can ship.
So the enquiry is now two questions rather than one:
- Why does the runtime switch fail, and is it supposed to work? (§2, §6.1, §6.2)
- Is 1080p30 MJPEG reachable on this SoC at all? (§5, §6.4)
Question 2 is the one that decides the product. We have set an internal acceptance criterion: JPEG streaming is offered only if it can run at 1080p30. If the RTL8735B cannot reach that, we will restrict JPEG to GetSnapshotUri and stop offering JPEG video streaming entirely. A clear “no” is as useful to us as a “yes” — it lets us close the question and remove the feature.
1. Summary
With one unchanged firmware build:
- Boot path — CH0 configured as JPEG before
videoInit() → 1080p MJPEG streams successfully.
- Runtime path — CH0 switched from H.264 to JPEG while running →
hal_video_open fails:
VOE command 0x206 fail ret 0xff
VOE_OPEN_CMD command fail 98600800
hal_video_open fail ret=98600800, group=3
Runtime switching between H.264 and H.265 using the same function works perfectly (28–29 fps). The failure is specific to JPEG.
2. The two execution paths
2.1 Boot path — 
succeeds
Camera.configVideoChannel(CHANNEL, config); // config._encoder = VIDEO_JPEG
Camera.videoInit();
Camera.channelBegin(CHANNEL);
cores/ambpro2/video_drv.c → cameraOpen():
video_control(p_priv, CMD_VIDEO_SET_PARAMS, &video_params);
mm_module_ctrl(p, MM_CMD_SET_QUEUE_LEN, fps * 3);
mm_module_ctrl(p, MM_CMD_INIT_QUEUE_ITEMS, MMQI_FLAG_DYNAMIC);
if (JPEG) mm_module_ctrl(p, CMD_VIDEO_SNAPSHOT, 0);
// CMD_VIDEO_APPLY is issued by channelBegin() -> cameraStart()
Result: 1920x1080 MJPEG on port 554. ffmpeg decodes 34 frames in 5 s (~6.8 fps). No hal_video_open failure in the serial log.
2.2 Runtime path — 
fails
Camera.configVideoChannel(CHANNEL, config); // config._encoder = VIDEO_JPEG
Camera.setQLen(CHANNEL, fps * 3);
Camera.snapshotCtrl(CHANNEL, 0); // added to match cameraOpen()
Camera.updateVideoParams(CHANNEL);
cameraUpdateParams():
mm_module_ctrl(p, CMD_VIDEO_STREAM_STOP, stream_id);
mm_module_ctrl(p, CMD_VIDEO_SET_PARAMS, &video_params);
mm_module_ctrl(p, CMD_VIDEO_APPLY, stream_id);
Result: hal_video_open fail ret=98600800, group=3. The encoder never opens → rtp_o_mjpeg_handler is never invoked → zero RTP packets. ffmpeg reports codec mjpeg, unspecified size.
2.3 The parameters are identical
Field-by-field, captured from the same build on both paths:
Boot: enc=2 (VIDEO_JPEG) res=6 (VIDEO_FHD) 1920x1080 fps=30 bps=2097152
Runtime: enc=2 res=6 1920x1080 fps=30 bps=2097152
The parameters are not the difference. The call sequence is.
3. The buffer arithmetic does not add up — and this is our strongest clue
video_open() (video_api.c, around the “jpeg max size = NV12 size” comment):
jpeg_out_buf_size = (enc_in_w * enc_in_h * 3) / 2; // 1920x1080 -> 3,110,400
jpeg_out_rsvd_size = jpeg_out_buf_size;
video_get_buffer_info() reports, on every measurement we have taken, on both the succeeding and the failing path:
out_buf = 2,097,152
out_rsvd = 393,216
required by video_open() 3,110,400
reported by get_buffer_info 2,097,152
difference ~1 MB
The requirement exceeds the reported allocation by roughly 1 MB, yet the boot path succeeds. Either the boot path obtains memory from somewhere video_get_buffer_info() does not report, or the requirement is not enforced on that path.
A third formula is involved: video_buf_calc() sizes the allocation at boot as (w*h)/8 + bps, which for our parameters is 259,200 + 2,097,152. That matches neither of the numbers above.
We cannot reconcile these three formulas, and we believe the answer to this question is also the answer to §5.
Note: our first enquiry listed “insufficient encoding buffer” as a ruled-out hypothesis because the reported value is identical on both paths. We now think that was the wrong conclusion to draw — identical reported values are consistent with the reporting function not covering the allocation that actually matters.
4. Hypotheses we have ruled out, with measurements
| # |
Hypothesis |
Measurement / result |
| 1 |
Bitrate too low |
2 MB → 4 MB; the boot path failed as well. Reverted. |
| 2 |
Single JPEG encoder restriction |
During a successful boot path, CH2 (snapshot=1, JPEG) runs at the same time without issue. |
| 3 |
CH1 occupying the VOE |
Pulled CH1 (port 555) with ffmpeg for 84 s at 30 fps; CH0 still failed on the runtime switch. Since then CH1 also carries a G.711 audio track and the result is unchanged. |
| 4 |
Resolution overhead |
Reduced to 720p; still 0 fps. |
| 5 |
StreamIO stack |
Added 64 KB setStackSize() to the MISO. Since then the audio pipeline was rebuilt as SIMO + MISO (two G.711 encoders). No change either time. |
| 6 |
Audio track interference |
Removed the G.711 track from port 554. No change. |
| 7 |
Queue not rebuilt |
Moved MM_CMD_SET_QUEUE_LEN + MM_CMD_INIT_QUEUE_ITEMS to before CMD_VIDEO_APPLY. No change. |
| 8 |
Missing CMD_VIDEO_SNAPSHOT(0) |
Called explicitly, execution confirmed in the serial log. No change. |
| 9 |
Different ctrl entry point |
Disassembled mm_module_ctrl: for cmd > 6 it fetches ctx->module->control and ctx->priv directly, which is functionally equivalent to video_control(p_priv, ...). |
“Insufficient encoding buffer” appeared on this list in our first enquiry. It has been moved to §3 as an open question, for the reason given there.
5. The frame rate we can reach on the path that DOES work
Even on the boot path, which opens successfully:
1920x1080 MJPEG, measured with ffmpeg over a fixed 5-second window
34 decoded frames in 5 s = 6.8 fps
For context:
ONVIF DTT's streaming tests require 7 frames within 1000 ms
Our reference device (AXIS FA51) advertises 1920x1080 @ 30 fps JPEG
Our CH1 at 1280x720 measured 30.2 fps
6.8 fps is below the certification threshold and far below the reference device. Our own product criterion is 1080p30.
Observation we cannot explain: 1280x720 at 30 fps is 27.6 Mpixel/s. 1920x1080 at 6.8 fps is 14.1 Mpixel/s — half the pixel throughput of the configuration that works. This does not look like a raw encoder-throughput limit, which is why we suspect it is related to §3.
6. Questions
6.1 Error code
- What is the exact meaning of
ret=0x98600800 (group=3, “Driver”)? video_get_error_group() extracts the group as error_id >> 27 & 0xF, but we do not have the mapping table for error IDs.
- For
VOE command 0x206 fail ret 0xff — is there a defined meaning for return value 0xff under 0x206 (VOE_OPEN_CMD)?
6.2 Runtime encoder switching
- Does the RTL8735B VOE support changing an active channel from H.264 to JPEG at runtime?
- If yes — what steps are required beyond
cameraUpdateParams() (STREAM_STOP → SET_PARAMS → APPLY)?
- If no — what is the recommended sequence?
video_close() then video_open(), or must the VOE module be re-initialised?
6.3 JPEG buffer allocation
- How do these three relate?
video_open() (w * h * 3) / 2 = 3,110,400
video_buf_calc() (w * h) / 8 + bps = 2,356,352
get_buffer_info() reports 2,097,152
- Does JPEG require memory reserved during initial boot that a runtime reconfiguration cannot obtain?
CMD_VIDEO_SET_VOE_HEAP is present but commented out in cores/ambpro2/video_drv.c. Is that the intended mechanism for raising the output buffer above 3,110,400, and what value should be used for 1080p JPEG?
6.4 
The question that decides the product
Can the RTL8735B produce 1920x1080 MJPEG at 30 fps at all?
- If yes — what VOE heap size, buffer configuration and channel layout does it require? We currently run CH0 (main), CH1 (MJPEG), CH2 (snapshot), CH3 (motion detection input, VGA RGB @ 10 fps), with roughly 41–60 MB free.
- If no — what is the realistic maximum for 1080p MJPEG on this SoC?
We are asking for a number, not a workaround. Our product criterion is 1080p30; a clear “not achievable” lets us drop JPEG video streaming and keep only GetSnapshotUri, which is all ONVIF Profile T actually requires (§7.19). We would rather close the question than keep engineering around it.
mmf2_video_example_v3_init.c uses V3_FPS = 5, V3_BPS = 2*1024*1024, V3_CHANNEL = 2 — if 5 fps is the intended design point for JPEG on this part, please say so directly.
7. Diagnostic material available on request
- Full serial logs, boot-path success vs runtime failure, same build
stream_fps.ps1 output (ffmpeg decoded-frame counts over a fixed window)- Complete ONVIF DTT 19.06 reports (366 tests)
- Individual serial logs for each ruled-out experiment
- A minimal reproducible sketch
8. Current status
Initialising CH0 as JPEG at boot works, but it is not usable as a product workaround for two separate reasons:
- ONVIF
SetVideoEncoderConfiguration requires runtime switching, and
- even the working path only reaches 6.8 fps at 1080p.
We have routed the ONVIF JPEG tests to a boot-time 720p JPEG channel as an interim measure. That satisfies the test tool but does not meet our product criterion, so the decision on whether to keep JPEG video streaming at all is waiting on §6.4.
Appendix — unrelated observation, low priority
Please treat this as a separate item; it is not part of the enquiry above and should not delay a reply to §6.
If CH1 and CH2 are stopped at the same time, the VOE stops responding entirely:
VOE command wait ACK timeout
hal_video_open ret=ffffffff group=15
After this, even H.264 channels cannot be opened. Are there constraints on which combinations of channels may be stopped while others remain active?
Long-term Risks of Continuous MJPEG Streaming