Send Audio to gemma4:e2b

Arduino AI
1

Application : AI plush toy

LLM Server : EdgeAI-Gallery App with gemma4-e2b (on Android phone)

EVB : AMB82-mini

APK: EdgeAI-Gallery .apk , EdgeAI-Gallery-1.0.19.zip added text/image/audio API by codex.

Code: Ollama_sendAudio.ino

Error :

== Rtl8735b IoT Platform ==

[test mode PG]
Download Image over UART1[tx=4,rx=3] baud=115200

== Rtl8735b IoT Platform ==

[Normal mode]
BootFromNORFlash
[Start Boot ROM…]
=== Load PARTBL ===
=== Load Done ===
=== Load ISP_IQ ===
[fcs chk pass]
ISP_IQ @ 0x8461080, 0x2ff80, 0x0
mfcs_data version 0x00010001
fcs_data version 0x00010101
=== Process ISP_IQ ===
=== Load Done ===
=== Load BL ===
[Image Start Table @ 0x18200]
=== Load Done ===

== Boot Loader ==
Sep 14 2026:19:14:06
=== Load FCS Para ===
=== Load Done ===
[crc pass]
=== Load ISP_IQ Sensor ===
ISP_IQ @ 0x8461080, 0x2ff80
=== Process ISP_IQ ===
=== Load Done ===
=== Load FW1 ===
FW_ISP_IQ @ 0x8061080, 0x31f80
=== Process FW_ISP_IQ ===
DRAM_TYPE is DDR2 128MB.
ddr_freq = 533
VOE flash @ 0x8093080, 0x82f80
FCS KM_status 0x00002081 err 0x0000200a
Wait KM fcs done 0 us
FCS TM_status 0x003f0000
store fcs data for application
It don’t do the sensor initial process
RAM TM_STATUS 0x00bf1208 err 0x00001208
read fcs_status 0x000000bf
read fcs_status 0x000000bf
=== Process VOE IMG ===
[Image Start Table @ 0x20106200]
RAM Load @ 0x8116100->0x20106200, 0x5dac
DDR Load @ 0x811c080->0x70100000, 0x252e0a
=== FW Load Done ===

Boot Loader <==

== RAM Start ==
Build @ 19:07:52, Sep 14 2026
set ASP print off

$8735b>set ASP print off

Attempting to connect to SSID: HITRON-DF90-2.4G
interface 0 is initialized

interface 1 is initialized

cfg_size_lib = 124, cfg_size_user = 124

Initializing WIFI …[Driver]: [HALMAC]

11692M HALMAC_MAJOR_VER = 1

HALMAC_PROTOTYPE_VER = 4

HALMAC_MINOR_VER = 20

HALMAC_PATCH_VER =

[Driver]: The driver include MP

[Driver]: Ver = libwlan:2026.09.14.19.30_b9.6_b5e42ffdfe93d306926f1a3b54269d29d305b1bf

RFE type = 0

start_addr=(0x4000), end_addr=(0x8000), buffer_size=(0x4000), smp_number_max=(2048)

WIFI initialized

[Driver]: set ssid [HITRON-DF90-2.4G]

[Driver]: rtw_joinbss_cmd MGMT_FRAME_PROTECTION_OPTIONAL no pmf.

[Driver]: Bcn (ie: 348, rsn: 0, wpa: 0) ch: 11, seq: 1432, ts: 1890011955211

[Driver]: start auth to f8:34:5a:ab:df:98

[Driver]: auth success, start assoc

[Driver]: listen_interval: 10, bcn_interval: 100

[Driver]: association success(res=4)

[Driver]: set pairwise key to hw: alg:4(WEP40-1 WEP104-5 TKIP-2 AES-4)

[Driver]: set group key to hw: alg:4(WEP40-1 WEP104-5 TKIP-2 AES-4) keyid:2

(0) Scan: 1, Auth: 1, Assoc: 1, 4way: 1, connect: 1, reason: 0

[Driver]: TSFValue = 1890016153612, tsf = 0, shift_set= 0x8000, bcn int = 100

Interface 0 IP address : 192.168.0.19

[LwIP_DHCP] dhcp offered_t0_lease: 604800[video_voe_presetting] fps:30 w:1920 h:1080

fwin(1),enc_en(0),IQ_OFFSET = 0x17b60

fwin(1),enc_en(0),SENSOR_OFFSET = 0x2fba0

sensor id 1 iq_data 17b60 sensor_data 2fba0

Register disk driver to Fatfs.

FATFS Register: disk driver 0

SD_Init 0

part_count = 0

Audio begin
[State] Listening…
Mic Level: 266
Mic Level: 348
Mic Level: 11082

TRIGGERED <<<
[State] Listening Stopped.
[State] Recording Setup
ctx->enc_info.frameLength * 2 = 2048

hal_voe_ready 0x0 0xbf1208

read fcs_status 0x000000bf

[video_init] uvcd iq is null, use default.

[video_init] uvcd SNR is null, use default.

IQ:FW size (98342)

sensor:date 2026/6/29 version:RTL8735B_VOE_1.7.1.0

sensor:FW size (5472)

sensor timestamp: 2026/06/29

iq timestamp: 2023/05/15 14:48:54

voe_heap malloc 0x707d9f00, size 21994496

ISP:1 ENC:1 H265:1 NN:1

hal_voe_ready 0x0 0xbf1208

voe :RTL8735B_VOE_1.7.1.0

sensor:RTL8735B_VOE_1.7.1.0

hal :RTL8735B_VOE_1.7.1.0

load time sensor:69us iq:1217us itcm:0us dtcm:0us ddr:0us ddr2:0us

Set H264 default HIGH profile
rc_version RC_v1

[video_pre_init_procedure] START

hal_voe_send2voe too long 38866 cmd 0x00000206 p1 0x00000000 p2 0x00000000

VOE command 0x206 fail ret 0x0

Start MP4 recording (1 files)

STORAGE_INIT

open file (0:/TestRecordingAudioOnly.mp4) len = 5 seconds

STORAGE_INIT → STORAGE_START

Stop MP4 recording

finish output task

hal_video_output_task closed
VOE command 0x207 fail ret 0x0

hal_voe_ready 0x0 0x1718

aac reset

ctx->enc_info.frameLength * 2 = 2048

[Error] Recording did not start or timed out.
[State] Listening…
Mic Level: 409
Mic Level: 339
Mic Level: 394
Mic Level: 417
Mic Level: 320
Mic Level: 341
Mic Level: 423
Mic Level: 419
Mic Level: 533

2

:waving_hand: Thanks for posting!

For documentation, SDK resources, FAQs, and community guidelines, please visit: here

For commercial users, if you would like to have dedicated technical support or to connect with us, please fill in the Private User Form

Happy building with Ameba!

:waving_hand: 感謝您的分享！

如需查閱技術文件、SDK 資源、常見問題及社群指南，請參考: 這裡

若您是商業用戶，希望獲得專屬技術支援或與我們聯繫，請填寫 Private User Form (商業用戶表單)

祝您使用 Ameba 開發順利！

3

Hi @rkuo2000 ,

Thanks for sharing the code and APK. Upon checking, the MP4 recording failure is due to incorrect configuration for mp4 object. For audio-only recording, calling of mp4.setRecordingDataType(STORAGE_AUDIO); is necessary. Please find the attached modified code for reference, the recording error should be resolved if the configurations are correct.

Ollama_sendAudioOnly.zip (3.9 KB)

Thank you.

4

Problem 1. HUB8735-Ultra did not support micLevel
/home/rkuo/Arduino/AMB82-mini/Ollama_sendAudioOnly/Ollama_sendAudioOnly.ino: In function ‘void doListening()’:
/home/rkuo/Arduino/AMB82-mini/Ollama_sendAudioOnly/Ollama_sendAudioOnly.ino:134:32: error: ‘class Audio’ has no member named ‘micLevel’
134 | int currentVol = audio.micLevel();
| ^~~~~~~~
exit status 1

Compilation error: ‘class Audio’ has no member named ‘micLevel’

Problem 2. AMB82-mini receive Error from gemma4
[Error] Ollama HTTP status: 400
{“error”:{“message”:“Malformed JSON or media data”}}

(the working python code is ollama_audio.py for your reference)

See below console messages:
== Rtl8735b IoT Platform ==

[test mode PG]
Download Image over UART1[tx=4,rx=3] baud=115200

== Rtl8735b IoT Platform ==

[Normal mode]
BootFromNORFlash
[Start Boot ROM…]
=== Load PARTBL ===
=== Load Done ===
=== Load ISP_IQ ===
[fcs chk pass]
ISP_IQ @ 0x8461080, 0x2ff80, 0x0
mfcs_data version 0x00010001
fcs_data version 0x00010101
=== Process ISP_IQ ===
=== Load Done ===
=== Load BL ===
[Image Start Table @ 0x18200]
=== Load Done ===

== Boot Loader ==
Sep 14 2026:19:14:06
=== Load FCS Para ===
=== Load Done ===
[crc pass]
=== Load ISP_IQ Sensor ===
ISP_IQ @ 0x8461080, 0x2ff80
=== Process ISP_IQ ===
=== Load Done ===
=== Load FW1 ===
FW_ISP_IQ @ 0x8061080, 0x31f80
=== Process FW_ISP_IQ ===
DRAM_TYPE is DDR2 128MB.
ddr_freq = 533
VOE flash @ 0x8093080, 0x82f80
FCS KM_status 0x00002081 err 0x0000200a
Wait KM fcs done 0 us
FCS TM_status 0x003f0000
store fcs data for application
It don’t do the sensor initial process
RAM TM_STATUS 0x00bf1208 err 0x00001208
read fcs_status 0x000000bf
read fcs_status 0x000000bf
=== Process VOE IMG ===
[Image Start Table @ 0x20106200]
RAM Load @ 0x8116100->0x20106200, 0x5a6c
DDR Load @ 0x811c080->0x70100000, 0x234a2a
=== FW Load Done ===

Boot Loader <==

== RAM Start ==
Build @ 14:56:20, Sep 22 2026
set ASP print off

$8735b>set ASP print off

Attempting to connect to SSID: HITRON-DF90-2.4G
interface 0 is initialized

interface 1 is initialized

cfg_size_lib = 124, cfg_size_user = 124

Initializing WIFI …[Driver]: [HALMAC]

11692M HALMAC_MAJOR_VER = 1

HALMAC_PROTOTYPE_VER = 4

HALMAC_MINOR_VER = 20

HALMAC_PATCH_VER =

[Driver]: The driver include MP

[Driver]: Ver = libwlan:2026.09.22.14.53_bmaster_78146548989f20fdbd7286451d67429c855d2a94

RFE type = 0

start_addr=(0x4000), end_addr=(0x8000), buffer_size=(0x4000), smp_number_max=(2048)

WIFI initialized

[Driver]: set ssid [HITRON-DF90-2.4G]

[Driver]: rtw_joinbss_cmd MGMT_FRAME_PROTECTION_OPTIONAL no pmf.

[Driver]: Bcn (ie: 348, rsn: 0, wpa: 0) ch: 11, seq: 1462, ts: 2268444057617

[Driver]: start auth to f8:34:5a:ab:df:98

[Driver]: auth success, start assoc

[Driver]: listen_interval: 10, bcn_interval: 100

[Driver]: association success(res=2)

[Driver]: set pairwise key to hw: alg:4(WEP40-1 WEP104-5 TKIP-2 AES-4)

[Driver]: set group key to hw: alg:4(WEP40-1 WEP104-5 TKIP-2 AES-4) keyid:1

(0) Scan: 1, Auth: 1, Assoc: 1, 4way: 1, connect: 1, reason: 0

Interface 0 IP address : 192.168.0.19

[LwIP_DHCP] dhcp offered_t0_lease: 604800[Driver]: TSFValue = 2268445081617, tsf = 0, shift_set= 0x8000, bcn int = 100

Register disk driver to Fatfs.

FATFS Register: disk driver 0

SD_Init 0

part_count = 0

Audio begin
[State] Listening…
Mic Level: 315
Mic Level: 428
Mic Level: 473
Mic Level: 14426
Mic Level: 14426

TRIGGERED <<<
[State] Listening Stopped.
[State] Recording Setup
ctx->enc_info.frameLength * 2 = 2048

Start MP4 recording (1 files)

STORAGE_INIT

open file (0:/TestRecordingAudioOnly.mp4) len = 5 seconds

STORAGE_INIT → STORAGE_START

STORAGE_START 4

INFO: No udta box callback defined. Skipping udta box.

STORAGE_STOP

video_len = 0 audio_len = 40 moov_len = 1661 total_size = 20201

STORAGE_STOP → STORAGE_IDLE

aac reset

ctx->enc_info.frameLength * 2 = 2048

[State] Recording Done
[State] Sending audio to Ollama…
SD_DeInit

Connect to 192.168.0.13

[INFO] Connect to Server successfully!
[INFO] Audio sent; waiting for transcription…
[Error] Ollama HTTP status: 400
{“error”:{“message”:“Malformed JSON or media data”}}
[State] Returning to Listen Mode
[State] Listening…
Mic Level: 325
Mic Level: 429

5

Hi @rkuo2000 ,

Problem 1: HUB8735-Ultra did not support micLevel

As the Arduino SDK for HUB8735 is outdated, you may try to apply this patch for the updated API. If you still encounter error when running example, you might need to request for version sync between the HUB8735 SDK and ameba-arduino-pro2.

Problem 2. AMB82-mini receive Error from gemma4

Issue : 400 Malformed JSON or media data

The request body was built by serializing a full ArduinoJson document, then hand-trimming the last 4 bytes with fixed byte arithmetic:

prefix.remove(prefix.length() - 4);        // which 4 chars? WRONG
prefix += ",{\"type\":\"input_audio\",\"input_audio\":{\"data\":\"";

serializeJson ends with 6 terminal chars ("}]}]}), so remove(4) chopped inside the "text" string’s closing quote, splicing the audio object into the value:

"text":"Transcribe the audio,{"type":"input_audio","input_audio":{"data":"...

→ invalid JSON → the Gallery server correctly rejected it with 400.

Solution

Build the whole body as one ArduinoJson document — no string arithmetic:

doc["model"] = model;
doc["stream"] = false;
JsonArray messages = doc["messages"].to<JsonArray>();
JsonObject userMsg = messages.add<JsonObject>();
userMsg["role"] = "user";
JsonArray content = userMsg["content"].to<JsonArray>();
JsonObject textPart = content.add<JsonObject>();
textPart["type"] = "text";
textPart["text"] = prompt;
JsonObject audioPart = content.add<JsonObject>();
audioPart["type"] = "input_audio";
JsonObject audioData = audioPart["input_audio"].to<JsonObject>();
audioData["data"] = encodedData;      // base64
audioData["format"] = "mp4";
serializeJson(doc, requestBody);       // valid JSON, real length()
// HTTP Content-Length = requestBody.length(), sent via writeAll()

Verified valid request:

{"model":"gemma4:e4b","stream":false,"messages":[{"role":"user","content":[
  {"type":"text","text":"Transcribe the audio"},
  {"type":"input_audio","input_audio":{"data":"<base64>","format":"mp4"}}]}]}

Issue — 2nd recording fails

SD card was being unmounted between cycles

  • The upload code called fs.end() after reading, and in AmebaFatFS.cpp:
void AmebaFatFS::end(void) {
    fatfs_sd_close();
    memset(&fatfs_sd, 0, sizeof(fatfs_sd_params_t));
    fatfs_sd.drv_num = -1;
    sd_gpio_deinit();      // ← de-inits the SD host GPIO
}
  • After the first upload, the SD host was powered down. The next mp4.begin() → f_open("0:/Clip2.mp4") failed because the disk driver was no longer registered, and the muxer logged Can't open the file.

Solution

Never unmount the SD card in the sketch. Remove every fs.end() call on the upload path, so the disk stays mounted for the whole sketch lifetime:

  • fs.begin() runs once (first send / whenever it’s missing).
  • Every error path just return — the next trigger’s mp4.begin() always finds the disk registered.
  • Uploaded clips are deleted after a successful transcription, but the disk stays mounted.

Please find the following attached fixed sample code for reference. Thank you.

Ollama_sendAudio_Fixed.zip (5.5 KB)

6

@KevinKL ,
Ollama_sendAudio_Fixed.zip works for AMB82-mini
but HUB8735-Ultra failed, audioLevel not triggering.

Due to AMB82-Mini dimension did not fit in OpenDuck-Mini form factor,
(also some small robot projects will need to use HUB8735-Ultra to fit in)
I still need HUB8735-Ultra working with gemma4:e2b.

7

Hi @rkuo2000 ,

Thanks for verifying. Would you mind providing the logs for HUB8735B-Ultra?

8

the patch AudioStream.h AudioStream.cpp can not directly replace HUB8735-Ultra’s code,
because their audio are different.

OTA also incompatible
/home/rkuo/.arduino15/packages/ideasHatch/hardware/AmebaPro2/4.1.1-Release/libraries/OTA/src/ota_drv.c: In function ‘http_update_ota_task’:
/home/rkuo/.arduino15/packages/ideasHatch/hardware/AmebaPro2/4.1.1-Release/libraries/OTA/src/ota_drv.c:34:15: warning: implicit declaration of function ‘https_update_ota’; did you mean ‘http_update_ota’? [-Wimplicit-function-declaration]
34 | ret = https_update_ota((char *)_server, _port, (char *)resource);
| ^~~~~~~~~~~~~~~~
| http_update_ota

Interface 0 IP address : 192.168.0.15
Register disk driver to Fatfs.

FATFS Register: disk driver 0
SD_Init 0
part_count = 0

Audio begin (STORAGE_AUDIO, trigger recording)

[State] Listening…
Mic Level: 36
Mic Level: 38
Mic Level: 40
Mic Level: 33
Mic Level: 39
Mic Level: 40

9

Hi @rkuo2000 ,

Would you mind replacing the pre-compiled libraries with the latest version from ameba-arduino-pro2/Arduino_package/hardware/variants/common_libs at dev · Ameba-AIoT/ameba-arduino-pro2 · GitHub ?

Kindly patch according to the required changes, ensure that the pins and board specific configurations remain unchanged.

Thank you.