AMB82 mini classification

我在嘗試用自製模型寫進板子裡面的時候發生以下錯誤
error: error: vip_create_butter_from_handle fail

以下是我的模型程式碼:
import numpy as np
import tensorflow as tf
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Conv2D, MaxPooling2D, Flatten, Input
from tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau
from tensorflow.keras.preprocessing.image import ImageDataGenerator

print(tf.version)

設定 np 亂數種子

np.random.seed(10)

載入訓練資料集

n = 2000
img_feature = np.fromfile(“array.features”, dtype=np.uint8)
img_feature = img_feature.reshape(n, 30, 30, 3)
img_label = np.fromfile(“array.labels”, dtype=np.uint8)
img_label = img_label.reshape(n, 1)

打散資料集

indexs = np.random.permutation(img_label.shape[0])
rand_img_feature = img_feature[indexs]
rand_img_label = img_label[indexs]

資料正規化

img_feature_normalized = rand_img_feature.astype(‘float32’) / 255

將 label 轉換為 onehot 表示

img_label_onehot = to_categorical(rand_img_label)

分割訓練和驗證資料

split_index = int(0.8 * n)
train_features = img_feature_normalized[:split_index]
train_labels = img_label_onehot[:split_index]
validation_features = img_feature_normalized[split_index:]
validation_labels = img_label_onehot[split_index:]

定義 EarlyStopping 和 Learning Rate Scheduler

early_stopping = EarlyStopping(monitor=‘val_loss’, patience=5, restore_best_weights=True)
lr_scheduler = ReduceLROnPlateau(monitor=‘val_loss’, factor=0.5, patience=3, min_lr=1e-6)

定義資料增強器

datagen = ImageDataGenerator(
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
horizontal_flip=True,
fill_mode=‘nearest’
)

datagen.fit(train_features)

建立一個線性堆疊模型

model = Sequential([
Input(shape=(30, 30, 3)),
Conv2D(filters=32, kernel_size=(3, 3), activation=‘relu’, padding=‘same’),
Dropout(rate=0.25),
MaxPooling2D(pool_size=(2, 2)),
Conv2D(filters=64, kernel_size=(3, 3), activation=‘relu’, padding=‘same’),
Dropout(rate=0.25),
MaxPooling2D(pool_size=(2, 2)),
Flatten(),
Dropout(rate=0.25),
Dense(1920, activation=‘relu’),
Dropout(rate=0.25),
Dense(2, activation=‘softmax’)
])

定義訓練方式

model.compile(loss=‘categorical_crossentropy’, # 損失函數
optimizer=‘adam’, # 最佳化方法
metrics=[‘accuracy’]) # 評估方式:準確度

顯示模型摘要

print(model.summary())

開始訓練模型

train_history = model.fit(
datagen.flow(train_features, train_labels, batch_size=50),
validation_data=(validation_features, validation_labels),
epochs=30, # 增加 epochs 數量
verbose=2,
callbacks=[early_stopping, lr_scheduler] # 加入 early stopping 和 learning rate scheduler
)

儲存模型

model.save(“./your/image/training/models.keras”)

model.save(“./your/image/training/models.h5”, include_optimizer=False)

轉換完成之後我用了範例程式RTSPImageClassification修改成客製化模型(CNN)
模型檔名我有改寫成img_class_cnn.nb這是從
~/Arduino15/packages/realtek/hardware/AmebaPro2/4.0.7/variants/common_nn_models/amebapro2_fwfs_nn_models.json
找到關於模型名稱設定的檔案修改完成的
所以確定有把模型成功寫入

錯誤代碼如下:
[video_pre_init_procedure] START

[VOE]ext_in = 0 sync = 0

[VOE][Ini set0]init dn 0 hdr 65535 mirrorflip 0xffff

[VOE]md init success

[VOE]algo ver 87e2381

[VOE]pack_v 0x0000 0x0002 cus_v 0 iq_id 0 dn 0 day 0 night 1 other 2 offset 224 length 16436 iq_size 32716

[VOE]Ver 0x0001000c Fast3A Cnt AE 1 AWB 1 period AE 5 AWB 1 delay 0 0

[VOE][isp_mod_sensor_init][1624]Errhdr_mode 0 is over sensor driver num 1

[VOE]hdr_mode 0 sensor driver num 1

[VOE]fps: 279825, pclk: 0, hts: 1077805056

[VOE]fps max 30.000000 min 5.000000

[VOE]exposure_step 29.629629

[VOE]change sensor mode => 1920x1080@30.000000fps - ‘linear’

[VOE]min_fps 5.000000, max_fps 30.000000, exp_step 29.629629

[VOE]md ver 0x6d640100

[VOE]ae ver 0x61650200

[VOE]awb ver 0x77620100

[VOE]cur_hdr_mode = 0
[VOE]VOE MEM Size = 20329 Used= 3552 KB Free=16777 KB (17179872)

[VOE]stream 0 buffer 0: 0x7079d500 size 3110400

[VOE]stream 0 buffer 1: 0x70a94c00 size 3110400

[VOE]first_config_osd2_block_num[0]: 1

[VOE]osd2_block_num[0]: 24

[VOE]NV12 1920x1080 1/30

[VOE]dynamic set fps 0 → 30 ok

[VOE]sensor power on
[VOE]zoom crop default setting

[VOE]status == 1718

[VOE]buffer size == (0x70dcbf60) (2356352 259200) queue(579)

[VOE]Set rate control: bpsqp -1 qpRange I[25, 48] PB[25, 48] 1048576 bps pic 1 skip 0 hrd 0 cpbSize 1000000 bitrateWindow 30 intraQpDelta -5 fixedIntraQp 0

[VOE]Set PreP: input 1920x1080 : offset 0x0 : format 1 : rotation 0cc 0 : scaling 0 : scaling format 0

[VOE]vcenc_set_ref_pic_set() NULL
hal_voe_send2voe too long 168344 cmd 0x00000206 p1 0x00000000 p2 0x00000000

inputRateNumer[10] < outputRateNumer[30], Set inputRateNumer → outputRateNumer

[VOE]ext_in = 0 sync = 0

[VOE]isp_device_probe had done
[VOE]isp_soc_start already done
[VOE]VOE MEM Size = 20329 Used= 18701 KB Free= 1628 KB (1667296)

[VOE]stream 4 buffer 0: 0x71668900 size 76032

[VOE]stream 4 buffer 1: 0x7167b300 size 76032

[VOE]RGB3 176x144 1/10

[VOE]status == 1718

[VOE]release s4 isp buffer 0

[VOE][WARN]useless release s4 slot0 status 0x00000000

[VOE]release s4 isp buffer 1

[VOE][WARN]useless release s4 slot1 status 0x00000000

error: error: vip_create_buffer_from_handle fail.

error: error: vip_create_buffer_from_handle fail.

error: error: vip_create_buffer_from_handle fail.

論壇上也沒找到有關於error: error: vip_create_butter_from_handle fail這段錯誤代碼的相關訊息
所以上來發問

Hi @Po8F
Here are some possible solutions:

  1. Check Model Format: Ensure the model format is correct and compatible with the hardware.
  2. Update Firmware and Software: Make sure all firmware and software are up to date.
  3. Check Memory: Verify that the hardware has sufficient memory to load the model.
  4. Seek Technical Support: If the above methods do not work, contact official technical support for assistance.