Inquiry regarding MobileNetV2 deployment workflow

Arduino AI
1

Hi Team,

I am currently working on deploying a MobileNetV2 model to the Pegasus NPU platform. I have prepared the .pt weight file and a conversion script. I would like to verify if my workflow is correct, especially when compared to the standard procedure used for YOLOv7-tiny.

My current workflow:

  1. PyTorch to ONNX: Use the convert2onnx.py script (see below) to export mobilenetv2.onnx.

    from mobilefacenet import MobileFaceNet
import torch
import time

if __name__ == '__main__':
    filename = 'weights/mobilefacenet.pt'
    print('loading {}...'.format(filename))
    start = time.time()
    model = MobileFaceNet()
    model.load_state_dict(torch.load(filename, map_location=torch.device('cpu')))
    print('elapsed {} sec'.format(time.time() - start))
    print(model)

    output_onnx = 'weights/MobileFaceNet.onnx'
    print("==> Exporting model to ONNX format at '{}'".format(output_onnx))
    input_names = ["input0"]
    output_names = ["output0"]
    inputs = torch.randn(1, 3, 112, 112)

    torch_out = torch.onnx._export(model, inputs, output_onnx, export_params=True, verbose=False,
                                   input_names=input_names, output_names=output_names, opset_version=10)

  2. Pegasus Import: Feed the generated ONNX file into the pegasus_import.sh script.

  3. Post-processing: Proceed with quantization and model compilation.

Questions:

  1. Does MobileFaceNet require a reparam (re-parameterization) step similar to YOLOv7-tiny before conversion? I would like to confirm this with your team.

  2. After generating the ONNX file for MobileNetV2 via convert2onnx.py, is the subsequent execution flow (importing to the conversion tools) identical to the workflow used for YOLOv7-tiny?

2

hi @t114c75035 ,

May I know if you are using MobileNetV2 or MobileFaceNet for your deployment?

Thank you.

3

Hi,

​For our current deployment, we are utilizing MobileNetV2.

​Thank you.

4

Hi @t114c75035 ,

Thanks for clarifying. For MobileNetV2 conversion, the overall execution flow is similar to YOLOv7-tiny, however, kindly refer to the mean and scale values below for the inputmeta.yml file changes.

Reparameterization step is not required for MobileNetV2 before conversion.

image
image831×648 10.5 KB

Thank you.

5

Hi @t114c75035 ,

To add on, please also use int16 for the quantization type for better results.

Thanks.

6

Sorry to bother you. While trying to use the converted .nb model with the “RTSPImageClassification” example, I encountered the following compilation error:

C:\Users\iron0\OneDrive\Documents\Codes\LAB\RTSPImageClassification\RTSPImageClassification.ino: In function 'void setup()':
C:\Users\iron0\OneDrive\Documents\Codes\LAB\RTSPImageClassification\RTSPImageClassification.ino:90:88: error: 'CUSTOMIZED_IMGCLASS_MOBILENETV2' was not declared in this scope
   90 |     imgclass.modelSelect(IMAGE_CLASSIFICATION, NA_MODEL, NA_MODEL, NA_MODEL, NA_MODEL, CUSTOMIZED_IMGCLASS_MOBILENETV2);
      |                                                                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
exit status 1

Compilation error: 'CUSTOMIZED_IMGCLASS_MOBILENETV2' was not declared in this scope

I would like to ask whether the AMB82-MINI currently does not support MobileNetV2.

Additionally, I understand from your suggestion that using int16 for the quantization type provides better results. However, I am currently using uint8 and would like to first confirm whether the model can run correctly with this setting before switching to int16. Would using uint8 still work for initial testing?

7

Hi @t114c75035 ,

MobileNetV2 is supported on AMB82-mini, but currently there is an issue to load the customized model from flash, would you mind to try loading your model from SD card while we try to resolve the issue as soon as possible?

–> Yes, you can still test with uint8 model

To load a customized mobilenetv2 from SD card,

  1. create a folder named NN_MDL
  2. copy your customized model into the folder
  3. rename your model to “mobilenetv2_int16.nb” (regardless of the quantization type)
  4. in your .ino file, under modelSelect function, change from “DEFAULT” to “CUSTOMIZED_IMGCLASS_MOBILENETV2”
  5. navigate to “Tools” → “NN Model Load from:” and select SD card as the option
  6. Compile and upload the code

Kindly also update ClassList and set USE_MODEL_META_DATA_EN to 0 if your model does not embed metadata.

Thank you.

8

Hi,

The issue has been resolved successfully. Thank you for your support!

I also wanted to ask if there is a version available for RTOS usage, or if there are any related resources or recommendations you could share.

Thanks again!

9

Hi @t114c75035 ,

You may refer to the image classification example in our FreeRTOS SDK, ameba-rtos-pro2/project/realtek_amebapro2_v0_example/src/mmfv2_video_example/mmf2_video_example_vipnn_classify_rtsp_init.c at main · Ameba-AIoT/ameba-rtos-pro2 · GitHub

Here is the guide to load model from SD card: NN Deployment — AmebaPro2's Documentation v0.1 documentation

Thank you.