Custom YOLOv7-tiny (2 classes) ghost boxes at top-left after Docker conversion — model verified correct pre-conversion (similar to #4667)

[navneet8875580017]

Hi Realtek team,

I’m deploying a custom-trained YOLOv7-tiny (2 classes: animal, person) on AMB82-Mini (RTL8735B), and getting the “ghost boxes clustered at top-left” issue described in thread #4667:

NPU Inference Discrepancy: Local conversion works, but Docker conversion results in "Ghost Boxes" at top-left

I am attempting to package my NPU conversion toolchain (PyTorch to .nb) into a Docker container. Input Model: YOLOv7 .pt file (already reparameterized , already fix Torch 2.6 loading issue in in yolo/models/experimental.py by setting weights_only=False **,**will be exported to onnx within Docker .). Scenario: Local Host Conversion: When I convert the model using my local environment, the generated .nb file works perfectly on the edge device. Docker Conversion: When I perform the exac…

I have systematically isolated the problem to the Acuity .nb conversion step. Everything before conversion is verified working:

VERIFIED WORKING

• Board + default YOLOv7-tiny model: detects correctly (person, 85% confidence). Board, camera, ISP, firmware all fine.
• My reparameterized model (best_reparam.pt) tested in PyTorch with detect.py: detects correctly, boxes on the right objects with correct labels. So training and reparameterization are correct.
• My exported ONNX: structurally correct.
  • Input: images [1, 3, 416, 416]
  • Outputs: [1,21,52,52], [1,21,26,26], [1,21,13,13] (21 = (2 classes + 5) x 3 anchors)
  • Detect.forward() modified per the docs to output raw [bs,255,H,W] layout (sigmoid only, no grid/stride decode).

THE PROBLEM (only appears after conversion)

• Converted .nb (uint8): loads fine (no “nn open fail”), runs, but ALL bounding boxes cluster at the top-left corner regardless of image content. Real objects in the frame are not detected. ~14 phantom detections per frame at duplicated coordinates. This is identical to thread #4667.
• int16 attempt: fails to load on the board with:
  “vip_create_buffer_from_handle fail, buffer size is bigger than handle size. 0xfd800 > 0x7ec00”
  (int16 model is ~9.8MB, appears too large for the NN memory region.)

CONVERSION DETAILS

• Toolkit: official Docker image ghcr.io/ameba-aiot/acuity-toolkit:6.18.8
• Board SDK: Realtek Ameba (AmebaPro2) 4.1.0
• ONNX export environment: PyTorch 2.11.0+cu128, numpy 2.3.5, onnx 1.22.0.
  Note: I had to use torch.onnx.export with dynamo=False (legacy exporter), because torch 2.11’s default dynamo exporter fails on this model.
• Reparam: reparam_yolov7-tiny.py from Yolov7_reparam_scripts, nc set to 2 in yolov7-tiny-deploy.yaml.
• inputmeta: scale 0.00392157, reverse_channel false, mean [0,0,0]
• Quantization: uint8, asymmetric_affine, 10 calibration images from my dataset.
• Export: pegasus export ovxlib, --optimize VIP8000NANONI_PID0XAD, --pack-nbg-unify.

QUESTIONS

1. Since the model is verified correct before conversion and only the .nb shows ghost boxes, what in the conversion/export environment causes this top-left coordinate shift for a 2-class model?
2. Is there a specific PyTorch / ONNX / numpy version combination required for the ONNX export? (Thread #4667’s working environment used numpy 2.2.6 / onnx 1.20.0 for export.)
3. Could the dynamo=False legacy ONNX export be the cause, and if so, what is the recommended export method for newer PyTorch?
4. Thread #4667 was resolved via the Docker toolkit, but I am already using the official Docker toolkit image and still see the issue. Is there an updated image or a known fix?

I can share the ONNX file, the inputmeta yml, or the full conversion logs if helpful. Thank you.

[system]

Thanks for your post!

For documentation, SDK resources, FAQs, and community guidelines, please visit: here

Happy building with Ameba!

---

感谢您的发帖！

如需查阅官方文档、SDK 资源、常见问题（FAQ）及社区使用指南，请参考: 這裏

祝您使用 Ameba 开发愉快！

[KevinKL]

Hi @navneet8875580017 ,

@navneet8875580017

My exported ONNX: structurally correct.

• Input: images [1, 3, 416, 416]
• Outputs: [1,21,52,52], [1,21,26,26], [1,21,13,13] (21 = (2 classes + 5) x 3 anchors)
• Detect.forward() modified per the docs to output raw [bs,255,H,W] layout (sigmoid only, no grid/stride decode).

May we know if you have exported the ONNX model using python export.py --weights weights/yolov7-tiny.pt --simplify --img-size H W

@navneet8875580017

1. Since the model is verified correct before conversion and only the .nb shows ghost boxes, what in the conversion/export environment causes this top-left coordinate shift for a 2-class model?
2. Is there a specific PyTorch / ONNX / numpy version combination required for the ONNX export? (Thread #4667’s working environment used numpy 2.2.6 / onnx 1.20.0 for export.)
3. Could the dynamo=False legacy ONNX export be the cause, and if so, what is the recommended export method for newer PyTorch?
4. Thread #4667 was resolved via the Docker toolkit, but I am already using the official Docker toolkit image and still see the issue. Is there an updated image or a known fix?

1. Could you verify that NNWIDTH and NNHEIGHT setting in SDK matches the input size of your model?
2. There is no official specification for ONNX export.
3. We would recommend you to follow the requirements from yolov7/requirements.txt at main · WongKinYiu/yolov7 · GitHub
4. No, ghcr.io/ameba-aiot/acuity-toolkit:6.18.8 is the latest released version.

Sure, if you are still having issues on model deployment, kindly share with us the files and logs as mentioned.

Thank you.

[navneet8875580017]

Hi @KevinKL, thank you for the guidance.

1. NNWIDTH / NNHEIGHT: confirmed both are 416 in the sketch, matching the model input size (416x416).

2/3. Understood regarding the ONNX export. My export environment was newer than yolov7/requirements.txt — I used PyTorch 2.11 + numpy 2.3.5 with torch.onnx.export(dynamo=False), because my GPU (RTX 5060 Ti / Blackwell) requires newer PyTorch to run. I will re-export the ONNX in a separate CPU-only environment that matches yolov7/requirements.txt (older torch / numpy / onnx), then re-convert and report back.

4. Understood that 6.18.8 is the latest toolkit.

Sharing my files for reference:

ONNX model (too large for the 10 MB forum limit, shared via Google Drive):

yolov7-tiny-onnx.zip

inputmeta.yml:

input_meta:
databases:

• path: dataset.txt
  type: TEXT
  ports:
  • lid: images_141
    category: image
    dtype: float32
    sparse: false
    tensor_name:
    layout: nchw
    shape:
    • 1
    • 3
    • 416
    • 416
      fitting: scale
      preprocess:
      reverse_channel: false
      mean:
      • 0
      • 0
      • 0
        scale: 0.00392157

My export command:
python export.py --weights best_reparam.pt --simplify --img-size 416 416
(with dynamo=False added to the torch.onnx.export call in export.py, required for torch 2.11)

I will report back after re-exporting with the older versions. Thank

[navneet8875580017]

Hi @KevinKL,

I re-exported the ONNX in a clean environment matching yolov7/requirements.txt (torch 2.2.0, numpy 1.26.4, onnx 1.22, no dynamo). I compared it against my original torch 2.11 export:

• Both: opset 12, 138 nodes, identical op types (58 Conv, 55 LeakyReLU, 14 Concat, 6 MaxPool, 3 Sigmoid, 2 Resize)
• Both: outputs [1,21,52,52], [1,21,26,26], [1,21,13,13]

The two ONNX files are operationally identical, so the export environment/torch version does not appear to be the cause of the ghost boxes.

Since the reparameterized model detects correctly in PyTorch, and the ONNX is verified correct, the issue appears to be in the Acuity conversion (import/quantize/export) itself. Could you please try converting my ONNX (shared above) on your end and check whether the resulting .nb produces the same top-left ghost boxes? That would confirm whether it’s the conversion step. Thank you.

[KevinKL]

Hi @navneet8875580017 ,

Thank you for your patience.

Thanks for providing your .onnx model and I have converted it into .nb using Ameba Offline Model Conversion Docker toolkit. Upon testing, I did not observe any ghost boxes and the model seems to work pretty well on recognising a person.

Could you verify that your inputmeta file is configured exactly as follow? Reference: Inputmeta File Modification — AmebaPro2's Documentation v0.1 documentation

input_meta:
  databases:
  - path: dataset.txt
    type: TEXT
    ports:
    - lid: images_141
      category: image
      dtype: float32
      sparse: false
      tensor_name:
      layout: nchw
      shape:
      - 1
      - 3
      - 416
      - 416
      fitting: scale
      preprocess:
        reverse_channel: false
        mean:
        - 0
        - 0
        - 0
        scale: 0.003921568627451
        preproc_node_params:
          add_preproc_node: true
          preproc_type: IMAGE_RGB888_PLANAR
          preproc_image_size:
          - 416
          - 416
          preproc_crop:
            enable_preproc_crop: false
            crop_rect:
            - 0
            - 0
            - 416
            - 416
          preproc_perm:
          - 0
          - 1
          - 2
          - 3
      redirect_to_output: false

Lastly, please find the attached converted model for verification on your end.

Thank you.

network_binary.zip (3.7 MB)