NPU Inference Discrepancy: Local conversion works, but Docker conversion results in "Ghost Boxes" at top-left

Arduino AI
1

I am attempting to package my NPU conversion toolchain (PyTorch to .nb) into a Docker container.

  • Input Model: YOLOv7 .pt file (already reparameterized , already fix Torch 2.6 loading issue in in yolo/models/experimental.py by setting weights_only=False **,**will be exported to onnx within Docker .).

  • Scenario:

    • Local Host Conversion: When I convert the model using my local environment, the generated .nb file works perfectly on the edge device.

    • Docker Conversion: When I perform the exact same conversion steps inside the Docker container, the generated .nb file behaves abnormally on the edge device.

  • Abnormal Behavior on NPU: Using the ObjDetectionLoop example:

    • The bounding boxes are incorrectly clustered at the top-left corner of the screen.

    • Even if there is a real object (e.g., a person/mask) in the center of the frame, it is NOT detected.

    • Instead, multiple ghost boxes (for both classes, e.g., no_mask, mask) appear at the top-left corner regardless of the input image content.

Here is the comparison between the Working Local Environment and the Failing Docker Environment.

1. Local Environment

OS / Machine

Linux (22.04)

1.1 Local – YOLO Environment python 3.13
PyTorch: 2.8.0

TorchVision: 0.23.0

TorchAudio: 2.8.0

NumPy: 2.2.6

SciPy: 1.14.1

ONNX: 1.20.0

OpenCV: 4.10.0

Ultralytics: 8.3.235

THOP: 0.1.1

Roboflow: 1.2.6

LabelImg: 1.8.6

Matplotlib / Seaborn / Pandas

1.2 Local – Tool Environment (Acuity / Conversion Tool)
Acuity Toolkit: 6.18.8

TensorFlow: 2.10.0

TensorBoard: 2.10.1

Keras: 2.10.0

NumPy: 1.21.6

SciPy: 1.10.1

ONNX: 1.12.0

ONNX Simplifier: 0.4.36

ONNX Optimizer: 0.3.1

PyTorch: 1.13.1+cpu

TorchVision: 0.14.1+cpu

libclang: 18.1.1

LMDB: 0.93

2.Docker Environment

2.1 Docker – YOLO Environment
PyTorch: 2.8.0

TorchVision: 0.23.0

TorchAudio: 2.8.0

NumPy: 1.23.5

SciPy: 1.15.3

ONNX: 1.20.0

ONNX GraphSurgeon: 0.5.8

ONNX Simplifier: 0.4.36

OpenCV: 4.11.0

2.2 Docker – Tool Environment
Acuity Toolkit: 6.18.8

TensorFlow: 2.10.0

TensorBoard: 2.10.1

Keras: 2.10.0

NumPy: 1.21.6

SciPy: 1.10.1

ONNX: 1.12.0

ONNX Runtime: 1.12.0

ONNX Simplifier: 0.4.36

ONNX Optimizer: 0.3.1

PyTorch: 1.13.1

LMDB: 0.93

libclang: 18.1.1

Since the Acuity/Toolchain environment (Section 2) is nearly identical between Local and Docker, the issue seems to stem from the Export Environment (Section 1). Could you advise on how to align the Docker environment or if there are specific PyTorch/ONNX versions required to avoid this “top-left corner” coordinate shift on the NPU?

2

Hi @t114c75035 ,

May I know if there is a need to perform conversion on a Docker environment? If yes, would you mind to tryout our Docker toolkit image to verify if this issue still persists? You may refer to the installation guide here. Remember to login to docker using your GitHub username and PAT before pulling the image.

Thank you.

3

The issue is resolved. Docker is working perfectly now. Thanks for the help!

1 Like