Started work on complete camera security system

[borud]

Hi,

I’m new here so please bear with me. I am not sure if this is the correct place to post this, but I thought that this might be interesting to anyone who uses the AMB82-MINI cameras.

I’ve started building a surveillance camera solution that includes

• Firmware for the camera with UI for configuration
• A server that can record video and play it back
• A new firmware flasher written in Go
  • a bit faster than the one that came with the SDK
  • comes with its own loader mini-app
  • also ported to JS so you can flash AMB82 boards from the browser
  • doesn’t require boot+reset buttons to be pressed on flashing
• Firmware management on the server and OTA updates for cameras
• Server can configure WiFi networks and credentials when flashing boards the first time

I haven’t started playing with the neural network bits yet, but those are next. I plan to split processing between camera and server so that the camera can do simple object/people recognition and the server can take care of heavier things like matching faces to embeddings etc.

Here are some screenshots:

This is the live-view

This is the replay view

Here is part of the config view for configuring individual cameras (the add camera button is somewhat superfluous as adding cameras is automatic when you flash them. They get the server address and the network credentials during the provisioning process)

Here is the firmware settings tab:

And here is the screen for flashing the AMB82 MINI from the browser, provisioning it with credentials and telling it where the server is

There is also an on-camera UI that can be used to configure the camera, but I had to skip the screenshot because there is a 5 image limit here.

I’m fairly new to this board so there are probably still some things that need ironing out. Also, as mentioned previously, all the neural network stuff is next. Plus additional integration interfaces and optimization of the video streaming (I have tested it for just 2 cameras and 5-6 consumers right now, but I suspect there is room for improvement).

Of course, I also have to add authentication, access control and I’ll have a look at whether the video streams from the camera to the server could perhaps be encrypted. I have no idea how this affects performance.

The entire thing can be shipped as a single binary. So you start the binary, it creates an SQLite database for its data, and stores video in the filesystem. Then you connect your cameras via USB the first time, flash and provision them, and after that you can flash them OTA. (You just need to set your server address and add some wifi networks and the cameras just connect automatically to the server).

The plan is to upgrade cameras automatically whenever I create a new firmware release for the AMB82.

I just thought I’d share this project here. Let me know if any of you find it interesting.

-Bjørn

[system]

Thanks for posting!

For documentation, SDK resources, FAQs, and community guidelines, please visit: here

For commercial users, if you would like to have dedicated technical support or to connect with us, please fill in the Private User Form

Happy building with Ameba!

---

感謝您的分享！

如需查閱技術文件、SDK 資源、常見問題及社群指南，請參考: 這裡

若您是商業用戶，希望獲得專屬技術支援或與我們聯繫，請填寫 Private User Form (商業用戶表單)

祝您使用 Ameba 開發順利！

[borud]

I’m considering open sourcing at least the flasher. Which also includes a command for creating the a directory with the HTML+JS for flashing via web browsers’ USB support which can serve as a starting point for making your own software and add browser-based (no tools to install) flashing.

Let me know if anyone is interested.

[boltsandbytes]

@borud

part of the config view for configuring individ

We are working on something similar , why did you not choose onvif ? This would allow easy compatibility with NVR’s / Frigate out there.

The problem we are facing is sourcing the actual hardware , tech is great but getting it other than devkits is not so easy.

[borud]

Simple: I forgot ONVIF existed :-). But I think it would be a good idea to add it both to the camera firmware and the server software.

I am somewhat puzzled why the Realtek chipset doesn’t seem to be more common in commercial products. Did realtek make newer chips that people use instead?

I am building this because I have tried to use various camera surveillance systems and they never quite work, so I want something that works for me and that I can change. I wrote it in Go and I like single binary systems that don’t have a lot of dependencies. It also uses SQLite as its database to eliminate the need to manage an external database. But I am probably going to add Postgres support.

-Bjørn

[boltsandbytes]

There are newer versions out there like RTL8735C , but not sure how to get those.

I think its high barrier of entry and hard to get silicon so not may open projects around , Realtek might be prioritizing larger setups , cant blame them, the hardware is really great though. In comparison For ex esp32-p4 can be easily bought at a lot of place if you do not heed h265 / NPU .

Your flasher does sound good , should help save time , will check .

[borud]

I’ve been trying to use various ESP32 cameras for the last years but almost none of them are any good since they lack proper video encoding in hardware. I would not try to do anything without video encoding in hardware because you end up blowing a lot of bandwidth on low framerates and not that great quality. I have tried transcoding to H264 on the server, but it tends to become fragile, complex and suck a lot of CPU when you start to stream lots of cameras.

There is a ESP32-P4 that I think does hardware H264 encoding, but I have never tried it. I might take a look at that one day.

The AMB82 MINI boards have turned out to be much better, but I did spend a long time getting all the gremlins out of the firmware. Initially the video feed was a bit choppy, but after optimizing the firmware for a few days I managed to get it really stable. My current firmware version has run for about a week.

This weekend I am doing some more measurements and code optimizations to make it use less CPU. Not so much because I need it to get better quality, but more because I want to have a bit of headroom for experimenting with neural nets on the camera.

The config that has worked well for me is:

• H265 VBR at 20fps
• 4 second keyframe interval
• Opus audio codec at 48khz
• RTSP over UDP

The keyframe interval is really mostly so the video files are “scrub friendly”. I am thinking about having much lower keyframe rate (8 sec) detecting if the frames are damaged/nonsense and use the fact that the management protocol I made for the cameras as support for “generate keyframe now” so it can recover dynamically.

I got a lot of frame dropouts when I used AAC encoding. I don’t know why so I am investigating that. Perhaps the AAC encoder is poorly implemented or it does something I don’t understand. AAC would be more convenient than Opus since Safari doesn’t support Opus but does support AAC. I could perhaps transcode sound in the server but that is a bit of a hassle. I’d like to keep things simple. 48khz also might be a bit too much. I tried to play with the analog gain levels but it just amplifies more noise.

I spent a LOT of time making RTSP over UDP behave. Mostly due to my own mistakes, like having different sampling framerate from video stream framerate. I could have used TCP, but if you have occasional dropouts TCP takes much longer to recover so any network choppiness just becomes horrific.

Below is the current statistics view on my server.

Late “Frame Delivery” used to look pretty horrible when I used AAC. In the plot below you can see how frame delays suddenly got much better around 16:18 when I switched from AAC to Opus. The stream wasn’t unusable, but it has some slight choppiness to it. After I switched to Opus the problems disappeared.

Hardware wishes

One thing that would be really interesting from a hardware perspective was if someone were to manufacture a board that is more suited for putting into real products. So:

• Slightly smaller board without soldered on header pins (solder pads is better)
• mounting holes on the PCB to make it easier to mount
• Camera that has slightly longer flex cable
• microphone connector instead of microphone on the board
• Programming header for pogo pins with UART and DTR and RTS pins for boot and firmware loading mode
• CAD files for board geometry so you can design enclosures around it.
• buck/boost converter and LiPo charging circuit
• proper power interface that can take 5-12V input
• audio output with optional class D amplifier chip that you can choose to populate

I am guessing you want to build something along those lines and this is where you ran into sourcing problems?

Let me know how you get on with this.

Firmware flasher

I’ll see if I can get it released this weekend. If not I’ll try next weekend. I have to check what license I have to release it under and make sure I don’t step on anyone’s toes in terms of licensing. I made a command line tool, a library (Go) and a web based firmware flasher (JS).

From a product point of view I think the web based flashing may be very neat since it makes setting up cameras really easy:

It uses the USB support in Chrome to talk to the camera, which means ordinary users can easily set up their cameras just by plugging them into the USB port. It flashes the firmware and it also sets up WiFi credentials and tells the camera where the server lives (so new cameras automatically turn up in the server interface).

Since I also implemented OTA in the server I am also thinking about using BLE just to set up the WiFi credentials and then do the rest over WiFi (works in a situation where your camera firmware is already on the camera when the user gets it).