Why Are Cameras Still So Passive?
Recently, I purchased a reCamera Gimbal from seeed studio. It is a gimbal camera. This camera is quite interesting. It can control the rotation of the gimbal on their web ui and also perform human body tracking. After my experience for some time, I found that this camera is still passive. For instance, if I want him to help me detect the need for someone to notify me, I need to do the secondary development myself.What if a camera could take instructions like an intelligent assistant?
Meet the AI Agent Camera
So I implanted an Agent in Gimbal, which can perform the following operations:
- Understand Natural Language
Simply describe what you want in plain language, and the agent translates your intent into actions without requiring predefined commands or complex configurations.
-
Control Physical HardwareThis Agent can directly interact with the onboard hardware. The following demonstrates the control of the LED lights.
-
Generate and Execute Code
Of course, you can also have the Agent help you write the script to realize your idea. I have asked it to write a script for an LED light to flash for ten seconds and provided an example of its operation.
- Plan Multi-Step Tasks You can also have the Agent help me carry out a series of tasks, such as: looking around, and taking a photo and sending it to me when it detects a person.
From Passive Camera to AI Agent In the past, those cameras could only execute simple commands. For instance, if you said to rotate 30 degrees to the left, they would indeed rotate by 30 degrees. However, if you said something like "look around", they couldn't understand it. This is why I call it AI Agent Camera.
Giving the Camera Tools
First, if you want the Agent to write its own code to control the local hardware, then you need to let the Agent know which hardware on the camera CAN be used and how to use it. This requires the use of "skills". You can first write a simple md document to describe the on-board hardware configuration and the CAN protocol of the pan-tilt. Here, I directly found the corresponding pdf document and wiki introduction from the seeed studio official website. By feeding these directly to the ai, the corresponding skill description document can be generated.Among the skills listed below, there are some that are related to my Gimbal.The part starting with "reCamera" is the skill I wrote for it myself.
Planning and Executing Tasks
It is not merely a tool that you say one thing and it does another. Instead, it can understand your needs and perform certain tasks at scheduled times.
For instance, you could ask him to take a photo every hour and send it to you. Or you could even instruct him to turn on the supplementary lighting if anyone is present during the photo-taking process.
[Read more](javascript:void(0))