We introduced Drone movement in the last post, we are going to try the video capture. It is also very simple with the help of DJITELLOPY and CV2 API. With the basic movement control and video capture, we can start face detection and face tracking very soon. Let’s show you how to capture video from the Drone Tello.
CV2, stands for OpenCV (Open Source Computer Vision Library), which is a popular open-source computer vision and machine learning software library. It provides a wide range of functions and tools for various computer vision tasks, image and video processing, machine learning, and more. OpenCV is widely used in the fields of computer vision, robotics, image processing, and artificial intelligence.
As we mentioned in the previous post, CV2 already installed when we installing the DJITELLOPY 2.50 package. We just need to import the CV2 library when start the program.
As we need to capture and display the video from drone by the same time it is flying, we need parallel progressing. THREADING is basic and common used Python.
How’s the program working
Video Stream from Tello
With DJITELLOPY to achieve the video stream is very simple, we use streamon() to instruct Tello to enable the video stream and get_frame_read() to read the existing video frame.
However, there may be a delay after calling streamon() in the DJITELLOPY library. When we call streamon(), we are initiating the video streaming from the Tello drone’s camera. The drone needs some time to establish the streaming connection and start sending video frames.
So, we setup a while loop to ensure the camera is ready and the first frame is being read before we proceed to the next step.
The get_frame_read() method in the djitellopy library returns a VideoFrame object that provides access to the current video frame from the Tello drone’s camera. Apart from the frame attribute, which contains the video frame data as a NumPy array, the VideoFrame object has other attributes that provide information about the frame. These attributes include:
- frame: The actual video frame data as a NumPy array.
- time: The timestamp of the frame in milliseconds.
- frame_number: The sequential number of the frame.
- h: The height of the video frame.
- w: The width of the video frame.
- channel: The number of color channels in the frame (usually 3 for RGB).
Video Capture and display
We define a function to read the frame from the drone and show as image in a separated window by CV2. There are two highlights,
- The frame we capture from Tello is in RGB colors but cv2 processing image as ‘BGR’ order. If we show the image directly, it will result as something bluish (Smurfs?). We need to convert this from RGB to BGR before we can show this properly.
- cv2.waitKey() must be excuted after the cv2.imshow(), otherwise, the image will not be displayed.
Since we need to capture the frame and display during the drone flying, we need parallel processing for capture_video().
We started the video capture thread just before the drone takeoff, so that we can see the video window when the drone take off and flying.
Again, that’s cool and easy, we just add few more lines to the last program and make it fly with video capturing. We believe that we can start doing face detection and tracking now.