Researchers at Florida Atlantic University (FAU) trained an AI model to find beluga whales in drone footage from the Canadian Arctic using just 34 hand-labeled images. The study ran in Frontiers in Marine Science on September 4, and FAU announced it on November 19. The team worked with Fisheries and Oceans Canada on 7,655 frames pulled from 4K video that DJI Phantom 4, Mini, Mavic 3E and Matrice drones shot in 2022 and 2023.
The method is called semi-supervised learning. Biologists label a small set of images, and the model then teaches itself on thousands more that nobody has marked up. The authors say it gets comparable results with five to ten times less labeling.
Calves are the hardest whales to spot from the air, and biologists need them counted because they show whether a population is reproducing. With 34 labeled images, the model found 27.8% of them.
SEMI-DETR Beat YOLO11 With 1% Of The Labeled Images #
SEMI-DETR, the semi-supervised model, scored 19.2 mean average precision on a 100-point scale with 34 labeled images, against 16.0 for YOLO11s, the best of four conventionally trained YOLO11 models. The gap peaked at 6.6 points with 171 labeled images, 33.6 against 27.0.
Those 34 images are 1% of a pool of 3,417 that marine mammal biologists labeled by hand, according to the paper. Another 3,383 images went to the model with no labels at all, and 855 frames from videos it never saw were held back for testing. The full set holds 27,118 marked whales, 20,685 adults and 6,433 calves, with as many as 31 in one frame.
The low overall scores hide where the gain came from. With 34 labeled images, YOLO11s found 1.3% of the calves in the test set and SEMI-DETR found 27.8%. For adults the two were close, at 60.2% and 63.6%.
Semi-supervised learning did not help by default. The team also tested a second semi-supervised method, Soft Teacher, and it scored below YOLO11s at every label count they tried.
Senior author Hanqi Zhuang, a professor in FAU’s College of Engineering and Computer Science, said in FAU’s announcement that “semi-supervised learning allows the technology to learn from a small amount of expert guidance.” FAU’s headline says the AI helps researchers track belugas. The paper tests detection in single frames, and it lists following individual whales from frame to frame as future work.
One In Three Calves Still Goes Uncounted With 1,709 Labeled Images #
Given half the labeled pool, 1,709 images, SEMI-DETR found 66.6% of the calves in the test set and 84.3% of the adults. That is the same recall measure as the 34-image result. YOLO11s found 43.9% of the calves with the same labels.
The paper’s error analysis, which matches detections to whales more loosely, shows what goes wrong. Of the 1,249 calves in the test set, 25.3% were missed outright and 2.7% were mistaken for adults. The authors put the misses down to pixels. Their permits from Fisheries and Oceans Canada authorized flights at 66 feet (20 meters) above the water. From that height the average calf covers 17.9 by 29 pixels of the frame, and the smallest are 1 by 2 pixels. Calves also swim tight against adults, glare and murky water blur their outlines, and sea ice produces false detections.
The paper recommends two fixes: fly future surveys lower where the rules allow it, or use cameras with more zoom.
Flying lower can disturb the whales. DroneXL reported in March 2023 on a University of Windsor study of a different beluga population, in Quebec’s St. Lawrence Estuary, in Low-Flying Drones Harass Endangered Belugas, Study Finds. Sudden dives happened on 4.3% of drone flights there, and on 14.3% of flights below 75 feet (23 meters). The FAU team flew under federal permit, at an altitude already below that line.
Co-author Greg O’Corry-Crowe, a research professor at FAU Harbor Branch Oceanographic Institute, said “the value of this technology goes far beyond simply counting animals.” He wants it to show where belugas are, how they move and how well they reproduce.
DroneXL’s Take #
A drone shoots 4K video faster than any biologist can label it, so the labeling is where a survey like this gets stuck, and a model that needs a fraction of the labels is real help to a small research team. I’d use this one today to count adults. I wouldn’t use it yet to count calves, and the paper’s own numbers are why. My read is that zoom is the fix and altitude is not. The Mavic 3E on the team’s own drone list carries a 162 mm equivalent tele camera, DJI’s specs show, and the St. Lawrence study is a good reason not to bring a drone any closer to a calf.
Sources: Frontiers in Marine Science, Florida Atlantic University, Phys.org DroneXL uses automated tools to support research and source retrieval. All reporting and editorial perspectives are by Haye Kesteloo. Read our editorial standards.
Advertise with DroneXL Your company here Reach 200,000+ drone readers a month with a sponsored article, a banner on your beat, or a partnership.See the options →