Google evaluated Perch 2.0 on marine audio tasks using a few-shot linear probe, comparing it with other pre-trained models in the Perch Hoplite repository. The tests covered underwater datasets including NOAA PIPAN, ReefSet and DCLDE, with the aim of measuring how well the models can support custom classifiers built from a small number of labelled examples.
The evaluation used embeddings from each candidate model, then trained a simple multi-class logistic regression model on top of them. For each class, the researchers used 4, 8, 16 or 32 examples and measured area under the receiver-operating characteristic curve (AUC_ROC), where values closer to 1 indicate stronger class separation.
The models compared included Perch 2.0, Perch 1.0, SurfPerch and the multispecies whale model.
NOAA PIPAN is an annotated subset of the NOAA NCEI Passive Acoustic Data Archive from the NOAA Pacific Islands Fisheries Science Center recordings. It includes labels used in earlier whale models and new annotations for baleen species such as common minke whale, humpback whale, sei whale, blue whale, fin whale and Bryde’s whale.
ReefSet was developed for SurfPerch model training and uses annotations from the Google Arts and Culture project: Calling in Our Corals. It includes biological reef noises, species and genera classes such as damselfish, dolphins and groupers, plus anthropomorphic noise and wave classes.
DCLDE was evaluated with three label sets: Species, Species Known Bio and Ecotype. The Ecotype set includes killer whale subpopulations such as Transient/Biggs, Northern Residents, Southern Residents, Southeastern Alaska killer whales and offshore killer whales.
The results showed that more examples per class improved performance across all models, except on ReefSet data, where performance was high even with only four examples per class for all models except the multispecies whale model. Perch 2.0 was consistently either the top or second-best performing model for each dataset and sample size.
Source: research.google.
Companies can share verified announcements through Newz9’s international press release submission page.
