AI Song Detector Accuracy: Our 65-Song Test

Our AI song detector answered 63 of 65 test songs correctly (96.9%) on October 21, 2026. It flagged none of the 40 human songs as AI and caught 23 of the 25 AI songs; Deezer's album label caught one more. No song landed in the Can't tell band. The sample is small, so treat these numbers as early evidence, not a guarantee.

The detector is the model behind our AI song detector: fakeprint, a small open-source model that looks for the trace AI music generators leave in the spectrum. The table above lists every song, its score, the checker's answer and how four other copies of the same song scored.

The test set has 25 AI songs and 40 human songs. The AI songs come from 17 acts that the artists, their creators, a streaming platform or the press identified as AI, almost all released in 2025 or 2026. The human songs were chosen to be hard: heavy Auto-Tune, vocoders, electronic tracks, and country, soul and gospel in the same styles as the AI songs.

The first 40 songs (20 AI, 20 human) are the core set. The pass line was fixed before testing: at least 80% correct and at most 10% of human songs flagged, or the checker wouldn't launch. The 25 extra songs came afterwards, to narrow the range of possible false alarms. Every song was scored from its official 30-second Apple Music preview, the same audio the checker downloads for a link. Using one source for both groups means the file format can't give the answer away. The list of AI artists says what was reported about each act.

Each song was scored once by the same model the checker runs, and the score was turned into the checker's three answers: 0.8 or above is Likely AI-generated, 0.2 or below is Likely human-made, and anything between is Can't tell. Accuracy is the share of songs answered correctly, with Can't tell counted as wrong.

Zero false alarms in 40 human songs sounds perfect, but a sample that size can't rule out a real rate of a few percent: the 95% Wilson interval runs from 0% to 8.8%. For AI songs, 2 misses in 25 put the miss rate somewhere between 2.2% and 25.0%.

The two groups were far apart. The highest human score was 0.0007 and the lowest score of a caught AI song was 0.873, so on these songs the exact cut-off barely mattered.

Deezer's label is read separately. Deezer marks albums its own detector found to contain AI-generated tracks, and the checker treats that label as AI whatever the fingerprint says.

It missed two of the 25 AI songs. TaTa's Pulse (feat. Timbaland) scored below 0.001; Rolling Stone reported that TaTa's songs start from Timbaland's own demos, which Suno extends, with human-written lyrics, so much of the audio may not be generated. Solomon Ray's Find Your Rest scored 0.119, and we can't explain why.

Deezer labels the album that Find Your Rest appears on as AI, so for that song the checker still answers Likely AI-generated when it finds the song on Deezer. Pulse has no Deezer label, and Hive and both Gemini models missed it as well. Songs like Pulse, where a person and an AI share the work, are the hardest case for any audio check.

Hardly for human songs, a little for AI songs. Every preview was re-encoded three ways, and Deezer's own MP3 previews were scored too. All 40 human songs stayed Likely human-made in every version. Depending on the version, 19 to 21 of the 25 AI songs stayed Likely AI-generated (18 of 22 on Deezer's previews); the rest moved to Can't tell or Likely human-made.

The biggest drops: King Willonius's BBL Drizzy fell to 0.12 as Opus at 64 kbps, and Breaking Rust's Livin' On Borrowed Time scored 0.998 on Apple's preview but 0.0006 on Deezer's MP3 preview. We don't know why; Deezer may carry a different master. That gap is why the checker uses Apple's preview first.

So heavy compression costs some AI catches, but for human songs it left AI song detector accuracy unchanged: none moved towards AI. The model reacts to the generator's trace, not to the codec.

On the 40 core songs, with each engine's default cut-off of 0.5, the fingerprint model was the most accurate at 95%. Deezer's album label came close on the songs it covers. Hive's general AI-audio detector missed 15 of 20 AI songs, and two Gemini models and two SONICS research models did worse than the fingerprint.

• Deezer's album label: 94% on the 35 core songs found on Deezer, with no human song flagged. It only covers music on Deezer, and only whole albums.
• fakeprint CNN (a second model in the same open-source project): 92.5%, but it flagged Billie Eilish's bad guy and Queen's Bohemian Rhapsody as AI. We don't use it.
• Gemini 4 Argon and Gemini 3.8 Flash, asked to listen and give a probability: 88.6% (on 35 usable replies) and 82.5%. Both can recognise famous songs, so their human answers on hits prove little, and they still flagged human songs, including Eminem's Lose Yourself.
• Hive: 62.5%, with no false alarms. We could only reach Hive's general AI-generated and deepfake audio detection, not its music-specific model, so this doesn't rate that product.
• SONICS (ICLR 2025): the 5-second model reached 57.5%, and the 120-second model 50% because it flagged every human song. They were trained on Suno and Udio songs from the SONICS dataset, and the 120-second model expects far more audio than a 30-second preview.
• AI or Not wasn't tested: its music model isn't open to the free plan.

These are results on 30-second previews of 40 songs, not general reviews of these products.

Five limits matter most. New generators can slip past it, songs that are only partly AI usually pass as human, the model's training data has licence questions, the test is small and was assembled by us, and Deezer's label comes from an undocumented interface that we looked at while choosing songs.

• New generators. The model card covers Suno up to version 5 and Udio up to version 1.5, and says the model needs retraining as generators change. Suno launched v6 on September 9, 2026, after these songs were made; we haven't tested it, or ElevenLabs Music, Google Lyria and Mureka. Expect misses until we retest.
• Part-AI songs. An AI voice over human production, or AI parts mixed into a real recording, usually scores as human. Pulse is the example in this test.
• Licences. The model's code and weights are MIT-licensed, but its training data includes the SONICS dataset, licensed CC BY-NC 4.0 for research only, and data the author doesn't publish. Deezer filed two patent applications for AI-detection methods in December 2024. We haven't checked whether they cover the method this model uses.
• A small sample we chose. 25 AI songs, all from acts named in the press and mostly from 2025, and 40 human songs we picked. A different set could score differently.
• Deezer's label. We read it through an undocumented interface that can change without notice, and it applies to whole albums. We looked at it while choosing the AI songs, so its AI-song result isn't a blind test; its result on human songs (none of 36 flagged) is.

Is This Song AI built and runs the checker tested here, and will earn from it if paid packs open, so we have an interest in good numbers. To keep ourselves honest we publish every song and score, picked hard human songs on purpose, and report the misses and the engines we decided not to use.

We have no business relationship with any company whose engine appears here. Hive ran on its free trial credit, and Gemini was reached through OpenRouter at standard prices. Deezer's claim that its own detector is 99.8% accurate is Deezer's figure, which we can't check. If the checker gets one of your songs wrong, use the Result wrong? button or email hello@isthissongai.app; the guide for artists explains what happens next.

In our 65-song test it answered 96.9% of songs correctly: no human song flagged as AI, 23 of 25 AI songs caught, and one more with Deezer's label. The AI song checker shows the same scores.

Not in our test: none of 40, including heavy Auto-Tune and electronic tracks. A test that size can't rule out a false alarm rate of up to about 8.8%, so treat a Likely AI-generated answer as one piece of evidence.

We don't know yet. Suno launched v6 on September 9, 2026, and the model covers Suno up to version 5. Until we retest, a v6 song may come back as Likely human-made.

On our core songs the fingerprint model was more accurate than both, costs nothing per check and runs in your browser, so your files never leave your device.