CommonLID Leaderboard

Results for the CommonLID and CommonLID-nano benchmarks. Headline metric: macro F1. Models are ranked by macro F1 within each tab; click a row to see per-language metrics.

🌐 Website • 📝 Blog post • 📄 Paper • 🆕 Add a model

CommonLID

Common Crawl's language identification benchmark, sampled from real-world web text and human-validated across hundreds of language varieties.

Reference • License: common-crawl-tou • Main score: macro_f1

Scoring scope

Scores are calculated over the whole dataset. Scores are calculated on the subset of language varieties covered by the model. (cov.)

commonlid — sorted by Macro F1

Model	Macro F1	Micro F1	Mean FPR (%)	Languages	Samples/s
commonlingua	60.4	70.4	0.07	1456	60383.4

Click a row to load per-language metrics.

Source: commoncrawl/commonlid-results @ HEAD.