Methodology · v2.0
How we rank text-to-speech APIs
We aggregate, we don't test. Every number on the board traces back to a public source, and the aggregation code is open source — anyone can re-run it and get the same ranking. For a deeper dive into the statistical framework, read the full methodology whitepaper.
The 3 boards
The ranking draws from 3 public leaderboards: artificialanalysis.com (Speech Arena Elo), voicearena.com (arena Elo, US English), humannessindex.vapi.ai (humanness score 0–100). Together they cover blind preference arenas (Artificial Analysis, Voice Arena — Elo) and a humanness panel (Humanness Index — humanness score 0–100). The source list is fixed per methodology version and only changes with a published update.
Name alignment
Boards name the same model differently — a model may appear under different names on different boards, or the same voice listed under two vendor labels. We maintain a public alias table mapping every board’s label to one canonical model ID; each alias is recorded with the snapshot it came from. Distinct variants (HD vs Turbo, streaming vs batch) are never merged.
The score: Bayesian pairwise rank aggregation
Every board contributes the same thing: a model’s rank. Each rank becomes a fraction — #1 on a board is always 1.0, the last row is 1/N — so a rank on a large board is worth more than the same rank on a small one. A coverage term pulls models that appear on few boards toward the field average, so a single strong placement can’t masquerade as a consensus. The aggregate is an ecosystem-normalized Borda count: the competitors each model defeats, weighted by board size, divided by the total competitors across every board. A model ranked #1 on every board scores exactly 100; miss a board and your ceiling drops, no matter how dominant you are there. The headline score is that 0–100 number.
Ties & confidence
Score ties are broken by Schulze pairwise comparison — the strongest chain of board wins between two models decides — then coverage, then best single rank. Every model also carries a confidence label: High (on at least two-thirds of boards, with the boards broadly agreeing), Med (on two or more boards), Low (single-board, or boards that disagree). Confidence is a recommendation of quality next to the score, never part of it.
Refresh & provenance
The boards are refreshed daily; the ranking recomputes when any of them changes. Every published ranking archives the exact snapshot of each board it was computed from — the full history is downloadable from the data page. The aggregation code is open source. Latency is deferred: the Humanness Index also publishes a latency signal, but too few models are measured on it yet for it to be a fair benchmark — we’ll add a latency dimension when more sources cover it.
Price transparency
Prices are listed as $X.XX per 1M characters (the column reads $/1M chars). Some providers price per minute of audio rather than per character; where a provider's listed price uses a per-minute format, we convert using a neutral benchmark — generate 1000 characters of audio through a reference model (Cartesia), measure the output duration in minutes, then derive the per-1M-characters equivalent. Converted prices are marked with an asterisk.
Independence. No vendor sponsorship, referral fees, or paid placement. There is no code path that can elevate or demote a specific vendor: every engine goes through the identical formula, the formula is public, and every published ranking links the exact source snapshots it was computed from.