How we rank text-to-speech APIs

The 3 boards

The ranking draws from 3 public leaderboards: artificialanalysis.com (Speech Arena Elo), voicearena.com (arena Elo, US English), humannessindex.vapi.ai (humanness score 0–100). Together they cover blind preference arenas (Artificial Analysis, Voice Arena — Elo) and a humanness panel (Humanness Index — humanness score 0–100). The source list is fixed per methodology version and only changes with a published update.

Name alignment

Boards name the same model differently — a model may appear under different names on different boards, or the same voice listed under two vendor labels. We maintain a public alias table mapping every board’s label to one canonical model ID; each alias is recorded with the snapshot it came from. Distinct variants (HD vs Turbo, streaming vs batch) are never merged.

The score: Bayesian pairwise rank aggregation

Every board contributes the same thing: a model’s rank. Each rank becomes a fraction — #1 on a board is always 1.0, the last row is 1/N — so a rank on a large board is worth more than the same rank on a small one. A coverage term pulls models that appear on few boards toward the field average, so a single strong placement can’t masquerade as a consensus. The aggregate is an ecosystem-normalized Borda count: the competitors each model defeats, weighted by board size, divided by the total competitors across every board. A model ranked #1 on every board scores exactly 100; miss a board and your ceiling drops, no matter how dominant you are there. The headline score is that 0–100 number.

Ties & confidence

Score ties are broken by Schulze pairwise comparison — the strongest chain of board wins between two models decides — then coverage, then best single rank. Every model also carries a confidence label: High (on at least two-thirds of boards, with the boards broadly agreeing), Med (on two or more boards), Low (single-board, or boards that disagree). Confidence is a recommendation of quality next to the score, never part of it.

Refresh & provenance

The boards are refreshed weekly; the ranking recomputes when any of them changes. Every published ranking archives the exact snapshot of each board it was computed from — the full history is downloadable from the data page. The aggregation code is open source. Latency is shown next to the score, never in it — see section 07.

Price transparency

Prices are listed as $X.XX per 1M characters (the column reads $/1M chars). Some providers price per minute of audio rather than per character; where a provider's listed price uses a per-minute format, we convert using a neutral benchmark — generate 1000 characters of audio through a reference model (Cartesia), measure the output duration in minutes, then derive the per-1M-characters equivalent. Converted prices are marked with an asterisk.

Latency

We aggregate latency the same way we aggregate quality: every figure is copied from a public benchmark, linked, and dated with when we read it. We never time an API ourselves, never estimate, and never fill a gap — a model no source lists shows a dash.

We read 3 public sources of time to first audio (how long after the request the first audible sound arrives, the number that decides whether a voice agent feels responsive):

1. Coval — Time to first audio, p50, us-east-1. Open-source harness run every 30 minutes from us-east-1; median over a rolling 7-day window. Streaming; connection handshakes excluded; counts leading silence until the first audible sample. Last read Oct 11, 2026, covering 20 models on the board.

2. Speko — Time to first audio, p50, US East. Sequential requests from US East, about 30 per run, re-run per model on the date shown. Last read Oct 11, 2026, covering 22 models on the board.

3. Humanness Index — Time to first audio, median. Region not stated. 50 sequential streaming API requests per model, measured once (the date shown). Last read Oct 11, 2026, covering 19 models on the board.

They do not measure the same thing in the same way: different harnesses, regions, sample sizes, and whether connection setup or leading silence counts. The same model can differ by several times between sources. So we never average or blend them. Every figure is shown under its own source, and the table is ordered by one source at a time. Coval is the primary source — the one highlighted by default and used for the "fastest" answer — because it re-measures every model continuously and publishes the largest sample (hundreds of requests per model each week) through a public API, from an open-source harness that treats every provider the same way. The others are shown beside it, never substituted into it.

A latency source labels models its own way, so each label is matched to a model on the board through a public alias table, or by an exact name match; anything else is left off, not guessed. Artificial Analysis is not a latency source here: it publishes generation speed (characters per second over a whole clip), not time to first audio.

Latency is best-effort. If a source is down at the weekly refresh, the board still publishes: that source keeps its previous values, marked with the date they were read, or shows dashes if there are none. Latency is never part of the 0–100 score — changing the score is a separate methodology decision.