How we test

• PingEvery hour

  What we ask

  Reply with exactly: ok

  What it measures

  How long the app takes to wake up and answer at all.

  This is not a writing-speed test. Most of a ping is the app starting up.

• SpeedEvery hour, every pinned model

  What we ask

  Count from 1 to 120 in plain text on a single line, numbers separated by single spaces. Output nothing else.

  What it measures

  How fast the model writes: wall tok/s, stream tok/s, and time to first word.

  The visible answer is about 240 tokens. Thinking tokens are extra and count only in wall tok/s.

Token

A token is a small chunk of text, roughly four letters. Models read and write text one token at a time.

Wall tok/s

Every token the model wrote, thinking included, divided by the full start-to-finish time. Higher is better. It is the only speed unit that means the same thing in all three apps, so our boards sort on it.

Stream tok/s

How fast text appears once it starts flowing. Claude counts visible answer tokens from the first word to the last. Grok does the same when its stream gives us a window. Codex sends no token stream, so its column covers the whole turn and carries a *. We show this number but never rank on it.

TTFT (time to first word)

How long you stare at a blank screen before the first visible word shows up. Thinking does not count as a word. Lower is better. Codex leaves this blank because it does not stream.

Wall time

Start to finish for the whole command, including the app booting. It is what you actually wait for. Lower is better.

Middle value (median)

For a window we report the middle sample, not the average, so one freak slow run cannot drag a whole row down.

Slots / n

How many finished checks sit behind a number. A small count moves a lot, so we always print it next to the value.

Status words like rate limit and wrong answer are spelled out on the Logs page. Test kit versions are spelled out on the Changelog.

Clean room

Every run starts in a new empty folder. No saved settings, no skills, no slash commands, no memory, no tools. Grok gets a throwaway home folder. If an app reaches for a tool anyway, the row fails and is labeled used tools.

Same job for everyone

Same prompts, same one-line system prompt — “Answer the user. Do not use tools. Do not add a preamble.” — and every model at the top of its own effort scale, so no app can quietly coast.

Paid sign-ins only

We run on plans we pay for, like any customer. If an API key is sitting in the environment, the run stops on purpose. A key would quietly move billing to the API and we would stop testing the paid app.

One try, fails stay in

We never retry a bad run and we never pick the nicer of two samples. Every fail is recorded once and labeled: rate limit, rate limit warning, timeout, auth, used tools, wrong answer, or parse. Dropping fails would make every tool look better than it really is.

Every round is graded

A speed round only counts if the answer is right. A plain script checks the count against the one correct answer — no human opinion, no AI judge. A fast wrong answer scores nothing, so no app can win by writing rubbish quickly.

Nothing gets blended

Write speed and the ping stay on separate boards. We never mash them into one winner score, because the trade-off is yours to make.

The models we pin

The exact names we ask for every round. If an app serves something else, we save the real name and mark the row remapped.

Featured on the boards

claude-fable-5 and claude-opus-5 in Claude Code, gpt-5.6-sol in Codex, and grok-4.6 in Grok Build. Every board leads with these four.

Also measured

claude-haiku-4-5, claude-sonnet-5, gpt-5.6-luna, gpt-5.6-terra, gpt-5.5, and grok-4.5. Same rounds, same rules. They live on the Other models page.

Effort

Every pin runs at the top of its own scale: xhigh for grok-4.6, high for everything else. grok-4.5 has no xhigh, so high is its honest ceiling.

Ping carriers

The ping rides on claude-haiku-4-5, gpt-5.6-luna, and grok-4.6. gpt-5.2 was retired on August 19, 2026 — OpenAI now refuses it on subscription accounts.

One lab, our numbers

Every number comes from one machine on a normal home connection, measured by us. Use it to compare the three apps against each other. These are not official figures from Anthropic, OpenAI, or xAI.

We time the app, not just the model

Start-up, the app’s own hidden prompt, and its network path all count. That is on purpose — it is what you wait for. It also means a quick model inside a slow app looks slow here.

Codex carries a big built-in prompt

Codex ships a large built-in agent prompt. Fresh input of roughly 9–12k tokens is that prompt, not a 250-token reply. Its CLI does not echo the model name back, so model_verified stays false on those rows.

Grok leftovers

A clean Grok 4.5 ping answered ok in 8.9s with 7987 input tokens. Its inspect output still lists a Codex plugin row and one hook. We name those leftovers on the sample. No tool ran. grok-4.5-build is the same family as grok-4.5.

We cannot see your quota

People talk about Claude and ChatGPT in 5-hour windows. SuperGrok is one weekly pool shared with Chat, Imagine, and Voice. We do not read anyone’s remaining quota from an API, so nothing here predicts when you will run out.

• Start here

  A one-minute tour of the boards, in plain words.

• Lab health

  Is the lab fresh right now, and is it still signed in?

• Logs

  Every sample we saved, fails included, plus our token spend.

• Changelog

  Every time we changed a rule on this page.