The bill routed my models for me

I stopped arguing about which model is smartest sometime in September. Not because I figured it out, or because the argument ended. I stopped because the invoice settled it. Three weeks of quota math taught me more about model selection than a year of benchmark threads, and the lesson was smaller than I expected. On a fixed budget the interesting question is never which model is best. It is which model earns its price on this task.

What a quota actually is

Subscription quotas are not tokens. They are windows, and each window has a shape. I pay for the 20x tier, twice the price of 5x, and it returns about 1.9 times the usage, not four times. Nobody prints that on the plan page. There is a five-hour limit, and a weekly limit that runs at roughly ten times the first. I read "20x" as linear for longer than I want to admit.

The windows reset on schedules that do not always match what support tells you. A shared account can also lose most of its weekly capacity in one session if someone else is working the same credentials. I learned that one the expensive way in the second week of September: a colleague and I both ran agents on the same account through the day, and the weekly quota was gone before Wednesday. One person per account now, and my personal accounts stay off shared machines.

The numbers that changed how I work

These are real figures from my logs and from people I compare notes with. They are the whole reason my routing looks the way it does.

A model that is cheap per token can still be expensive per task. I watched a job that should take twenty minutes eat a full five-hour window. The model was a fraction of the price and it did finish, but it looped and re-read context the way cheap models do, and the job still cost me a day of capacity. Price per token is a useless number without the token count, and the token count depends on how well the model stays on task.

Effort levels inside one model move the price more than swapping models does. I had been leaving things on max out of habit. Running the same tasks at the extra-high setting cost about a third of what max cost for the same work, and dropping the fleet down one level cut my monthly bill by a third with no visible quality change.

Higher effort is not monotonic either. A teammate measured a mid-tier model at medium versus high and found medium more complete. High had started drifting into overthinking. Effort is a per-role knob for me now, not a global dial, and I test it instead of assuming.

Then there is the number I keep around to remember what scale looks like. One person I follow reported a single session that burned four billion tokens across twenty-six hours and 392 subagents. I have never gone near that. It is a ceiling marker. If a session is drifting into that shape, something is wrong with my orchestration, not my model choice.

Blocking the caps by role

My routing table exists because I stopped believing that one model should do everything. Three jobs, three price points.

The supervisory layer checks in rarely, recovers context from history, and watches for repeated mistakes. It does not need a frontier model. Mine runs cheap, and I think of it as a log reader with an opinion.

The lead plans, coordinates, and accepts work. It needs judgment and it touches every decision, so it gets a strong mid-tier model. It is also the role where my biggest single figure showed up: I once had a heavyweight model running both supervisor and lead duties, and it spent nine hours and twenty-two dollars across one working day. Fine when the day produces something. Absurd when it is supervising three blocking tasks.

The peers do the bulk of the tokens, and here the arithmetic is brutal. Peers are where volume lives. A frontier model on every peer means the workers cost more than the planning. I staff peers with the cheapest model that reliably handles bounded implementation, and spend the saved budget on the models that make decisions.

My lineup reads like a depth chart: strongest model at the top for arbitration, mid-tier for lead, cheap for peers. When a cheaper model proved excellent at reading code and following structure, it took the peer slot the same week. When another cheap model turned out to be great at finding bugs and weak at writing, it got the bug-hunting tasks and nothing else.

The sentence that ended the benchmark arguments

For weeks I had a running fight with two people about two model families. One insisted family A was stronger, the other had evidence for family B, and both kept quoting numbers that did not line up. What ended it was not a benchmark I ran. It was a line from a colleague who had gotten tired of the thread: the model is one, the harness is five.

That one stuck. Most of what I had been calling model quality was harness behavior. The same model in two different setups gives two different experiences. The router, the memory handling, the tool permissions, the container the agent runs in, the prompts it wakes up with, all of it shapes the result before the model does anything. When someone tells me model X is bad today and was good yesterday, the harness changed, or the account's quota posture did, or the tools shifted underneath. Almost never the weights.

So I stopped ranking models on feel and started measuring them per role, on my own tasks, with my own harness held constant. That is a much smaller and more useful exercise. Rankings from other people calibrate where I start and nothing more.

Where the money actually goes

The grey market around AI accounts is its own economy, and watching it taught me as much about pricing as any vendor page. Accounts are traded, resold, bundled, and occasionally swapped for hardware in group chats. People run portfolios, five to ten accounts each, juggling renewals and resets to keep working without hitting a wall. Quota functions as currency. I have seen a course priced at a week of quota, and a service contract settled in tokens.

That economy also carries real risk. Resellers get their packages revoked. Shared credentials get flagged and blocked. A bulk purchase turns out to be a shared account already in use elsewhere, and a batch can zero out overnight when someone finds the same credentials in a public dump. Every account I buy now comes with one question answered first: is this account exclusive to me, and what happens to my work when it is not. The cheap route is only cheap until it costs a day.

I also stopped buying compute I do not need. A managed agent host at a flat monthly rate looked reasonable until I priced the same workload against a machine I already owned and a tunnel back to it. At my volume the hosted option stays more expensive for years. That is a decision everyone has to run with their own numbers.

What I do now

Three habits hold the whole thing together.

Route by role, and revisit the table monthly, because prices move. The model that was the smart pick last month may be the expensive pick now.

Measure cost per task, not per token, and log it where I can see it. A routing choice that saves tokens and burns hours is not a saving. I keep one number per role: cost to finish a standard task. When it moves, I look.

Keep the harness constant when comparing models, and change one thing at a time. Half the model debates I had were harness debates wearing a model's name. Once I pinned the harness, the disagreements got small enough to test, and testing ended them in an afternoon.

The invoice is not a nuisance at the end of the month. It is the most honest evaluation signal I have. Benchmarks measure what a model can do in someone else's harness. The bill measures what mine just did.