LLM cost and speed audit
I find the waste in your LLM stack and fix it
A fixed-price audit for teams that serve their own models with vLLM, NVIDIA Dynamo, or similar. It is also for teams that spend a lot on hosted LLM APIs.
You get measured numbers, a ranked list of problems, and patches for the ones I can fix in the time.
My fixes are merged in vLLM and NVIDIA Dynamo, two of the serving stacks this audit covers.
USD 2,500, fixed price. Delivered in two to three weeks.
Scope: one service or one request path, agreed before the start.
Describe your stack, your traffic, and the symptom.
What I look at
If you serve your own models (vLLM, NVIDIA Dynamo, or similar)
- CPU time on the request path: tokenizer, detokenizer, router, queues, streaming.
- Copies and allocations that grow with prompt length, batch size, or worker count.
- Lock contention under concurrent load.
- Batching, prefix cache, and scheduler settings against your real traffic.
If you call hosted APIs (Anthropic, OpenAI, Bedrock, and others)
- Where the tokens go, by feature and by part of the request.
- Prompt cache hit rate, and what breaks the cache.
- Requests that can move to a cheaper model or to batch pricing.
- Retries, duplicate calls, and usage that never reaches your billing.
I wrote about what breaks a prompt cache and how to check the hit rate: How Prompt Caching Works in Coding Agent Harnesses.
How I find problems
- I read the request path for work that repeats on every request: copies of data that the code only reads, and scans inside code that runs for each request.
- I check that the work is not needed: what owns the data, what else reads it, and what triggers the path.
- I check for duplicates. If a fix is already in progress upstream, I leave it alone.
- I measure each candidate with a probe on the real code. An idea that does not hold up in the numbers does not go in the report.
Your code
Your code stays in your environment. I use AI coding tools on your code only with your written approval. You own every patch.
What you get
- A report with ranked findings. Each finding has evidence, an estimate of the effect, and the effort to fix it.
- A plain list of what I did not measure.
- Before and after numbers for every fix I make, with the steps to reproduce them.
- Patches as pull requests to your repository.
- Fixes to open-source dependencies go upstream, if you agree.
How it works
- You email a short description of your stack and the problem.
- I reply in two working days with a yes or a no, and the access I need.
- Half of the fee is due before I start. The other half is due on delivery. If the audit finds no problem worth fixing, you do not pay the second half.
- I work in your staging environment or on a reproduction. I sign an NDA if you need one.
What I do not do
- GPU kernel work, quantization, or model training.
- Open-ended retainers. If the audit finds more work, we scope it as a separate job.
About me
I am a backend engineer with more than six years of production work. I maintain pg_strict and uids-postgres, two Postgres extensions. I write at saybackend.com.
Start with an email
USD 2,500, fixed price. Delivered in two to three weeks.
Scope: one service or one request path, agreed before the start.
Describe your stack, your traffic, and the symptom.