Staff Software Engineer

LLM optimization is a young field with no settled playbook, so the optimizations you invent here are genuinely new - and they ship into production, not a paper. And they matter: real enterprises run enormous volumes of LLM traffic and feel every wasted token, so the work you do lands on live customer requests and solves a problem they actually have. It's about as close to the frontier, and as close to the product, as engineering gets.

03What you'll do
Invent and ship optimizations. Design new strategies across every layer of a request - prompt and context compression, tool pruning, file and dev-command trimming, semantic caching, cache-prefix stabilization - and take them all the way to the hot path. Each one has to cut tokens on real traffic without degrading output.
Prove every win. Build the evals and the token/quality instrumentation that separate a genuine saving from a plausible-sounding one. An optimization only ships when the numbers back it.
Live in the caching trade-offs. Provider prompt caches only pay off on byte-stable prefixes, and a naive rewrite can cost more than it saves. Knowing exactly when an optimization actually wins - deterministic transforms, per-session decision pins, semantic dedup - is a lot of the job.
Build and own the gateway that runs them. The real-time proxy that carries production LLM traffic - streaming, retries, backpressure - executing every optimization on the live request path.
Keep it fast and deploy anywhere. Defend a tight latency budget on the hot path, and package the whole thing so it installs cleanly into any customer's cloud - AWS, GCP, Azure, containers, Kubernetes - and upgrades without drama.
Own systems end to end. From first commit to code serving live customer traffic - plus the spend-visibility and governance product built on top - with a direct line to the founders.