Inference Performance Engineer

About the Company

We are an early-stage, still-in-stealth team building an AI platform that gives organizations control of their AI by delivering special models that are faster, more cost-efficient, and more secure than one-size-fits-all alternatives, and that customers can own and run on their own terms.

If you are passionate about high-scale infrastructure, hardware-software co-optimization, and shaping the foundation of the modern AI stack, we would love to talk.

Role Overview

As an Inference Performance Engineer, you will own and evolve our core execution platform and LLM serving infrastructure. You will design, deploy, and optimize highly available, resilient multi-cloud GPU and Kubernetes systems. Working closely with systems engineers and research teams, you will tackle bare-metal, network, and framework bottlenecks to keep models serving at maximum throughput and ultra-low latency.

Core Responsibilities

• Inference & Performance Optimization: Profile, benchmark, and optimize high-throughput, latency-sensitive LLM workloads like vLLM, smart routing layers, and KV-cache orchestration.
• Systems Architecture: Design, build, and operate resilient, k8s-native platforms and custom automation across multi-cloud environments.
• Low-Level & Hardware Efficiency: Locate and eliminate system-level bottlenecks to squeeze maximum performance out of compute and GPU resources.
• Developer Platform & Tooling: Build robust, self-service platform tools to enable rapid experimentation, deployment, and scalability.

Requirements

• 5+ years of software engineering, systems backend, or performance engineering experience.
• Deep expertise in Kubernetes managing production workloads including CRDs, Helm, Karpenter, affinity/anti-affinity, and custom scheduling.
• Strong background in Performance Engineering with proven experience profiling and squeezing efficiency out of high-throughput systems handling thousands of requests per second.
• Proficiency in Go or Python.
• Hands-on experience with major cloud platforms like AWS or GCP and production observability stacks like Prometheus and Grafana.

Preferred Qualifications

• Experience with GPU orchestration on Kubernetes using NVIDIA device plugins, DCGM, or GPU scheduling.
• Deep familiarity with LLM inference mechanics and procedures such as prefill/decode phases, TTFT, ITL, and custom engine tweaks.
• Experience with multi-cloud or hybrid-cloud architectures.