Senior Software Engineer

About the Job

Join Impala AI, an innovative startup building a fully-managed LLM-inference platform, that enables data heavy enterprises to perform any AI task at any scale without limits.

We're looking for an Experienced Software Engineer to join our founding team. You’ll be responsible for building and optimizing scalable cloud infrastructure solutions tailored for AI workloads. This role offers a unique opportunity to directly shape our infrastructure strategy, improve system reliability and performance, and contribute to establishing Impala AI as a leader in adaptive AI compute management.

Join us to tackle the magic that make AI tick under the hood and build the backbone powering the AI revolution.

• Design and implement the control plane that orchestrates distributed inference across cloud GPU infrastructure.
• Own backend systems responsible for job scheduling, SLA tracking, usage metering, tenancy, auth, and data coordination.
• Build robust APIs that glue together infra automation, ML workloads, and user-facing dashboards.
• Work across multiple cloud providers and BYOC environments—build systems that expect chaos and still work.
• Collaborate with infra, ML, and product teams to abstract complex infra flows into clean interfaces.
• Help define system architecture and lay the groundwork for a platform that adapts to varied customer deployments.

• 5+ years of experience building backend systems and APIs in cloud production environments.
• Fluency in Python, with solid experience in designing clean service boundaries.
• Experience building distributed systems or control planes—Kubernetes operators, service meshes, autoscaling logic, etc.
• Familiarity with cloud-native tooling: Terraform, Docker, Helm, Prometheus, etc.
• Strong systems thinking: ability to model complex workflows and build abstractions that last.
• Comfort navigating messy multi-cloud (AWS - Must), multi-tenant, BYOC architectures.
• Ability to thrive in ambiguity and take ownership from idea to deployment.
• Self-motivated and able to operate independently in a fast-moving startup environment.
• A collaborative team player with strong interpersonal skills, a positive and easygoing attitude, and the potential to grow into a leadership role.

• Background in infrastructure, DevOps, or ML platforms.
• Experience with serverless orchestration and async compute models.
• Passion for building developer tools and internal platforms.
• Contributions to open-source infra projects or devtools.
• Familiarity with SLA-driven compute or HPC-style schedulers.