Senior Inference Systems Engineer – KV Cache Optimization

Company Description Lightbits is a technology company focused on high‑performance, software-defined storage and data platforms for modern cloud and AI workloads. The team builds scalable, resilient infrastructure solutions that help organizations run data-intensive applications efficiently and reliably. Lightbits emphasizes innovation in low-latency systems, flash storage optimization, and cloud-native architectures. Team members work in a collaborative, engineering-driven environment with opportunities to solve complex performance and reliability challenges. The company values technical excellence, ownership, and continuous learning in a fast-moving domain.
Role Description The Senior Inference Systems Engineer – KV Cache Optimization is a full-time hybrid role based in Kfar Saba, with flexibility for partial work from home. This role focuses on designing, implementing, and optimizing key-value (KV) cache systems used in large-scale inference workloads, ensuring low latency, high throughput, and robust reliability. Day-to-day responsibilities include analyzing system performance, profiling inference pipelines, tuning cache strategies, and collaborating with software, hardware, and DevOps teams to integrate optimized caching into production environments. The engineer will troubleshoot complex system issues, improve observability and monitoring for inference and caching components, and contribute to the overall systems architecture to support evolving AI and data workloads. The role also involves documenting designs, participating in code and design reviews, and mentoring other team members in systems engineering and performance optimization best practices.
Qualifications

• Strong Systems Engineering and Systems Design skills for building and scaling high-performance inference and KV cache architectures.
• Hands-on System Administration and Technical Support experience with Linux-based environments, containers, and production-grade distributed systems.
• Advanced Troubleshooting capabilities for diagnosing performance bottlenecks, latency issues, and reliability problems in complex, multi-layered systems.
• Proficiency in performance profiling and optimization of caching layers, storage systems, and network I/O in large-scale AI or data platforms.
• Solid programming skills in languages such as C/C++, Rust, or Go, and familiarity with scripting (e.g., Python, Bash) for tooling and automation.
• Experience with inference frameworks and serving systems (e.g., TensorRT, ONNX Runtime, Triton, or similar) and their integration with caching and storage.
• Knowledge of distributed systems concepts (consistency, partitioning, replication) and modern cloud-native technologies (Kubernetes, microservices, observability tools).
• Bachelor’s or Master’s degree in Computer Science, Electrical Engineering, or related field, or equivalent practical experience.
• Ability to work effectively in a hybrid environment, communicate clearly with cross-functional teams, and take ownership of complex technical challenges.