Senior Software Engineer- AI Datacenter Orchestration

Description

Location: Tel Aviv

#Hybrid

DriveNets is a leader in high-scale disaggregated networking solutions. Founded in 2015, DriveNets modernizes the way service providers, cloud providers and hyperscalers build networks. Supporting the largest network in the world, more than half of AT&T’s backbone traffic is running on DriveNets’ Network Cloud open disaggregated architecture. Raising $587 million in three funding rounds, DriveNets is disrupting the networking market from high-scale architecture to AI platforms, and is bringing onboard the most talented people. We are seeking people that want to make an impact on the world’s leading communication networks and are experienced in networking architecture or AI infrastructure solutions.

Responsibilities And Duties

• Design and build the profiled network infrastructure that teams run high-performance LLM serving services on in production.
• Build the data-path and memory-fabric infrastructure that gives teams the primitives to implement KV cache strategies — paged attention, prefix caching, eviction policies — and hit their efficiency and latency targets.
• Provision and profile the network fabric and cluster infrastructure that inference frameworks (vLLM, TGI, TensorRT-LLM, Triton) are deployed on across GPU clusters.
• Build the scheduling and network infrastructure that exposes the throughput primitives teams need to implement batching strategies (continuous batching, dynamic batching) under SLA constraints.
• Build the compute and memory-bandwidth infrastructure profiles that give teams the headroom to evaluate and apply quantization techniques (GPTQ, AWQ, FP8, INT8) with clear production tradeoffs.
• Build network-level observability infrastructure — TTFT, TPOT, tokens/sec, GPU utilization, cache hit rates — that teams instrument their inference services against.
• Design and build the transport layer (SSE, gRPC, WebSocket) that teams use to expose real-time inference APIs.
• Build the storage and network infrastructure — sharding, format conversion, runtime configuration — that model teams use to move checkpoints to production endpoints.
  
  

• 5+ years of backend engineering, with 2+ years specifically in ML inference systems.
• Deep understanding of transformer attention mechanics as they relate to KV cache design.
• Hands-on experience with at least one major inference engine (vLLM, TGI, TRT-LLM, Triton).
• Strong Python skills; ability to read and modify inference engine internals; C++/CUDA familiarity.
• Experience with paged/virtual KV cache, prefix caching, speculative decoding, or disaggregated prefill/decode.
• Production experience with GPU clusters (A100/H100/H200) and CUDA memory management.
• Experience with container orchestration (Kubernetes) and GPU scheduling.
• Strong fundamentals in building observable, production-grade microservices: health checks, structured logging, distributed tracing, metrics.
  
  

• Strong cross-functional collaboration — ability to work effectively with model research and platform teams.
• Ownership mindset: comfortable driving production tradeoffs and making decisions under uncertainty.
• Clear technical communication: able to explain complex systems to both engineering and non-engineering stakeholders.
  
  

• Experience with tensor parallelism (TP), pipeline parallelism (PP), or multi-node inference.
• Contributions to open-source inference projects (vLLM, SGLang, etc.).
• Familiarity with attention variants: GQA, MLA, sliding window, MoE routing.
• Experience with NVIDIA NIM or Triton Inference Server deployment at scale.