Deep Learning Engineer

Description:

Our team is looking for a Deep Learning Engineer.

AI21 is one of the few companies to have trained multi-billion parameter Large Language Models (LLMs). This feat requires top-tier engineering and deep learning algorithmic knowledge. Serving these LLMs efficiently requires cutting-edge technology as well. As a deep learning engineer on the team, you will be responsible for improving and maintaining our training infrastructure, developing/scaling/testing new ideas such as performance improvements and new training methods, adapting our code to run on and best utilize the newest and most advanced hardware accelerators, and running training and evaluation experiments to optimize LLMs on different tasks.

• Develop, improve and maintain our training framework and related infrastructure, as part of our applied research projects
• Optimize training performance and quality in a variety of aspects, such as adding parallelism methods, improving training SDK, integrating RL environments, optimizing new model architectures, research and development of training methods, etc.
• Optimize model serving to support high throughput and low latency in production and training
• Take part in critical post-training efforts along with applied researchers

• 3+ years developing software for production systems, with strong Python and solid engineering fundamentals (testing, debugging, performance work, reading unfamiliar codebases).
• Hands-on understanding of how modern LLM training works end to end in PyTorch: data/model/tensor/pipeline parallelism and when each applies, mixed precision, memory vs. compute tradeoffs, and what a framework like FSDP, Megatron, or DeepSpeed is actually doing under the hood. You don't need to have used a specific one.
• A deep understanding of  LLM architectures — attention variants, MoE, positional encodings, KV cache.
• Ability to debug across the stack: a loss spike, a NCCL hang, a logprob mismatch, or a kernel that's slower than it should be.
• At least a BSc in computer science, E.E. or a degree in a similar field.
• Bonus – experience working in a distributed, containerized GPU training environment (Slurm, Kubernetes, or similar): launching and debugging multi-node jobs, dealing with images, storage, and scheduling constraints.

Significant experience in at least one of the following

• Distributed training: scaling training jobs across nodes, working with any of FSDP/Megatron/DeepSpeed/JAX-based stacks, diagnosing communication and scaling inefficiencies.
• Post-training and RL: SFT/DPO/GRPO-style pipelines, online vs. offline training, rollout–trainer coupling, frameworks such as verl, slime, prime-RL, TRL, or in-house equivalents.
• Inference systems: serving frameworks (vLLM, SGLang, TensorRT-LLM), batching and scheduling, KV cache management, latency/throughput optimization.
• Model efficiency: quantization, distillation, kernel-level optimization, spec-decoding, model export/serialization for deployment.