AI Algorithms Engineer

Description

Role Overview

We are looking for a Senior/Staff AI Algorithms Engineer to join our research and engineering team, focusing on the development, training, and optimization of large-scale language models (LLMs) on distributed networks. The ideal candidate combines deep theoretical understanding with hands-on engineering excellence.

Must-Have Requirements

Education & Experience

• M.Sc. or Ph.D. in Computer Science, Electrical Engineering, Applied Mathematics, or a related field
• 5+ years of industry or research experience in machine learning / deep learning
• Demonstrated track record of delivering production-quality ML systems or publishing in top-tier venues (NeurIPS, ICML, ICLR, TMLR, etc.)

Core AI/ML Expertise

• Deep understanding of transformer architectures (encoder-only, decoder-only, encoder-decoder), attention mechanisms, positional encodings (RoPE, ALiBi, etc.), and normalization strategies
• Hands-on experience training large-scale models (1B–70B+ parameters) from scratch
• Familiarity with pre-training, instruction tuning, RLHF, DPO, and related alignment techniques
• Knowledge of model evaluation: perplexity, downstream benchmarks (MMLU, HellaSwag, etc.), and ablation methodology

Distributed Training & Systems

• Strong practical experience with distributed training paradigms:
• Data Parallelism (DDP, FSDP)
• Tensor Parallelism (Megatron-style)
• Pipeline Parallelism
• ZeRO (Stage 1/2/3 with optimizer/gradient/parameter sharding)
• Proficiency with modern training frameworks: PyTorch, DeepSpeed, Megatron-LM, Hugging Face Accelerate / Transformers
• Experience managing large-scale GPU clusters (A100/H100/B200 or equivalent), including job scheduling, multi-node communication (NCCL), and GPU utilization monitoring

Engineering Skills

• Expert-level Python programming; clean, testable, modular code
• Proficiency with data pipelines for LLM pre-training
• Solid understanding of profiling and debugging training runs: loss spikes, gradient norms, throughput bottlenecks (MFU), dead nodes

Strong Advantage (Nice-to-Have)

Research & Innovation

• First/co-author publications in LLM training, efficient transformers, or distributed ML at top venues
• Experience with novel architecture exploration: SSMs (Mamba), MoE, hybrid architectures
• Familiarity with continual learning or domain adaptation
• Experience with federated learning or layer-wise / alternative training strategies