RL Environment Engineer - Agentic AI Security

Frontier AI agents learn how to act, safely or not, inside reinforcement learning environments. The quality of those environments sets the ceiling on what agents learn and how much any test of them can be trusted. Building environments that are realistic, verifiable, and hard to game is now one of the most important engineering problems in AI.

Innodata's Agentic Evaluation and Safety practice works directly with the world's leading frontier AI labs. As an RL Environment Engineer, you build the environments frontier agents are trained and evaluated in, with a focus on AI security, cyber, and agentic safety. You turn scenario designs from offensive security engineers and alignment scientists into stateful, reproducible environments with rewards that hold up under optimization.

You report to the Head of AI Cyber Agentic Environments & Testing and work closely with the CTO. You will help shape a team at the cutting edge of AI security.

What you'll do

• Build agentic environments. Implement stateful, containerized environments where agents use tools, browsers, terminals, and APIs to complete multi-step tasks. Define clean observation and action spaces, tool schemas, and task specs.
• Design rewards and verifiers. Write programmatic graders and verifiers that check end state, not just final answers. Combine rule-based checks, state diffs, and model-based grading where needed, and make them robust to reward hacking.
• Make environments reproducible. Ensure deterministic seeding, reliable reset and snapshot, and hermetic execution with no hidden network access, so every episode starts from the same state.
• Calibrate difficulty. Run frontier and open models through environments, measure pass rates and variance, and tune tasks so they produce real learning signal instead of being trivial or impossible.
• Analyze trajectories. Inspect rollouts to find shortcuts, broken graders, leaked answers, and cues that reveal a task is a test. Fix them before anything ships.
• Scale the pipeline. Build tooling for generating environment variants, running many environments in parallel, logging trajectories, and packaging environments for delivery to labs.
• Collaborate closely. Work with offensive security engineers, red teamers, alignment scientists, and platform engineers to turn scenario designs into training- and evaluation-ready environments.

What you bring

• 4+ years of software engineering, with strong Python and solid systems skills: Docker, Linux, networking, and databases.
• A working understanding of reinforcement learning for LLMs: environments, episodes, rewards, verifiers, rollouts, and post-training methods such as RLHF, RLVR, and GRPO.
• Hands-on experience with AI agents: tool calling, agent loops and harnesses, MCP, and computer-use or coding agents.
• An adversarial mindset about your own work. You instinctively ask how an agent could game a reward or exploit a shortcut.
• Experience building test infrastructure, simulators, sandboxes, or evaluation systems where correctness and reproducibility matter.
• Comfort running and analyzing experiments, including pass rates, variance, and failure analysis across many runs.
• Clear writing. Your environment specs and design decisions are easy for others to build on.
• Mentor junior SWEs, cyber researchers, and RL engineers.

Nice to have

• Built RL environments or agent benchmarks that were used for training or evaluation of frontier or open models.
• Experience with environment and evaluation frameworks such as Gymnasium, Inspect, Harbor, verifiers, or OSWorld-style harnesses.
• Background in security: CTFs, pentesting, or building vulnerable-by-design systems.
• Experience training models with RL yourself, even at small scale.
• Kubernetes and experience running large fleets of parallel sandboxes.
• Familiarity with AI safety and alignment research: reward hacking, evaluation awareness, agentic risk.
• Publications or open-source contributions in RL, agents, or evaluation.

We’re an equal opportunity employer.

Only relevant applicants will be contacted