Technical Solutions Architect, Evals & Fine-Tuning

Innodata partners with leading foundation model labs, hyperscalers, and enterprise AI teams to build the data, evaluation, and post-training systems that make modern LLMs trustworthy and production-ready.

As a Technical Solutions Architect for Evals & Fine-Tuning, you are the technical face of Innodata to our most demanding customers. You sit at the intersection of client AI/ML teams, our research scientists and ML engineers, our subject-matter expert workforce, and our platform teams. You translate ambiguous customer goals - “improve factuality on long-context legal QA,” “build a safety eval suite for our next model release,” “design a DPO pipeline for our coding assistant” - into concrete, scoped, deliverable engagements.

This is a senior individual-contributor role for someone who has done the work: built fine-tuning pipelines, designed eval harnesses, argued with stakeholders about benchmark validity, and earned credibility with sophisticated ML buyers.

This remote role is based in Israel, with travel to client sites as needed. You will be part of the Generative AI Solutions team and report directly to the VP of LLM Data Services.

What you’ll do

• Lead technical discovery with prospective and existing customers - foundation model labs, frontier AI teams, and large enterprises - to understand model objectives, gaps, and constraints.
• Design end-to-end solutions across the post-training stack: SFT data curation, preference data collection for RLHF/DPO, golden datasets, custom benchmarks, LLM-as-judge pipelines, human-in-the-loop evaluation, red teaming, and multimodal eval (text, image, audio, video, long-context).
• Architect engagements that combine Innodata’s platforms (GenAI Test & Evaluation Platform, Annotation Platform, GenAI Workbench) with our global SME workforce across 85+ languages and domains.
• Author technical proposals, SOWs, solution diagrams, and pricing models in partnership with sales, delivery, and finance.
• Run technical workshops, POCs, and pilot designs that de-risk larger programs and prove value quickly.
• Serve as the ongoing technical advisor during delivery, partnering with applied research scientists, AI/ML research engineers, language data scientists, and program managers to keep solutions aligned with the original intent.
• Feed customer signal back into Innodata’s R&D and product roadmap- what benchmarks customers actually want, where eval methodology is breaking, what new fine-tuning paradigms are gaining traction.
• Stay current on the state of the art in evals (e.g., dynamic and agentic benchmarks, capability vs. safety evals, long-context and tool-use evaluation) and post-training (SFT, RLHF, DPO, RLAIF, rejection sampling, distillation).
• Represent Innodata externally - at customer reviews, conferences, and in technical content.

What you’ll bring

• 7+ years of experience in applied ML, ML engineering, ML research, or technical solutions roles, with at least 2+ years focused specifically on LLM evaluation and/or post-training.
• Hands-on experience fine-tuning LLMs (SFT at minimum; preference optimization methods like RLHF, DPO, or KTO strongly preferred) and designing the data pipelines that feed them.
• Deep familiarity with LLM evaluation methodology: public benchmarks and their limitations, custom benchmark construction, LLM-as-judge design and its failure modes, inter-annotator agreement, and human eval workflow design.
• Strong fluency in Python and the modern LLM toolchain (Hugging Face, PyTorch, vLLM, evaluation frameworks such as lm-evaluation-harness, lighteval, or equivalents).
• Excellent technical communication. You can hold your own in a room with research scientists at a frontier lab and, an hour later, brief a non-technical executive on the same engagement.
• A consultative mindset: you ask sharp questions, you push back when a customer’s stated request won’t actually solve their problem, and you are comfortable owning a recommendation.
• Bachelor’s or advanced degree in computer science, machine learning, computational linguistics, or related field- or equivalent demonstrated experience.

Nice to have

• Experience working directly with a foundation model lab or frontier AI team.
• Multimodal evaluation experience (vision-language, audio, video).
• Background in a regulated domain (healthcare, finance, legal) where evaluation rigor is non-negotiable.
• Open-source contributions to evaluation frameworks, fine-tuning libraries, or benchmark datasets.
• Experience scoping and pricing services engagements; comfort partnering with sales.
• Multilingual evaluation experience or familiarity with cross-lingual benchmark design.

Why Innodata

You’ll work with the teams building and shipping the most capable AI systems in the world, on the problems they are actively losing sleep over. You’ll have access to a 5,000+ SME workforce, proprietary platforms, and a research org that publishes open-source toolkits and datasets. And you’ll help shape what “good” looks like for evaluation and post-training across the industry.