Automation & AI Evals Engineer

We’re hiring: Automation & AI Evals Engineer

Israel / Hybrid

Caura.ai is building AI-native products and workflows, and we’re looking for an Automation & AI Evals Engineer to own quality across both product and AI systems.

This is not a classic testing role.

We’re looking for someone strong in automation testing, but also AI-native — someone who actively works with tools like Cursor, Claude Code, Codex, and AI coding agents, and is excited about testing LLMs, agents, prompts, and AI workflows.

What you’ll do:

• Build E2E, API, regression, smoke, and integration tests

• Test AI-powered features, workflows, and production systems

• Create evals for LLM apps, agents, prompts, and structured outputs

• Detect regressions, hallucinations, edge cases, and reliability issues

• Work closely with R&D to improve release confidence and product quality

What we’re looking for:

• 4+ years in automation / technical testing

• Strong E2E + API testing experience

• Experience with Playwright, Cypress, Selenium, Postman, etc.

• Strong debugging skills across the stack

• Curiosity and hands-on interest in AI, LLMs, agents, and evals

Strong plus:

Experience with AI evals, RAG, prompt testing, TypeScript, Python, CI/CD, Docker, Kubernetes, or cloud environments