Careers › Engineering

Senior AI Engineer (LLM & Evaluation)

Full-time · Remote (IT/EU) · Talent pool: opens as we grow

Grace's heart is an AI evaluation engine: every user prompt is judged across 7 dimensions by LLMs. You will make that engine more accurate, more consistent, cheaper and multi-model.

What you will do

Design and optimise the evaluation system prompts (rubrics, score calibration, cross-model consistency)
Manage the multi-provider pipeline (Anthropic primary, OpenAI-compatible fallbacks) and per-evaluation costs
Build eval sets and metrics: how do we measure that the AI judge judges well?
Moderation and safety of generated and evaluated content
Explore new capabilities: image, voice and agent evaluation

What we are looking for

4+ years of software development, at least 1-2 on LLM applications in production
Real experience with structured prompt engineering and output evaluation (evals)
Knowledge of the main providers' APIs (Anthropic, OpenAI) and their economics
Ability to reason about quality and bias of an automated judging system
Fluent English; Italian appreciated

Think you qualify?

The selection is already open: 5 real senior scenarios evaluated by Grace's AI, minimum score 70/100, retries allowed. Test your skills now →

Interested? Send your CV or LinkedIn profile to [email protected]: when we start hiring, we call first those who stepped forward.

Other positions

From our network

AGORÀ Intelligence: Enterprise AI Governance Platform

Govern AI at scale: policies, adoption and measurable results on your data. Built for boards and C-suite.

Visit agora-intelligence.com →

From the Agora Intelligence blog