AI Evaluation Jobs

Why this is a real AI job: The role explicitly focuses on building and scaling GenAI observability and evaluation programs for clients, requiring deep AI expertise and MLOps experience. The company itself is an AI observability platform.

About Arize AI is rapidly transforming the world. As generative AI reshapes industries, teams need powerful ways to monitor, troubleshoot, and optimize their AI systems. That’s where we come in. Arize AI is the leading AI & Agent Engineering observability and evaluation platform , empowering AI eng…

Details Open source / apply

Why this is a real AI job: The role explicitly focuses on designing, implementing, and scaling GenAI observability and evaluation programs for clients. The job description heavily emphasizes AI/ML engineering, MLOps, and working with AI applications in production. The company itself is…

About Arize AI is rapidly transforming the world. As generative AI reshapes industries, teams need powerful ways to monitor, troubleshoot, and optimize their AI systems. That’s where we come in. Arize AI is the leading AI & Agent Engineering observability and evaluation platform , empowering AI eng…

Details Open source / apply

Why this is a real AI job: The role explicitly focuses on designing, implementing, and scaling GenAI observability and evaluation programs for clients. The job description heavily emphasizes AI/ML engineering, MLOps, and working with AI applications in production. The company itself is…

About Arize AI is rapidly transforming the world. As generative AI reshapes industries, teams need powerful ways to monitor, troubleshoot, and optimize their AI systems. That’s where we come in. Arize AI is the leading AI & Agent Engineering observability and evaluation platform , empowering AI eng…

Details Open source / apply

Why this is a real AI job: The role is explicitly focused on building and shipping AI products, defining AI strategy, and leading AI-focused workstreams (AI Trust, AI Building/Super Agent). The requirements heavily emphasize AI depth, experience with AI tools, and a track record of shi…

The AI orchestration of your wildest imagination. n8n is the open workflow orchestration platform built for the new era of AI. We give technical teams the freedom of code with the speed of no-code, so they can automate faster, smarter, and without limits. Backed by a fiercely inventive community an…

Details Open source / apply

Why this is a real AI job: The role is explicitly focused on improving AI-powered features, designing prompts, running evaluations, and operating model releases. The job description heavily emphasizes practical AI/ML production experience and working directly with LLMs.

About Mistral At Mistral AI, we believe in the power of AI to simplify tasks, save time, and enhance learning and creativity. Our technology is designed to integrate seamlessly into daily working life. We democratize AI through high-performance, optimized, open-source and cutting-edge models, produ…

Details Open source / apply

Why this is a real AI job: Die Masterarbeit konzentriert sich direkt auf die automatisierte Bewertung generativer KI, was einen klaren und zentralen KI-Fokus darstellt.

Master Thesis Automated Generative AI Evaluation (f/m/x) Informatiker/in

Details Open source / apply

Why this is a real AI job: The role explicitly focuses on designing, implementing, and scaling GenAI observability and evaluation programs for clients. The job description heavily emphasizes AI/ML engineering, MLOps, and working with AI applications in production. The company itself is…

About Arize AI is rapidly transforming the world. As generative AI reshapes industries, teams need powerful ways to monitor, troubleshoot, and optimize their AI systems. That’s where we come in. Arize AI is the leading AI & Agent Engineering observability and evaluation platform , empowering AI eng…

Details Open source / apply

Why this is a real AI job: Die Rolle ist stark auf die Entwicklung und Analyse von Metriken und Systemen fuer ein AI-Plattform-Framework ausgerichtet, mit klarem Fokus auf die Verbesserung von AI-Plattformleistungen, Entwicklererfolgen und Sicherheit. Die Aufgaben umfassen die Definiti…

About the Team OpenAI’s mission is to ensure AGI benefits all of humanity. The API organization is one of the highest-leverage ways we do that: we put frontier intelligence in the hands of builders who turn it into products, businesses, and services that reach people everywhere. We build the infras…

Details Open source / apply

Why this is a real AI job: The role is deeply embedded in building the data infrastructure that powers AI models, specifically GenAI. The PM will directly shape AI model quality through the products they build. The description explicitly mentions AI/ML, data labeling, model training, a…

Scale AI builds the data infrastructure that powers the world’s most advanced AI. We are the trusted data partner behind frontier model makers and enterprise AI teams — providing the high-quality training data, evaluation frameworks, and human-feedback systems that make models smarter, safer, and m…

Details Open source / apply

Why this is a real AI job: The role is explicitly focused on GenAI test & evaluation, owning the roadmap for evaluation capabilities, and improving the performance of agentic applications. The qualifications heavily emphasize technical expertise in AI systems and evaluation methodologi…

At Scale, our mission is to develop reliable AI systems for the world’s most important decisions. The Public Sector team is at the forefront of this mission, partnering with government agencies to deploy mission-critical agentic solutions. Role Overview The Public Sector GenAI T&E Product Manager w…

Details Open source / apply