Labeling & Evaluation Agent (German)

David Kennedy Recruitment is working with a leading technology company specialising in artificial intelligence that is seeking to onboard a Labeling & Evaluation Agent to support the development and continuous improvement of its AI models.

The role focuses on evaluating AI-generated outputs, including chatbot responses, and providing high-quality annotations and feedback. You will assess response quality, factual accuracy, relevance, safety and policy compliance, helping improve the performance and reliability of large language models (LLMs).

Position: Labeling & Evaluation Agent (German)

Location: Lisbon, Portugal

Work model: On-site

Employment type: Full-time

DUTIES AND RESPONSIBILITIES:

- Evaluate AI-generated responses using detailed quality assessment and annotation guidelines

- Label and annotate language model outputs accurately and consistently

- Assess generated content for quality, relevance, factual accuracy, safety and policy compliance

- Apply complex annotation guidelines across different scenarios and content types

- Identify inconsistencies, edge cases and potential issues, escalating them when necessary

- Analyse structured text, language data and AI-generated content

- Maintain strict confidentiality and comply with project standards and data security requirements

- Provide accurate evaluations, annotations and actionable feedback to support ongoing AI model improvement

REQUIREMENTS:

- Native-level proficiency in German

- English proficiency at B2 level or above

- Strong understanding of Generative AI, large language models (LLMs) and AI annotation or evaluation platforms

- Excellent analytical and critical thinking skills, with strong attention to detail

- Ability to make objective, evidence-based decisions

- Ability to interpret and consistently apply complex written guidelines and policies

- Strong reading comprehension and written language skills

- Ability to evaluate structured text, datasets and AI-generated content

- Ability to meet productivity, accuracy and quality targets

- Professionalism, integrity and discretion when handling confidential information

BENEFITS :

- Modern office in city center with open spaces, easy to access with public transportation

- Comprehensive initial training and guidance to make sure you're well prepared and comfortable at a new workplace

- Full-time employment in Lisbon

- 24/7 rotational working schedule

- 2 rotational days off

- Opportunity to work in the rapidly developing field of Generative AI and Large Language Models

- International and multicultural working environment

- Support from colleagues with coaching and mentoring opportunities

- Health and Wellbeing initiatives

- Monthly meal allowance

- And much more!!!