AI Tester (AI Validation & RAG Testing)
Role Overview
We are seeking an AI Tester to support the validation and quality assurance of AI-powered applications within a large enterprise environment. You will focus on evaluating AI model behaviour, ensuring response accuracy, and identifying risks such as hallucinations and model drift.
This role is critical in ensuring that AI systems are reliable, scalable, and fit for production use across multiple use cases.
Key Responsibilities
- Work closely with application teams to test and validate AI-driven solutions
- Perform RAG evaluation, including identification of hallucinations, inaccuracies, and grounding issues
- Detect and monitor model drift and response consistency over time
- Execute functional and exploratory testing of AI outputs
- Support stress and load testing scenarios by analysing system response under high user query volumes
- Validate AI responses against test datasets and benchmark expectations
- Collaborate with engineering teams to improve model performance and output quality
- Contribute to defining AI testing frameworks, metrics, and validation methodologies
Required Skills & Qualifications
- Experience in AI testing, QA, or data validation roles
- Strong understanding of LLMs, RAG systems, and AI behaviour
- Experience identifying hallucinations, bias, and model inconsistencies
- Familiarity with API testing tools (e.g., Postman)
- Understanding of performance and stress testing concepts
- Analytical mindset with strong problem-solving skills
- Ability to work in cross-functional Agile teams
