AI Quality & Reliability Engineer (QA/SRE) – UAE

Location

United Arab Emirates

Job Category

• Information Technology (IT) & Software
• Engineering & Technical
• Science & Research

Job Overview

We are seeking an AI Quality & Reliability Engineer (QA/SRE) to support the quality, reliability, performance, and operational readiness of AI-powered applications and cloud infrastructure in the UAE. The role combines software test automation, Site Reliability Engineering, cloud infrastructure, AI evaluation, observability, and performance engineering.

The successful candidate will develop and automate AI testing using Python and frameworks such as PyTest, Selenium, or Robot Framework. The role will also involve evaluating AI model outputs, monitoring cloud reliability, developing real-time dashboards, supporting performance testing, and identifying inefficient or high-cost infrastructure usage during testing.

This position offers long-term career growth at the intersection of AI quality engineering, SRE, cloud infrastructure, and automated AI evaluation. Professionals will have opportunities to strengthen their expertise in AI testing, LLM evaluation, cloud reliability, observability, performance engineering, and FinOps practices.

Key Responsibilities

• Design and implement automated testing strategies for AI-powered applications.
• Develop AI testing automation using Python.
• Build automated test frameworks using PyTest, Selenium, Robot Framework, or equivalent technologies.
• Test AI model outputs and evaluate response quality.
• Apply prompt engineering techniques for AI testing and evaluation.
• Implement RAG evaluation and AI quality assessment methodologies.
• Work with ROUGE, BLEU, custom LLM benchmarks, or equivalent evaluation methods.
• Develop automated AI evaluation workflows using DeepEval, Ragas, LangSmith, or similar frameworks.
• Support cloud infrastructure reliability across Azure or AWS environments.
• Validate cloud networking, scaling, and serverless reliability.
• Develop and maintain monitoring and reliability dashboards.
• Work with Grafana, Prometheus, or native cloud monitoring tools.
• Identify reliability issues and support root-cause analysis.
• Implement CI/CD automation using GitHub Actions.
• Support performance testing using K6, JMeter, or equivalent tools.
• Review and validate Terraform configurations and infrastructure changes.
• Identify expensive test workloads and inefficient cloud resource utilization.
• Support FinOps-related optimization during testing and validation.
• Collaborate with engineering, AI, DevOps, SRE, and platform teams.
• Contribute to production-readiness, quality standards, and continuous improvement.

Requirements & Qualifications

• Strong proficiency in Python; Python is mandatory.
• Experience with AI testing and automation.
• Strong experience with PyTest, Selenium, Robot Framework, or equivalent automation frameworks.
• Hands-on experience with Azure or AWS cloud infrastructure.
• Strong understanding of cloud networking, scaling, and serverless reliability.
• Understanding of AI/ML concepts and AI model evaluation.
• Knowledge of prompt engineering.
• Experience with RAG evaluation, ROUGE/BLEU scores, custom LLM benchmarks, or similar methodologies.
• Experience with Grafana, Prometheus, or native cloud monitoring platforms.
• Strong understanding of reliability monitoring and dashboard development.
• Awareness of FinOps and cloud resource optimization.
• Ability to identify inefficient cloud resource usage during testing.
• Experience with Bash scripting.
• Experience with GitHub Actions and CI/CD.
• Ability to read and validate Terraform configurations.
• Experience with K6, JMeter, or equivalent performance testing tools.
• Familiarity with DeepEval, Ragas, LangSmith, or similar AI evaluation frameworks.
• Strong analytical, debugging, and problem-solving skills.
• Ability to work collaboratively across technical teams.

Salary, Benefits & Career Growth

Salary details have not been provided by the employer.

The role provides opportunities for career progression across AI quality engineering, SRE, cloud reliability, AI evaluation, performance engineering, and DevOps. Professionals can further develop their expertise through hands-on exposure to automated AI testing, LLM evaluation, cloud observability, performance testing, infrastructure validation, and FinOps.

Training, certification, and skill-development opportunities may be available depending on project requirements and organizational programs.

Application Process

Application Process (Website)

Apply only through the official job link.

Click Apply Now on the website to access the employer’s application instructions.

HR Email for Application

Send your updated CV directly