Senior QA Engineer (AI Solutions)
Cape Town or Johannesburg, South Africa
ZAR 700K-860K (estimate) Senior-level Full Time
Tasks
- Build and maintain automated test suites
- Coach engineers on testing observability and evals
- Collaborate with backend frontend AI platform engineers
- Define agent quality metrics accuracy grounding safety latency cost
- Design LLM and agent evaluation harnesses
- Establish deployment gates with quality evidence
- Execute accessibility visual regression and security checks
- Feed defect patterns evaluation results telemetry into priorities
- Integrate LLM quality metrics into test reporting
- Maintain golden datasets regression suites red teaming scenarios
- Own end to end QA strategy for agent platform
- Partner with security and compliance on safety reviews
- Perform AI assisted triage clustering failures root cause
- Practice prompt engineering for QA test generation
- Run performance load and chaos testing
- Shift left into requirements and design testing
- Shift right using production traces evals user feedback
- Stress test guardrails prompt injection defenses tool use restrictions data boundaries
- Use AI tooling for test generation and synthetic data
Perks/Benefits
- Employee Share Purchase Plan ESPP
- Employee Stock Options Grant
- Flexible working hours
- Growth and development opportunities
- Health benefits plan
- Internal LMS and knowledge hubs
- Modern equipment
- Pension plan
- Performance bonus
Skills/Tech-stack
API Testing | AWS | Accessibility testing | Azure | Azure DevOps | CI/CD | Chaos Testing | Contract testing | DAST | Event Driven | Event-driven architecture | GitHub Actions | GitLab CI | Golden Dataset | Hallucination detection | JMeter | JUnit | Java | K6 | Kubernetes | LLM Evaluation | LLM-as-judge | Load Testing | Locust | Logs | Metrics | Microservices | Model Drift | Observability | Performance Testing | Playwright | Prompt engineering | Prompt injection | Pytest | Python | RAG | Red Teaming | Regression testing | RestAssured | Retrieval-Augmented Generation | Rubric Based Evaluation | SAST | Service virtualization | Traces | TypeScript | Visual regression | Visual regression testing
Education
N/A
Roles
Engineer | QA Engineer | SDET | SDET Engineer | Senior QA | Senior QA Engineer
Regions
Countries
States
Related jobs
-
Anomaly Detection | Data Modeling | Data Pipelines | Docker | JavaScriptCareer growth opportunities | Flexible work environment | Remote workMid-level Full TimeSouth Africa R18h ago
-
API Gateway | AWS Lambda | Access Control | Agent systems | Amazon APISenior-level ContractJohannesburg, South Africa1d ago
-
Accelerate | Deep learning | Diffusers | Generative AI | Hugging FaceAutonomy and ownership | Career growth opportunities | Flexible globally distributed work environment | Fully remote work | Global collaborationMid-level Full TimeSouth Africa R1d ago
-
Data Curation | Deep learning | Distributed Training | GPU Training | JAXAccess to high performance GPU clusters | Collaborative global culture | Flexible work schedule | Fully remote | Professional growth opportunitiesSenior-level Full TimeSouth Africa R1d ago
-
AWS | Azure | Embeddings | GCP | Hugging FaceFlexible work environment | Fully remote | Growth opportunities | Remote distributed team experienceSenior-level Full TimeSouth Africa - Remote R1d ago
-
Data Engineer ZAR 420K-500KApache Airflow | Apache Spark | Data Governance | Data Modeling | DockerCredit and Criminal Record Check | Curro School Fees Benefit | Employee assistance programme | Financial Services Assistance | Free Sleep CoachingMid-level Full TimeCape Town, South Africa1d ago
-
AWS Lambda | Amazon CloudWatch | Amazon EKS | Amazon Redshift | Amazon SageMakerOn-call rotationSenior-level Full TimeCape Town, Western Cape, South Africa4d ago
-
API Security | Access Control | Airflow | Amazon Redshift | AuthenticationFlexible schedule | Remote workSenior-level Full TimeSouth Africa - Remote R4d ago
-
Application Support Engineer ZAR 300K-420KAgile | Ataccama | Cause analysis | Change Management | CollibraMid-level Full Time15 Alice Lane, South Africa4d ago
-
Learning Experience Designer (AI & Automation) ZAR 216K-240KAudit Logging | Generative AI | Human-in-the-loop | Microsoft 365 | Microsoft CopilotEntry-level Full TimeJohannesburg, GP, South Africa5d ago
-
AWS | AWS Bedrock | AWS SageMaker | Cloud Computing | Computer VisionFlexible scheduling | Fully remote | Paid training | Stable internet freedomMid-level Full TimeSouth Africa R5d ago
-
Machine Learning Engineer ZAR 420K-600KAirflow | Amazon SageMaker | CI/CD | Docker | Feature EngineeringAgile environment | Collaborative culture | Feedback and empathy | Freedom and responsibility | Self-organizationMid-level Full TimeSouth Africa - Remote R5d ago
-
Data Engineer ZAR 420K-480KAPI | Amazon Redshift | Amazon Web Services | Amazon Web Services (AWS) | Apache SparkMid-level Full TimePinelands, South Africa5d ago
-
Senior-level Full TimePretoria, South Africa6d ago
-
Senior Cloud Data Engineer ZAR 780K-920KApache Airflow | Apache Beam | Apache Spark | Cloud platform | Data GovernanceIn-office workSenior-level Full TimeCape Town, Western Cape, South Africa6d ago
-
Apache Flink | Apache Spark | Data Analysis | Exploratory Data Analysis | Feature EngineeringCross-functional collaboration | Knowledge sharing | MentorshipSenior-level Full TimeJohannesburg, GP, South Africa6d ago
-
Intermediate Data Engineer ZAR 396K-480KApache Airflow | Azure Data | Azure Data Factory | Azure DevOps | Azure FunctionsHybrid work modelMid-level Full TimeSandton, GP, South Africa6d ago
-
Senior-level Full TimeSouth Africa - Remote R6d ago
-
Head Modelling & Analytics ZAR 750K-840KALM | Cash Flow | Cash flow modeling | Classification | Data AnalysisExecutive-level Full Time15 Alice Lane, South Africa7d ago
-
Embedded Software Development Engineer ZAR 300K-420KARM | C# | Debugging | Embedded Systems | Embedded firmwareDisability programs | Hybrid work | Life insurance | Private medical care | Tailored financial programsMid-level Full TimeZA-Century City-Cape Town-The Matrix 2G and …7d ago
-
Mid-level Full TimeJohannesburg, South Africa7d ago
-
AI Engineer-2 ZAR 396K-480KAWS | Agentic Systems | Cloud platform | Conversational AI | Copilot StudioChallenging work | Culture of innovation and continuous learning | Ethical AI focus | Inclusive development culture | Opportunities to network and collaborateMid-level Full TimeFNB Building, 1 Enterprise Road, Fairland, …7d ago
-
Senior Data Engineer I ZAR 500K-600KAWS | Airflow | Apache Spark | Azure | C#Flexible working hours | Medical aid | Modern family benefits | Retirement plan | SabbaticalSenior-level Full TimeSouth Africa - Durban7d ago
-
Senior Data Engineer I ZAR 500K-600KAWS | Apache Airflow | Apache Kafka | Azure | C#Flexible working hours | Medical aid | Modern family benefits | Retirement plan | SabbaticalSenior-level Full TimeSouth Africa - Durban7d ago
-
Data Engineer ZAR 420K-500KApache Airflow | Apache Spark | Cloud Platforms | Data Governance | Data ModelingEmployee assistance programme | Financial Services Assistance | Free daily meals | Free massages on-site | Free on-site gymMid-level Full TimeCape Town, South Africa7d ago