Senior QA Engineer (AI Solutions)
Cape Town or Johannesburg, South Africa
ZAR 700K-860K (estimate) Senior-level Full Time
Tasks
- Build and maintain automated test suites
- Coach engineers on testing observability and evals
- Collaborate with backend frontend AI platform engineers
- Define agent quality metrics accuracy grounding safety latency cost
- Design LLM and agent evaluation harnesses
- Establish deployment gates with quality evidence
- Execute accessibility visual regression and security checks
- Feed defect patterns evaluation results telemetry into priorities
- Integrate LLM quality metrics into test reporting
- Maintain golden datasets regression suites red teaming scenarios
- Own end to end QA strategy for agent platform
- Partner with security and compliance on safety reviews
- Perform AI assisted triage clustering failures root cause
- Practice prompt engineering for QA test generation
- Run performance load and chaos testing
- Shift left into requirements and design testing
- Shift right using production traces evals user feedback
- Stress test guardrails prompt injection defenses tool use restrictions data boundaries
- Use AI tooling for test generation and synthetic data
Perks/Benefits
- Employee Share Purchase Plan ESPP
- Employee Stock Options Grant
- Flexible working hours
- Growth and development opportunities
- Health benefits plan
- Internal LMS and knowledge hubs
- Modern equipment
- Pension plan
- Performance bonus
Skills/Tech-stack
API Testing | AWS | Accessibility testing | Azure | Azure DevOps | CI/CD | Chaos Testing | Contract testing | DAST | Event Driven | Event-driven architecture | GitHub Actions | GitLab CI | Golden Dataset | Hallucination detection | JMeter | JUnit | Java | K6 | Kubernetes | LLM Evaluation | LLM-as-judge | Load Testing | Locust | Logs | Metrics | Microservices | Model Drift | Observability | Performance Testing | Playwright | Prompt engineering | Prompt injection | Pytest | Python | RAG | Red Teaming | Regression testing | RestAssured | Retrieval-Augmented Generation | Rubric Based Evaluation | SAST | Service virtualization | Traces | TypeScript | Visual regression | Visual regression testing
Education
N/A
Roles
Engineer | QA Engineer | SDET | SDET Engineer | Senior QA | Senior QA Engineer
Regions
Countries
States
Related jobs
-
Optimization Engineer ZAR 480K-540K5G-NR | Actix | Air interface | Atoll | Drive testGlobal team collaboration | Knowledge sharing | Mentorship | On-call supportMid-level Full TimeJohannesburg,Gauteng,South Africa1d ago
-
Data Engineer ZAR 840K-1170KAzure Synapse | Azure Synapse Analytics | CDC | CI/CD | Change Data CaptureSenior-level Full TimeSandton - 1 Discovery Place, GP, …1d ago
-
Junior Specialist: Analytics & Data Science ZAR 240K-288KAI | Agile | Dashboard Development | Data Cleansing | Data MiningEntry-level Full TimeAbsa Towers West, South Africa1d ago
-
Machine Learning Engineer ZAR 264K-360KAutocomplete | Classification | Cloud platform | Embeddings | Google CloudMid-level Full TimeJohannesburg, Gauteng, South Africa2d ago
-
Bash | Cloud infrastructure | Data Processing | Docker | GCPAsynchronous culture | Friendly work environment | Remote or distributed workMid-level Full TimeCape Town, South Africa2d ago
-
Access Control | Airflow | Amazon Redshift | Authentication | BigQueryFlexible working hours | Remote workSenior-level Full TimeSouth Africa - Remote R2d ago
-
Mid-level ContractJohannesburg, South Africa4d ago
-
Senior-level Full TimeSandton, South Africa4d ago
-
AI Implementation Expert ZAR 745K-860KAPI Integration | Azure Machine Learning | Azure OpenAI | ChatGPT | Copilot StudioSenior-level Full TimeSouth Africa4d ago
-
AI Implementation Expert ZAR 745K-860KAPI Integration | Azure AI | Azure Machine Learning | Azure OpenAI | ChatGPTSenior-level Full TimeSouth Africa4d ago
-
Airflow | Apache Superset | BigQuery | Cloud Migration | Cloud platformAnnual vacation | English language discounts | Fully remote | Health reimbursement | International engineering cultureSenior-level Full TimeSouth Africa R4d ago
-
AI/ML | AWS | Access Control | Asana | AzureRemote work | Video interview requirementSenior-level Full TimeArgentina - Buenos Aires5d ago
-
AWS | Amazon Redshift | Apache Airflow | Apache Spark | BigQueryFlexible time zones | Remote workMid-level Full TimeSouth Africa - Cape Town5d ago
-
AWS | Apache Spark | BigQuery | C++ | CI/CDSenior-level Full TimePretoria, South Africa6d ago
-
A/B | A/B Testing | AWS | Amazon Athena | B testingCareer development | Collaborative & Inclusive Culture | Cross-functional collaboration | Flexible working environment | Learning and professional growthSenior-level Full TimeSouth Africa6d ago
-
AWS | Amazon Redshift | Amazon S3 | Bitbucket | ConfluenceRotational training programEntry-level Contract Full TimeJohannesburg, GP, South Africa6d ago
-
AI Engineer (Data Scientist) ZAR 700K-860KAI Foundry | AML | AWS Bedrock | AWS Bedrock Agents | Agentic AIAI practitioner community participation | Mentoring junior data scientists | Responsible AI governance forum involvementSenior-level Full Time15 Alice Lane, South Africa6d ago
-
Senior Data Engineer (Databricks & Cloud Data Platforms) ZAR 875K-1100KAWS Glue | Amazon Redshift | Amazon S3 | Apache Spark | AzureMentorship | Professional development | Sponsored events | Technical communitySenior-level Contract Full TimeJohannesburg, Gauteng, South Africa7d ago
-
Senior Data Engineer (Databricks & Cloud Data Platforms) ZAR 875K-1100KAWS Glue | Airflow | Amazon Redshift | Amazon S3 | AzureMentorship | Ongoing professional development | Sponsored events | Technical communitiesSenior-level Contract Full TimeJohannesburg, Gauteng, South Africa7d ago
-
AWS | Azure | CI/CD | Data Governance | Data LakesConference access | Fully remote work | High-impact projects | International collaboration | Modern work equipmentMid-level Full TimeSouth Africa R7d ago
-
Senior Full Stack Engineer (NLP) ZAR 600K-800KAPI Integration | CSS | Database Schema | Database Schema Design | DjangoAnnual leave | Education stipend | Employee assistance program | Internet benefits | Laptop providedSenior-level Full TimeRemote, South Africa R8d ago
-
Bash | Data Ingestion | Data Ingestion Pipelines | Data Processing | DockerMid-level Full TimeJohannesburg, South Africa8d ago
-
Senior-level Full TimeSandton - 1 Discovery Place, GP, …8d ago
-
Data Engineer (MLOps / Analytics Focus) ZAR 384K-420KAPI Integration | API Monitoring | Azure DevOps | CI/CD | Data GovernanceMid-level ContractJohannesburg, South Africa11d ago
-
API Integration | AWS | Analytics | Dashboard Development | Data ArchitectureMid-level Full TimeCape Town, South Africa11d ago