QA Analyst (AI Systems)
Tasks
- Build Python scripts for agent testing
- Create agent evaluation question templates
- Create agentic task datasets
- Create ground-truth datasets
- Run bulk testing
- Run load testing
- Test RESTful APIs
- Test agent payloads for logic failures
- Test for prompt injection risks
- Track agent performance KPIs
- Validate JSON inputs outputs
- Write integration tests
- Write unit tests
Perks/Benefits
Skills/Tech-stack
Agentic Workflows | Docker | Integration Testing | JMeter | JSON | LLM orchestration | Langchain | Langfuse | Langgraph | Langsmith | Locust | OpenTelemetry | Playwright | Prompt engineering | Pydantic AI | Pytest | Python | RAG | REST APIs | React | Retrieval-Augmented Generation | Selenium | Unit Testing | Vector Databases
Education
N/A
Roles
AI | AI QA | AI QA Engineer | Analyst | Engineer | QA Analyst | QA Engineer
Related jobs
-
Ingeniero de Datos SSIS Senior MXN 216K-216KAgile | Azure | Azure Data | Azure Data Factory | Azure SynapseHybrid workSenior-level Full TimeCDMX, México R1h ago
-
Software Engineer 2 - (Python, LLM, Machine Learning) USD 201K-246KAPI Integration | AWS | Agentic Workflows | Azure | CrewAI401k match | Educational assistance | Federal Holidays | Healthcare coverage | Paid time offMid-level Full TimeAnnapolis Junction, Maryland R4h ago
-
Senior-level Full TimeAustin, TX R4h ago
-
AI/ML | AKS | Audit Logs | Azure DevOps | Azure MonitoringLocal only candidates required | Onsite 2 days per week | Remote 3 days per weekSenior-level Full TimeAustin, TX R4h ago
-
Test Data Management Engineer MXN 950K-1300KAPI Testing | Data Generation | Data Management | Data Provisioning | Data ValidationAgile delivery focus | Continuous learning | Fully remoteSenior-level Full TimeCiudad de México, CDMX, MX R5h ago
-
IDEA Data Analyst USD 56K-72KData Governance | Data Quality | Data Validation | Data Visualization | Microsoft ExcelDental insurance | Disability insurance | Flexible spending accounts | Health insurance | Health savings accountMid-level Full TimeColorado Springs R7h ago
-
IDEA Data Analyst USD 56K-72KCU Central Information Warehouse | Campus Solutions | Canvas | LMS | Microsoft ExcelDental insurance | Health insurance | Paid Holidays | Paid parental leave | Paid time offMid-level Full TimeColorado Springs R7h ago
-
Entanglement | JSON | Qiskit | Quantum Computing | Quantum circuitsRemote workSenior-level Full TimeUnited States R13h ago
-
Machine Learning Experts: Linear Algebra Exam Audit USD 150K-180KComponent analysis | Eigenvalues | JSON | Linear Algebra | Matrix OperationsRemote workSenior-level Full TimeUnited States R13h ago
-
Quantum Computing Experts: Review for a Beginner Exam USD 139K-219KEntanglement | JSON | Logic gates | Qiskit | Quantum ComputingRemote workSenior-level Full TimeUnited States R13h ago
-
Senior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL R13h ago
-
AI tools | Agile | Dashboard Development | Data Governance | Data ModelingContinuous improvement opportunities | Cross-functional project exposure | Discount perks | Employee assistance program | Fully remote in CanadaSenior-level Full TimeCanada R15h ago
-
Software Engineer, Embedded Systems USD 197K-270KAAC | AC3 | API Design | AV1 | Adaptive streamingCommuter benefits | Disability benefits | Health insurance | Life insurance | Paid time offSenior-level Full TimeSan Jose, California R16h ago
-
AWS | Access Control | Azure | CI/CD | DBTAgile collaborative environment | Data governance exposure | Fully remote | Opportunity to work with modern cloud data technologies | Professional growthMid-level Contract Full TimeBrazil R16h ago
-
Automation | Business Intelligence | Dashboards | Data Analysis | Data ManipulationMid-level Full TimeSantiago, Chile R19h ago
-
Staff AI Engineer USD 151K-332KC++ | CUDA | CUDA kernel | CUDA kernel development | Cache ManagementHybrid work | In-person work | Remote work | Work-life balance supportSenior-level Full TimeSeattle (WA), United States R20h ago
-
Staff AI Engineer - Zoom AI Services Platform USD 177K-387KAI Inference | AI Platform | AI Platform Engineering | API Design | C++Hybrid work | Remote workSenior-level Full TimeSeattle (WA), United States R20h ago
-
Senior AI Engineer USD 113K-182KAWS | AWS CDK | Agent Orchestration | Autogen | AzureDental insurance | Employer-funded retirement | Free tuition program | Medical insurance | Mental health resourcesSenior-level Full TimeRemote (United States), United States R20h ago
-
AI Solutions Architect (LLMs & AI Agents) MXN 950K-1300KAWS | Agent Orchestration | Agent guardrails | Autogen | AzureFlexible work arrangements | Hybrid work | In-person collaboration | Inclusive workplaceSenior-level Full TimeMX446 - DXC Mexico City Lago … R20h ago
-
Data Engineer, Mid USD 77K-176KAgentic AI | Apache Spark | Artificial Intelligence | Data Mining | Data ModelingDependent care | Disability insurance | Health insurance | Life insurance | Paid leaveMid-level Full TimeUSA, CO, Aurora (3855 Lewiston St), … R20h ago
-
Scientific Data Engineer Consultant USD 134K-180KAWS | Amazon Aurora | Amazon Glue | Amazon Redshift | Data IngestionMid-level Full TimeRaleigh, United States R20h ago
-
Staff Software Engineer - Agentic AI Platform USD 165K-209KAWS | Agent systems | Agile | Apache Spark | C#Career development | Employee resource groups | Flexible WFH policy | Generous PTO policy | Paid volunteer timeSenior-level Full TimeUS-Texas-Austin, United States R20h ago
-
Senior Machine Learning Engineer USD 150K-241KC++ | Cloud Architecture | Computer Vision | Data Pipelines | Fine TuningDiscretionary paid time off | Emotional and mental wellness support | Employer 401k match | Fitness programs | Learning and development programsSenior-level Full TimeSeattle, Washington, United States R20h ago
-
Staff Data Engineer CAD 120K-144KApache Kafka | CI/CD | Data Modeling | Data Observability | Data QualityDiversity and inclusion working groups | Flexible working practices | Paid volunteer time off | Parental leaveSenior-level Full TimeCanada - Remote R21h ago
-
Senior Solution Engineer USD 165K-216KAnalytics | Cloud Computing | Data Lake | Data Warehouse | Data fabricTravelSenior-level Full TimeUS-GA-Remote R21h ago