AI Automation QA
Taguig, National Capital, Philippines
PHP 250K-250K (estimate) Senior-level Full Time
Tasks
- Align testing logging and monitoring with privacy audit and regulatory requirements
- Apply LLM evaluation frameworks
- Build automated evaluation harnesses for agent performance
- Collaborate with developers to improve prompts and workflow design
- Define quality criteria for accuracy latency safety compliance
- Define testing and quality assurance strategies for AI workflows
- Develop automated test suites with Playwright and custom scripts
- Develop runbooks for common failure modes
- Implement continuous production monitoring for anomalies
- Produce quality dashboards and reports
Perks/Benefits
- N/A
Skills/Tech-stack
API Testing | Data Analysis | DeepEval | Evidently | LLM Evaluation | Playwright | Production Monitoring | Python | Ragas | Regression testing | Safety Compliance | Test automation | UI Testing
Education
N/A
Related jobs
-
Azure Blob | Azure Blob Storage | Azure Cloud | Azure Cloud Security | Azure DataSenior-level Full TimeMakati City, Metro Manila, Philippines R23h ago
-
ADLS | Azure Data | Azure Data Factory | Azure Databricks | Azure DevOpsHybrid workSenior-level Full TimeMakati City, Metro Manila, Philippines R23h ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksMid-level Full TimeMakati City, Metro Manila, Philippines R23h ago
-
Mid-level Full TimePhilippines - Quezon City - E. …23h ago
-
Junior AI Engineer PHP 300K-420KAPI Integration | AWS | Artificial Intelligence | Azure | Cloud ComputingEntry-level Full TimePhilippines - Quezon City - E. …23h ago
-
AWS | CI/CD | Continuous Monitoring | DevSecOps | GitNetworking opportunities | Training and developmentEntry-level Full TimeAsia1d ago
-
Data & Analytics Engineer PHP 540K-576KAzure Data | Azure Data Factory | CDS Views | Data Factory | Data ModelingOnsite work setupMid-level Full TimeSoutheast Asia, Philippines1d ago
-
Azure Engineer (Databricks) PHP 4375K-4375KAgile | Azure Data | Azure Data Factory | Azure Data Lake | Azure DatabricksSenior-level Full TimeSoutheast Asia, Philippines1d ago
-
Ansible | Apache Spark | Avro | Azure Blob | Azure Blob StorageHybrid work | Night shift | WFHMid-level Full TimeMakati City, Metro Manila, Philippines R1d ago
-
ADLS | AWS Glue | AWS Lambda | AWS S3 | Amazon RedshiftHealth insurance | Hybrid work setup | Life insurance | Mentoring and coaching | Paid time offMid-level Full TimeManila, PH1d ago
-
Senior Data Engineer PHP 1200K-1440KAWS | Advanced SQL | Amazon Aurora | Amazon CloudWatch | Amazon ECS13th month pay | HMO coverage | Hybrid work | Monthly miscellaneous allowance | SSS Pag Ibig Philhealth contributionsSenior-level Full TimeMandaluyong, Metro Manila, Philippines2d ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous work culture | Flexible, hands-off management | Friendly laid-back atmosphere | Impact on learning accessibilityMid-level Full TimeCebu, Philippines2d ago
-
Data Engineer PHP 420K-480KAPI Development | AWS | Amazon Web Services | Apache Airflow | Apache SparkGlobal Training Network | Health insurance | Mentorship | Sports discount | Sports voucherMid-level Full TimeAlabang, Philippines (Hybrid) R4d ago
-
Data Engineer PHP 216K-216KAgile | Amazon Redshift | Amazon Web Services | Big Data | Data ModelingFully onsiteMid-level Full TimeOrtigas Center, Metro Manila, Philippines4d ago
-
AI Developer PHP 150K-200KLangchain | Language Models | Large Language Models | MySQL | PostgreSQLInternational team collaborationEntry-level Full TimeMonterrey, Monterrey, Mexico5d ago
-
Alerting | Azure Data | Azure Data Factory | Azure Fabric | Azure SynapseAccidental death disability cover | Company share plan | Day 1 HMO healthcare coverage | Discounted share purchase | Employee assistance programMid-level Full TimePHL Manila5d ago
-
Command Line | Command-line Interface | Data Pipelines | Data Quality | ELTDay shift schedule | Hybrid work modelSenior-level Full TimeTaguig City R5d ago
-
API Development | AWS | Agile | Azure | Azure DataHMO on day 1 | Hybrid work | International work from another location up to 30 days per year | Two free dependent coverageSenior-level Full TimePasig, PH6d ago
-
AWS | Agile | Azure Data | Azure Data Factory | Azure Data WarehouseHMO Day 1 with dependent coverage | Work from another location up to 30 days per yearMid-level Full TimePasig, PH6d ago
-
Agentic AI | Bias Mitigation | Data Ingestion | Data Modeling | Data TransformationMid-level Full TimeLOC5028: MSSC, Boulevard Díaz Ordaz 130, …6d ago
-
Agile | Azure | Azure Data | Azure Data Factory | Azure Data WarehouseHMO | Work from another locationMid-level Full TimePasig, PH6d ago
-
API Development | Agile | Azure | Azure Data | Azure Data Factory30 days work from another location | HMO on day 1 | Hybrid workSenior-level Full TimePasig, PH6d ago
-
API Orchestration | AWS | Agent workflows | Anthropic API | Asynchronous processingSenior-level Full TimeManila, Metro Manila, Philippines8d ago
-
Senior Data Engineer / Analyst PHP 600K-1080KAWS | Data Analysis | Data Management | Data Modeling | Data VisualizationCompany events | HMO coverage | Health and safety focus | Life insurance | Recognition programsSenior-level Full TimePhilippines, PH - Manila8d ago
-
Mid-level Full TimeManila (One Ayala Tower 2), Philippines8d ago