AI Software Engineer in Test
Tasks
- Analyze codebases for test coverage gaps
- Build automation components and shared libraries
- Create automated test workflows
- Create evaluation datasets and golden answer sets
- Debug test failures using logs
- Define verification criteria and pass fail checks
- Design AI enabled application test strategies
- Establish test traceability
- Generate automated test scripts
- Identify testing risks across functional and operational areas
- Implement quality gates for builds and releases
- Integrate tests into CI/CD pipelines
- Maintain automated test frameworks
- Modify and refactor automated tests
- Perform responsible AI testing audits
- Run regression testing for prompts and model upgrades
- Translate requirements into test plans and scenarios
- Troubleshoot failing pipeline jobs
- Validate AI outputs for accuracy and policy compliance
Perks/Benefits
- 401(k) retirement benefits
- Disability insurance
- Life insurance
- Medical/Dental/Vision insurance
- Paid Holidays
- Paid sick time
- Paid time off
Skills/Tech-stack
Azure DevOps | Bias Testing | CI/CD | Cypress | Data Validation | Explainability | Fairness testing | GitLab CI | JUnit | Java | JavaScript | Jenkins | LLM | Log Analysis | Model Evaluation | Playwright | Prompt testing | Pytest | Python | RAG | REST Assured | Responsible AI | SQL | Selenium | Test automation | Test frameworks | TestNG | Traceability
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
AC DC Sizing | AC/DC | Arc Flash | Artificial Intelligence | AutomationMultiple project extensions | Paid on time | Remote work optionSenior-level Full TimeJuno Beach, FL R5h ago
-
Senior Machine Learning Engineer USD 160K-200KAWS | CI/CD | Computer Vision | Data Processing | DockerCommuter benefits | Dental insurance | Employee assistance program | Equity | Health insuranceSenior-level Full TimeArlington, VA R22h ago
-
Senior Computer Vision & Machine Learning Engineer USD 140K-206KAblation Studies | CI/CD | Data Pipelines | Docker | Error AnalysisAutonomy | Client project impact | Support for professional growthSenior-level Full TimeRemote, US R22h ago
-
Computer Vision & Machine Learning Engineer USD 150K-200KAblation Studies | CI/CD | Computer Vision | Data Pipelines | DockerEntry-level Full TimeRemote, US R22h ago
-
Sr Staff AI Engineer - Veza USD 190K-334KAgent Orchestration | Algorithms | C++ | Data Structures | Distributed Systems401k plan with match | ESPP | Family leave programs | Flexible time away | Health plansSenior-level Full TimeMinneapolis, Minnesota, United States R23h ago
-
AI/ML Engineer USD 152K-220K3D Reconstruction | 3D Scene | 3D Scene Understanding | AI Agents | Cloud ComputingAgile team collaboration | Fully remote | Production monitoring ownershipMid-level Full TimeSan Francisco, CA (Hybrid) or United … R23h ago
-
Senior GTM AI Engineer USD 154K-213KAgent Orchestration | Agent Performance | Agent Performance Monitoring | Anthropic Claude | App Development401k match | Catered lunch | Company events | Desk setup stipend | Flexible spending accountsSenior-level Full TimeOakland, California, United States R1d ago
-
Data Science Engineer USD 121K-154KData Visualization | Documentation | Entity Resolution | Fine Tuning | Git401k | Education reimbursement program | Flexible schedules | Hybrid schedule | Relocation assistanceEntry-level Full TimeLivermore, CA, United States R1d ago
-
Senior Software Engineer - Frontend USD 130K-300KAI coding | AI coding tools | Algorithms | Coding Tools | Data StructuresCareer development opportunities | Community guilds | Employee stock purchase plan | Free mental health benefit | Hybrid work environmentSenior-level Full TimeNew York, New York, USA R1d ago
-
Staff Diagnostics and Embedded Cyber Security Engineer USD 150K-283KAUTOSAR | Automotive Ethernet | BLE | C# | C++Employee resource groups | Flexible family care days | Paid parental leave | Paid time off | Subsidized back-up child careSenior-level Full TimePalo Alto, CA, United States R1d ago
-
Senior Analytics Engineer USD 153K-206KAWS | Data Quality | ETL | Healthcare Claims | PySpark401k plan | Commuter benefits | Dental insurance | Disability insurance | Flexible spending accountSenior-level Full TimeRemote- US R1d ago
-
Senior Data Engineer USD 190K-210KAccess Control | Alerting | CI/CD | DBT | Data Governance401k | Flex PTO | Learning and development reimbursement | Medical, dental, vision plans | Paid parental leaveSenior-level Full TimeRemote (US) R1d ago
-
Senior AI Analytics Engineer USD 149K-212KAI Agents | Agent Frameworks | Automation | DBT | Python401k match | Employee assistance program | HSA contribution | Health insurance | Hybrid workSenior-level Full TimePlano,Texas,United States R1d ago
-
800-37 | 800-53 | AI Governance | AI orchestration | AI red teaming401k | Dental insurance | Disability insurance | Employee assistance program | Hybrid work scheduleSenior-level Full TimeWashington, DC R1d ago
-
IAA - Principal Data Engineer USD 171K-256KAzure | Azure Synapse | Azure Synapse Analytics | CI/CD | Data GovernanceHybrid work schedule | Onsite expectation 2 days per weekSenior-level Full TimeChicago, IL (Hybrid) R1d ago
-
Senior Analytics Engineer USD 149K-195KAWS | Alerting | Azure | Cloud Data | Cloud Data PlatformsAnnual performance bonuses | Company equity grants | Hybrid work schedule | Unlimited vacationSenior-level Full TimeNew York, New York, United States R1d ago
-
Full-Stack AI Engineer USD 150K-200KAWS | Authentication | Azure | CI/CD | CSSD E And I Initiatives | Fertility and menopause leave | Fitness allowance | Health benefits | Hybrid workingMid-level Full TimeNew York R1d ago
-
Software Engineer I USD 97K-122KBatch inference | Cloud infrastructure | Docker | GPU Optimization | GoConference reimbursement | Education reimbursement | Employee assistance program | Flexible time off | Hybrid workEntry-level Full TimeSeattle R1d ago
-
Applied AI Architect, Startups USD 240K-315KAI architecture | Context engineering | Integration Patterns | LLM Evaluation | Language ModelsFlexible working hours | Generous vacation | Optional equity donation matching | Parental leaveSenior-level Full TimeSan Francisco, CA | New York … R1d ago
-
Data Scientist USD 160K-170KAPI Integration | APIs | Agentic Workflows | Anomaly Detection | Artificial Intelligence401k retirement plan | Employee assistance program | Health Savings Account (HSA) | Life insurance | Long-term disabilityMid-level ContractPhoenix, Arizona, USA R1d ago
-
Mid-level Full TimeRemote, US R1d ago
-
Data Engineer USD 153K-207KAirflow | Automated testing | CI/CD | DBT | Data GovernanceFlexible hybrid schedule | Hybrid work arrangement | Onsite Days Per WeekMid-level Full TimeSan Francisco, CA R1d ago
-
Adobe Analytics | Adtech | Attribution Modeling | Data Infrastructure | Data VisualizationAnnual Tuition Stipend | Annual learning stipend | Group RRSP contributions | Healthcare Dental Vision | Internet stipendSenior-level Full TimeNew York R1d ago
-
PME Data Engineer USD 116K-147KData Modeling | Data Validation | Data cleaning | Data integration | Data pipelineDocumentation and team knowledge sharing | Hybrid work model | Telecommuting optionsMid-level Full TimeChinook Building 401 5th Avenue Seattle R1d ago
-
AWS | Agent systems | Apache Spark | Azure | Cloud platformOn site customer collaboration | Remote work | Travel for customer visitsSenior-level Full TimeWashington, D.C. R1d ago