Senior Quality Engineer, Data & AI Platform
Lawrenceville, Georgia, United States
USD 125K-150K (estimate) Senior-level Full Time
Tasks
- Apply statistical testing to pipeline output validation
- Build regression suites for concept drift label drift and feature degradation
- Coordinate builds deployments and migrations for releases and hotfixes
- Create prompt regression tests for model and prompt changes
- Define acceptance thresholds for precision recall and F1
- Define and instrument production monitoring for model performance
- Design and automate data contract tests for schema and data quality
- Design and execute test strategies for AI ML features
- Develop and maintain automated test scripts using Katalon and Playwright
- Evaluate LLM outputs for accuracy hallucination and format compliance
- Implement automated validation checkpoints in Databricks pipelines
- Maintain testing tools and automation frameworks
- Perform functional regression integration and exploratory testing
- Review logs to identify defects
- Support CI/CD quality gates
- Test ETL processes data warehouse workflows and API validation
- Validate AI generated outputs against acceptance criteria
- Write data validation logic in Python or SQL
Perks/Benefits
- N/A
Skills/Tech-stack
Agile | Apache Kafka | Azure DevOps | CI/CD | Concept drift | DBT | Data Contracts | Data Drift | Databricks | Delta Lake | ETL | Great Expectations | Katalon | LLM Evaluation | Label Drift | MLflow | Machine Learning | Microsoft SQL | Microsoft SQL Server | Monitoring | Null Checks | Playwright | Postman | Prompt Regression | Prompt Regression Testing | Python | Referential Integrity | Regression testing | Row Counts | SOAPUI | SQL | SQL Server | Schema Validation | Scrum | Statistical Testing | Swagger
Education
Roles
Related jobs
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R1d ago
-
Machine Learning Engineer V USD 231K-382KAWS | Agent Orchestration | Automated testing | Azure | CI/CDBonus eligibility | Disability insurance | Life insurance | Paid parental leave | Paid time offSenior-level Full TimeRemote, United States R16h ago
-
SAP Analytics & Reporting SME — Alteryx USD 101K-142KABAP CDS | Alteryx Designer | Alteryx Server | Analytics Cloud | Analytics documentationDoD clearance eligibility | Training and enablement supportMid-level Full TimeScott Air Force Base, United States20h ago
-
Cloud Security | Cloud infrastructure | IT Operations | Linux | Network Automation401k | Dental insurance | Disability coverage | Life insurance | Medical insuranceSenior-level Full TimeLaurel, United States20h ago
-
Mid-level Full TimeUS-NY-New York21h ago
-
Analytics & Insights Engineer II (Healthcare EDI) USD 97K-171KData Governance | Data Quality | Data integration | Data interchange | ETLMid-level Full TimeNashville, Tennessee, United States21h ago
-
Databricks Engineer USD 97K-162KAPI | Cloud Data | Cloud Data Platforms | Data Governance | Data ModelingMid-level Full TimeArlington/Rosslyn, Virginia, United States; Baltimore, Maryland, …21h ago
-
Databricks Engineer USD 126K-171KAPIs | Agile | Apache Spark | Data Governance | Data ModelingTravel 10 to 15 percentMid-level Full TimeGilbert, Arizona, United States; Lake Mary, …21h ago
-
Computer Vision Engineer, Reality Labs USD 147K-208K3D Reconstruction | C++ | Camera Calibration | Camera Pose Estimation | Computer VisionSenior-level Full TimeRedmond, WA22h ago
-
Senior Software Machine Learning Engineer, DeepMind USD 174K-253K2D Games | 3D Games | AI Agents | C++ | Generative AISenior-level Full TimeMountain View, CA, USA23h ago
-
Staff Software Engineer, AI/ML GenAI, Google Cloud USD 207K-301KComputer Vision | Data Preparation | Data Processing | Data Storage | DebuggingSenior-level Full TimeSunnyvale, CA, USA23h ago
-
Forward Deployed Engineer, GenAI, YouTube GTM Operations USD 128K-185KAgentic Workflows | Cloud Computing | Containerization | DevOps | EmbeddingsEquity | Health insurance | Paid time off | Retirement planMid-level Full TimeNew York, NY, USA; Mountain View, …23h ago
-
Senior Software Engineer, Google Cloud Storage USD 174K-253KAccess Control | As-a-Service | C++ | Chaos Testing | Cloud FunctionsSenior-level Full TimeRaleigh, NC, USA; Durham, NC, USA23h ago
-
AI Research Lead USD 96K-104KAPIs | AWS | Cloud infrastructure | Code review | Copyright ProtectionCompany paid life insurance | Medical, dental & vision coverage | Mental well-being resources | Paid parental leave | Paid sick daysSenior-level Full TimeNew York, NY, US, 102811d ago
-
Analytics Engineer USD 100K-160KAWS | Analytics engineering | Cohorting | DBT | Data ClassificationFlexible time off | Medical, dental, vision benefits | Paid Holidays | Paid parental leaveMid-level Full TimeUSA1d ago
-
Supply Chain Master Data Analyst USD 129K-171KData Analysis | Data Auditing | Data Dictionary | Data Governance | Data ManagementMid-level Full TimeCosta Mesa, California, United States1d ago
-
AI Engineer USD 149K-184KAPI Integration | EHR Integration | Evaluation | Language Models | Large Language ModelsSenior-level Full TimeNew York, NY1d ago
-
Senior AI Engineer USD 182K-220KAgent workflows | Arize Phoenix | Compliance | Cost Optimization | EmbeddingsSenior-level Full TimeNew York, NY1d ago
-
AI Engineer USD 78K-82KAPI Integration | CI/CD | Containerization | Csharp | Function Calling401k match | Flexible spending account | Health savings account | Legal ID Shield | Life insuranceEntry-level Full TimeTroy, MI, US1d ago
-
AWS | Alerting | Black box monitoring | Black-box | CI/CDBackup childcare | Financial coaching | Mental health support | Mentoring | Onsite health and wellness centersSenior-level Full TimeJersey City, NJ, United States1d ago
-
Senior Embedded Engineer USD 165K-218KADC | AXI | AXI Stream Interface | AXI-Stream | Board SupportHealth benefits | On-site benefits | Recovery Benefits | U S Secret Clearance eligibilitySenior-level Full TimeHudson, New Hampshire, United States1d ago
-
Embedded Software Engineer USD 166K-220KARM | Board Design | Bootloader | C plus plus | C#Dental insurance | Health insurance | Paid time off | Recovery Benefits | Security clearance sponsorshipMid-level Full TimeLexington, Massachusetts, United States1d ago
-
Data Engineer USD 101K-224KApache Kafka | Apache Spark | Data Governance | Data Lineage | Data ModelingConstant learning | Flexible working models | Health and well-being | Skill growthSenior-level Full TimeBellevue, WA, US, 980041d ago
-
Machine Learning Engineer USD 101K-224KAgentic Workflows | Embeddings | Fine Tuning | Hugging Face | LLM orchestrationSenior-level Full TimeBellevue, WA, US, 980041d ago
-
Machine Learning Operations Engineer (MLOps) USD 101K-224KAlerting | Azure | CI/CD | Docker | Inference OptimizationSenior-level Full TimeBellevue, WA, US, 980041d ago