Senior Quality Engineer, Data & AI Platform
Lawrenceville, Georgia, United States
USD 125K-150K (estimate) Senior-level Full Time
Tasks
- Apply statistical testing to pipeline output validation
- Build regression suites for concept drift label drift and feature degradation
- Coordinate builds deployments and migrations for releases and hotfixes
- Create prompt regression tests for model and prompt changes
- Define acceptance thresholds for precision recall and F1
- Define and instrument production monitoring for model performance
- Design and automate data contract tests for schema and data quality
- Design and execute test strategies for AI ML features
- Develop and maintain automated test scripts using Katalon and Playwright
- Evaluate LLM outputs for accuracy hallucination and format compliance
- Implement automated validation checkpoints in Databricks pipelines
- Maintain testing tools and automation frameworks
- Perform functional regression integration and exploratory testing
- Review logs to identify defects
- Support CI/CD quality gates
- Test ETL processes data warehouse workflows and API validation
- Validate AI generated outputs against acceptance criteria
- Write data validation logic in Python or SQL
Perks/Benefits
- N/A
Skills/Tech-stack
Agile | Apache Kafka | Azure DevOps | CI/CD | Concept drift | DBT | Data Contracts | Data Drift | Databricks | Delta Lake | ETL | Great Expectations | Katalon | LLM Evaluation | Label Drift | MLflow | Machine Learning | Microsoft SQL | Microsoft SQL Server | Monitoring | Null Checks | Playwright | Postman | Prompt Regression | Prompt Regression Testing | Python | Referential Integrity | Regression testing | Row Counts | SOAPUI | SQL | SQL Server | Schema Validation | Scrum | Statistical Testing | Swagger
Education
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R11d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R24d ago
-
Sr. Embedded Software/Firmware Engineer USD 180K-240KARM | ASPICE | AUTOSAR | C# | C++Competitive compensation | Dental insurance | Flexible hours | Medical insurance | Pre-IPO stock optionsSenior-level Full TimeUnited States - Remote R7h ago
-
Sr. Software Engineer, AI Infrastructure USD 144K-236KApache Beam | Apache Flink | Apache Spark | C# | C++Career development | Hybrid work | Stock grantsSenior-level Full TimeSunnyvale, CA, United States8h ago
-
Google Cloud Platform (GCP) Data Engineer USD 100K-203KApache Beam | BigQuery | Broadcast joins | CI/CD | CachingMid-level Full TimeArlington, VA8h ago
-
Google Cloud Platform (GCP) Data Engineer USD 100K-203KApache Beam | BigQuery | CI/CD | Cloud Storage | Cloud platformSecurity clearance providedMid-level Full TimeArlington, VA8h ago
-
AI/ML Engineer USD 91K-184KAutomated Machine Learning | Continuous Deployment | Continuous integration | Data Pipelines | MLOpsMid-level Full TimeArlington, VA9h ago
-
Senior Data Engineer USD 144K-187KAWS Lambda | Amazon DynamoDB | Amazon ECS | Amazon EKS | Amazon S3401k employer contribution | Dental insurance | Flexible Remote Friendly Work Options | Health Savings Account employer contribution | Health insuranceSenior-level Full TimeAustin, Texas11h ago
-
Senior Data Engineer - Finance USD 172K-203KARR | AWS | Apache Iceberg | Apache Spark | CRMDental insurance | Employee discounts | Flexible wellness time off | Health insurance | Hybrid workSenior-level Full TimeHybrid - San Francisco, California R11h ago
-
Sr. Data Engineer (HYBRID) USD 145K-160KAWS | AWS Glue | AWS Lambda | Amazon Athena | Amazon DynamoDBComplimentary club membership | Hybrid work schedule | Personal training | Pilates | Shop discountsSenior-level Full TimeNew York, NY, United States R12h ago
-
Staff Data Engineer - Data Engineering USD 207K-275KAccess Control | Apache Fluss | Apache Hudi | Apache Iceberg | Apache Paimon401k matching | Dental insurance | Disability insurance | Employee stock purchase program | Flexible PTOSenior-level Full TimeLivingston, NJ / New York, NY …12h ago
-
Cheminformatics | Cloud Computing | Data Analysis | Data Visualization | Deep learningSenior-level Full TimeRedwood City, California, United States12h ago
-
AI Developer USD 99K-198KAWS | Agentic AI | Azure | CI/CD | Data LakesHybrid work schedule | Leadership development | Professional developmentMid-level Full TimeWashington, DC12h ago
-
AI Engineer I (Contractor) - 40526R0024893 USD 130K-183KAI Agents | AI Inventory | AI monitoring | Documentation | Enterprise SearchHybrid work model | Work standard business hoursEntry-level ContractAustin, TX, United States13h ago
-
Senior-level Full TimeOrlando, FL, United States13h ago
-
AI Requirements Engineer USD 120K-145KAWS | Acceptance criteria | Agile | Analytics | Artificial IntelligenceMid-level Full TimeArlington, VA, United States13h ago
-
Algorithms | Concurrency | Data Structures | Database systems | Distributed SystemsCompany-matched 401k | Flexible schedule | Health coverage | Parental leave | Remote work flexibilitySenior-level Full TimeUnited States R14h ago
-
Data Analytics & Engineering Opportunities USD 65K-105KBusiness Intelligence | Microstrategy | Python | R | SQL401k match | Dental insurance | Disability insurance | Firm paid holidays | Health insuranceEntry-level Full TimeWashington, DC, United States14h ago
-
Artificial Intelligence | C plus plus | CUDA | Camera Calibration | Computer VisionVisa sponsorship not availableMid-level Full TimeKenner, LA, United States16h ago
-
Solutions Engineer USD 95K-130KAgile | Audit controls | Bug Fixing | Dashboards | Data LineageOnsite positionMid-level Full TimeNew Albany, OH, United States17h ago
-
Senior-level Full TimeSouthfield, Michigan, USA, 4807617h ago
-
Reindeer- Software Engineer (Infrastructure) USD 180K-250KAPI Integration | AWS | Authentication | Authorization | AzureSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL17h ago
-
Reindeer- AI Solutions Engineer (IL) USD 80K-170KApplication Integration | Cloud Computing | Communication | Compliance | Data AnalysisSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL18h ago
-
Agent systems | Alignment Training | C++ | CoT prompting | Computer VisionSenior-level Full TimeSeattle, Washington, United States18h ago
-
C++ | DPO | GNN | Graph Neural Networks | Heterogeneous GraphsEntry-level Full TimeSeattle, Washington, United States18h ago