Senior Quality Engineer, Data & AI Platform
Lawrenceville, Georgia, United States
USD 125K-150K (estimate) Senior-level Full Time
Tasks
- Apply statistical testing to pipeline output validation
- Build regression suites for concept drift label drift and feature degradation
- Coordinate builds deployments and migrations for releases and hotfixes
- Create prompt regression tests for model and prompt changes
- Define acceptance thresholds for precision recall and F1
- Define and instrument production monitoring for model performance
- Design and automate data contract tests for schema and data quality
- Design and execute test strategies for AI ML features
- Develop and maintain automated test scripts using Katalon and Playwright
- Evaluate LLM outputs for accuracy hallucination and format compliance
- Implement automated validation checkpoints in Databricks pipelines
- Maintain testing tools and automation frameworks
- Perform functional regression integration and exploratory testing
- Review logs to identify defects
- Support CI/CD quality gates
- Test ETL processes data warehouse workflows and API validation
- Validate AI generated outputs against acceptance criteria
- Write data validation logic in Python or SQL
Perks/Benefits
- N/A
Skills/Tech-stack
Agile | Apache Kafka | Azure DevOps | CI/CD | Concept drift | DBT | Data Contracts | Data Drift | Databricks | Delta Lake | ETL | Great Expectations | Katalon | LLM Evaluation | Label Drift | MLflow | Machine Learning | Microsoft SQL | Microsoft SQL Server | Monitoring | Null Checks | Playwright | Postman | Prompt Regression | Prompt Regression Testing | Python | Referential Integrity | Regression testing | Row Counts | SOAPUI | SQL | SQL Server | Schema Validation | Scrum | Statistical Testing | Swagger
Education
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R11d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R24d ago
-
Senior-level Full TimeOrlando, FL, United States3h ago
-
AI Requirements Engineer USD 120K-145KAWS | Acceptance criteria | Agile | Analytics | Artificial IntelligenceMid-level Full TimeArlington, VA, United States3h ago
-
Data Analytics & Engineering Opportunities USD 65K-105KBusiness Intelligence | Microstrategy | Python | R | SQL401k match | Dental insurance | Disability insurance | Firm paid holidays | Health insuranceEntry-level Full TimeWashington, DC, United States3h ago
-
Artificial Intelligence | C plus plus | CUDA | Camera Calibration | Computer VisionVisa sponsorship not availableMid-level Full TimeKenner, LA, United States5h ago
-
Solutions Engineer USD 95K-130KAgile | Audit controls | Bug Fixing | Dashboards | Data LineageOnsite positionMid-level Full TimeNew Albany, OH, United States6h ago
-
Reindeer- Software Engineer (Infrastructure) USD 180K-250KAPI Integration | AWS | Authentication | Authorization | AzureSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL7h ago
-
Reindeer- AI Solutions Engineer (IL) USD 80K-170KApplication Integration | Cloud Computing | Communication | Compliance | Data AnalysisSenior-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL7h ago
-
Agent systems | Alignment Training | C++ | CoT prompting | Computer VisionSenior-level Full TimeSeattle, Washington, United States8h ago
-
C++ | DPO | GNN | Graph Neural Networks | Heterogeneous GraphsEntry-level Full TimeSeattle, Washington, United States8h ago
-
Forward Deployed Engineer IV, Applied AI, Google Cloud USD 207K-301KAPI Integration | Agent systems | Cloud platform | Conversational AI | DebuggingTravel up to 50%Senior-level Full TimeSan Francisco, CA, USA; Austin, TX, …9h ago
-
AI Inference | CUDA | Compilers | Computer Vision | Embedded SoftwareSenior-level Full TimeMountain View, CA, USA9h ago
-
Senior Software Engineer, AI/ML, Search Ecosystems USD 174K-253KAlgorithms | C++ | Data Processing | Data Structures | Data debuggingSenior-level Full TimeMountain View, CA, USA9h ago
-
C++ | Data Processing | Debugging | Distributed Computing | Machine LearningSenior-level Full TimeSunnyvale, CA, USA; New York, NY, …9h ago
-
Deep learning | Isaac Sim | Jetson | Language Models | Large Language ModelsSenior-level Full TimeMountain View, CA, USA9h ago
-
Artificial Intelligence | Data Engineering | Data Pipelines | Deployment | Generative AISenior-level Full TimeMountain View, CA, USA9h ago
-
Senior Lead Data Architect USD 170K-205KAI-assisted validation | Access Controls | Agent Toolchains | Application development | Artificial IntelligenceBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimePlano, TX, United States13h ago
-
Data Analytics Architect USD 136K-220KArtificial Intelligence | Business Intelligence | Data Engineering | Data Governance | Data LakeHybrid work | One day per week remoteSenior-level Full TimeCenter, Center District, IL14h ago
-
Principal Data Engineer USD 129K-180KAdobe Analytics | Adobe XD | Amazon Web Services | Bitbucket | CI/CDRelocation Support N/A | Remote work permittedSenior-level Full TimeUnited States R16h ago
-
Mid-level Full TimeRichardson, TX, United States17h ago
-
ASIL | Admittance control | Auto Docking | BLDC | C++Mid-level Full TimeIrvine, CA18h ago
-
Product Design & HRI Engineer, Robotics Hardware USD 119K-300K3D Printing | Acoustic Performance Evaluation | Audio Systems | CAD | CESenior-level Full TimeIrvine, CA18h ago
-
Analytics Engineer USD 118K-177KAWS | Airflow | Apache Spark | Census | DBT401k match | Dental insurance | Health insurance | Life insurance | Tuition reimbursementSenior-level Full TimeNewton Office , United States20h ago
-
Lead AI Solutions Systems Engineer USD 142K-188KAPI Design | Agentic AI | Artificial Intelligence | Bias detection | CI/CDSenior-level Full TimeHerndon, VA, United States20h ago