Lead Data Engineer
New York - 150 E 42nd, United States
USD 94K-152K Senior-level Full Time
Tasks
- Build automated validation frameworks and quality tracking pipelines
- Build data models for consumer and household identity linkage
- Build ground truth evaluation and match quality dashboards
- Continuously monitor data completeness match consistency and anomaly detection
- Create regression detection across engine versions
- Design build and maintain Snowflake pipelines for identity assets
- Design implement test and deploy via CI CD
- Document data models pipeline logic and algorithm decisions
- Enhance AI assisted matching engine feature design scoring logic and evaluation
- Implement and test matching algorithm improvements
- Investigate data anomalies and trace match decisions to source signals
- Measure match quality using precision recall and asset impact
- Perform code reviews and architectural discussions
- Support QA and on call responsibilities for production pipelines
- Translate identity requirements into reliable scalable solutions
- Write SQL and Python for transformation deduplication and enrichment
Perks/Benefits
- 401k
- Dental insurance
- Flexible paid time off
- Life insurance
- Long-term disability
- Medical insurance
- Paid Holidays
- Paid parental leave
- Paid sick leave
- Short-term disability
- Vision insurance
Skills/Tech-stack
AWS | Agile | Bash | CI/CD | Data Deduplication | Data Enrichment | Data Governance | Data Modeling | Data Quality | Data pipeline | Deterministic matching | Git | Identity matching | LLM | Machine Learning | Privacy | Probabilistic matching | Prompt engineering | Python | Record Linkage | SQL | Snowflake | Snowpark | Streams | Tasks | Unix
Education
N/A
Roles
Data Engineer | Data Engineering | Data Engineering Manager | Engineer | Engineering Manager | Lead | Lead Data | Lead Data Engineer | Manager
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R12d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R25d ago
-
Mid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL2h ago
-
Senior-level ContractRaleigh, United States3h ago
-
AI Agents | AI Governance | Artificial Intelligence | Cybersecurity compliance | Enterprise SearchHybrid work | Local residency requirementEntry-level ContractAustin, United States3h ago
-
SYSTEM ENGINEER - Computer Network Support - AI/ML - 6+ yrs of Experience - TS/SCI w/Poly clearance is required - ES - 032726-2 A USD 136K-140KAI/ML | Agile | Confluence | Jira | LLM401k retirement plan | Dental insurance | Disability insurance | Federal Holidays | Floating holidaysMid-level Full TimeFort George G Meade, United States4h ago
-
Senior Manager, Data Science USD 165K-256KA/B | A/B Testing | Ad Spend | Artificial Intelligence | Attribution401k matching | Disability insurance | Life insurance | Medical/Dental/Vision insurance | Paid Company HolidaysSenior-level Full TimeLocations: Redwood City, California, United States …4h ago
-
AI experimentation | Alerting | Artificial Intelligence | Data Annotation | Data ArchitectureSenior-level Full TimeNew York4h ago
-
Senior Software Engineer (C++/DS/Algorithm) USD 119K-164KAlgorithms | C# | C++ | Compiler concepts | Compiler optimizationsHealth and wellness benefits | Hybrid work modeSenior-level Full TimeAustin - Texas - United States …4h ago
-
Project GenAI Engineer III USD 113K-188KAWS | Bedrock | Generative AI | Machine LearningMentorship | Onsite work | Professional development | Travel up to 25%Senior-level Full TimeArlington/Rosslyn, Virginia, United States4h ago
-
AWS | Agentic AI | Agile | Azure | Business IntelligenceDiscretionary incentive program | Hybrid work model | Mentorship | Professional developmentSenior-level Full TimeAustin, Texas, United States; Dallas, Texas, …4h ago
-
AWS | Artificial Intelligence | Cloud platform | Data Architecture | Data GovernanceTravel 10 to 15 percentSenior-level Full TimeArlington/Rosslyn, Virginia, United States; Washington, District …4h ago
-
Project Lead Data Engineer III USD 119K-198KBTEQ | Database Administration | Patch Management | Performance Tuning | Query OptimizationSenior-level Full TimeSacramento, California, United States4h ago
-
Senior Manager, Data Science USD 142K-222KA/B | A/B Testing | Artificial Intelligence | B testing | Channel attributionCommunity health | Free games | Hybrid work schedule | Medical insurance | Mental health supportSenior-level Full Time地点：Redwood City, California, United States of …5h ago
-
Machine Learning Engineer / AI Model Developer (Mid to Senior)(Top Secret Clearance Required) (Hybrid) USD 77K-176KAPI Development | Agile | Backend Development | Cloud Computing | Data Preparation401k | Employee discounts | Employee referral rewards | Flexible work schedule | Health savings accountMid-level Full TimeQuantico, VA, US R6h ago
-
Machine Learning Engineer / AI Model Developer (Mid to Senior)(Top Secret Clearance Required) (Hybrid) USD 78K-176KAPIs | Agile methodologies | Backend Development | Cloud Computing | Data Preparation401k | Employee discount program | Employee referral rewards | Flexible spending account | Flexible work scheduleMid-level Full TimeFort Belvoir, VA, US R6h ago
-
Computer Vision Engineer USD 170K-251K3D Modeling | AR/VR | Computer Graphics | Computer Vision | Data AnalysisSenior-level Full TimeBurlingame, CA6h ago
-
Software Engineer, Systems ML Engineering USD 170K-251KAlerting | Benchmarking | C++ | CUDA | DashboardingSenior-level Full TimeSunnyvale, CA | Bellevue, WA | …6h ago
-
AI/HPC Network Performance Engineer USD 147K-226KAI Training | Alerting | Auto-remediation | C++ | Configuration ManagementOncall rotationSenior-level Full TimeMenlo Park, CA6h ago
-
Forward Deployed Engineer III, Google Cloud, Applied AI USD 174K-253KAPI Integration | Agent systems | Agentic Workflows | Chatbots | Cloud platformBenefits | Bonus | Equity | Travel up to 50 percentSenior-level Full TimeSan Francisco, CA, USA; Atlanta, GA, …6h ago
-
Product Manager, CTM Intelligence USD 192K-279KAPI Design | Agentic AI | Classifier Models | Intent detection | LLM orchestrationSenior-level Full TimeSunnyvale, CA, USA6h ago
-
Research Software Engineer, Multimodal AI USD 174K-253KAgent Orchestration | Audio | C++ | Few-Shot Learning | Few-shotMid-level Full TimeSan Jose, CA, USA7h ago
-
Product Manager, Responsible AI Infra USD 163K-237KAI Evaluation | Artificial Intelligence | Automated compliance | Compliance | Data GovernanceMid-level Full TimeMountain View, CA, USA7h ago
-
ML Engineer, GenAI Ads, Search Personalization USD 207K-301KC++ | Clustering Algorithms | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA7h ago
-
Software Engineering Manager II, AI/ML, Google Cloud USD 207K-301KC# | C++ | Cloud platform | Data Processing | DebuggingSenior-level Full TimeSunnyvale, CA, USA7h ago