Lead Data Engineer
New York - 150 E 42nd, United States
USD 94K-152K Senior-level Full Time
Tasks
- Build automated validation frameworks and quality tracking pipelines
- Build data models for consumer and household identity linkage
- Build ground truth evaluation and match quality dashboards
- Continuously monitor data completeness match consistency and anomaly detection
- Create regression detection across engine versions
- Design build and maintain Snowflake pipelines for identity assets
- Design implement test and deploy via CI CD
- Document data models pipeline logic and algorithm decisions
- Enhance AI assisted matching engine feature design scoring logic and evaluation
- Implement and test matching algorithm improvements
- Investigate data anomalies and trace match decisions to source signals
- Measure match quality using precision recall and asset impact
- Perform code reviews and architectural discussions
- Support QA and on call responsibilities for production pipelines
- Translate identity requirements into reliable scalable solutions
- Write SQL and Python for transformation deduplication and enrichment
Perks/Benefits
- 401k
- Dental insurance
- Flexible paid time off
- Life insurance
- Long-term disability
- Medical insurance
- Paid Holidays
- Paid parental leave
- Paid sick leave
- Short-term disability
- Vision insurance
Skills/Tech-stack
AWS | Agile | Bash | CI/CD | Data Deduplication | Data Enrichment | Data Governance | Data Modeling | Data Quality | Data pipeline | Deterministic matching | Git | Identity matching | LLM | Machine Learning | Privacy | Probabilistic matching | Prompt engineering | Python | Record Linkage | SQL | Snowflake | Snowpark | Streams | Tasks | Unix
Education
N/A
Roles
Data Engineer | Data Engineering | Data Engineering Manager | Engineer | Engineering Manager | Lead | Lead Data | Lead Data Engineer | Manager
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R11d ago
-
Featured Feat. Associate Director, Data Labs USD 167K-167KAWS | Cloud Computing | Compute Infrastructure | Data Analysis | LLM GovernanceConference speaking opportunities | Hybrid work schedule | Media appearancesSenior-level Full TimeWashington, District of Columbia, 20004, United … R24d ago
-
Computer Vision Engineer USD 170K-251K3D Modeling | AR/VR | Computer Graphics | Computer Vision | Data AnalysisSenior-level Full TimeBurlingame, CA2h ago
-
Software Engineer, Systems ML Engineering USD 170K-251KAlerting | Benchmarking | C++ | CUDA | DashboardingSenior-level Full TimeSunnyvale, CA | Bellevue, WA | …2h ago
-
AI/HPC Network Performance Engineer USD 147K-226KAI Training | Alerting | Auto-remediation | C++ | Configuration ManagementOncall rotationSenior-level Full TimeMenlo Park, CA2h ago
-
Forward Deployed Engineer III, Google Cloud, Applied AI USD 174K-253KAPI Integration | Agent systems | Agentic Workflows | Chatbots | Cloud platformBenefits | Bonus | Equity | Travel up to 50 percentSenior-level Full TimeSan Francisco, CA, USA; Atlanta, GA, …2h ago
-
Product Manager, CTM Intelligence USD 192K-279KAPI Design | Agentic AI | Classifier Models | Intent detection | LLM orchestrationSenior-level Full TimeSunnyvale, CA, USA2h ago
-
Research Software Engineer, Multimodal AI USD 174K-253KAgent Orchestration | Audio | C++ | Few-Shot Learning | Few-shotMid-level Full TimeSan Jose, CA, USA2h ago
-
Product Manager, Responsible AI Infra USD 163K-237KAI Evaluation | Artificial Intelligence | Automated compliance | Compliance | Data GovernanceMid-level Full TimeMountain View, CA, USA2h ago
-
ML Engineer, GenAI Ads, Search Personalization USD 207K-301KC++ | Clustering Algorithms | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeMountain View, CA, USA2h ago
-
Software Engineering Manager II, AI/ML, Google Cloud USD 207K-301KC# | C++ | Cloud platform | Data Processing | DebuggingSenior-level Full TimeSunnyvale, CA, USA2h ago
-
Senior Software Engineer, AI/ML, Geo and Gemini App USD 174K-253KA/B | A/B Testing | B testing | C++ | Data AnalysisSenior-level Full TimeNew York, NY, USA2h ago
-
Customer Engineer, Data/Analytics, Google Public Sector USD 152K-222KApache Spark | Batch Processing | Big Data | Cloud Architecture | Data GovernanceBonus target | Equity | Health benefits | Paid time off | Retirement benefitsSenior-level Full TimeReston, VA, USA; Atlanta, GA, USA2h ago
-
Senior Software Engineer, AI/ML, Ads Training USD 174K-253KC++ | Distributed Systems | GPU | JAX | Machine LearningSenior-level Full TimeMountain View, CA, USA2h ago
-
Principal Machine Learning Engineer, Content Safety USD 295K-345KComputer Vision | Content Moderation | Data Pipelines | Deep learning | Language ModelsEquity compensationSenior-level Full TimeSan Mateo, CA, United States R6h ago
-
Software Engineer, Product USD 208K-269KAgentic Frameworks | Anthropic | Claude Code | Cursor | DocumentationDental insurance | Health insurance | Paid time off | Retirement plan | Vision insuranceMid-level Full TimeSan Francisco, CA7h ago
-
Sr. Manager Data Engineering USD 125K-188KAutomated testing | Bitbucket | CI/CD | Cloud Functions | Cloud RunSenior-level Full TimeSouthlake, TX, United States9h ago
-
Senior Data Science Manager USD 160K-200KCausal Inference | Coded Quality | Data Pipelines | Data Visualization | Experiment designSenior-level Full TimeCupertino11h ago
-
Data Analysis | Data Visualization | GenAI | Java | Jupyter NotebookSenior-level Full TimeCupertino12h ago
-
Customer Analytics Manager USD 107K-154KBigQuery | Cross-sell | Cross-sell modeling | Customer Health | Customer health scoringMid-level Full TimeRemote - USA R12h ago
-
Forward Deployed AI Engineer USD 180K-230KAPI | AWS | Azure | CI/CD | Cloud platform5-day workweek | Collaborative work culture | Flexible working hours | Supportive work environmentSenior-level Full TimeNew York, New York, United States12h ago
-
Principal AI Engineer USD 200K-245KAgent-based | Agent-based architecture | Amazon Web Services | Azure | CI/CD5 days per week | Collaborative culture | Flexible working hours | Supportive work environmentSenior-level Full TimeNew York, New York, United States12h ago
-
Machine Learning Engineer, Sensor Pipelines USD 175K-215KC++ | Computer Vision | JAX | Machine Learning | PyTorchHybrid work scheduleMid-level Full TimeMountain View, CA, USA13h ago
-
Sr. Embedded Software/Firmware Engineer USD 180K-240KARM | ASPICE | AUTOSAR | C# | C++Competitive compensation | Dental insurance | Flexible hours | Medical insurance | Pre-IPO stock optionsSenior-level Full TimeUnited States - Remote R13h ago
-
AI Engineer USD 105K-115KAPI Cost Modeling | API Integration | API Wrappers | Anthropic API | Conversation DesignCalm health employee membership | Company social events | Comprehensive benefits package | Employee referral incentive program | Employee resource groupsEntry-level Full TimeCincinnati, OH, US13h ago