Lead Data Engineer
New York - 150 E 42nd, United States
USD 94K-152K Senior-level Full Time
Tasks
- Build automated validation frameworks and quality tracking pipelines
- Build data models for consumer and household identity linkage
- Build ground truth evaluation and match quality dashboards
- Continuously monitor data completeness match consistency and anomaly detection
- Create regression detection across engine versions
- Design build and maintain Snowflake pipelines for identity assets
- Design implement test and deploy via CI CD
- Document data models pipeline logic and algorithm decisions
- Enhance AI assisted matching engine feature design scoring logic and evaluation
- Implement and test matching algorithm improvements
- Investigate data anomalies and trace match decisions to source signals
- Measure match quality using precision recall and asset impact
- Perform code reviews and architectural discussions
- Support QA and on call responsibilities for production pipelines
- Translate identity requirements into reliable scalable solutions
- Write SQL and Python for transformation deduplication and enrichment
Perks/Benefits
- 401k
- Dental insurance
- Flexible paid time off
- Life insurance
- Long-term disability
- Medical insurance
- Paid Holidays
- Paid parental leave
- Paid sick leave
- Short-term disability
- Vision insurance
Skills/Tech-stack
AWS | Agile | Bash | CI/CD | Data Deduplication | Data Enrichment | Data Governance | Data Modeling | Data Quality | Data pipeline | Deterministic matching | Git | Identity matching | LLM | Machine Learning | Privacy | Probabilistic matching | Prompt engineering | Python | Record Linkage | SQL | Snowflake | Snowpark | Streams | Tasks | Unix
Education
N/A
Roles
Data Engineer | Data Engineering | Data Engineering Manager | Engineer | Engineering Manager | Lead | Lead Data | Lead Data Engineer | Manager
Regions
Countries
States
Cities
Related jobs
-
Data Engineer USD 100K-115KCrystal Reports | Data Warehouse | ETL | Oracle | Power BI401k matching | Accidental insurance | Critical illness insurance | Employee discount | Equity programMid-level Full TimeRomeoville, IL, United States5h ago
-
Lead Product Manager (Data Platform) USD 129K-186KAgile | Capital management | Data analytics | Enterprise SaaS | Feature FlagsPerformance bonuses | Restricted stock units | Travel opportunities | Work flexibilitySenior-level Full TimeUnited States R5h ago
-
Software Developer - Embedded USD 80K-135K.NET | C Sharp | C# | Embedded Software | Ethernet401k Plan Employer Match | Benefits active on day 1 | Disability insurance | Health savings account | Life insuranceMid-level Full TimeLas Vegas, NV, United States5h ago
-
Computer Scientist II USD 120K-144KAPI Design | Agile | Angular | Azure DevOps | Azure DevOps Pipelines401k employer match | Dental insurance | Disability insurance | Health insurance | Health savings accountMid-level Full TimeLas Vegas, NV, United States5h ago
-
Computer Scientist III USD 97K-130KC++ | Configuration Management | Integration Testing | Linux | Python401k plan | Dental insurance | Disability insurance | Health savings account | Life insuranceSenior-level Full TimeNorth Las Vegas, NV, United States5h ago
-
Computer Scientist I USD 123K-145KC++ | Configuration Management | Development Lifecycle | Integration Testing | Linux401k match | Disability insurance | Health savings accounts | Life insurance | Paid time offMid-level Full TimeEdwards AFB, CA, United States5h ago
-
Computer Scientist I USD 122K-150KASP Classic | C# | CSS | Django | HTML401k match | Dental insurance | Disability insurance | Flexible spending account | Health savings accountMid-level Full TimeEdwards AFB, CA, United States5h ago
-
Computer Scientist III USD 91K-130KC# | C++ | Data Structures | Documentation | Hardware Design401k match | Dental insurance | Disability insurance | Health savings account | Immediate vestingSenior-level Full TimeLas Vegas, NV, United States5h ago
-
Computer Scientist I USD 120K-144KC# | C++ | Development Lifecycle | Hardware documentation | Integration Testing401k match | Dental insurance | Disability insurance | Flexible spending account | Health savings accountMid-level Full TimeLas Vegas, NV, United States5h ago
-
Computer Scientist - openMCT USD 135K-165KCSS3 | Data Visualization | Data integration | Git | HTML5401(k) plan match | Dental insurance | Disability insurance | Employer matching contributions | Flexible spending accountEntry-level Full TimeEdwards AFB, CA, United States5h ago
-
Computer Scientist - openMCT USD 135K-165KBrowser Developer Tools | CSS3 | Data Visualization | Developer tools | Git401k matching | Dental insurance | Disability insurance | Flexible spending account | Health savings accountMid-level Full TimeEdwards AFB, CA, United States5h ago
-
Data Engineer / BI Developer USD 91K-130KAmazon Web Services | Apache Airflow | Cloud platform | DBT | Data ModelingMid-level Full TimeCenter, Center District, IL7h ago
-
Embedded Software Engineer - Body Control Modules USD 79K-178KA/D | ASIL A D | AUTOSAR | C# | CI/CDAdoption and surrogacy expense reimbursement | Employee resource groups | Fertility treatments | Flexible family care days | Medical, dental & vision coverageSenior-level Full TimeDearborn, MI, United States8h ago
-
Software Engineer III - Senior Java Spark Developer USD 113K-188KAgile | Apache Spark | CI/CD | Concurrency | Distributed SystemsSenior-level Full TimeJersey City, New Jersey, United States9h ago
-
Data Engineer USD 97K-163KBTEQ | Data Partitioning | Database Administration | Performance Tuning | Query OptimizationSenior-level Full TimeSacramento, California, United States9h ago
-
Data Scientist AI Execution USD 140K-190KArtificial Intelligence | Classification | Data Engineering | Data analytics | Language ProcessingSenior-level Full TimeAuburn Hills, MI, United States10h ago
-
Machine Learning Engineer Graduate (E-Commerce Supply Chain & Logistics) - 2026 Start (PhD) USD 129K-246KData Mining | Deep learning | Graph Neural Networks | Knowledge graphs | LLMEntry-level Full TimeSan Jose, California, United States10h ago
-
API Integration | Agent Orchestration | Database Design | Fine Tuning | JavaScriptMid-level Full TimeNew York, New York, United States10h ago
-
Software Engineer, Video USD 141K-251KAV1 | AV2 | Artificial Intelligence | Audio CODEC | Automated testingMid-level Full TimeBellevue, WA | Menlo Park, CA …11h ago
-
Software Engineer, AI System Hacker, GenAI, DeepMind USD 174K-253K2D Games | 3D Games | AI Agents | C++ | CSSMid-level Full TimeMountain View, CA, USA11h ago
-
Staff Software Engineer, Data Center Orchestration USD 207K-301KC++ | Data Storage | Data integration | Distributed Systems | Generative AISenior-level Full TimePittsburgh, PA, USA11h ago
-
Attribution Modeling | Causal Inference | Dashboarding | Data pipeline | Data triangulationBonus | Equity compensation | Health insurance | Paid time off | Retirement planSenior-level Full TimeNew York, NY, USA; Atlanta, GA, …11h ago
-
ML Software Engineer, GenAI for Youth USD 207K-301KAdversarial Machine Learning | Automated testing | Benchmarking | Data Pipelines | Distributed ComputingSenior-level Full TimeMountain View, CA, USA11h ago
-
Software Engineer, AI/ML, Ads Data USD 147K-211KC++ | Data Structures | Data Structures and Algorithms | Debugging | Distributed ComputingMid-level Full TimeLos Angeles, CA, USA11h ago
-
Customer Engineer, Data Analytics, Financial Services USD 152K-222KApache Spark | Batch Processing | Big Data | Cloud Computing | Data GovernanceSenior-level Full TimeNew York, NY, USA; Atlanta, GA, …11h ago