Lead Data Engineer
Tasks
- Apply domain driven design for data modeling
- Build bronze ingestion silver transformation gold domain layers
- Collaborate with analytics to define business ready gold layer
- Coordinate with vendors and internal stakeholders
- Design and implement Lakehouse architecture
- Develop data pipelines for ingestion and transformation
- Enforce schema and manage data lineage
- Govern lakehouse with Unity Catalog access controls
- Identify architectural risks and propose resolutions
- Implement Delta Lake tables and transformations
- Maintain documentation and status updates
- Manage Delta Live Tables pipelines
Perks/Benefits
- N/A
Skills/Tech-stack
Airflow | Astro | Cloud Storage | Data Contracts | Data Governance | Data Lineage | Data Modeling | Data pipeline | Databricks | Delta Lake | Delta Live Tables | Delta Live) | Domain-Driven Design | GCS | Google Cloud | Google Cloud Storage | PySpark | SQL | Schema Enforcement | Unity Catalog
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Senior-level Full TimeCA-ON-Toronto3h ago
-
AI Integration - Backend Services USD 207K-300KC++ | Cloud platform | Distributed Systems | Docker | GoCross-functional collaboration | Mentorship | On-call rotationSenior-level Full TimeUS-CA-San Jose3h ago
-
AI and Data Science Engineer III USD 110K-218KArtificial Intelligence | Containerization | Data Analysis | Data Pipelines | DockerDiscretionary annual incentive program | Mentorship programs | Professional development opportunities | US travel opportunitiesSenior-level Full TimeAustin, Texas, United States; Boston, Massachusetts, …4h ago
-
AWS | Agile | Airtable | Anthropic | Artificial IntelligenceMentorship | Professional development | Travel opportunitySenior-level Full TimePittsburgh, Pennsylvania, United States4h ago
-
Engineering Lead-Data, Analytics & AI Platform USD 125K-155KAgentic AI | Cloud Native | Data Architecture | Data Governance | Data LineageSenior-level Full TimeJersey City, New Jersey, United States4h ago
-
Data Engineer, Product Analytics (University Grad) USD 171K-196KAI tools | Compliance | Data Modeling | Data Pipelines | Data PrivacyCareer Growth & Development | Work authorization supportNone Full TimeBellevue, WA | Menlo Park, CA …5h ago
-
Analytics | Data Analysis | Data Visualization | Data Workflow | Data strategySenior-level Full TimeSan Bruno, CA, USA5h ago
-
C++ | Cloud infrastructure | Computer Networking | Data Storage | Data StructuresSenior-level Full TimeSunnyvale, CA, USA5h ago
-
Data Engineer USD 76K-107KAmazon Web Services | Apache Airflow | Apache Kafka | Cause analysis | Cloud Composer401k match | Basic life insurance | Dental insurance | Disability coverage | Life insuranceMid-level Full TimePiscataway, NJ, US8h ago
-
Data Engineer USD 76K-121KAWS | Airflow | Apache Airflow | Cloud Data | Cloud Data Warehouse401k matching | Dental insurance | Disability coverage | Life insurance | Medical insuranceMid-level Full TimePiscataway, NJ, US8h ago
-
Advanced Analytics - Vet USD 109K-155KAutomation | Cloud Storage | Data Engineering | Data Governance | Data ModelingMid-level Full TimeOverland Park, KS, US8h ago
-
Advanced Analytics - Retail USD 109K-155KAWS | Cloud Computing | Cloud Storage | Data Governance | Data Pipelines401k matching | Dental insurance | Disability insurance | Life insurance | Medical insuranceMid-level Full TimeOverland Park, KS, US8h ago
-
AWS | Alerting | CI/CD | Databricks | DatadogSenior-level Full TimeJersey City, NJ, United States14h ago
-
Data Engineer, Infrastructure FinOps USD 146K-194KAPI Design | BigQuery | CI/CD | CloudFormation | DBTMid-level Full TimeCosta Mesa, California, United States15h ago
-
Senior Software Engineer, Data Platform USD 166K-220KAWS | Amazon Athena | Apache Iceberg | Apache Spark | AzureSenior-level Full TimeCosta Mesa, California, United States15h ago
-
Staff Data Scientist (Pricing) USD 179K-269KBehavioral Modeling | Causal Inference | Causal measurement | Choice Modeling | Contextual Bandits401k matching | Dental insurance | Employee learning development programs | Employee recognition programs | Financial assistance for hybrid workSenior-level Full TimeSan Francisco, CA16h ago
-
Senior Data Engineer USD 135K-168KAWS | Airflow | Amazon Redshift | Data Modeling | Data Warehousing401k | Flexible work arrangement | Health insurance | Mentorship program | Paid time offSenior-level Full TimePlano, TX18h ago
-
Lead Insurance Data Scientist USD 120K-150KAzure Databricks | Dashboards | Data Analysis | Data Visualization | Data cleaning401k matching | Mentorship program | Paid parental leave | Paid time off | Tuition reimbursementSenior-level Full Time14881 Quorum Drive Suite 850, Dallas, …19h ago
-
Lead Insurance Data Scientist USD 120K-150KAzure | Data Analysis | Data Visualization | Databricks | Excel401k matching | Dental insurance | Employee assistance program | Employee recognition programs | Medical insuranceSenior-level Full Time25 Deforest Avenue, Suite 203, Summit, …19h ago
-
Lead Insurance Data Scientist USD 120K-150KAzure Databricks | Dashboard Development | Data Analysis | Data Visualization | Database technologies401k matching | Early-Closure Days | Employee assistance programs | Employee recognition programs | Family planningSenior-level Full Time5909 Peachtree Dunwoody Rd Bldg D …19h ago
-
Lead Insurance Data Scientist USD 120K-150KAzure Databricks | Dashboarding | Data Analysis | Data Cleansing | Data Visualization401k matching | Employee assistance program | Identity theft fraud protection | Legal planning | Mentorship programSenior-level Full Time6111 Bollinger Canyon Road, Suite 300 …19h ago
-
AI Full Stack Developer & Architect USD 140K-161KCloud Run | Kubernetes | MLOps | Machine Learning | Next.jsSenior-level Contract Full TimeSan Jose, CA, United States21h ago
-
Data & AI Platform Engineer USD 95K-155KAI Search | APIs | AWS | Airflow | ArcGIS401k matching | Dental insurance | Health insurance | Life insurance | Paid HolidaysSenior-level Full TimeRemote, United States R21h ago
-
Senior Data Engineer USD 140K-160KApache Airflow | CI/CD | Change Data Capture | Cluster Sizing | Data Capture401k | Dental insurance | Health insurance | Hybrid work | Paid leaveSenior-level Full TimeNorth Hollywood, CA, United States21h ago
-
Senior Data Engineer USD 140K-160KAirflow | Auto Loader | CI/CD | Change Data Capture | Data CaptureSenior-level Full TimeNew York, NEW YORK, United States21h ago