Lead Data Engineer
Tasks
- Apply domain driven design for data modeling
- Build bronze ingestion silver transformation gold domain layers
- Collaborate with analytics to define business ready gold layer
- Coordinate with vendors and internal stakeholders
- Design and implement Lakehouse architecture
- Develop data pipelines for ingestion and transformation
- Enforce schema and manage data lineage
- Govern lakehouse with Unity Catalog access controls
- Identify architectural risks and propose resolutions
- Implement Delta Lake tables and transformations
- Maintain documentation and status updates
- Manage Delta Live Tables pipelines
Perks/Benefits
- N/A
Skills/Tech-stack
Airflow | Astro | Cloud Storage | Data Contracts | Data Governance | Data Lineage | Data Modeling | Data pipeline | Databricks | Delta Lake | Delta Live Tables | Delta Live) | Domain-Driven Design | GCS | Google Cloud | Google Cloud Storage | PySpark | SQL | Schema Enforcement | Unity Catalog
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R4d ago
-
Data Visualization & Integration Specialist USD 115K-150KAPI Integration | Bill of Materials | Data Governance | ETL | ExcelHybrid work scheduleMid-level Full TimeMarietta, GA, United States7h ago
-
Senior-level Full TimeHoboken, NJ, United States8h ago
-
AWS Glue | AWS IAM | AWS Kinesis | AWS Lambda | AWS Step Functions100 percent on siteSenior-level ContractMc Lean, United States9h ago
-
AI | Adversarial Machine Learning | Bot Detection | Data Analysis | Data IntegrityMid-level Full TimeMountain View, CA, USA11h ago
-
Staff Software Engineer, AI/ML, Google Cloud Security USD 207K-301KAgentic AI | Cloud platform | Data Processing | Debugging | Distributed SystemsSenior-level Full TimeReston, VA, USA; New York, NY, …11h ago
-
Senior-level Full TimeUS - Remote; Canada - Remote R17h ago
-
Senior-level Full TimeMountain View, CA19h ago
-
Staff Software Engineer, AI Integrations USD 170K-240KAgile | Alerting | Amazon Web Services | C++ | CSSSenior-level Full TimeMountain View, CA19h ago
-
Principal / Sr Principal Data Analytics Engineer USD 98K-184KAgile | CI/CD | Data Governance | Data Modeling | Data MonitoringCompany-Paid Holidays | Disability insurance | Health insurance | Hybrid work | Life insuranceSenior-level Full TimeFLME229, United States22h ago
-
Sr. Data Engineer USD 109K-145KAWS | Agile | CI/CD | Compliance | Data Governance401k matching | Employee assistance program | Flexible working hours | Learning platform access | Medical, dental, and vision insuranceSenior-level Full TimeRemote, TX, United States R22h ago
-
Senior Machine Learning Engineer USD 134K-168KData Drift | Databricks | Ethical AI | Feature Engineering | KubernetesSenior-level Full TimeNew York, NY, United States22h ago
-
Hadoop Big Data Developer USD 100K-150KAirflow | Apache Flink | Apache HBase | Apache Hive | Apache Hudi100 percent remote | Career growth | Full-time W2 employmentSenior-level Full TimeUnited States - Remote R22h ago
-
Hadoop Big Data Developer USD 100K-150KAirflow | Apache Flink | Apache Hive | Apache Hudi | Apache IcebergSenior-level Full TimeUnited States - Remote R22h ago
-
AI Data Engineer USD 100K-150KActive Learning | Apache Beam | Apache Spark | CI/CD | Code reviewCareer growth | Health benefits | Remote workMid-level Full TimeUnited States - Remote R22h ago
-
AI Data Engineer USD 100K-150KApache Beam | CI/CD | Data Ingestion | Data Lineage | Data ModelingRemote workMid-level Full TimeUnited States - Remote R22h ago
-
AI Data Engineer USD 100K-150KApache Beam | Apache Spark | CI/CD | Caching | Code review100 percent remote | Benefits | Career growth potentialMid-level Full TimeUnited States - Remote R22h ago
-
Senior-level Full TimeUnited States - Remote R22h ago
-
Data Engineer USD 88K-102KAWS | Data Lake | Data Modeling | Data Transformation | Data WarehouseEmployee discounts | Health care options | Paid leave | Remote work option | Retirement contributionsMid-level Full TimeIthaca (Main Campus), United States R22h ago
-
Senior Clinical Data Science Lead USD 110K-138KAcceptance Testing | Change Control | Clinical Trial Design | Clinical data | Clinical data managementDisability coverage | Employee Assistance programmes | Health and Wellbeing Programmes | Learning and development opportunities | Life assuranceSenior-level Full TimeUS, Blue Bell (ICON), United States22h ago
-
Senior Data Integration Operations Engineerr USD 130K-189KAWS | Access Control | Automation | Azure | Best practices24 7 availability on rotating basis | Documentation and knowledge management support | Hybrid work schedule | Rotating on-callSenior-level Full TimeNU Boston Campus, United States22h ago
-
AI Infrastructure Engineer USD 100K-150KApache Beam | Apache Spark | CI/CD | Code review | Data LineageMid-level Full TimeUnited States - Remote R22h ago
-
AWS SageMaker | Actimize | Apache Hive | Apache Spark | Data VisualizationHybrid workSenior-level Full TimeMalvern, Pennsylvania, United States1d ago
-
Associate Software Engineer (Python + AWS) USD 143K-194KAWS Lambda | AWS S3 | AWS SNS | AWS SQS | Amazon Web ServicesHybrid work | Work authorization not requiring visa sponsorship | Work from home up to three days per weekMid-level Full TimeBaltimore, Maryland, United States1d ago
-
Senior Staff Generative AI Scientist USD 185K-210KAWS | Apache Spark | Artificial Intelligence | Azure | Cloud platform401k matching | Dental insurance | Disability insurance | Health insurance | Life insuranceSenior-level Full TimeRemote, United States R1d ago