Lead Data Engineer (Databricks)
Tasks
- Architect for scale and reliability
- Coach through code review and pairing
- Create technical documentation and reference architectures
- Design scalable data pipelines
- Drive dataops practices
- Enforce automated testing pipeline observability and CI CD
- Identify delivery blockers
- Implement lakehouse architectures
- Lead client data engagements
- Manage technical scope and timelines
- Mentor data engineers
- Own architecture and build
Perks/Benefits
- N/A
Skills/Tech-stack
AWS Glue | AWS Redshift | Amazon S3 | Apache Spark | Automated testing | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Synapse | Azure Synapse Analytics | CI/CD | Cloud Storage | Data Factory | Data Lake Storage | Data Modeling | Data Warehousing | Databricks | Databricks Workflows | Delta Lake | ELT | ETL | Google BigQuery | Google Cloud | Google Cloud Storage | Google Dataflow | Infrastructure as Code | Lakehouse Architecture | Observability | Python | Scala | Synapse Analytics | Unity Catalog | Version control | “as-code”
Education
N/A
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Research Engineer, SysML - FAIR USD 141K-208KAgent Orchestration | Bias Mitigation | C# | C++ | CUBLASMid-level Full TimeMenlo Park, CA | Remote, US R11h ago
-
AI Foundry | AWS Bedrock | Agent Frameworks | Agent Orchestration | AnthropicMid-level Full TimeHouston, TX, United States R18h ago
-
Embedded Semiconductor Engineer USD 120K-150KAutomation | BSP | C# | C++ | CloudCareer growth | Mentorship | Remote workSenior-level Full TimeUnited States - Remote R23h ago
-
Data Analytics Manager USD 95K-150KAPI Design | Agile | Azure | Azure Data | Azure Data FactoryComprehensive benefits package | Professional development programs | Remote work optionMid-level Full TimeDallas, Texas, United States R1d ago
-
Staff Data Engineer, Corporate Engineering USD 180K-252KAWS S3 | Automated reconciliation | CI/CD | Cloud Data | Cloud Data Warehouse401k employer match | Family planning support | Flexible vacation | Gender-affirming care | Healthcare benefitsSenior-level Full TimeRemote - United States R1d ago
-
Senior AI Systems Engineer USD 130K-195KAlerting | Bash | CI/CD | CMMC | Configuration ManagementHybrid work option | Remote work optionSenior-level Full TimeRaleigh, North Carolina, United States; Albuquerque, … R1d ago
-
Senior AI Technologist USD 145K-208KAI Agents | AI Foundry | AWS Bedrock | Apache Airflow | Apache SparkCross-functional collaboration | Fully remote or hybrid or onsite optionSenior-level Full TimeRaleigh, North Carolina, United States; Albuquerque, … R1d ago
-
Data Intelligence Lead USD 114K-140KAdvanced Analytics | Azure | Azure Data | Azure Data Factory | Azure Data LakeSenior-level Full TimeRemote, United States R1d ago
-
Data Engineer USD 120K-150KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | CI/CDSenior-level Full TimeRemote, United States R1d ago
-
API | AWS Glue | Amazon Redshift | Amazon Web Services | Apache AirflowFully remote | Mentorship opportunities | On-call supportSenior-level Full TimeOrlando, FL, United States R1d ago
-
Senior Data Engineer – Hardware & Supply Chain USD 147K-203KAWS | Apache Iceberg | DBT | Data Governance | Data ModelingFlexible wellness time off | Paid Holidays | Paid sick leave | Paid time off | Parental leaveSenior-level Full TimeHybrid - San Francisco, California R1d ago
-
Principal Database Engineer USD 102K-203KASH | AWR | AWS IAM | AWS KMS | AWS RDSFlexible hybrid work model | Health and life insurance | Paid time off | Pension/retirement benefits | Personal/Family Care leavesSenior-level Full TimeTampa, FL, United States R1d ago
-
Computational Scientist USD 90K-105KC++ | Documentation | Fortran | HPC clusters | LinuxConference attendance | Open source contributions | Professional development opportunities | Student mentoringEntry-level Full TimeBlacksburg, Virginia, Hybrid R1d ago
-
AI Architect Lead (Hybrid-within BankUnited's footprint) USD 140K-205KAI Foundry | AWS | AWS Bedrock | Agentic AI | Architecture ReviewHybrid work modelSenior-level Full TimeMiami Lakes, FL, United States R1d ago
-
Software Developer Sr USD 95K-169K.NET | ARM | Apache Spark | Automated testing | Azure401k plan | Charity programs | Dental insurance | Life insurance | Medical insuranceSenior-level Full TimeUnited States R1d ago
-
Software Developer Sr USD 95K-169K.NET | ARM | Apache Spark | Automated testing | Azure401k | Charity support | Dental insurance | Global Employee Stock Purchase Plan | Life insuranceSenior-level Full TimeUnited States R1d ago
-
Senior Machine Learning Engineer II USD 156K-250KAWS | Azure | Batch Processing | Container Orchestration | Data PipelinesDiscretionary paid time off | Emotional and mental wellness support | Employee resource groups | Fitness programs | Learning and development programsSenior-level Full TimeSeattle, Washington, United States R1d ago
-
AI orchestration | Alerting | Caching | Distributed Systems | DockerDental insurance | Medical insurance | Paid time off | Savings plan options | Vision insuranceSenior-level Full TimeSan Francisco, CA, United States R1d ago
-
A/B | A/B Testing | APIs | Anthropic | B testingDental insurance | Medical insurance | PTO | Remote | Savings plan optionsMid-level Full TimeSan Francisco, CA, United States R1d ago
-
Alerting | Batching | Caching | Distributed Systems | DockerDental insurance | Medical insurance | Paid time off | Remote work | Savings planMid-level Full TimeSan Francisco, CA, United States R1d ago
-
Security Operations AI Engineer, Contract USD 151K-225KAI Governance | AI RMF | AI Security | Adversarial Attacks | Adversarial Machine LearningMid-level Full TimeRemote, United States R1d ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DatabricksCollaborative cross-functional team | Exposure to large scale insurance analytics | Fully remote work | High-impact leadership role | Long term contract with extension potentialSenior-level Contract Full TimeCanada R1d ago
-
Data Engineer USD 97K-122K.NET | API Development | API Gateway | AWS | AWS Glue401k match | Commuter benefits | Dental insurance | Dependent Care Savings Account | Education assistanceMid-level Full TimeRemote, United States R1d ago
-
AWS | Alerting | Azure | BigQuery | CI/CDEquity opportunities | Fully remote within Canada | Health, dental, vision coverage | Paid parental leave | Performance-Based IncentivesSenior-level Full TimeCanada R1d ago
-
API Design | Background Processing | CI/CD | Celery | DockerFully remote within Canada | High autonomy | High impact AI strategy involvementSenior-level Full TimeCanada R1d ago