Databricks Lead Data Engineer
Tasks
- Build Databricks platform components
- Collaborate with data scientists and analysts
- Design analytical data models star schemas
- Design develop test deploy scalable batch streaming data pipelines
- Develop data ingestion validation transformation pipelines
- Document code artifacts and user documentation runbooks
- Enforce Databricks best practices notebook development job orchestration
- Implement logging monitoring exception handling
- Integrate Databricks with cloud services
- Lead technical planning & requirements gathering
- Participate in design demos prototypes training workshops
- Perform code reviews and CI CD pipeline development
- Serve as technical lead mentor
- Troubleshoot deployment across environments
Perks/Benefits
- N/A
Skills/Tech-stack
AWS Glue | AWS Lambda | Amazon S3 | Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOps | Azure Event | Azure Event Hubs | CI/CD | Data Factory | Data Lake Storage | Data Lakehouse | Databricks | ELT | ETL | Event Hubs | Git | Jira | Medallion Architecture | Notebook Repositories | PostgreSQL | PySpark | Python | SQL | Unit Testing
Education
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer | Senior Data Engineer
Related jobs
-
Senior Software Engineer, AI Platform Engineering USD 160K-240KAWS | Amazon SageMaker | Containerization | Docker | EC2401k matching | Dental insurance | Life insurance | Medical insurance | Paid HolidaysSenior-level Full TimeNew York5h ago
-
C++ | Data Preparation | Data Processing | Debugging | GenAISenior-level Full TimeMountain View, CA, USA7h ago
-
C++ | Data Processing | Debugging | Embedding | Information RetrievalSenior-level Full TimeMountain View, CA, USA; San Bruno, …7h ago
-
Algorithms | C++ | Data Processing | Data Structures | DebuggingSenior-level Full TimeMountain View, CA, USA; San Bruno, …7h ago
-
Member of Technical Staff (AI Software Engineer, Agents) USD 220K-405KAI Evaluation | Browser technologies | CDP | Code Quality | Context engineeringSenior-level Full TimeSan Francisco14h ago
-
Data Engineer USD 165K-200KAirbyte | Airflow | BigQuery | CI/CD | Cloud ComposerCareer growth opportunities | Learning and development budget | Paid parental leave | Regular events and social activities | Team assistance programSenior-level Full TimeAustin, Texas; San Francisco Bay Area, …16h ago
-
Senior Embedded Engineer USD 165K-218KADC | AXI | AXI-Stream | Bootloader | DDREquity grants | Health insurance | Paid time off | Recovery supportSenior-level Full TimeHudson, New Hampshire, United States16h ago
-
Forward Deployed Engineer USD 120K-158KAngular | Code Reviews | Customer enablement | Documentation | GitMid-level Full TimeAtlanta, Georgia, United States; Chicago, Illinois, …17h ago
-
Mid-level Full TimeRedwood City, California, United States17h ago
-
Software Technical Account Manager - Wichita, Kansas USD 93K-150KAI-powered applications | Analytics | Cause analysis | Cloud Computing | Customer adoption401k employer match | Discretionary paid time off | Emotional and mental wellness support | Fitness programs | Learning and development programsMid-level Full TimeKansas-Remote R17h ago
-
Software Engineer, Propulsion Simulation & Data Analysis USD 125K-175K.NET | Angular | C# | C++ | Combustion Engineering401k retirement plan | Dental insurance | Employee stock purchase plan | Health insurance | Paid HolidaysSenior-level Full TimeHawthorne, CA18h ago
-
.NET | Angular | C# | C++ | CI/CD401k retirement plan | Company stock options | Dental insurance | Employee stock purchase plan | Life insuranceSenior-level Full TimeHawthorne, CA18h ago
-
Senior Data Engineer USD 165K-216KAccess Control | Data Governance | Data Lineage | Data Modeling | Data QualitySenior-level Full TimeHouston, TX, 77040, USA18h ago
-
Senior GenAI Software Engineer (North America) USD 165K-230KA/B | A/B Testing | B testing | Debugging | EvaluationEquity | Health, dental, and vision benefits | In person team gatherings quarterly | Remote-first work | Wellness stipendsSenior-level Full TimeUnited States R19h ago
-
Senior Embedded Software Engineer - Ethernet Retimers USD 133K-185KBash | C# | C++ | Clang | DiagnosticsSenior-level Full TimeSan Jose, California, United States19h ago
-
Lead - POC Data Science USD 200K-280KA/B | A/B Testing | Anomaly Detection | Apache Spark | B testing401k matching | Dental insurance | Flexible paid time off | Health and wellness stipend | Health insuranceSenior-level Full TimeUnited States R19h ago
-
Data Analytics Engineer USD 160K-195KATE | Advantest | Anomaly Detection | Cause analysis | CredenceSenior-level Full TimeSan Jose, California, United States19h ago
-
Data & Digital Solutions Engineer USD 86K-118KAPI | AWS | Application development | Artificial Intelligence | AutomationMid-level Full TimeUS > Arizona > Phoenix20h ago
-
Integrated Analytics and Insights Expert USD 107K-232KDashboards | Data Modeling | Data Pipelines | Data Quality | Data VisualizationSenior-level Full TimeUnited States20h ago
-
Lead Data Engineer USD 135K-160KApache Iceberg | Automation | CI/CD | Cloud Platforms | Columnar Storage401k match | Commuter benefits | FSA | HSA | Paid HolidaysSenior-level Full TimeDenver, CO, United States20h ago
-
AI Agents | AWS | Azure | Cloud Computing | Data Modeling401k matching | Dental insurance | Life insurance | Medical insurance | Paid time offSenior-level Full TimeUnited States21h ago
-
AI Agents | AWS | Azure | Cloud Computing | Cycle management401k match | Dental insurance | Health insurance | Life insurance | Paid parental leaveSenior-level Full TimeUnited States21h ago
-
Mid Level AI Engineer USD 98K-158KAWS | Anthropic | Autogen | Azure OpenAI | Backend Development401k match | Commuter benefits | Dental insurance | Employee Discount Marketplace | FSAMid-level Full TimeUS - Orlando21h ago
-
Machine Learning Platform Engineer USD 135K-160KAmazon SageMaker | Apache Flink | C++ | CI/CD | Cloud PubSub401k match | Annual bonus | Company equipment provided | Company medical dental vision plans | Disability benefitsMid-level Full TimeAtlanta, GA preferred, Remote R21h ago
-
Senior Research Engineer, Olmo + Molmo USD 146K-220KAgentic Systems | Amazon Web Services | Cloud Computing | Cloud platform | ContainerizationFamily leave | Paid sick leave | Paid vacationSenior-level Full TimeSeattle, WA21h ago