Azure Databricks Lead
Tasks
- Build batch and near real time data pipelines
- Define end-to-end data architecture
- Design lakehouse and warehouse patterns
- Develop medallion style data models
- Implement data quality checks
- Implement monitoring and alerting
- Mentor team members
- Optimize Databricks compute performance
- Optimize Snowflake cost and performance
- Support enterprise production analytics
Perks/Benefits
- Company-Paid Holidays
- Dental insurance
- Disability insurance
- Employee assistance program
- Life insurance
- Medical insurance
- Paid time off
- Personal days
- Retirement plan
- Sick leave
- Vision insurance
Skills/Tech-stack
Apache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Databricks | Azure Key Vault | Azure SQL | Azure Synapse | Azure Synapse Analytics | Data Factory | Data Governance | Data Lake Storage | Data Modeling | Delta Lake | Entra ID | Key Vault | Medallion Architecture | Microsoft Entra | Microsoft Entra ID | Notebook orchestration | Python | Role-based security | SQL | Snowflake | Synapse Analytics | Unity Catalog | Virtual Warehouses
Education
N/A
Roles
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R13d ago
-
Data Engineer, gUP Engineering USD 106K-151KApache Flume | Apache Spark | Cloud Dataflow | Data Analysis | Data ModelingMid-level Full TimeBoulder, CO, USA1h ago
-
Data Engineer, GCS Data Science USD 106K-151KBig Data | C plus plus | Data Documentation | Data Modeling | Data ValidationEntry-level Full TimeNew York, NY, USA; Mountain View, …1h ago
-
Senior Software Engineer, AI/ML GenAI, Google Cloud USD 174K-253KC++ | Computer Vision | Data Processing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA1h ago
-
Senior Data Engineer USD 170K-200KApache Airflow | Apache Spark | ClickHouse | Data Contracts | Data Modeling401k | Catered lunches | Commuter benefits | Continued education benefit | Equity compensationSenior-level Full TimeNew York, New York, United States7h ago
-
Senior Data Engineer USD 170K-200KApache Airflow | Apache Spark | Data Contracts | Data Modeling | Data Quality401k | Catered lunches | Commuter benefits | Company wellness day off | Continued education benefitSenior-level Full TimeLos Angeles, California, United States7h ago
-
Senior Data Engineer USD 170K-200KApache Airflow | Apache Spark | Data Contracts | Data Modeling | Data Quality401k | Catered lunches | Commuter benefits | Continued education benefit | FSASenior-level Full TimeSan Francisco, California, United States7h ago
-
AI Engineer USD 113K-180KAngular | Apache Kafka | CI/CD | Context Management | DevOps401k matching | Charitable matching | Community outreach groups | Flexible paid time off | Life insuranceMid-level Full TimeLeawood, KS, US R7h ago
-
Applied AI Engineer USD 80K-120KAPIs | Agent Orchestration | Data Pipelines | Evaluation | FedRAMPDental insurance | Equity compensation | Health insurance | Travel benefits | Vision insuranceMid-level Full TimeSF Bay Area, CA, USA8h ago
-
AI Solutions Architect USD 80K-120KAPI Integration | Agent Orchestration | Cloud infrastructure | DPA | Data PipelinesDental insurance | Equity | Flexible work schedule | Health insurance | Travel allowanceSenior-level Full TimeSF Bay Area, CA, USA8h ago
-
Data Architect | Contract | Hybrid in Los Angeles USD 100K-161KAPI Integration | AWS architecture | Access Control | Amazon Redshift | Azure ArchitectureContract role | Hybrid work location | Onsite work three days per weekSenior-level Full TimeLos Angeles, California, United States R11h ago
-
Azure Data Engineer USD 119K-219KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Autonomy | Career growth | Collaborative team | Community | Supportive cultureMid-level Full TimeIndianapolis, IN, US11h ago
-
Data Insights & Analytics Lead USD 120K-180KAmplitude | Blockchain Analytics | Contract analytics | Data Visualization | Dune AnalyticsCollaborative work culture | Dental insurance | Global team | Health insurance | Professional developmentSenior-level Full TimeCayman R12h ago
-
Principal Technical Lead, Software Engineering (Data Infrastructure) - North America Software Center USD 166K-241KAmazon S3 | Apache Airflow | Apache Flink | Apache Hudi | Apache Iceberg12 holidays per year | 401k match | Dental insurance | Fitness Center | Game roomSenior-level Full TimeVancouver, WA, US, 9868312h ago
-
Senior Data Science Engineer USD 116K-198KApache Airflow | CI/CD | DBT | Databricks | Databricks AutoloaderFlexible work optionsSenior-level Full TimeRemote (United States) R13h ago
-
AWQ | Benchmarking | CUDA | CUDA kernels | Chunked prefill401k match | Career growth | Collaborative culture | Disability insurance | FlexibilitySenior-level Full TimePalo Alto, California, United States13h ago
-
CUDA | Checkpointing | DTensor | Data parallelism | DeepSpeed401k plan | Career growth and learning opportunities | Collaborative culture | Disability insurance | Flexibility and ownershipSenior-level Full TimePalo Alto, California, United States13h ago
-
APIs | Behavior Trees | C++ | Dashboards | Data PipelinesSenior-level Full TimeIrvine, CA14h ago
-
Machine Learning Engineer, Local Search & Marketplace USD 190K-300KA/B | A/B Testing | Apache Spark | B testing | C++Senior-level Full TimeMountain View, California, United States14h ago
-
Machine Learning Engineer, Local Search & Marketplace USD 190K-300KA/B | A/B Testing | Algorithms | B testing | C++Senior-level Full TimeMountain View, California, United States14h ago
-
AI Engineer USD 230K-315KAPI Design | Caching | Concurrent programming | Context Window Pruning | Context caching401k matching | Comprehensive medical/dental/vision coverage | Fertility benefits | Flexible PTO | Lunch CreditSenior-level Full TimeSan Francisco, CA14h ago
-
Member of Technical Staff, Applied AI Research USD 203K-299KAgentic technology | Apache Cassandra | Apache Kafka | Caching | Capacity PlanningSenior-level Full TimeSan Francisco, CA; Sunnyvale, CA14h ago
-
Senior-level Full TimeColumbia, MD14h ago
-
AI Data Engineer USD 133K-258KAWS | Agile | Apache Kafka | Apache Spark | AzureAnnual bonus eligibility | Health and wellness benefits | Hybrid work schedule | Paid time off | Retirement planSenior-level Full TimeBaltimore, Maryland, United States14h ago
-
Senior Software Engineer, Data Enablement USD 160K-283KAWS | Apache Spark | CloudEvents | DBT | DatabricksFlexible time off | Medical insurance | Modern family planning | Remote work | Retirement savings plansSenior-level Full TimeBay Area, CA, United States of …15h ago