Lead Data Engineer (Databricks)
Chicago - Illinois - Wacker, United States
USD 112K-140K Senior-level Full Time
Tasks
- Configure and manage Profisee MDM
- Develop and optimize Databricks notebooks
- Drive automation and CI/CD pipelines with Azure DevOps
- Enhance and support BI reporting and dashboards
- Implement data cataloging and governance with Azure Purview
- Implement data storage solutions in Azure Data Lake
- Implement new technologies in data engineering
- Lead SQL Server database optimization
- Lead data architecture and engineering
- Manage and optimize ETL processes and data pipelines
- Mentor data engineers and team
- Set up Change Data Capture processes
Perks/Benefits
Skills/Tech-stack
Automation | Azure Data | Azure Data Factory | Azure Data Lake | Big Data | Big data analytics | CI/CD | Data Factory | Data Governance | Data Lake | Data Modeling | Data Processing | Data Warehousing | Data analytics | Database Management | Databricks | Delta Parquet | ETL Processes | Leadership | Machine Learning | Predictive Analysis | Profisee MDM | SQL Server | SQL Server Management | Server management | Tableau
Education
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Automation Testing | CI/CD | CSS | Cypress | Feature DevelopmentMedical, dental & vision coverage | Paid time off | Parental leave | Reimbursement programs | Retirement planMid-levelRaleigh, United States R7d ago
-
Software Engineer, Docs AI, Workspace USD 147K-211KAI | Application development | C++ | Distributed Computing | Full StackBenefitsMid-level Full TimeNew York, NY, USA3h ago
-
Senior Staff Software Engineer, Drive GenAI USD 262K-365KAlgorithms | Data Structures | Generative AI | Language Models | Language ProcessingCareer growth opportunities | Flexible work hours | Health insuranceSenior-level Full TimeBoulder, CO, USA; New York, NY, …3h ago
-
Software Engineer, GenAI, Ads Forecasting USD 147K-211KData Preparation | GenAI | Machine Learning | Machine Learning Infrastructure | ProgrammingBenefitsMid-level Full TimeLos Angeles, CA, USA3h ago
-
Staff Software Engineer, AI/ML, Workforce Identity USD 207K-300KAlgorithms | Cloud Security | Data Structures | Distributed Systems | Generative AIBenefits | Bonus | EquitySenior-level Full TimeSunnyvale, CA, USA3h ago
-
Software Engineer, BigQuery Metadata USD 147K-211KC++ | Cloud infrastructure | Data analytics | Databases | Distributed SystemsBenefits | Bonus | EquityMid-level Full TimeSeattle, WA, USA3h ago
-
Senior Software Engineer, BigQuery, Agentic AI USD 174K-252KCloud services | Data Warehousing | Distributed Systems | Generative AI | GoBenefitsSenior-level Full TimeNew York, NY, USA3h ago
-
Artificial Intelligence | Automation | C++ | Cloud Computing | Distributed SystemsBenefits | Bonus | EquitySenior-level Full TimeKirkland, WA, USA3h ago
-
Software Engineer - AI Data Platform USD 130K-180KAWS | Azure | Big Data | C# | C++Health insurance | Life insurance | Paid time off | Performance incentives | Retirement plansMid-level Full TimeSan Jose, CA, US, 951287h ago
-
Machine Learning Engineer, Integrity USD 266K-555KAlgorithms | Data Structures | Deep learning | Machine Learning | PyTorchMid-level Full TimeSan Francisco8h ago
-
Machine Learning Infrastructure Engineer - #4694 USD 190K-255KAWS | Bazel | C# | C++ | Distributed ComputingFlexible time off | Medical, dental & vision coverage | Mindfulness programs | Retirement planSenior-level Full TimeMenlo Park, CA8h ago
-
Staff ML Engineer, Frontier AI USD 250KDeep learning | Language Processing | Machine Learning | Model Optimization | Natural Language401k match | Flexible time off | Medical/Dental/Vision | Parental leave | Remote workSenior-level Full TimeSan Francisco9h ago
-
Senior Analytical Engineer USD 149K-240KAWS | Airflow | Azure | Databricks | Generative AI401k | Dental | FSA/HSA | Life insurance | MedicalSenior-level Full TimeSan Francisco, CA, United States11h ago
-
Data Engineer USD 155K-200KBigQuery | Cloud Computing | Cloud platform | DBT | Data ArchitectureFlexible time off | Health and wellbeing support | Learning and development budget | Parental leave | Team eventsSenior-level Full TimeAustin, Texas; San Francisco Bay Area, …11h ago
-
C++ | CUDA | Computer Architecture | Deep learning | Hardware accelerationCutting-edge technology access | Impact on transportation advancement | Supportive environmentSenior-level Full TimeSanta Clara, CA13h ago
-
Sr. AI Engineer (24 months fixed-term) USD 137K-232KBias Mitigation | Communication | Data Analysis | Education Technology | Generative AIFlexible schedule | Health insurance | Inclusive culture | Learning and development opportunities | Paid time offSenior-level TemporaryMountain View, CA / Remote (Continental … R13h ago
-
Staff Data Engineer USD 131K-210KAWS | Apache Spark | Azure | CI/CD | Data Governance401k | Dental insurance | FSA | HSA | Hybrid workSenior-level Full TimeBellevue, WA, United States14h ago
-
Sr. Software Dev Engineer, Leo AI Foundations USD 168K-227KAWS | Automated testing | Data Ingestion | Data Lakehouse | Data ModelingSenior-level Full TimeRedmond, Washington, USA14h ago
-
AWS | Big Data | Data Governance | Distributed Systems | Server-lessFlexible work hours | Inclusive culture | MentorshipSenior-level Full TimeEast Palo Alto, California, USA14h ago
-
Application development | Cloud Platforms | Deep learning | Distributed Systems | Docker401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States14h ago
-
Senior Staff Machine Learning Engineer USD 261K-330KAI architectures | Deep learning | Embeddings | Experimentation | Language ModelsCompetitive compensation | Equity grants | Flexible work options | Health benefits | Remote workSenior-level Full TimeUnited States - Remote R14h ago
-
Staff Machine Learning Engineer, Robotics USD 215K-364KComputer Vision | Data Science | Deep learning | Language Processing | Machine LearningCompetitive compensation | Cutting-edge technology | Impact on transportation and robotics | Snacks and activities | Supportive environmentSenior-level Full TimeSanta Clara, CA15h ago
-
Machine Learning Engineer - Sr. Consultant Level USD 152K-243KAI | Big Data | C++ | CI/CD | Flink401k | Dental | FSA/HSA | Life insurance | MedicalSenior-level Full TimeAustin, TX, United States15h ago
-
Sr. Data Platform Engineer I (6366) USD 170K-239KAgile | Backup & Recovery | CI/CD | Database security | Kubernetes401k plan | Flexible work | Health, dental, vision insurance | Paid time off | Parental leaveSenior-level Full TimeHybrid - Local to D.C. metro … R16h ago
-
Big Data Engineer / Developer USD 76K-136KBigQuery | Data Engineering | Data Processing | ETL Tools | GCPInclusive work environment | Industry training | Professional certificationsSenior-level Full TimeWashington, DC17h ago