Lead Data Engineer (Databricks)
Chicago - Illinois - Wacker, United States
USD 112K-140K Senior-level Full Time
Tasks
- Configure and manage Profisee MDM
- Develop and optimize Databricks notebooks
- Drive automation and CI/CD pipelines with Azure DevOps
- Enhance and support BI reporting and dashboards
- Implement data cataloging and governance with Azure Purview
- Implement data storage solutions in Azure Data Lake
- Implement new technologies in data engineering
- Lead SQL Server database optimization
- Lead data architecture and engineering
- Manage and optimize ETL processes and data pipelines
- Mentor data engineers and team
- Set up Change Data Capture processes
Perks/Benefits
Skills/Tech-stack
Automation | Azure Data | Azure Data Factory | Azure Data Lake | Big Data | Big data analytics | CI/CD | Data Factory | Data Governance | Data Lake | Data Modeling | Data Processing | Data Warehousing | Data analytics | Database Management | Databricks | Delta Parquet | ETL Processes | Leadership | Machine Learning | Predictive Analysis | Profisee MDM | SQL Server | SQL Server Management | Server management | Tableau
Education
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Automation Testing | CI/CD | CSS | Cypress | Feature DevelopmentMedical, dental & vision coverage | Paid time off | Parental leave | Reimbursement programs | Retirement planMid-levelRaleigh, United States R7d ago
-
Staff Software Engineer, AI/ML, Workforce Identity USD 207K-300KAlgorithms | Cloud Security | Data Structures | Distributed Systems | Generative AIBenefits | Bonus | EquitySenior-level Full TimeSunnyvale, CA, USA1h ago
-
Artificial Intelligence | Automation | C++ | Cloud Computing | Distributed SystemsBenefits | Bonus | EquitySenior-level Full TimeKirkland, WA, USA1h ago
-
Software Engineer - AI Data Platform USD 130K-180KAWS | Azure | Big Data | C# | C++Health insurance | Life insurance | Paid time off | Performance incentives | Retirement plansMid-level Full TimeSan Jose, CA, US, 951285h ago
-
Machine Learning Engineer, Integrity USD 266K-555KAlgorithms | Data Structures | Deep learning | Machine Learning | PyTorchMid-level Full TimeSan Francisco6h ago
-
Machine Learning Infrastructure Engineer - #4694 USD 190K-255KAWS | Bazel | C# | C++ | Distributed ComputingFlexible time off | Medical, dental & vision coverage | Mindfulness programs | Retirement planSenior-level Full TimeMenlo Park, CA7h ago
-
Staff ML Engineer, Frontier AI USD 250KDeep learning | Language Processing | Machine Learning | Model Optimization | Natural Language401k match | Flexible time off | Medical/Dental/Vision | Parental leave | Remote workSenior-level Full TimeSan Francisco7h ago
-
Senior Analytical Engineer USD 149K-240KAWS | Airflow | Azure | Databricks | Generative AI401k | Dental | FSA/HSA | Life insurance | MedicalSenior-level Full TimeSan Francisco, CA, United States9h ago
-
Data Engineer USD 155K-200KBigQuery | Cloud Computing | Cloud platform | DBT | Data ArchitectureFlexible time off | Health and wellbeing support | Learning and development budget | Parental leave | Team eventsSenior-level Full TimeAustin, Texas; San Francisco Bay Area, …10h ago
-
C++ | CUDA | Computer Architecture | Deep learning | Hardware accelerationCutting-edge technology access | Impact on transportation advancement | Supportive environmentSenior-level Full TimeSanta Clara, CA12h ago
-
Sr. AI Engineer (24 months fixed-term) USD 137K-232KBias Mitigation | Communication | Data Analysis | Education Technology | Generative AIFlexible schedule | Health insurance | Inclusive culture | Learning and development opportunities | Paid time offSenior-level TemporaryMountain View, CA / Remote (Continental … R12h ago
-
Staff Data Engineer USD 131K-210KAWS | Apache Spark | Azure | CI/CD | Data Governance401k | Dental insurance | FSA | HSA | Hybrid workSenior-level Full TimeBellevue, WA, United States12h ago
-
Sr. Software Dev Engineer, Leo AI Foundations USD 168K-227KAWS | Automated testing | Data Ingestion | Data Lakehouse | Data ModelingSenior-level Full TimeRedmond, Washington, USA12h ago
-
AWS | Big Data | Data Governance | Distributed Systems | Server-lessFlexible work hours | Inclusive culture | MentorshipSenior-level Full TimeEast Palo Alto, California, USA12h ago
-
Application development | Cloud Platforms | Deep learning | Distributed Systems | Docker401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States13h ago
-
Senior Staff Machine Learning Engineer USD 261K-330KAI architectures | Deep learning | Embeddings | Experimentation | Language ModelsCompetitive compensation | Equity grants | Flexible work options | Health benefits | Remote workSenior-level Full TimeUnited States - Remote R13h ago
-
Staff Machine Learning Engineer, Robotics USD 215K-364KComputer Vision | Data Science | Deep learning | Language Processing | Machine LearningCompetitive compensation | Cutting-edge technology | Impact on transportation and robotics | Snacks and activities | Supportive environmentSenior-level Full TimeSanta Clara, CA13h ago
-
Machine Learning Engineer - Sr. Consultant Level USD 152K-243KAI | Big Data | C++ | CI/CD | Flink401k | Dental | FSA/HSA | Life insurance | MedicalSenior-level Full TimeAustin, TX, United States14h ago
-
Sr. Data Platform Engineer I (6366) USD 170K-239KAgile | Backup & Recovery | CI/CD | Database security | Kubernetes401k plan | Flexible work | Health, dental, vision insurance | Paid time off | Parental leaveSenior-level Full TimeHybrid - Local to D.C. metro … R15h ago
-
Big Data Engineer / Developer USD 76K-136KBigQuery | Data Engineering | Data Processing | ETL Tools | GCPInclusive work environment | Industry training | Professional certificationsSenior-level Full TimeWashington, DC16h ago
-
Data Engineer USD 129K-186KAWS | CI/CD | Cloud Data | Cloud data platform | DBTBenefits package | Collaborative environment | On-site position in Charlotte | Professional development supportMid-level Full TimeCharlotte, North Carolina, United States16h ago
-
Cloud Data Engineer USD 122K-160KAzure | Azure Data | Azure Data Factory | Data Architecture | Data FactoryMid-level Full TimeSouthfield, Michigan, United States16h ago
-
Senior DevOps Engineer USD 180K-200KAWS | Bash | CI/CD | Cloud Security | CloudWatch401k plan | Complete laptop workstation | Flexible PTO | In-office breakfast and snacks | Medical/Dental/Vision insuranceSenior-level Full TimeNew York, New York16h ago
-
Agile Development | Data Science | Java | JavaScript | Machine LearningMentorship | Paid apprenticeship | Remote work | Tuition coverageEntry-level ContractRemote (United States) R16h ago
-
Mid-level Full TimeWoodinville, Washington, United States16h ago