Lead Data Engineer (Python, Databricks, SQL)
IL-70 W Madison St, 8th Fl, United States
USD 150K-165K Senior-level Full Time
Tasks
- Build analytics and reporting data consumption
- Develop ETL pipelines
- Drive proofs of concept for innovation
- Ensure security and privacy by design
- Implement automated testing
- Improve SDLC throughput using intake estimation
- Manage KPIs for delivery performance
- Modernize data platform and reduce technical debt
- Optimize data pipelines performance
- Oversee risk management and regulatory compliance adherence
- Own end to end data engineering delivery
- Provide technical leadership and hands-on development
- Set up CI/CD pipelines
- Translate business objectives into secure scalable cloud data solutions
Perks/Benefits
- 401k
- Dental insurance
- Disability insurance
- Health savings account
- Holidays
- Hybrid work arrangement
- Life insurance
- Medical insurance
- Paid time off
- Parental leave
- Sick leave
- Vision insurance
- Wellbeing and recognition program
- Wellbeing support
Skills/Tech-stack
Automation | Azure | Azure DevOps | Azure SQL | CI/CD | Cognos | Databricks | ETL | GitHub Copilot | Jira | Observability | Power BI | PySpark | Python | REST APIs | SDLC | SQL | SQL Server | SSIS | Scala | Tableau | Testing
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Featured Feat. Applied AI Engineer - Bay Area USD 211K-263KArtificial Intelligence | C plus plus | C# | Embeddings | Feature Engineering401k | Comprehensive health and wellness benefits | Learning and development opportunities | Unlimited time offMid-level Full TimeHQ (San Francisco)24d ago
-
Linguistic Engineer USD 103K-156KBias Mitigation | Ethical AI | Hack | Knowledge graphs | Machine LearningMid-level Full TimeRedmond, WA3h ago
-
Partner Engineer, Generative AI USD 159K-223KAWS | Agent Orchestration | Azure | Bias Mitigation | C++Senior-level Full TimeMenlo Park, CA3h ago
-
Senior Embedded Flight Software Engineer USD 160K-240KAMD Xilinx | AMD Xilinx FPGAs | AMD/Xilinx SoC | Agile | Aircraft systems401k | Casual dress code | Dental insurance | FSA | Free daily lunchSenior-level Full TimeHawthorne, CA10h ago
-
Senior Data Scientist, Machine Learning USD 194K-218KAWS | Active Learning | Airflow | Amazon Redshift | Automated Labeling100% TelecommutingSenior-level Full TimeRedwood City, CA R12h ago
-
Principal Scientist, Machine Learning - Biomolecules USD 208K-286KAWS Batch | AWS ECS | AWS EKS | AWS S3 | AWS SageMakerAnnual incentive program | Healthcare coverage | Retirement benefitsSenior-level Full TimeCambridge, MA USA12h ago
-
AI Engineer USD 125K-201KAWS | Agent Frameworks | Agent SDK | Agent coordination | Claude Agent SDKCollaboration with little supervision | Startup environment | Work on cutting-edge AIEntry-level Full TimePittsburgh, Pennsylvania, United States13h ago
-
Mid-level Full TimeSan Francisco13h ago
-
Mid-level Full TimeSan Francisco14h ago
-
Data Engineer Data Pipelines and ETL USD 99K-147KAnomaly Detection | Apache Airflow | CDC | Cloud Composer | Data Governance401k plan | Disability benefits | Life insurance | Life insurance coverage | Medical/Dental/VisionMid-level Full TimeBurbank, CA, US, 9150514h ago
-
Sr. Software Engineer, Data Streaming Systems USD 130K-195KAutoscaling | Blocking I/O | CI/CD | Concurrency | Distributed Systems401k plan | Dental insurance | Disability benefits | Life insurance | Medical insuranceSenior-level Full TimeBurbank, CA, US, 9150514h ago
-
Medicaid AI Lead USD 71K-156KArtificial Intelligence | Automation | Capacity Planning | Data Analysis | Data dashboards401k match | Dental insurance | Employee assistance program | Flexible spending account | Health savings accountSenior-level Full TimeSpringfield, IL, US14h ago
-
Analytics | Business Intelligence | Dashboard Development | Data Modeling | Data QualitySenior-level Full TimeMarietta, GA, US15h ago
-
Senior Analytics Engineer, GTM USD 176K-265KAPI Integration | CI/CD | DBT | Data Modeling | Data QualityCompany-sponsored team events | Flexible time off | Wellness resourcesSenior-level Full TimeSanta Clara, California15h ago
-
Senior AI Engineer – Developer Products USD 170K-250KAPI Design | LLMOps | Language Models | Large Language Models | MLOpsGenerous benefits | Health benefits | Remote work options | Stock optionsSenior-level Full TimeSan Francisco, California15h ago
-
ETL Engineer_Woonsocket, RI USD 44K-154KAWS | Apache Spark | CI/CD | CloudFormation | Data Quality401k retirement plan | Dental benefits | Medical benefits | Paid Holidays | Paid time offSenior-level Full TimeUnited States15h ago
-
Machine Learning Engineer USD 140K-190KApache Flink | Apache Kafka | Apache Spark | Bigtable | CI/CDMid-level Full TimeRemote - USA R15h ago
-
Staff AI/ML Engineer USD 240K-270KAWS | Agentic Workflows | Azure | Data Curation | Deep learning401k | Commuter benefits | Dog-friendly office | Equity | FSA benefitsSenior-level Full TimeNew York City, NY15h ago
-
Staff AI/ML Engineer USD 240K-270KAWS | Agentic Workflows | Cloud platform | Deep learning | Foundation Models401k | Commuter benefits | Dog-friendly office | Equity | Flexible spending accountSenior-level Full TimeSan Francisco, CA15h ago
-
Senior AI/ML Engineer USD 240K-270KAWS | Agentic Workflows | Azure | Data Curation | Deep learning401k | Commuter benefits | Dog-friendly office | Equity | FSA benefitsSenior-level Full TimeNew York City, NY15h ago
-
Senior AI/ML Engineer USD 240K-270KAWS | Agentic Workflows | Artificial Intelligence | Azure | Cloud infrastructure401k | Commuter benefits | Dog-friendly office | Equity | Flexible spending accountSenior-level Full TimeSan Francisco, CA15h ago
-
BigQuery | Cloud Functions | Compute Engine | DBT | Fivetran11 paid holidays | 401k matching | Basic life insurance | Commuter benefits | Disability insuranceMid-level Full TimeNew York, NY15h ago
-
Research Engineers, Data USD 150K-250KData Annotation | Data Drift | Data Modeling | Data Pipelines | Data Quality401k | Access to modern AI tools | Commuter benefits | In-office lunch | Medical, dental & vision coverageSenior-level Full TimeSan Francisco15h ago
-
VP, Embedded Software USD 290K-350KBLE | CI/CD | Controls engineering | Device Software | Device to Cloud401k matching | Company holidays | Dental insurance | Employee stock purchase program | Flexible spending accountsExecutive-level Full TimeNeedham, MA, United States16h ago
-
Senior Embedded Software Engineer - Future Forward USD 153K-201KAgile | Authentication | Board Bring-up | Bring-up | C#Senior-level Full TimeSunnyvale, CA, United States R16h ago