Lead Data Engineer (Python, Databricks, SQL)
IL-70 W Madison St, 8th Fl, United States
USD 150K-165K Senior-level Full Time
Tasks
- Build analytics and reporting data consumption
- Develop ETL pipelines
- Drive proofs of concept for innovation
- Ensure security and privacy by design
- Implement automated testing
- Improve SDLC throughput using intake estimation
- Manage KPIs for delivery performance
- Modernize data platform and reduce technical debt
- Optimize data pipelines performance
- Oversee risk management and regulatory compliance adherence
- Own end to end data engineering delivery
- Provide technical leadership and hands-on development
- Set up CI/CD pipelines
- Translate business objectives into secure scalable cloud data solutions
Perks/Benefits
- 401k
- Dental insurance
- Disability insurance
- Health savings account
- Holidays
- Hybrid work arrangement
- Life insurance
- Medical insurance
- Paid time off
- Parental leave
- Sick leave
- Vision insurance
- Wellbeing and recognition program
- Wellbeing support
Skills/Tech-stack
Automation | Azure | Azure DevOps | Azure SQL | CI/CD | Cognos | Databricks | ETL | GitHub Copilot | Jira | Observability | Power BI | PySpark | Python | REST APIs | SDLC | SQL | SQL Server | SSIS | Scala | Tableau | Testing
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Related jobs
-
Featured Feat. Applied AI Engineer - Bay Area USD 211K-263KArtificial Intelligence | C plus plus | C# | Embeddings | Feature Engineering401k | Comprehensive health and wellness benefits | Learning and development opportunities | Unlimited time offMid-level Full TimeHQ (San Francisco)24d ago
-
AI Search | Agile | Angular | Azure AI | Azure AI Search100 percent onsite | Public trust clearance requiredSenior-level ContractWoodlawn, United States1h ago
-
Senior Software Engineer - DevOps & Data Engineering USD 140K-190KAgentic Workflows | Automation | CI/CD | Claude | Cloud infrastructureSenior-level ContractChandler, United States2h ago
-
Azure Databricks Developer USD 125K-198KApache Spark | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageSenior-level Full TimeLouisville, Kentucky, United States2h ago
-
Entry-level Full TimeUS-Kansas-Wichita2h ago
-
Mid-level Full TimeUS-Kansas-Wichita3h ago
-
Data Engineer II USD 101K-176KAWS | Agile | Airflow | Azure | Azure DevOps401k matching | Health & dental insurance | Onsite fitness center | Time off | Tuition reimbursementSenior-level Full TimeUS-Rhode Island-Providence3h ago
-
C++ | Cloud Native | Container Orchestration | Deep learning | Distributed SystemsCareer growth | Open Source contribution | World Class CollaborationEntry-level Full TimeSan Jose, California, United States3h ago
-
Linguistic Engineer USD 103K-156KBias Mitigation | Ethical AI | Hack | Knowledge graphs | Machine LearningMid-level Full TimeRedmond, WA4h ago
-
Partner Engineer, Generative AI USD 159K-223KAWS | Agent Orchestration | Azure | Bias Mitigation | C++Senior-level Full TimeMenlo Park, CA4h ago
-
Senior Embedded Flight Software Engineer USD 160K-240KAMD Xilinx | AMD Xilinx FPGAs | AMD/Xilinx SoC | Agile | Aircraft systems401k | Casual dress code | Dental insurance | FSA | Free daily lunchSenior-level Full TimeHawthorne, CA12h ago
-
Senior Data Scientist, Machine Learning USD 194K-218KAWS | Active Learning | Airflow | Amazon Redshift | Automated Labeling100% TelecommutingSenior-level Full TimeRedwood City, CA R13h ago
-
Principal Scientist, Machine Learning - Biomolecules USD 208K-286KAWS Batch | AWS ECS | AWS EKS | AWS S3 | AWS SageMakerAnnual incentive program | Healthcare coverage | Retirement benefitsSenior-level Full TimeCambridge, MA USA14h ago
-
AI Engineer USD 125K-201KAWS | Agent Frameworks | Agent SDK | Agent coordination | Claude Agent SDKCollaboration with little supervision | Startup environment | Work on cutting-edge AIEntry-level Full TimePittsburgh, Pennsylvania, United States14h ago
-
Mid-level Full TimeSan Francisco15h ago
-
Mid-level Full TimeSan Francisco15h ago
-
Data Engineer Data Pipelines and ETL USD 99K-147KAnomaly Detection | Apache Airflow | CDC | Cloud Composer | Data Governance401k plan | Disability benefits | Life insurance | Life insurance coverage | Medical/Dental/VisionMid-level Full TimeBurbank, CA, US, 9150515h ago
-
Sr. Software Engineer, Data Streaming Systems USD 130K-195KAutoscaling | Blocking I/O | CI/CD | Concurrency | Distributed Systems401k plan | Dental insurance | Disability benefits | Life insurance | Medical insuranceSenior-level Full TimeBurbank, CA, US, 9150515h ago
-
Medicaid AI Lead USD 71K-156KArtificial Intelligence | Automation | Capacity Planning | Data Analysis | Data dashboards401k match | Dental insurance | Employee assistance program | Flexible spending account | Health savings accountSenior-level Full TimeSpringfield, IL, US15h ago
-
Lead Analyst, Insights & Analytics USD 82K-172KAd-Hoc | Ad-hoc analysis | Analytics | Analytics Metrics | Business RequirementsSenior-level Full TimeMadisonville Office Building, United States15h ago
-
Analytics | Business Intelligence | Dashboard Development | Data Modeling | Data QualitySenior-level Full TimeMarietta, GA, US16h ago
-
Manager II, Data Science (CECL Modeling) USD 78K-138KChange Management | Competing Risk Models | Credit loss forecasting | Data Mining | Data ReconciliationMid-level Full TimeVienna, VA, United States16h ago
-
Senior Analytics Engineer, GTM USD 176K-265KAPI Integration | CI/CD | DBT | Data Modeling | Data QualityCompany-sponsored team events | Flexible time off | Wellness resourcesSenior-level Full TimeSanta Clara, California16h ago
-
Senior AI Engineer – Developer Products USD 170K-250KAPI Design | LLMOps | Language Models | Large Language Models | MLOpsGenerous benefits | Health benefits | Remote work options | Stock optionsSenior-level Full TimeSan Francisco, California16h ago
-
ETL Engineer_Woonsocket, RI USD 44K-154KAWS | Apache Spark | CI/CD | CloudFormation | Data Quality401k retirement plan | Dental benefits | Medical benefits | Paid Holidays | Paid time offSenior-level Full TimeUnited States16h ago