Lead Data Engineer (Databricks, SQL)
IL-70 W Madison St, 8th Fl, United States
USD 150K-170K Senior-level Full Time
Tasks
- Build ETL analytics and data consumption
- Define secure scalable data architecture
- Develop solutions using Databricks Python PySpark and SQL Server
- Drive modernization via proofs of concept
- Embed risk management and security privacy by design
- Ensure regulatory compliance and partner with compliance and audit
- Implement automation testing CI CD and observability
- Improve SDLC throughput via intake estimation and KPI management
- Modernize and reduce technical debt
- Own end to end delivery of data strategy capabilities
- Translate business objectives into secure scalable cloud data solutions
Perks/Benefits
- 401k
- Disability insurance
- Health savings account
- Hybrid work arrangement
- Life insurance
- Medical/Dental/Vision insurance
- Paid time off
- Parental leave
- Sick leave
- Wellbeing support
Skills/Tech-stack
Advanced Analytics | Agile | Automation | Azure | Azure Blob | Azure Blob Storage | Azure DevOps | Azure Functions | Azure SQL | Blob Storage | CI/CD | Cognos | Data Ingestion | Data Optimization | Data Pipeline Development | Data pipeline | Databricks | DevOps | ETL | GitHub | GitHub Copilot | Integration Services | Jira | Observability | Pipeline Development | Power BI | PySpark | Python | REST APIs | Reporting | SDLC | SQL | SQL Server | SQL Server Integration Services | SQL Server integration | SSIS | Scala | Server Integration Services | Tableau | Testing
Education
Associate Degree | Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Data Engineer | Engineer | Lead | Lead Data | Lead Data Engineer
Regions
Countries
States
Cities
Related jobs
-
Automation Testing | CI/CD | CSS | Cypress | Feature DevelopmentMedical, dental & vision coverage | Paid time off | Parental leave | Reimbursement programs | Retirement planMid-levelRaleigh, United States R11d ago
-
Robotics Test & Data Engineer, Mapping USD 70K-300K3D Point Cloud Processing | 3D point cloud | Bash | Cloud processing | Cloud validationMid-level Full TimeIrvine, CA4h ago
-
Fullstack - Data Platform (Autonomy) USD 170K-210KAWS RDS | Amazon DocumentDB | Database Query | Database Query Optimization | DjangoDental insurance | Health insurance | Paid time off | Vision insuranceSenior-level Full TimeSouth San Francisco, California, USA9h ago
-
Deployed Engineer (Seattle) USD 150K-250KAWS | Agent Orchestration | Containers | Demo Delivery | Failure handling401k plan | Dental insurance | Flexible vacation | Health insurance | Life insuranceMid-level Full TimeSeattle, WA10h ago
-
Systems Software Engineer - Robotic Platform USD 150K-200K*nix | C++ | Deterministic builds | Docker | LinuxSenior-level Full TimeAustin, Texas, United States11h ago
-
Software Engineer - Robotic Applications USD 160K-245KAutonomy Models | Behavior systems | C++ | Controls | LocalizationSenior-level Full TimeAustin, Texas, United States11h ago
-
Senior Developer, Data & IT - AI Solutions USD 120K-142KAI Agents | API Integration | AWS | AWS Bedrock | AWS SageMakerDental insurance | Dependent Care Account | Health insurance | Health savings account | Mental health counseling supportSenior-level Full TimeNew York, NY, United States11h ago
-
Senior Data Engineer USD 120K-142KApache Airflow | C# | Continuous integration | DBT | Data Modeling401b retirement savings program | Dental insurance | Dependent Care Account | Employer match | Flexible spending accountsSenior-level Full TimeNew York, NY, United States12h ago
-
Technical Support Engineer USD 90K-125KBusiness Intelligence | CTE | Data Modeling | Data Visualization | Database401k | Commuter benefits | Dog-friendly office | Equity | FSA benefitsSenior-level Full TimeSan Francisco, CA12h ago
-
Technical Support Engineer USD 90K-125KAWS | Amazon Redshift | Business Intelligence | CTE | Cloud Computing401k | Annual bonus | Commuter benefits | Dog-friendly office | EquitySenior-level Full TimeNew York City, NY12h ago
-
AI Research Engineer USD 190K-280KAgentic AI | Clinical data | Data Pipelines | Data integration | Deep learningDiversity and inclusion initiatives | Flexible work environment | Friendly work environment | Professional developmentMid-level Full TimeSeattle, Washington, United States; South San …12h ago
-
Principal AI Engineer - Nexus Black USD 135K-160KCI/CD | Cloud Native | Cloud Native Architecture | Distributed Systems | EvaluationHybrid workSenior-level Full TimeItasca, United States13h ago
-
Generative AI Inference Engineer USD 152K-287KAWS | CUDA | Cloud platform | Diffusion Models | DockerSenior-level Full TimeUnited States13h ago
-
Distinguished Software Engineer, Data Infrastructure USD 248K-406KAI Inference | AI Training | Batch Processing | Compliance | Data InfrastructureExecutive-level Full TimeMountain View, CA, United States14h ago
-
Senior Machine Learning Engineer USD 108K-160KA/B | A/B Testing | AWS Lambda | Algolia | Amazon OpenSearch401k | Dental insurance | Disability insurance | Life insurance | Medical insuranceSenior-level Full TimeRemote US R14h ago
-
Senior-level Full TimeMorristown, NJ, United States14h ago
-
Staff AI/ML Engineer USD 240K-270KAI Agents | AWS | Azure | Cloud infrastructure | Data Curation401k | Commuter benefits | Dog-friendly office | Equity | FSA benefitsSenior-level Full TimeSan Francisco, CA15h ago
-
Staff AI/ML Engineer USD 240K-270KAWS | Agentic Workflows | Azure | Cloud infrastructure | Data Curation401k | Commuter benefits | Dog-friendly office | Equity | FSA benefitsSenior-level Full TimeNew York City, NY15h ago
-
Data Engineer I USD 85K-100KAWS Glue | AWS Lambda | Amazon Athena | Amazon Redshift | Amazon S3401k match | Direct student loan payment program | Employee recognition rewards | Flexible vacation | HRAMid-level Full TimeUnited States15h ago
-
Senior AI/ML Engineer USD 240K-270KAI Agents | AWS | Azure | Cloud infrastructure | Data Curation401k | Commuter benefits | Dog-friendly office | Equity | FSA benefitsSenior-level Full TimeSan Francisco, CA15h ago
-
Data Science Manager USD 155K-225KEvaluation Frameworks | Experiment design | GenAI | Logging | Machine LearningMid-level Full TimeRemote, United States R15h ago
-
Staff Data Engineer - Information Security USD 152K-248KAzure | Big Data | C# | C++ | Data PipelinesHealth and wellness programs | Time away from workSenior-level Full TimeSunnyvale, CA, United States15h ago
-
Manager, Data Engineering USD 130K-166KAWS | Access Controls | Apache Airflow | Audit Logging | AzureCollaborative team culture | Remote work | Work-life balanceSenior-level Full TimeRemote, United States R15h ago
-
Senior Software Engineer - Data Platform USD 150K-195KAPIs | BigQuery | C++ | CI/CD | Cloud Storage401k company contribution | Disability insurance | Fertility and infertility benefits | Industry competitive PTO | Learning and development opportunitiesSenior-level Full TimeUnited States R15h ago
-
AI Search | API Development | AWS | AWS Bedrock | Azure401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceSenior-level Contract Full TimeRidgefield Park, NJ, United States16h ago