Lead Software Engineer - Databricks/Spark/AWS
Tasks
- Build reusable Python or Java libraries frameworks and APIs
- Deploy infrastructure with Terraform
- Design secure data ingestion and transformation frameworks
- Drive Spark performance engineering
- Drive adoption of secure AI assisted engineering practices
- Enforce data quality, lineage, and governance
- Establish lakehouse patterns with Delta Lake
- Implement CI CD for data projects with Git workflows
- Lead architecture and delivery of data pipelines
- Orchestrate jobs with Databricks Workflows
- Own Databricks cluster strategy and setup
Perks/Benefits
- Backup childcare
- Financial coaching
- Health care coverage
- Mental health support
- Retirement savings plan
- Tuition reimbursement
Skills/Tech-stack
AWS Glue | AWS IAM | AWS Lambda | AWS S3 | AWS Secrets | AWS Secrets Manager | Amazon CloudWatch | Amazon Kinesis | Amazon MSK | Apache Spark | Azure? N/A | CI/CD | Databricks | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Git | Infrastructure as Code | JUnit | Java | Kafka | N A | Pytest | Python | SQL | Secrets Manager | Structured Streaming | Terraform | Unity Catalog | “as-code”
Education
N/A
Roles
Engineer | Lead | Lead Software Engineer | Software Engineer
Related jobs
-
Senior-level Full TimeNorth America, United States3h ago
-
SOFTWARE DATA ENGINEER - Enterprise Infrastructure - 5+ yrs of Experience - TS/SCI w/Poly clearance is required - ES A USD 168K-172KCloud Security | Cloud infrastructure | Linux | Network Automation | OpenStack401k retirement plan | Dental insurance | Health insurance | Life insurance | Long-term disabilityMid-level Full TimeLaurel, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | REducation budget | Fitness budget | Flextime | Mentorship | Personalized growth roadmapsSenior-level Full TimeLos Angeles, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KCybersecurity | Data Visualization | Machine Learning | PyTorch | PythonEducation budget | Fitness budget | Flexible schedule | Mentorship | Professional growthSenior-level Full TimeSan Francisco, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | RFlexible schedule | Mentorship | Office work | Professional growth | Remote workSenior-level Full TimeBoca Raton, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | REducation budget | Fitness budget | Flextime | Mentorship | Personalized growth roadmapsSenior-level Full TimeOrlando, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | RFlexible schedule | Mentorship | Professional growth | Remote and office options | TechtalksSenior-level Full TimeBaltimore, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | REducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeBlacksburg, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | REducation budget | Fitness budget | Flexible schedule | Mentorship | Professional growthSenior-level Full TimeBoston, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | PyTorch | Python | R | SQLFlexible schedule | Mentorship | Office options | Professional growth | Remote work optionsSenior-level Full TimeChicago, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | REducation budget | Fitness budget | Flexible schedule | Mentorship | Personalized growth roadmapsSenior-level Full TimeRichmond, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KCybersecurity | Data Visualization | Machine Learning | PyTorch | PythonFlextime | Mentorship | Personalized growth roadmaps | Remote work options | TechtalksSenior-level Full TimePort Charlotte, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | RFlexible schedule | Mentorship | Professional growth roadmap | Remote work options | TechtalksSenior-level Full TimeWest Palm Beach, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Modeling | Data Visualization | Machine Learning | PyTorch | PythonEducation budget | Fitness budget | Flexible schedule | Mentorship | Office optionsSenior-level Full TimeHouston, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | RFlexible schedule | Mentorship | Office options | Personalized growth roadmaps | Professional growthSenior-level Full TimeTampa, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | REducation budget | Fitness budget | Flexible schedule | Mentorship | Office work optionsSenior-level Full TimeFort Lauderdale, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KCybersecurity | Data Visualization | Machine Learning | PyTorch | PythonEducation budget | Fitness budget | Flexible schedule | Mentorship | Professional growthSenior-level Full TimeTexas City, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | RFlexible schedule | Professional growth | Remote and office optionsSenior-level Full TimeIrving, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KCloud Security | Cloud Security Posture | Cloud Security Posture Management | Data Visualization | Machine LearningFlextime | Mentorship | Office work options | Personalized growth roadmaps | Remote work optionsSenior-level Full TimeMiami, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | RFlextime | Professional growth | Remote work optionsSenior-level Full TimeTallahassee, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | RFlextime | Mentorship | Office options | Personalized growth roadmaps | Professional growthSenior-level Full TimeAustin, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | Predictive Modeling | PyTorch | PythonFlextime | Mentorship | Office options | Personalized growth roadmaps | Professional growthSenior-level Full TimeNew York, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KPyTorch | Python | R | SQL | TensorFlowFlexible schedule | Mentorship | Personalized growth roadmaps | Professional growth | Remote work optionSenior-level Full TimeJacksonville, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | RFlexible schedule | Mentorship | Office options | Personalized growth roadmaps | Professional growthSenior-level Full TimeJersey City, United States4h ago
-
Senior Data Scientist ID71658 USD 160K-190KData Visualization | Machine Learning | PyTorch | Python | REducation budget | Fitness budget | Flexible schedule | Personalized growth roadmaps | Professional growthSenior-level Full TimeAtlanta, United States4h ago