Lead Software Engineer - Databricks/Spark/AWS
Tasks
- Build reusable Python or Java libraries frameworks and APIs
- Deploy infrastructure with Terraform
- Design secure data ingestion and transformation frameworks
- Drive Spark performance engineering
- Drive adoption of secure AI assisted engineering practices
- Enforce data quality, lineage, and governance
- Establish lakehouse patterns with Delta Lake
- Implement CI CD for data projects with Git workflows
- Lead architecture and delivery of data pipelines
- Orchestrate jobs with Databricks Workflows
- Own Databricks cluster strategy and setup
Perks/Benefits
- Backup childcare
- Financial coaching
- Health care coverage
- Mental health support
- Retirement savings plan
- Tuition reimbursement
Skills/Tech-stack
AWS Glue | AWS IAM | AWS Lambda | AWS S3 | AWS Secrets | AWS Secrets Manager | Amazon CloudWatch | Amazon Kinesis | Amazon MSK | Apache Spark | Azure? N/A | CI/CD | Databricks | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Git | Infrastructure as Code | JUnit | Java | Kafka | N A | Pytest | Python | SQL | Secrets Manager | Structured Streaming | Terraform | Unity Catalog | “as-code”
Education
N/A
Roles
Engineer | Lead | Lead Software Engineer | Software Engineer
Related jobs
-
Senior-level Full TimeRemote, Canada; Computer/Software; Canada, Canada; Remote, … R4h ago
-
ANSI SQL | Amazon Redshift | Analytical modeling | Cloud Data | Cloud Data PlatformsCareer growth | Onsite work | Possible contract extension | Successful Completion Bonus | Support from consultantsMid-level Full TimeJuno Beach, FL; Juno Beach, FL; …4h ago
-
AI Governance | APIs | AWS | Agentic Workflows | Agile deliveryRemote workSenior-level Full TimePlainsboro, NJ. R5h ago
-
AWS | Bitbucket | Confluence | Data Analysis | Eclipse401k employer contribution | Direct contribution to 401k vesting | Educational assistance | Federal Holidays | Healthcare benefitsSenior-level Full TimeAnnapolis Junction, Maryland; Annapolis Junction, Maryland; …5h ago
-
Software Engineer 1 - (Python, UI/UX, LLM) USD 169K-207KAWS | Argo CD | CI/CD | Containerization | Docker401k match | Educational assistance | Federal Holidays | Healthcare coverage | PTOMid-level Full TimeAnnapolis Junction, Maryland5h ago
-
Senior Applied AI Engineer 3 - (Python, LLMs, RAG) USD 241K-295KAI Agents | APIs | Backend Development | Data Pipelines | Docker401k matching | Education assistance | Healthcare coverage | Paid time offSenior-level Full TimeAnnapolis Junction, Maryland5h ago
-
Cloud Software Engineer 3 - (Cloud, Hadoop, Java) USD 228K-278KAnsible | Apache Kafka | Apache Lucene | Apache Solr | Avro401k matching | Educational assistance | Federal Holidays | Healthcare coverage | Paid time offSenior-level Full TimeFort Meade, Maryland R5h ago
-
Technical Delivery Lead USD 160K-225KAI Safety | Agent reasoning | Agent registries | Amazon CloudWatch | Automated testing401k retirement plan | AD&D coverage | Dental insurance | Disability coverage | Employer matchSenior-level Full TimeRemote, United States R9h ago
-
Software Engineer III, AI/ML, Platforms and Devices USD 147K-210KAndroid | C++ | Data Processing | Debugging | Language ProcessingSenior-level Full TimeSeattle, WA, USA10h ago
-
AI Strategy Lead USD 192K-278KAgentic AI | Artificial Intelligence | As-a-Service | Business Intelligence | Data AnalysisSenior-level Full TimeChicago, IL, USA; Sunnyvale, CA, USA10h ago
-
Agents | Artificial Intelligence | Automated Evaluation | C++ | Code reviewSenior-level Full TimeSunnyvale, CA, USA10h ago
-
API Development | Artificial Intelligence | C# | C++ | Computer VisionMid-level Full TimeSunnyvale, CA, USA; Atlanta, GA, USA10h ago
-
Staff ML Engineer, Search Ads Shopping Relevance Models USD 207K-300KC++ | CI/CD | Data Analysis | Data Processing | Deep learningSenior-level Full TimeMountain View, CA, USA10h ago
-
Software Engineer, ML Accelerators, DeepMind USD 174K-252KC++ | Hardware Accelerators | JAX | Kernel development | Machine LearningMid-level Full TimeMountain View, CA, USA; San Francisco, …10h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous work culture | Flexible management approach | Professional growth opportunities | Remote or distributed workMid-level Full TimePalo Alto, CA, USA14h ago
-
Bash | Cloud infrastructure | Data Ingestion | Data Ingestion Pipelines | Data ProcessingAsynchronous culture | Career growth | Impactful mission | Remote-friendly cultureMid-level Full TimeLos Angeles, CA, USA14h ago
-
Bash | Cloud platform | Data Processing | Docker | GCPAsynchronous culture | Friendly work environment | Remote workMid-level Full TimeBellevue, WA, USA15h ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Bonus | Equity | Remote-friendlyMid-level Full TimeDenver, CO, USA15h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous culture | Friendly laid-back atmosphere | Opportunity for career growthMid-level Full TimeAnaheim, CA, USA15h ago
-
Bash | Data Ingestion | Data Ingestion Pipelines | Data Processing | DockerAsynchronous culture | Flexible management approach | Mission driven product impact | Supportive team environmentMid-level Full TimeProvo, UT, USA15h ago
-
Bash | Data Processing | Docker | GCP | Large Scale DataAsynchronous culture | Competitive benefits | Flexible management | Remote workMid-level Full TimeFort Worth, TX, USA15h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Career growth | Remote workMid-level Full TimeHouston, TX, USA15h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerMid-level Full TimeCollege Park, MD, USA15h ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerMid-level Full TimeSaint Paul, MN, USA15h ago
-
Bash | Cloud platform | Data Processing | Data Processing Pipelines | DockerAsynchronous culture | Competitive benefits | Flexible management | Remote-friendly | Supportive teamMid-level Full TimeDurham, NC, USA15h ago