Lead Software Engineer - Databricks/Spark/AWS
Tasks
- Build reusable libraries and APIs with testing
- Design secure data ingestion and transformation frameworks
- Drive Spark performance engineering
- Enforce data quality, lineage, and governance
- Establish lakehouse patterns with Delta Lake
- Implement CI/CD for data projects
- Lead architecture and delivery of data pipelines
- Orchestrate Databricks workflows and integrate AWS eventing
- Own Databricks cluster strategy and setup
Perks/Benefits
- Backup childcare
- Financial coaching
- Health care coverage
- Mental health support
- On-site health and wellness centers
- Retirement savings plan
- Tuition reimbursement
Skills/Tech-stack
AQE | AWS CloudWatch | AWS Glue | AWS IAM | AWS Lambda | AWS S3 | AWS Secrets | AWS Secrets Manager | Amazon Kinesis | Amazon MSK | Apache Spark | Autoscaling | Broadcast joins | CI/CD | Data Governance | Data Lineage | Databricks | Delta Lake | Delta Lake ACID | Delta Live Tables | Delta Live) | Git | JUnit | Java | Kafka | Pytest | Python | SQL | Secrets Manager | Shuffle Tuning | Spark SQL | Spark Streaming | Streaming SQL | Structured Streaming | Structured Streaming SQL | Terraform | Unity Catalog
Education
N/A
Related jobs
-
Senior Machine Learning Engineer USD 155K-200KC++ | Data Engineering | Data Ingestion | Data Retrieval | Data Storage401k employer contributions | All-hands meetings | Healthcare coverage | Team Member Appreciation AwardsSenior-level Full TimeFremont, CA5h ago
-
Senior Embedded Software Engineer - SATCOM USD 139K-193KBootloader | C# | C++ | CI/CD | Continuous integrationDental insurance | Disability insurance | FSA | HSA | Health insuranceSenior-level Full TimeWestminster, CO7h ago
-
Senior Embedded Software Engineer - Spacecraft USD 160K-200KAttitude Dynamics | C plus plus | C# | Code profiling | Communication SystemsDental insurance | Disability insurance | FSA | HSA | Health insuranceSenior-level Full TimeWestminster, CO7h ago
-
Embedded Software Engineer II - SATCOM USD 125K-175KBoard Bring-up | Bootloader | Bring-up | CI/CD | Configuration ManagementCompany stock options | Dental insurance | Disability insurance | FSA | HSAEntry-level Full TimeWestminster, CO7h ago
-
Embedded Software Engineer II USD 105K-145KBoard Bring-up | Bring-up | C# | C++ | CIDental insurance | Disability insurance | FSA | HSA | Health insuranceMid-level Full TimeAustin, TX8h ago
-
Embedded Software Engineer II USD 105K-145KBootloader | Bring-up | C# | C++ | CI/CDDental insurance | Disability insurance | FSA | HSA | Health insuranceMid-level Full TimeWestminster, CO8h ago
-
Embedded Software Engineer II USD 105K-145KBootloader | C# | C++ | CI/CD | Code reviewDental insurance | Disability insurance | FSA | HSA | Health insuranceMid-level Full TimeEl Segundo, CA8h ago
-
Software Engineer, Data Flywheel Platform USD 283K-335KAirflow | Dagster | Data orchestration | Databricks | Distributed SystemsSenior-level Full TimeSunnyvale12h ago
-
Artificial Intelligence | C# | C++ | CAD | Design of ExperimentsHealth insurance | Paid time off | Recovery BenefitsSenior-level Full TimeCosta Mesa, California, United States21h ago
-
Senior Software Engineer, Robotics Platform USD 175K-269KC# | C++ | CUDA | DDS | Embedded SystemsOn-site workSenior-level Full TimeSan Francisco1d ago
-
Mechatronic/Robotics Engineer or Systems Integration Engineer - Camera and Sensor Calibration USD 139K-211KAutomation systems | Bash | C++ | Camera Calibration | Control SystemsCollaborative team environment | Hands-on engineering role | On site work full timeMid-level Full TimeMountain View, CA1d ago
-
Senior-level Full TimeMountain View, CA1d ago
-
Mid-level Full TimeRedwood City, CA1d ago
-
Mid-level Full Time200 Varick St #802, New York, …1d ago
-
AC DC Sizing | AC/DC | Arc Flash | Artificial Intelligence | AutomationMultiple project extensions | Paid on time | Remote work optionSenior-level Full TimeJuno Beach, FL R1d ago
-
AI Agents | API Development | Azure | CI/CD | LLMOpsCollaborative cross-functional team | Fully remoteMid-level Full TimeCanada, Ontario; Toronto, None; Toronto, Ontario; … R1d ago
-
API Integration | AWS Bedrock | Auditing | Authentication | AuthorizationMentoring | MentorshipMid-level Full TimeMenlo Park, CA, United States1d ago
-
Lead Palantir AI and Data Science Engineer II USD 159K-265KAI workflows | Analytics | Cloud Computing | Dashboards | Data GovernanceProfessional development | Travel opportunitiesSenior-level Full TimeRosslyn, Virginia, United States1d ago
-
Analytics and Insights Engineer III USD 131K-218KApache Spark | Data Governance | Data Modeling | Data Pipelines | Data QualitySenior-level Full TimeRichmond, Virginia, United States1d ago
-
3D Rendering | A/B | A/B Testing | Agentic AI | AlgorithmsSenior-level Full TimeSan Jose, California, United States1d ago
-
C++ | Computer Vision | Data Processing | Deep learning | Domain AdaptationEntry-level Full TimeSan Jose, California, United States1d ago
-
Senior Software Engineer, Google Cloud Dataproc USD 174K-252KApache Hive | Apache Spark | C++ | Data Warehouse | Data Warehouse InternalsSenior-level Full TimeSunnyvale, CA, USA1d ago
-
C++ | Cloud Platforms | Data Protection | Disaster Recovery | Distributed SystemsSenior-level Full TimeSunnyvale, CA, USA1d ago
-
Agentic Orchestration | C++ | Data Onboarding | Data Pipeline Engineering | Data pipelineSenior-level Full TimeSan Jose, CA, USA1d ago
-
Senior Software Engineer, AI/ML, Core USD 174K-252KC++ | Data Processing | Data Structures | Data Structures and Algorithms | DebuggingSenior-level Full TimeMountain View, CA, USA1d ago