Lead Software Engineer - Databricks
Wilmington, DE, United States
USD 175K-200K (estimate) Senior-level Full Time
Tasks
- Build reusable Python and Java libraries
- Deliver high throughput low latency data pipelines
- Design secure data ingestion and transformation
- Enforce data quality, lineage, and governance
- Establish lakehouse patterns with Delta Lake
- Implement CI/CD for data projects
- Integrate workflows with AWS services
- Lead architecture for data pipelines
- Orchestrate and automate pipeline jobs
- Own Databricks cluster strategy and configuration
- Tune Spark and Databricks performance
Perks/Benefits
- Backup childcare
- Comprehensive healthcare coverage
- Financial coaching
- Mental health support
- On-site health and wellness centers
- Retirement savings plan
- Tuition reimbursement
Skills/Tech-stack
AWS EMR | AWS Glue | AWS Glue Catalog | Airflow | Apache Spark | Automated testing | CI/CD | Databricks | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Encryption | Git | Glue Catalog | Java | Networking | Python | SQL | SQL Warehouses | Secrets management | Security groups | Structured Streaming | Terraform | Unity Catalog | VPC | VPC Subnets
Education
N/A
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R1h ago
-
Data Engineer III (Top Secret Clearance Required) USD 107K-179KAWS | Azure | Cloud Integration | Computer Vision | Data PipelinesSenior-level Full TimeArlington/Rosslyn, Virginia, United States4h ago
-
Delivery Consultant, Data Engineering and Gen AI USD 107K-179KAgentic AI | Amazon Web Services | Apache Spark | Artificial Intelligence | Cloud ComputingLimited sponsorship available | Travel opportunitiesMid-level Full TimeGilbert, Arizona, United States; Lake Mary, …4h ago
-
Mid-level Full TimeArlington/Rosslyn, Virginia, United States4h ago
-
Software Engineer, AI/ML Computer Vision, Pixel Camera USD 147K-211KC++ | Computer Vision | Data Processing | Debugging | Image classificationMid-level Full TimeMountain View, CA, USA6h ago
-
Cloud Data and AI Engineer, Professional Services USD 127K-183KAlloyDB | Aurora | Bigtable | C++ | Capacity PlanningTechnical workshops and training delivery | Travel up to 30 percentMid-level Full TimeReston, VA, USA6h ago
-
API Design | Apache Flink | Apache Presto | Apache Spark | AutoscalingSenior-level Full TimeSunnyvale, CA, USA6h ago
-
Infra Engineer- US USD 145K-170KAWS | Cloud Cost Optimization | Cloud platform | Cost Optimization | GoMid-level Full TimeNew York City, NY, US9h ago
-
Machine Learning Engineer USD 136K-215KAgentic AI | Autonomous Vehicles | Bayesian Inference | Code review | Deep learningMid-level Full TimeChicago, IL, US11h ago
-
Forward Deployed AI Engineer USD 140K-240KAPIs | AWS | Azure | C# | Computer Vision401k matching | Comprehensive healthcare | Education stipend | Stock awards | Wellness stipendMid-level Full TimeMountain View, CA14h ago
-
Director, AI / Machine Learning Data Engineer USD 127K-250KCloud Computing | Continuous Delivery | Continuous integration | Data Engineering | Data PipelinesExecutive-level Full TimeNew York, NY, United States16h ago
-
Senior-level Full TimeCedar Rapids, IA16h ago
-
Senior Analytics Engineer USD 87K-161KAutomation | CI/CD | DBT | Data Ingestion | Data Modeling401k | Flexible workplace options | Hybrid work options | Paid time off | Remote work optionsSenior-level Full TimeRemote-FL, United States R17h ago
-
Senior AI Data Science Engineer USD 149K-201KAnomaly Detection | Apache Airflow | Apache Spark | Cloud Computing | Computer VisionSenior-level Full TimeSunnyvale, CA, United States17h ago
-
AI Deployment Engineer, Enterprise USD 197K-278KAgents | Cost Optimization | Data Governance | Debugging | EvaluationMid-level Full TimeSan Francisco18h ago
-
AI Deployment Engineer, Digital Natives USD 197K-278KEnterprise Architecture | Generative AI | JavaScript | OpenAI API | Project ManagementHybrid work model | Relocation assistanceSenior-level Full TimeNew York City18h ago
-
AI Deployment Engineer, Digital Natives USD 197K-278KAPI Integration | Enterprise Architecture | Generative AI | JavaScript | OpenAI APIsHybrid work model | Relocation assistanceSenior-level Full TimeSan Francisco18h ago
-
Access Controls | Azure | Azure Data | Azure Data Factory | Azure DevOpsHybrid work model | No benefits for contract position | Paid relocation not available | Remote work optionsSenior-level ContractDetroit R18h ago
-
BI & Data Engineer USD 110K-130KBI Data modeling | BI Gateway | CI/CD | Container Services | DAXHybrid work | Onsite 3 days per week | St Louis onsiteSenior-level Full TimeSt. Louis, MO18h ago
-
Analytics Engineer USD 90K-110KAPI Development | Access Control | DBT | Data Governance | Data LineageAnnual cash bonus | Comprehensive benefits package | Hybrid work | Onsite 3 days per weekSenior-level Full TimeSt. Louis18h ago
-
Ansible | BGP | Bash | Ceph | Ceph RBDEducation reimbursement | Health plans | Hybrid work model | In-office perks | Parental leave optionsSenior-level Full TimeBellevue, Washington, USA; San Jose, California, …18h ago
-
Sr. Forward Deployed Engineer (FDE) - Public Sector USD 182K-250KAWS | Apache Spark | Azure | CI/CD | DatabricksSenior-level Full TimeWashington, D.C.19h ago
-
Scientific Software Engineer - AMO Simulations USD 125K-180KAPIs | C++ | CUDA | Chebyshev polynomials | Code reviewMid-level Full TimeBoston, MA, USA19h ago
-
Lead Software Engineer - Databricks/Snowflake/AWS USD 175K-186KAPI | API Gateway | AWS ECS | AWS Lambda | Amazon APIBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site wellness centersSenior-level Full TimePlano, TX, United States20h ago
-
A/B | A/B Testing | B testing | Computer Vision | Deep Neural NetworksSenior-level Full TimeSan Francisco, CA, US; Palo Alto, …20h ago