Lead Software Engineer - Python/PySpark/Databricks/AWS
Wilmington, DE, United States
USD 170K-186K (estimate) Senior-level Full Time
Tasks
- Apply software development life cycle toolchain capabilities
- Automate remediation of recurring issues
- Build and operate Databricks batch and streaming pipelines
- Deliver CI CD and infrastructure as code for Databricks and AWS
- Design Databricks lakehouse solutions
- Develop secure production code
- Drive adoption of AI assisted engineering practices
- Ensure production readiness with testing monitoring and incident response
- Execute software solutions design development technical troubleshooting
- Implement governance security controls in Databricks
- Lead communities of practice
- Lead vendor and internal evaluation sessions
- Optimize Spark Delta performance and cost
- Review debug others code
Perks/Benefits
- N/A
Skills/Tech-stack
Access Management | Agile | Alerting | Amazon S3 | Amazon Web Services | Apache Spark | Automated testing | Backfill | CDC | CI/CD | Cause analysis | Checkpointing | Contract testing | Data Quality | Databricks | Databricks Jobs | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Idempotency | Idempotent Pipelines | Identity and Access Management | Identity and access | Incident Response | Infrastructure as Code | Integration Testing | Key Management | Key Management Service | Least Privilege | Monitoring | PII Classification | Peer review | PySpark | Python | Retry) | Root Cause Analysis | Root cause | SLO/SLA | Schemata Evolution | Secure Coding | Security groups | Structured Streaming | Test automation | Unit Testing | Unity Catalog | VPC | Web Services | “as-code”
Education
N/A
Roles
Engineer | Lead | Lead Software Engineer | Software Engineer
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R20d ago
-
Staff Software Engineer USD 110K-165KAI Safety | AWS | Agentic Workflows | Airflow | Automated testingHealth insurance | Onsite work presence required | Paid time off | Retirement contributionsSenior-level Full TimeBurlington, Massachusetts, US, 018037h ago
-
Senior-level Full TimeSeattle, WA11h ago
-
Senior-level Full TimeRedmond, WA11h ago
-
Senior Data Engineer USD 100K-118KAWS | Amazon Redshift | Aurora PostgreSQL | Data Modeling | Data PipelinesCareer development opportunities | Dental insurance | Life insurance | Medical insurance | Paid HolidaysSenior-level Full TimeNew York, NY, US11h ago
-
Senior Software Engineer, Data Engineering USD 172K-230KApache Airflow | Apache Spark | Flink | Presto | Python401k matching | Catered meals | Company holidays | Disability insurance | Fertility benefitsSenior-level Full TimeMenlo Park, CA11h ago
-
Lead Software Engineer - Java, AI Lead USD 170K-195KAWS | AWS EC2 | AWS Lambda | AWS S3 | Access ManagementBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimePlano, TX, United States12h ago
-
Staff ML Engineer, Agent Training & Environments USD 250K-280KApache Kafka | CI/CD | Cloud platform | Direct Preference Optimization | Google CloudCareer growth tied to impact | Hybrid work model | Ownership and autonomySenior-level Full TimeSan Francisco Bay Area R13h ago
-
Senior AI Engineer USD 170K-200KAPI Integration | Anomaly Detection | Artificial Intelligence | Caching | Embeddings401k match | Flexible paid time off | Gym access | HSA contribution | Medical, dental, and visionSenior-level Full TimeRemote, USA R13h ago
-
Senior Software Engineer USD 119K-234KAnomaly Detection | Apache Flink | Apache Kafka | Apache Spark | AzureHybrid workSenior-level Full TimeUnited States, Washington, Redmond14h ago
-
Senior Space Embedded Linux Software Engineer USD 169K-216KAnalog electronics | Bare Metal | C# | C++ | CAN401k employer match | Dental insurance | EV charging | Flexible PTO | HSA employer matchSenior-level Full TimeEl Segundo, California, United States14h ago
-
Senior Software Engineer USD 119K-234KApache Flink | Apache Kafka | Apache Spark | Azure | Azure BlobSenior-level Full TimeUnited States, Washington, Redmond14h ago
-
Space Embedded Linux Software Engineer USD 131K-184KAnalog electronics | Bare Metal | C# | C++ | CAN401k employer match | EV charging | Flexible PTO | HSA employer match | Medical/Dental/Vision insuranceEntry-level Full TimeEl Segundo, California, United States14h ago
-
Senior/Staff Software Engineer, Data Platform USD 155K-185KAWS | AWS CloudFormation | Airflow | Amazon S3 | Apache Spark11 paid holidays | 401k plan | Commuter benefits | Dental insurance | Flexible vacationSenior-level Full TimeBoulder, Colorado14h ago
-
AWS Glue | AWS S3 | Amazon Kinesis | Ansible | Apache Flink401k | Dental insurance | Medical insurance | Paid sick hours | Vision insuranceMid-level Contract Full TimeEnglewood Cliffs, NJ, United States14h ago
-
Senior AI/ML Engineer USD 134K-197KAWS | Automated testing | Azure | CI/CD | Containerization401k match | Flexible PTO | Flexible spending account | Health, dental, and vision insurance | Paid Cloud Developer AccountsSenior-level Full TimeArlington, VA15h ago
-
AI/ML Engineer USD 86K-180KAPI Development | AWS | Automated testing | Azure | CI/CD401k match | Access to sports tickets | Dental insurance | Federal Holidays | Flexible PTOMid-level Full TimeArlington, VA15h ago
-
DataOps Engineer USD 110K-140KAWS Glue | AWS S3 | Amazon Kinesis | Ansible | Apache FlinkOn-call rotation | Relocation requiredMid-level Contract Full TimeEnglewood Cliffs, NJ, United States15h ago
-
Lead Data Analyst USD 110K-171KAgile | Automation | Claims analytics | Confluence | Data Governance401k | Flexible work arrangements | Health insurance | Hybrid work | Mentorship programsSenior-level Full TimeSan Francisco, CA | Lehi, UT …15h ago
-
Lead Software Engineer - Big Data USD 175K-186KACID | Apache Cassandra | Apache HBase | Apache Iceberg | Apache SparkBackup childcare | Health care coverage | Mental health support | Retirement savings plan | Tuition reimbursementSenior-level Full TimePlano, TX, United States15h ago
-
Analytics Engineer USD 138K-215KAccess Control | Data Documentation | Data Glossary | Data Governance | Data Lineage401k match | Dental insurance | Health insurance | Health savings account | Paid time offSenior-level Full TimeCottonwood Heights, Utah, United States15h ago
-
Senior System Analyst - Robotic Controls Engineer USD 125K-201KC# | C++ | Control Theory | Electromechanical Systems | Feedback controlSenior-level Full TimeSunnyvale, CA, United States15h ago
-
Senior DataOps Engineer USD 94K-136KAWS | Access Control | Amazon Aurora | Amazon DynamoDB | Amazon RDS401k employer match | AD and D insurance | Dental insurance | LTD insurance | Life insuranceSenior-level Full TimeRemote, United States R15h ago
-
Senior Applied AI/ML Engineer USD 180K-230KAgentic Workflows | Artificial Intelligence | Benchmarking | Compliance | Data Privacy401k retirement plan | Flexible time off | Health, dental and vision coverage | Opportunities for company events and connection | Remote or hybrid flexibilitySenior-level Full TimeSanta Monica, CA16h ago
-
Sr. AI/ML Engineer II (6761) USD 152K-195KArtificial Intelligence | MLOps | Machine Learning | PyTorch | Python401k match | Employee Assistance and Wellness Programs | Flexible work arrangements | Health, dental, and vision insurance | Paid time offMid-level Full TimeHybrid - Local to D.C. metro … R16h ago