Software Engineer III - Python, Databricks and AWS
Jersey City, NJ, United States
USD 170K-185K (estimate) Senior-level Full Time
Tasks
- Build data ingestion pipelines at scale
- Configure Databricks Workflows and Jobs
- Develop Delta Live Tables pipelines
- Enable CDC ingestion from AWS sources
- Establish CI CD using Terraform and IaC
- Implement batch and streaming processing
- Implement data governance and security controls
- Implement monitoring, alerting, and incident response
- Maintain Databricks Lakehouse pipelines
- Maintain runbooks and documentation
- Manage cluster sizing and storage layout
- Productionize workloads with retries checkpointing and idempotency
- Set up auditing lineage and metadata tracking
- Tune Spark and Delta performance
- Write unit and integration tests and data quality checks
Perks/Benefits
- Backup childcare
- Financial coaching
- Health care coverage
- Mental health support
- On-site health and wellness centers
- Retirement savings plan
- Tuition reimbursement
Skills/Tech-stack
AWS IAM | AWS KMS | AWS S3 | CI/CD | Cause analysis | Checkpointing | CloudWatch | Data Quality | Databricks | Delta Lake | Delta Lake ACID | Delta Lake partitioning | Delta Live Tables | Delta Live) | Idempotency | Incident Response | Infrastructure as Code | Integration Testing | Monitoring | Python | Root Cause Analysis | Root cause | S3 | SLA management | SLO/SLA | SLO/SLA Management | Schema evolution | Security groups | Spark | Structured Streaming | Terraform | Unit Testing | Unity Catalog | VPC | “as-code”
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Featured Feat. Principal Knowledge & Data Architect USD 174K-284KAWS Neptune | Canonicalization | Chunking | Cypher | DBTBenefits including health and wellness programs | Health, wellness, and retirement plansSenior-level Full TimeHeadquarters - Chevy Chase, MD R22d ago
-
APIs | AWS | Agentic AI | Azure | Claude401k | Dental insurance | Fully remote US based | Generous PTO | HolidaysSenior-level Full TimeRemote USA, United States R3h ago
-
Staff Data Engineer USD 190K-250KAWS | Agent systems | Airflow | Alation | Amazon KinesisDental insurance | Flexible working hours | Health insurance | Hybrid options | Learning resourcesSenior-level Full TimeDallas, TX, United States3h ago
-
Staff Software Engineer USD 190K-250KAPI Design | AWS | Autogen | Azure | C#Dental insurance | Flexible working hours | Growth and learning resources | Health insurance | Hybrid optionsSenior-level Full TimeChicago, IL, United States3h ago
-
Staff Software Engineer USD 190K-250KAWS | Autogen | Azure | C# | CI/CDDental insurance | Flexible working hours | Health insurance | Hybrid options | Learning resourcesSenior-level Full TimeChicago, IL, United States3h ago
-
Staff Software Engineer USD 190K-250KAPI Design | AWS | Autogen | Azure | C#Dental insurance | Flexible working hours | Health insurance | Hybrid options | Learning resourcesSenior-level Full TimeChicago, IL, United States3h ago
-
.NET | ArgoCD | Automation | Bash | C#Mid-level Full TimeDallas, Texas, United States; Hermitage, Tennessee, …3h ago
-
AI & Machine Learning Engineer USD 98K-140KAmazon Web Services | Apache Kafka | Azure | C++ | CI/CD401k match | Company paid life insurance | Medical, dental, and vision insurance | Paid Holidays | Paid time offMid-level Full TimeUSA-WISCONSIN-Green Bay4h ago
-
Data Engineer, Technical Leadership (Ranking, AI) USD 182K-235KAI workflows | Data Architecture | Data Governance | Data Modeling | Data QualitySenior-level Full TimeMenlo Park, CA5h ago
-
Software Engineer, SystemML - AI Networking USD 170K-287KC# | C++ | CUDA | Data-parallel | Distributed Data ParallelMid-level Full TimeMenlo Park, CA5h ago
-
Senior Performance Co-Design Engineer, LLM Serving USD 174K-252KC++ | Co-design | Computer Architecture | Deep learning | Deep learning inferenceSenior-level Full TimeSunnyvale, CA, USA5h ago
-
Automated testing | C# | C++ | CI/CD | Continuous DeploymentMid-level Full TimeMountain View, CA, USA5h ago
-
Data Processing | Debugging | Distributed Computing | Information Retrieval | Language ModelsSenior-level Full TimeSunnyvale, CA, USA5h ago
-
C++ | Data Processing | Debugging | Generative AI | Google CloudHealth insurance | Learning and development opportunities | Paid time off | Relocation assistance | Retirement planSenior-level Full TimeMountain View, CA, USA5h ago
-
C++ | Data Processing | Debugging | Language Models | Language ProcessingSenior-level Full TimeMountain View, CA, USA5h ago
-
Senior Software Engineer, AI/ML GenAI, Google Workspace USD 174K-252KAlgorithms | C++ | Data Processing | Data Structures | DebuggingSenior-level Full TimeSunnyvale, CA, USA5h ago
-
Senior Software Engineer, Google Cloud Storage USD 174K-252KAlgorithms | C# | C++ | Cloud Computing | Data IntegritySenior-level Full TimeNew York, NY, USA5h ago
-
Software Engineer III, AI/ML, AI and Infrastructure USD 147K-210KAlgorithms | C++ | Data Processing | Data Structures | DebuggingSenior-level Full TimeSunnyvale, CA, USA5h ago
-
Senior Performance Co-design Engineer, LLM Training USD 174K-252KC++ | Co-design | Computer Architecture | Data parallelism | Distributed TrainingSenior-level Full TimeSunnyvale, CA, USA5h ago
-
Artificial Intelligence | C++ | Cloud Security | Cybersecurity | Data ProcessingSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA5h ago
-
Alerting | Automated remediation | Cause analysis | Chaos Engineering | Distributed StorageSenior-level Full TimeCambridge, MA, USA5h ago
-
Data Engineer - Emergency Preparedness and Response USD 117K-151KAzure | Azure Data | Azure Data Factory | Azure Databricks | Big DataEmergency response availability | Health safety wellbeing focus | Hybrid work scheduleMid-level Full TimeDetroit, MI, US9h ago
-
Software Engineer, Embedded Agentic AI USD 170K-210KAgent Orchestration | Agent systems | C# | C++ | Cloud PlatformsDisability benefits | Equity awards | Health insurance | Life insurance | Paid time offEntry-level Full TimeSan Jose, California12h ago
-
Staff Engineer, GenAI Tooling & DevEx (R5469) USD 121K-197K*nix | AWS | Access Management | Active Directory | Azure GovSenior-level Full TimeWashington, DC14h ago
-
Staff Machine Learning Engineer, Personalization USD 152K-277KAirflow | Apache Spark | BigQuery | Contrastive Learning | Deep learning401k match | Disability insurance | Electric Car Charging Station | Employee assistance program | Flexible spending accountSenior-level Full TimeMountain View, USA16h ago