Senior Data Engineer
Tasks
- Build distributed data-processing pipelines
- Collaborate with cross-functional teams
- Conduct code reviews and enforce code standards
- Contribute to product improvements
- Create functional and design specifications
- Design data processing solutions
- Develop scalable database systems
- Identify technical risks and implement mitigation plans
- Implement Spark Structured Streaming solutions
- Lead functional and architectural design
- Optimize Spark workloads
- Perform Spark SQL query performance optimization
- Provide effort estimations and contribute to project planning
- Resolve technical crisis situations
Perks/Benefits
- 401k matching
- Employee assistance program
- Life and disability insurance
- Medical, dental & vision coverage
- Paid time off
Skills/Tech-stack
Apache Spark | Asset bundles | Data Processing | Databricks (asset) | Databricks Asset Bundles | Distributed data | Distributed data processing | Linux | Query Optimization | Spark SQL | Spark Structured Streaming | Structured Streaming | Terraform
Education
N/A
Roles
Related jobs
-
Automation Testing | CI/CD | CSS | Cypress | Feature DevelopmentMedical, dental & vision coverage | Paid time off | Parental leave | Reimbursement programs | Retirement planMid-levelRaleigh, United States R10d ago
-
Databricks Pipeline Architect USD 150K-180KAWS Glue | AWS Lambda | AWS S3 | Agile | Amazon Web ServicesPublic trust clearance support | Remote workSenior-level Full TimeWork from home, VA, United States R8h ago
-
Software Engineer, Ads Data Application USD 136K-205KArtificial Intelligence | Backend Development | Batch Processing | Data Processing | Data WarehouseMid-level Full TimeSan Jose, California, United States12h ago
-
Research Engineer, FAIR AI & Society USD 117K-173KAI alignment | Apache Hive | Apache Spark | Artificial Intelligence | DPOEntry-level Full TimeMenlo Park, CA | New York, …13h ago
-
Software Engineer, Audio Embedded DSP - Reality Labs USD 117K-173KAndroid | Audio Processing | Audio Software | Audio software engineering | C#Mid-level Full TimeSunnyvale, CA13h ago
-
Software Engineer, Systems USD 221K-240KAlgorithms | CSS | Data Analysis | Data Modeling | Data ProcessingEntry-level Full TimeBellevue, WA13h ago
-
Staff Software Engineer, AI/ML, YouTube USD 207K-300KAudio Processing | Data Processing | Debugging | Distributed Systems | Fine TuningSenior-level Full TimeSan Bruno, CA, USA13h ago
-
Senior Staff Software Engineer, AI/ML, Google Cloud USD 262K-365KData Processing | Data Structures | Data Structures and Algorithms | Debugging | Fine TuningSenior-level Full TimeSunnyvale, CA, USA13h ago
-
Senior Software Engineer, AI/ML, Google Cloud AI USD 174K-252KC++ | Data Processing | Data Structures | Data structures algorithms | DebuggingSenior-level Full TimeMountain View, CA, USA13h ago
-
Staff Software Engineer, AI/ML Data Processing USD 207K-300KAPIs | Checkpointing | Cloud technologies | Data Processing | DebuggingSenior-level Full TimeSunnyvale, CA, USA13h ago
-
Senior-level Full TimeSan Jose, CA21h ago
-
Senior Software Engineer - Data Platform USD 186K-218KAccess Control | Airflow | Apache Kafka | Apache Spark | CachingSenior-level Full TimeRemote - USA R22h ago
-
Software Engineer - Reference Data (Java/SQL) USD 175K-200KAPI Development | Distributed Systems | Git | HBase | HadoopMid-level Full TimeNew York22h ago
-
Senior-level Full TimeHybrid (Salt Lake City, UT, US) R23h ago
-
Robotics Hardware Integration Engineer USD 120K-158KAWS IoT | AWS IoT Core | AWS IoT Greengrass | C++ | CANDirect impact on shipped products | Early-stage company | Equity | Health, dental, and vision insurance | Permissive time off policyMid-level Full TimeAtlanta, Georgia, United States - Remote R1d ago
-
Lead AI Engineer (ML Ops) USD 116K-170KAPI Development | AWS | Azure | CI/CD | Cloud services401k match | Charity match | Employee assistance program | Employee stock purchase plan | Health and wellness allowanceSenior-level Full TimeIrving - 6011 Connection, United States1d ago
-
Senior-level Full TimeMcLean, VA, United States1d ago
-
Lead Data Engineer USD 179K-225KAWS | Agile | Amazon Redshift | Azure | CassandraHealth benefits | Inclusive work environment | Long-term incentives | Performance bonusSenior-level Full TimeMcLean, VA, United States1d ago
-
AWS | CI/CD | Cloud platform | Database Administration | Google CloudSenior-level ContractSunnyvale, CA1d ago
-
AI/ML Subject Matter Expert USD 169K-229KAI Risk Management Framework | AWS Bedrock | AWS Lambda | AWS SageMaker | AirflowSenior-level Full TimeUSA LA Home Office (LAHOME), United …1d ago
-
DevOps Engineer, Infrastructure & Security USD 236K-295KAWS | Azure | Azure DevOps | Bash | CI/CDCommuter stipend | Comprehensive health, dental and vision coverage | Generous PTO | Learning and development stipend | Retirement benefitsMid-level Full TimeSan Francisco, CA; New York, NY1d ago
-
AI, Data & Technology - AI-ML Engineer USD 112K-179KAWS | Apache Spark | Azure | BigQuery | CI/CDHybrid work model | In person work two days per week | Travel up to 40 percentSenior-level Full TimeChicago, IL, United States1d ago
-
Alteryx | Apache Spark | Data Mining | Data Querying | Data analyticsBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersMid-level Full TimeWilmington, DE, United States1d ago
-
Staff Analytics Engineer - Foundational Data USD 194K-240KAirflow | Apache Spark | Business Intelligence | Data Modeling | Data ProcessingEmployee travel credits | Remote eligibleSenior-level Full TimeUnited States1d ago
-
Staff Software Engineer, Ads Data Platform USD 221K-279KApache Airflow | Apache Spark | Big Data | DBT | DatabricksAnnual equity refresh grants | Equity grants | Flexible work location | Remote workSenior-level Full TimeUnited States - Remote R1d ago