Software Engineer, Managed Service for Apache Spark
Tasks
- Architect implement and operate Dataproc Control Plane
- Build customer facing features for running Hadoop Spark Flink AI ML in the cloud
- Design and implement features for Dataproc
- Enable global scale resource provisioning and cluster orchestration
- Ensure high availability fault tolerance and state management
Perks/Benefits
- N/A
Skills/Tech-stack
API Integration | Apache Flink | Apache Hadoop | Apache Spark | Apache YARN | BigQuery | Cluster Orchestration | Compute Engine | Data Governance | Data Lineage | Distributed Computing | Distributed File System | Fault Tolerance | File System | Google Compute | Google Compute Engine | Hadoop Distributed File System | High Availability | Presto | State management
Education
Roles
Related jobs
-
Senior Data Engineer (Remote) USD 155KAgile | Apache Spark | BigQuery | Cassandra | Data Governance401k match | Dental insurance | Employee assistance program | Employee stock purchase plan | Flexible scheduleSenior-level Full TimeWork From Home, United States R6h ago
-
Apache Flink | Apache Spark | Automation | C++ | Cause analysisSenior-level Full TimeSan Jose, California, United States11h ago
-
Agentic data | Apache Hive | Apache Spark | Coding Data | Data CurationSenior-level Full TimeMenlo Park, CA12h ago
-
Staff Software Engineer, Agentic AI, Trust and Safety USD 207K-301KAgentic AI | Anti-abuse | Anti-abuse systems | Architecture ownership | Artificial IntelligenceSenior-level Full TimeKirkland, WA, USA12h ago
-
Customer Engineer, AI/ML, Startups, Google Cloud USD 104K-151KAI Studio | Agent systems | Agentic Workflows | Artificial Intelligence | AutogenMid-level Full TimeAustin, TX, USA12h ago
-
Staff Software Engineer, YouTube Ads Machine Learning USD 207K-301KData Processing | Distributed Computing | Embeddings | Fine Tuning | Information RetrievalSenior-level Full TimeMountain View, CA, USA12h ago
-
Senior Staff Software Engineer, Dataproc, Control Plane USD 262K-365KC++ | Cloud APIs | Cloud Computing | Data Structures | Data Structures and AlgorithmsSenior-level Full TimeSunnyvale, CA, USA; Kirkland, WA, USA12h ago
-
Applied AI ML Researcher Lead USD 177K-215KAblation Testing | Agent systems | Artificial Intelligence | Distributed Computing | Error AnalysisBackup childcare | Financial coaching | Health care coverage | Mental health support | On-site health and wellness centersSenior-level Full TimePalo Alto, CA, United States17h ago
-
Lead Analytics Engineer USD 123K-175KCI/CD | DBT | Data Governance | Data Modeling | NetsuiteFull-time telecommuting | Remote work optionSenior-level Full TimePalo Alto, California19h ago
-
Software Development Engineer, Aurora Storage USD 143K-194KAWS | Amazon Aurora | Distributed Systems | High Availability | MySQLCareer growth | Flexible work schedule | Mentorship | Work-life balanceMid-level Full TimeRedmond, Washington, USA23h ago
-
Data Platform Engineer USD 182K-240KAWS | Amazon Kinesis | Apache Airflow | Apache Flink | Apache Kafka401k | Dental insurance | Family leave | Flexible paid time off | Free food and snacksSenior-level Full TimeOrlando, Florida, United States - Remote R23h ago
-
Mid-level Full TimePENNANT PARK, ATLANTA - 9141, United …23h ago
-
Manager Data Engineering USD 102K-177KAWS | Airflow | Automated testing | Azure | Azure DataInclusive work environment | Remote work possibility | Travel up to 20 percentMid-level Full TimeUS014 MA Raynham - 325 Paramount …23h ago
-
Senior Data Engineer, Cat Digital USD 112K-183KAWS | BigQuery | Cloud Data | Cloud Data Pipelines | Cloud platform401k savings plan | Adoption benefits | Career development | Employee assistance program | Employee discountsSenior-level Full TimeChicago, Illinois, United States23h ago
-
Product Analytics Data Builder, Senior Associate USD 86K-146KApache Airflow | Data Lineage | Data Modeling | Data Quality | Data ValidationBook clubs | Collaborative workspaces | Commuter support | Employee assistance program | Employee resource groupsSenior-level Full TimeBoston MA, United States23h ago
-
Data Engineer USD 84K-153KAzure | Azure Data | Azure Data Lake | Azure Data Lake Storage | CI/CD401k | Childcare subsidy | College coaching | Commuter and transit programs | Employee stock grantEntry-level Full TimeWA-Headquarters, Building 5, United States23h ago
-
Senior Data Engineer - Athlete (REMOTE) USD 83K-138KData Governance | Data Lineage | Data Modeling | Data Privacy | Data TestingSenior-level Full TimeRemote - US, United States R23h ago
-
Senior-level ContractNew Jersey, NJ23h ago
-
Senior Data Engineer IS - Remote USD 122K-208KAPI | API Integration | Access Control | Alerting | BashRemote work | Rotational on-call supportSenior-level Full TimeRenton, WA, United States R23h ago
-
Senior-level Full TimeUnited States - Remote R1d ago
-
Sr Data Engineer USD 115K-145KAWS S3 | Apache Airflow | Azure Blob | Azure Blob Storage | BigQuery401k | Dental insurance | Discounts | Fully remote | Health insuranceSenior-level Full TimeNew York, NEW YORK, United States R1d ago
-
Mid-level Full TimeLinthicum Heights, MD1d ago
-
Data Engineer USD 119K-168KAI Agents | Azure Data | Azure Data Factory | Azure Functions | Azure Lakehouse401k | Dental coverage | Health coverage | Paid parental leave | Paid time offMid-level Full TimeSterling Heights1d ago
-
Staff Software Engineer, AI Runtime USD 190K-265KCUDA | Checkpointing | Data parallelism | DeepSpeed | Distributed SystemsSenior-level Full TimeMountain View, California; San Francisco, California1d ago
-
Senior Software Engineer, AI Runtime USD 160K-225KAlgorithms | Checkpointing | Collective communication | Data Structures | Data parallelismSenior-level Full TimeMountain View, California; San Francisco, California1d ago