Software Engineer, Managed Service for Apache Spark
Tasks
- Architect implement and operate Dataproc Control Plane
- Build customer facing features for running Hadoop Spark Flink AI ML in the cloud
- Design and implement features for Dataproc
- Enable global scale resource provisioning and cluster orchestration
- Ensure high availability fault tolerance and state management
Perks/Benefits
- N/A
Skills/Tech-stack
API Integration | Apache Flink | Apache Hadoop | Apache Spark | Apache YARN | BigQuery | Cluster Orchestration | Compute Engine | Data Governance | Data Lineage | Distributed Computing | Distributed File System | Fault Tolerance | File System | Google Compute | Google Compute Engine | Hadoop Distributed File System | High Availability | Presto | State management
Education
Roles
Related jobs
-
Computer Vision | Data Processing | Data Storage | Debugging | Deep learningSenior-level Full TimeSunnyvale, CA, USA3h ago
-
Data Engineer, Product Data Warehouse, Go-To-Market USD 156K-226KApache Flume | Apache Spark | Business Intelligence | Code review | DashboardsMid-level Full TimeNew York, NY, USA; Atlanta, GA, …3h ago
-
Staff Machine Learning Engineer 2, Ads USD 159K-309KAWS | Airflow | Apache Spark | BigQuery | Cloud Platforms401k plan company match | Disability insurance | Electric Car Charging Station | Employee assistance program | Flexible spending accountSenior-level Full TimeMountain View, USA13h ago
-
Staff Machine Learning Engineer 2, Ads USD 164K-282KAWS | Airflow | Amazon SageMaker | Apache Spark | BigQuery401k plan with company match | Dental insurance | Disability insurance | Electric car charging | Employee assistance programSenior-level Full TimeMountain View, USA13h ago
-
Data Lead (Defense) USD 96K-198KAPI Design | Airflow | Anomaly Detection | Apache Flink | Apache KafkaSenior-level Full TimeHawaii, US15h ago
-
Lead Software Engineer - Data Platform USD 150K-175KAPIs | AWS | CI/CD | Data Governance | Data Modeling401k match | Company holidays | Dental insurance | Disability insurance | Employee assistance programSenior-level Full TimeUnited States R16h ago
-
People Systems Developer USD 145K-215KAPI Integration | Anthropic | Automation | Google Gemini | Google WorkspaceContinuous career development | Flexible time off | Friendly inclusive workplace culture | Gym reimbursement | Hybrid workMid-level Full TimeBoston, Massachusetts, USA; Denver, Colorado, USA; …16h ago
-
AI Data Engineer USD 133K-185KAmazon Redshift | BigQuery | CI/CD | CRM Integration | Churn Prediction401(k) self contribution | BetterUp coaching | Company wide summer break | Company-wide winter break | Development planSenior-level Full TimeAustin, TX17h ago
-
Senior Data Engineer USD 125K-160KAWS CDK | AWS CodeCommit | AWS CodePipeline | AWS Glue | AWS LambdaAgile team collaborationSenior-level Full TimeUnited States17h ago
-
Data Engineer USD 122K-180KAzure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOps401k with company match | Collaborative and independent work mix | Community outreach opportunities | Company paid disability and life insurance | Comprehensive medical, dental and vision coverageMid-level Full TimeRemote-US R18h ago
-
Software Engineer, Data Infrastructure USD 170K-225KAirflow | Apache Beam | Apache Flink | Apache Spark | BigQueryCo-working stipend | Health and dental benefits | Open, inclusive culture | Parental leave top-up | Personal enrichment benefitsMid-level Full TimeNew York19h ago
-
AI Specialist - Data & AI Team USD 67K-80KBigQuery | Cloud platform | Data Ingestion | Data Quality | Data Transformation100 Percent Healthcare Coverage | 401k match | Bonus eligible | Education support | Free lunchMid-level Full TimeEnglewood Cliffs, NJ, United States20h ago
-
Senior Software Engineer, AI Solutions USD 120K-142KAI Agents | API Integration | AWS Bedrock | Airflow | Amazon SageMakerDental insurance | Dependent Care Account | Employer match | Flexible spending account | Health insuranceSenior-level Full TimeNew York, NY, United States20h ago
-
Senior Data Engineer USD 120K-142KAPIs | Apache Airflow | DBT | Data Governance | Data Modeling401b retirement savings program | Employer match | Flexible spending accounts | Health, dental, vision insurance | Mental health counseling supportSenior-level Full TimeNew York, NY, United States20h ago
-
Senior AI Engineer USD 120K-180KAPI Integration | Agent Orchestration | Agent systems | Autogen | Compliance checks401k match | Career growth | Dental insurance | Employee resource groups | Health insuranceSenior-level Full TimeRemote, United States R21h ago
-
Senior Cloud Database Engineer USD 144K-180KAWS | AWS Step Functions | Amazon Aurora | CI/CD | Database AdministrationDevelopment dollars stipend | Employee stock purchase program | Financial coaching | Flexible time off | Free therapy sessionsSenior-level Full TimeRemote, USA R21h ago
-
Senior Cloud Database Engineer USD 144K-180KAWS | AWS Step Functions | Amazon Aurora | CI/CD | Database AdministrationFinancial coaching | Flexible time off | Free therapy sessions | Legal advice | Parental leaveSenior-level Full TimeRemote, USA R21h ago
-
Senior DataOps Engineer- Remote US USD 130K-168KAWS CloudFormation | Amazon EMR | Amazon Web Services | Ansible | Apache Airflow401k contribution | Employee assistance program | Flexible Time Away From Work | Health & medical benefits | Learning programsSenior-level Full TimeRemote US R22h ago
-
Principal AI Platform Engineer USD 190K-225KACR | API Integration | Audit Logging | Azure | Azure DevOps401k employer match | Career growth and professional development opportunities | Employee assistance program | Medical Dental Vision Coverage Options | Paid HolidaysSenior-level Full TimeRemote (United States) R22h ago
-
Senior Data Engineer, Core Experimentation USD 293K-325KAmazon S3 | Apache Airflow | Apache Flink | Apache Hadoop | Apache SparkHybrid work modelSenior-level Full TimeSeattle22h ago
-
Staff Data Engineer, Analytics USD 248K-279KAlerting | Anomaly Detection | BI reporting | BigQuery | CI/CDSenior-level Full TimeSan Francisco Bay Area22h ago
-
Apache Iceberg | Apache OpenLineage | Apache Spark | Columnar file formats | DuckDBSenior-level Full TimeNew York, New York, USA1d ago
-
Data Engineer USD 159K-200KAWS DMS | AWS Glue | Amazon ECR | Amazon ECS | Amazon RedshiftFlexible remote weeks around holidays | Generous perks | In-office collaborationSenior-level Full TimePhiladelphia, Pennsylvania, United States1d ago
-
Bill of Materials | Change Management | Data Cleansing | Data Governance | Data Management401k | Collaborative team environment | Cross training | Dental insurance | Medical insuranceMid-level Full TimeWichita, KS, United States1d ago
-
Senior Software Engineer, ML Infrastructure, Core Infra USD 174K-252KC++ | Data Processing | Data Storage | Debugging | Distributed ComputingSenior-level Full TimeSunnyvale, CA, USA1d ago