Staff Software Engineer - Streaming
Tasks
- Design and build new stream processing operators
- Design and build stream processing state management
- Drive stream processing adoption across product portfolio
- Guide architecture and technical debt decisions
- Improve streaming engine architecture latency and throughput
- Improve streaming engine cost efficiency
- Mentor and lead engineers
- Raise engineering quality and operability for production
- Set technical vision for Spark Structured Streaming
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Spark | Data Storage | Database systems | Distributed Systems | Latency optimization | Operational excellence | Performance optimization | Software Architecture | Spark Structured Streaming | Structured Streaming | Throughput Optimization
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Software Engineer, API Multimodal USD 293K-385KConcurrency | Distributed Systems | Go | Latency | ObservabilitySenior-level Full TimeSan Francisco11h ago
-
Machine Learning Engineer I, Network USD 151K-189KClassification | Data Pipelines | Deep learning | Experiment tracking | Feature Engineering401k match | Commuting support | Flexible PTO | Free lunch | Gym accessSenior-level Full TimeSan Francisco, CA13h ago
-
Senior Software Engineer, Data Lake USD 196K-230KAWS S3 | Apache Flink | Apache Kafka | Apache Spark | Change Data Capture401k matching | Disability insurance | Fertility benefits | Health insurance | Life insuranceSenior-level Full TimeBellevue, WA14h ago
-
Principal AI Engineer USD 185K-265KAPI Integration | Agent systems | Artificial Intelligence | Cloud Native | Cloud-native ArchitecturesEducation reimbursement | Health plans | Paid time off | Parental leave | Remote workSenior-level Full TimeRemote - USA R14h ago
-
AI Engineer - FDE (Forward Deployed Engineer) USD 152K-210KAWS | Agent systems | Apache Spark | Azure | Databricks Intelligence PlatformRemote work | Travel opportunitiesSenior-level Full TimeUnited States14h ago
-
Senior Data Engineer II, AI Native USD 148K-216KAmazon Web Services | Apache Airflow | Apache Spark | Azure | BigQuery401k matching | Dental insurance | Disability insurance | Employee assistance program | Equipment reimbursementSenior-level Full TimeRemote, USA R14h ago
-
API Design | AWS | Access Control | Amazon S3 | Apache AirflowEmployee discounts | Employee equity | Medical, dental & vision coverage | Pet insurance | Unlimited PTOSenior-level Full TimeNashville, TN15h ago
-
API Design | AWS | Apache Airflow | Apache Flink | Apache KafkaDental insurance | Employee discounts | Employee equity | Health insurance | Pet insuranceSenior-level Full TimeRemote - United States R15h ago
-
API Integration | Apache Airflow | Apache Spark | Big Data | Classified EnvironmentsHealth insurance | Paid leave | RetirementSenior-level Full TimeHanover, MD15h ago
-
Senior Data Engineer USD 115K-175KAWS Fargate | AWS Glue | AWS IAM | AWS Lambda | AWS MWAACharitable giving program | Company-Paid Holidays | Dental insurance | Flexible PTO | Life insuranceSenior-level Full TimeMassachusetts - Boston R16h ago
-
Desk Apps Developer USD 95K-130KApache Spark | Data pipeline | FastAPI | HTML5 | Hadoop401k company match | Dental | Life insurance | Long-term disability | MedicalMid-level Full TimeHouston, TX, United States16h ago
-
Autonomy | C++ | DDS | Data Ingestion | Edge Computing401k | Healthcare coverage | Paid Holidays | Paid time off | Phone stipendMid-level Full TimeSan Carlos - Hybrid R17h ago
-
A/B | A/B Testing | API Development | AWS Bedrock | Access Control401k plan | Healthcare premium coverage | Paid Company Holidays | Paid time off | Phone stipendSenior-level Full TimeSan Carlos - Hybrid R17h ago
-
Autonomy | C++ | CPU GPU | CPU GPU Debugging | DDS401k | Healthcare | Paid Holidays | Paid time off | Phone stipendSenior-level Full TimeSan Carlos - Hybrid R17h ago
-
Senior Software Engineer - Machine Learning USD 183K-223KApache Flink | Apache Spark | Apache Storm | Deep learning | Generative AI401k match | Dental insurance | Health insurance | Life insurance | Long-term disabilitySenior-level Full TimeSan Francisco, CA 94158, United States17h ago
-
Senior Staff Engineer - Senior Data Engineer USD 130K-166KAmazon Web Services | Apache Airflow | Apache Kafka | Apache Spark | DatabricksSenior-level Full TimeGrand-Prairie, Texas, United States20h ago
-
Staff Data Engineer USD 190K-250KAWS | Agent systems | Airflow | Alation | Amazon KinesisDental insurance | Flexible working hours | Health insurance | Hybrid options | Learning resourcesSenior-level Full TimeDallas, TX, United States21h ago
-
Staff Software Engineer USD 190K-250KAPI Design | AWS | Autogen | Azure | C#Dental insurance | Flexible working hours | Growth and learning resources | Health insurance | Hybrid optionsSenior-level Full TimeChicago, IL, United States21h ago
-
Staff Software Engineer USD 190K-250KAWS | Autogen | Azure | C# | CI/CDDental insurance | Flexible working hours | Health insurance | Hybrid options | Learning resourcesSenior-level Full TimeChicago, IL, United States21h ago
-
Staff Software Engineer USD 190K-250KAPI Design | AWS | Autogen | Azure | C#Dental insurance | Flexible working hours | Health insurance | Hybrid options | Learning resourcesSenior-level Full TimeChicago, IL, United States21h ago
-
AWS | Agile | Apache Hive | Apache Spark | Cloudera Data PlatformRemote workSenior-level ContractLincoln, United States R21h ago
-
Middle Data Engineer (Azure Databricks) USD 160K-223KApache Spark | Azure Data | Azure Data Factory | Azure Databricks | Azure DevOpsCertification programs | Health insurance | Mentorship | Professional development opportunities | Relocation programMid-level Full TimeMadrid, MD, Spain R22h ago
-
Staff Software Engineer, YouTube Ads Machine Learning USD 207K-300KData Processing | Data Storage | Distributed Computing | Embeddings | Fine TuningBonus | Equity | Health insurance | Paid time off | Retirement plansSenior-level Full TimeMountain View, CA, USA23h ago
-
Data Structures | Data Structures and Algorithms | Distributed query | Distributed query processing | Linux KernelSenior-level Full TimeMountain View, CA, USA; Zürich, Switzerland23h ago
-
Staff Software Engineer, Storage Intelligence Platform USD 207K-300KC++ | Data Structures | Data Structures and Algorithms | Distributed Systems | Large-scaleSenior-level Full TimeSeattle, WA, USA23h ago