Senior Software Engineer - Data Platform (Spark & Trino)
Tasks
- Build end to end tracing metrics and alerting
- Build large scale distributed data processing platform in cloud
- Design fault-tolerant, high-availability architectures
- Develop data processing algorithms for large datasets
- Implement multi tenant isolation for elastic workloads
- Implement resiliency and graceful degradation for Spark and Trino clusters
- Optimize resource allocation and cluster auto scaling
Perks/Benefits
Skills/Tech-stack
AWS | Apache Spark | Cloud platform | Distributed Systems | Google Cloud | Google Cloud Platform | Grafana | JMX | Java | Linux | OpenTelemetry | Presto | Prometheus | SQL | Scala | Trino
Education
Regions
Countries
States
Related jobs
-
Bash | Cloud infrastructure | Data Ingestion | Data Processing | DockerAsynchronous culture | Distributed team | Flexible management | Remote-friendlyMid-level Full TimeMontreal, Canada1d ago
-
Senior Software Engineer, Data CAD 180K-230KAPI Development | Agentic coding | Airflow | Alerting | AlloyDBFully remoteSenior-level Full TimeRemote - Canada R1d ago
-
ADC | AWS | CI/CD | Control Theory | DebuggingRemote workMid-level Full TimeCalgary, Alberta, Canada1d ago
-
AI Engineer ll CAD 120K-170KAI Services | APIs | Automation | Azure AI | Azure AI ServicesHybrid work | Paid time off | Paid volunteering days | Wellness benefitsSenior-level Full TimeToronto1d ago
-
Agile | Azure Blob | Azure Blob Storage | Azure Data | Azure Data LakeEmployee assistance program | Financial wellness services | Global career opportunities | Health and wellness programs | Paid time offEntry-level Full TimeCanada, British Columbia1d ago
-
Agile | Azure Blob | Azure Blob Storage | Azure Data | Azure Data LakeEmployee assistance program | Employee stock purchase plan | Health & wellness programs | Paid time off | RRSP/RPP matchingEntry-level Full TimeCanada, Ontario1d ago
-
ABAC | AWS Bedrock | AWS Glue | AWS S3 | Amazon RedshiftEquity or stock options | Health dental and wellness benefits | Professional development opportunities | Remote first within CanadaSenior-level Full TimeCanada2d ago
-
BI Dashboards | DBT | Data Governance | Data Marts | Data ModelingCross-functional collaboration | Fully remote | Health benefits coverage | High ownership | Performance bonusSenior-level Full TimeCanada R2d ago
-
Amazon Web Services | C++ | CUDA | Cloud platform | Deep learningFlexible work arrangements | Fully remoteMid-level Full TimeCanada R2d ago
-
Amazon Web Services | Argo CD | Argo Workflows | Bash | CI/CDEquity opportunities | Family support benefits | Flexible work arrangements | Health, dental, vision coverage | Learning and development supportSenior-level Full TimeCanada R2d ago
-
AI Specialist CAD 77K-100KAmazon Web Services | Cloud Computing | Cloud platform | D3.js | Data ModelingEmployee discounts and other rewards programs | Employer paid health and dental benefits | Extensive training programs | Family oriented team environment | Retirement benefits programMid-level ContractOttawa, Canada; CORE2d ago
-
Senior Staff Machine Learning Platform Engineer CAD 248K-341KAWS | Airflow | Cloud Architecture | Databricks | DatadogComprehensive benefits | Equity | Hybrid work | Professional development | Remote work flexibilitySenior-level Full TimeKitchener-Waterloo, ON; Toronto, ON R2d ago
-
Senior Backend Software Developer, ML Platform CAD 152K-171KAWS | Amazon SageMaker | Amazon Web Services | CI/CD | Infrastructure as CodeSenior-level Full TimeMontreal (Province of Quebec, Canada)2d ago
-
Staff AI Platform & Agent Runtime Engineer CAD 120K-170KAI Foundry | API Design | Agent Orchestration | Autogen | AzureSenior-level Full TimeToronto2d ago
-
Data Engineer CAD 85K-105KCI/CD | Data Modeling | Database Design | ETL | GitHybrid work environment | Inclusion and accessibility supportMid-level Full TimeToronto, Ontario, Canada2d ago
-
Senior Platform Engineer, Technology CAD 90K-110KAgile | Azure | CI/CD | Cloud Architecture | ComputeSenior-level Full TimeCanada2d ago
-
Senior-level Full TimeToronto, Ontario, Canada R2d ago
-
Python Developer – AI Operations CAD 100K-130KAgentic Workflows | Artificial Intelligence | As-a-Service | Asynchronous Python | BashHybrid work environment | International team collaborationMid-level Full TimeMontréal, Quebec2d ago
-
Machine Learning Engineer IV, Data CAD 137K-172KAWS | Computer Vision | Computer Vision Model Serving | Data Annotation | DatabricksEmployee stock purchase program | Healthcare Dental Vision Life ADND | Long-term disability | Parental leave top-up | RRSP retirement planningMid-level Full TimeToronto, ON, CA2d ago
-
Machine Learning Engineer III, Data CAD 111K-139KAWS | Computer Vision | Data Annotation | Databricks | DockerCritical illness coverage | Employee stock purchase program | Generous parental leave top up pay | Life and ADandD insurance | Long-term disabilitySenior-level Full TimeToronto, ON, CA2d ago
-
C++ | CI/CD | Cloud Computing | Distributed Systems | DockerSenior-level Full TimeWaterloo, ON, Canada; Toronto, ON, Canada3d ago
-
AWS | Alerting | Azure | BigQuery | CI/CDEquity opportunities | Fully remote within Canada | Health, dental, vision coverage | Paid parental leave | Performance-Based IncentivesSenior-level Full TimeCanada R3d ago
-
Big Data | Continuous Delivery | Data Engineering | Data Governance | Data LakesFlexible working arrangements | Health, dental, and vision insurance | Mentorship and career growth | Professional development | Remote-friendlySenior-level Full TimeCanada3d ago
-
Consultant, Data Engineer CAD 92K-120KApplication development | Azure Databricks | Code review | Data Engineering | Data pipelineBanking benefits | Employee share purchase plan | Paid day off for professional development | Pension plan | Social recognition programSenior-level Full TimeToronto-81 Bay, 15th Floor, Canada3d ago
-
Principal MCP/AI Developer CAD 153K-224KAPIs | Agent Frameworks | Auditability | Cost Optimization | Data platformsSenior-level Full TimeAMER - Canada - Ontario - …3d ago