PySpark Big Data Senior Developer - Vice President
5900 HURONTARIO STREET MISSISSAUGA, Canada
CAD 120K-170K Senior-level Full Time
Tasks
- Build CI/CD pipelines
- Build event-driven architectures
- Create dimensional models
- Deploy and monitor data processing jobs
- Design Big Data solutions
- Design data warehousing solutions
- Develop and optimize data pipelines
- Ensure data quality and security
- Implement real-time data streaming
- Implement testing
- Lead technical discussions
- Mentor junior engineers
- Perform code reviews
- Select distributed storage technologies
- Translate business requirements into technical specifications
- Troubleshoot distributed big data issues
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS Glue | Amazon Redshift | Amazon S3 | Apache Hive | Apache Kafka | Apache Spark | Avro | Azure | Azure Data | Azure Data Factory | Azure Databricks | Azure Synapse | Azure Synapse Analytics | BigQuery | Cassandra | Continuous Deployment | Continuous integration | Data Factory | Data Processing | Data processing performance optimization | Databricks SQL | Dataproc | Delta Lake | Docker | EMR | Event Driven | Event-driven architecture | Git | Google Cloud | Google Dataflow | HBase | HDFS | Hadoop | Iceberg | Kubernetes | MongoDB | NoSQL | ORC | Parquet | Performance optimization | PySpark | Python | REST API | Real Time | Real-time Streaming | SQL | Snowflake | Spark SQL | Spark Streaming | Synapse Analytics | YARN
Education
Bachelor of Engineering | Bachelor of Science | Master of Science | PhD
Related jobs
-
Ansible | Apache Airflow | CI/CD | DBT | Data CleansingFlexible vacation | Flexible work options | Hackathons | Mentorship programs | Recognition programsEntry-level InternshipOttawa, ON, Canada8h ago
-
Infrastructure Engineer - DataOps Team CAD 180K-200KAWS | Application Firewall | ArgoCD | Bash | CI/CDOn-site work in TorontoMid-level Full TimeToronto, Ontario, Canada13h ago
-
Lead Data Engineer CAD 150K-241KBig Data | Continuous Delivery | Data Architecture | Data Engineering | Data GovernanceCareer development support | Learning and development | Mentoring | Remote workSenior-level Full TimeToronto, Canada R13h ago
-
Senior-level Full TimeVancouver, British Columbia, Canada R21h ago
-
Senior Data Engineer CAD 120K-145KApache Hadoop | Apache Kafka | Apache Spark | Azure Fabric | CI/CDFlexible working hours | Health and wellness benefits | Learning and development | Parental leave | Retirement planningSenior-level Full TimeToronto23h ago
-
Software Engineer II - Model Platform CAD 100K-159KAWS | Asynchronous Communication | Azure | Data Pipelines | Data ProcessingMid-level Full TimeRemote - Canada R23h ago
-
Mid-level Full TimeCA-ON-Toronto1d ago
-
Senior Engineer, Data CAD 120K-140KAzure | Data Modeling | Data Quality | Data Warehousing | Data integrationAssociate discount program | Continuous development | Diverse, inclusive work environment | Training and mentoringSenior-level Full TimeBrampton, ON, Canada1d ago
-
Senior Software Engineer (Typescript / FrontEnd) - AI/ML CAD 140K-195KAI Inference | API Design | AWS | Azure | Cloud deploymentFlexible time off | Global gatherings | Healthcare | Home office setup | Remote-friendlySenior-level Full TimeCanada R1d ago
-
AI Agents | Access Management | Azure OpenAI | C# | ComplianceContinuous learning | Dental insurance | Flexible work arrangements | Fully remote | Health insuranceMid-level Full TimeCanada R1d ago
-
AWS | Amazon Bedrock | Code review | Conversational State | Distributed SystemsFlexible paid time off | Health spending account | Long-term disability insurance | Medical, dental & vision coverage | RRSP employer matchingSenior-level Full TimeCanada1d ago
-
Airflow | CI/CD | Cloud Architecture | Data Pipelines | DockerAccess to continuous learning | Collaborative consulting environment | Hybrid or remote friendly work arrangements | Professional growth opportunities | Thought leadership and presentation opportunitiesSenior-level Full TimeCanada1d ago
-
Senior Data Platform Engineer USD 148K-227KAWS | Airbyte | Ansible | Apache Airflow | DBT401k matching | Dental benefits | Health benefits | IT Setup Stipend | Internet and mobile stipendSenior-level Full TimeRemote - Canada Only R1d ago
-
Bilby | CAMB | Class | DOLFINx | Differential EquationsFreelance projects | Part-time schedule | Project based workSenior-level FreelanceCanada - Remote R1d ago
-
Staff Data Eng, Data Systems, TCGplayer CAD 142K-190KApache Kafka | Data Architecture | Data Engineering | Data Governance | Data Lifecycle ManagementSenior-level Full TimeRemote Canada R1d ago
-
Big Data Engineer CAD 100K-120KAWS S3 | Agile | Amazon EC2 | Amazon EMR | Amazon RDSGenerous time off | Hybrid work | Leaves | On-call rotationSenior-level Full TimeToronto - 100 Adelaide St W, …1d ago
-
AI Engineer CAD 129K-142KAI vector search | Agent Development Kit | Agentic Systems | CI/CD | Cloud platformFlexible benefitsMid-level Full TimeToronto - 181 Bay St, Canada1d ago
-
Ingénieur en machine learning/Machine Learning Engineer, ProServe Shared Delivery Team - Data & AI CAD 99K-166KAWS | Artificial Intelligence | Cloud Migration | Data Preparation | Generative AIMid-level Full TimeMontreal, Quebec, CAN1d ago
-
Senior Software Engineer (Data Engineering) CAD 140K-210KAWS | AWS Infrastructure | Apache Avro | Apache Flink | Apache IcebergFlexible work schedule | In office collaboration time | Remote work optionSenior-level Full TimeCanada, BC, Vancouver1d ago
-
Machine Learning Engineer (with Vertex AI Experience) CAD 152K-180KAuthentication | Automl | BigQuery | CI/CD | Cloud BuildCareer development opportunities | Entrepreneurial environment | Equal employment opportunity | High responsibilitySenior-level Full TimeCanada - Remote R2d ago
-
.NET | API Integration | AWS | Agentic Systems | AngularCollaborative innovation culture | Inclusive supportive environment | Mentorship | Professional growth | Remote-first work environmentSenior-level Full TimeCanada2d ago
-
Analyst/Consultant/Senior Consultant, Quantitative Market Risk Models (12 Month Contract) CAD 56K-126KBinomial trees | C# | C++ | Capital Markets | Counterparty CreditDevelopment and Innovation Days | Flexible work arrangements | Hybrid work structure | Learning days | Mental health support benefitsSenior-level ContractToronto, ON, CA, M5C 3G72d ago
-
Staff Firmware Engineer - memory constrained embedded system development/data center interconnectivity CAD 98K-130KBash | C# | Debugging | Device Driver Development | Device DriversSenior-level Full TimeCA-ON - Ottawa, Canada2d ago
-
MATLAB | NumPy | Pandas | Python | RPart-time project-based workSenior-level FreelanceCanada - Remote R2d ago
-
Sr. Machine Learning Ops Engineer CAD 99K-132KAWS | Alerting | Azure | Batch inference | CI/CDHealth benefits | Paid time off | Retirement planSenior-level Full TimeMississauga, ON, CAN - 2300 Meadowvale …2d ago