PySpark Big Data Senior Developer - Vice President
CAD 120K-170K Senior-level Full Time
Tasks
- Build CI/CD pipelines
- Build event-driven architectures
- Create dimensional models
- Deploy and monitor data processing jobs
- Design Big Data solutions
- Design data warehousing solutions
- Develop and optimize data pipelines
- Ensure data quality and security
- Implement real-time data streaming
- Implement testing
- Lead technical discussions
- Mentor junior engineers
- Perform code reviews
- Select distributed storage technologies
- Translate business requirements into technical specifications
- Troubleshoot distributed big data issues
Perks/Benefits
- N/A
Skills/Tech-stack
AWS | AWS Glue | Amazon Redshift | Amazon S3 | Apache Hive | Apache Kafka | Apache Spark | Avro | Azure | Azure Data | Azure Data Factory | Azure Databricks | Azure Synapse | Azure Synapse Analytics | BigQuery | Cassandra | Continuous Deployment | Continuous integration | Data Factory | Data Processing | Data processing performance optimization | Databricks SQL | Dataproc | Delta Lake | Docker | EMR | Event Driven | Event-driven architecture | Git | Google Cloud | Google Dataflow | HBase | HDFS | Hadoop | Iceberg | Kubernetes | MongoDB | NoSQL | ORC | Parquet | Performance optimization | PySpark | Python | REST API | Real Time | Real-time Streaming | SQL | Snowflake | Spark SQL | Spark Streaming | Synapse Analytics | YARN
Education
Bachelor of Engineering | Bachelor of Science | Master of Science | PhD
Related jobs
-
Senior Data Engineer CAD 86K-118KAWS DMS | AWS Glue | AWS Lambda | Agile | Amazon AthenaAccident and life insurance | Health insurance | Other location specific benefitsSenior-level Full TimeCalgary, Alberta, Canada; Winnipeg, Manitoba, Canada; …21h ago
-
Staff Software Engineer - Applied AI CAD 140K-180KAWS | Amazon Bedrock | CI/CD | Claude | Data Pipelines401k match | Charitable donation match | Commuter benefits | Flexible time off | Paid parental leaveSenior-level Full TimeRemote - Ontario, Canada R1d ago
-
Senior-level Full TimeToronto, Ontario, Canada1d ago
-
Machine Learning Engineer - IV (Biometrics) CAD 106K-154KAWS | Airflow | Algorithmic Bias Mitigation | Algorithmic bias | Amazon EC2Mid-level Full TimeMontreal1d ago
-
Senior-level Full TimeToronto, Ontario, Canada R1d ago
-
GTM AI Engineer CAD 141K-191KAPIs | Agents | Attribution | Automation | CRM SyncChange management support | Hybrid work option | Playbooks | Remote work flexibility | Stakeholder trainingSenior-level Full TimeCanada - Remote (0002) R1d ago
-
Senior Software Engineer | Azure Data Analytics CAD 114K-203KC# | C++ | Database optimization | Distributed Systems | JavaSenior-level Full TimeVancouver, BC, CA1d ago
-
Senior Software Engineer, AI/LLM USD 202K-227KAPI Development | API Orchestration | Automated testing | Best practices | CI/CDFlexible PTO | Health, dental, vision coverage | Mental health & wellness benefits | Parental leave | Professional development stipendSenior-level Full TimeRemote (US/Canada) R1d ago
-
Senior Analytics Engineer | Canada | Remote CAD 129K-155KAPM | AWS | BigQuery | Cloud platform | DBT100% remote | Annual leave | Global culture | In-person onboarding | Remote-firstSenior-level Full TimeCanada (Remote) R1d ago
-
Data Developer - TG Quality Engineering TGQF CAD 120K-161K.NET | Agile | Algorithms | Amazon Web Services | Apache FlumeSenior-level Full TimeQuebec City, QC, Canada1d ago
-
Senior-level Full TimeMontreal, QC, Canada1d ago
-
Entry-level Full TimeQuebec City, QC, Canada1d ago
-
.NET | AWS | Agile | Apache Flume | Apache IcebergEntry-level Full TimeQuebec City, QC, Canada1d ago
-
Software Engineer, ML Ops CAD 100K-130KC# | C++ | Computer Vision | Data Preprocessing | Deep learningMid-level Full TimeToronto1d ago
-
C++ | Chat systems | Data Processing | Debugging | Distributed SystemsSenior-level Full TimeWaterloo, ON, Canada1d ago
-
Professional, Engineering - Multidisciplinary Design, Analysis & Optimization (MDAO / MDO) CAD 77K-127KBayesian optimization | CI/CD | Fidelity optimization | GEMSEO | GitDisability insurance | Employee assistance program | Health insurance | Life insurance | Retirement savings planMid-level Full TimeDorval, Québec, CA, H4S 1Y91d ago
-
Database Management | Debugging | Distributed Computing | Docker | GitHubMid-level Full TimeKitchener1d ago
-
Apache Hadoop | Apache Hive | Apache Kafka | Apache NiFi | Apache PinotSenior-level Full TimeKitchener1d ago
-
AI Engineer - New Grad CAD 56K-70KAPI | Agent Frameworks | Fine Tuning | Language Models | Large Language ModelsDiscount program | Flexible Medical Health Spending Account | Health benefits | RRSP matching | Tuition reimbursementEntry-level Full TimeGreater Toronto Area, ON, Canada1d ago
-
Senior-level Full TimeCAN - Ontario - Toronto, Canada2d ago
-
APIs | Deployment | Design Patterns | Java | Machine LearningMid-level Full TimeToronto, Ontario, CAN2d ago
-
MTS Software Engineer, Data Platform CAD 142K-190KAgile | Algorithms | Apache Spark | Data Structures | Distributed SystemsMedical benefits | Paid time off | Parental leave | RRSP eligibilitySenior-level Full TimeToronto, Canada2d ago
-
Automation | Data Visualization | Data analytics | Data cleaning | Database ManagementFlexible hours | On-site work | University enrollment requirementEntry-level Full Time InternshipCA-QC-LONGUEUIL-J01 ~ 1000 Blvd Marie-Victorin ~ …2d ago
-
Airflow | Automation | Credit Rating Modeling | Credit Risk | Credit ratingHealthcare coverage | Hybrid work environment | Matched donations | Paid volunteer days | Parental leaveMid-level Full TimeToronto, ON, CA2d ago
-
Advanced Database Engineer CAD 83K-138KBackup monitoring | Business Suite | Database Administration | Database Migration | Database PatchingMid-level Full TimeCA Canada (ResMed Halifax)2d ago