Associate Director, Principal Data Engineer
RBC CENTRE, 155 WELLINGTON ST W:TORONTO, Canada
CAD 55K-90K (estimate) Mid-level Full Time
Tasks
- Apply Databricks best practices
- Build batch and real time data pipelines with structured streaming
- Build incremental ingestion patterns
- Conduct Spark job performance benchmarking and tuning
- Design data quality frameworks
- Develop ML engineering pipelines with MLflow
- Develop and optimize Spark code
- Evaluate emerging Databricks and Spark capabilities
- Implement CI CD pipelines for data engineering workloads
- Implement Delta Lake solutions
- Implement Unity Catalog governance
- Implement feature store and model serving
- Lead end to end data engineering architecture and delivery on Databricks
- Provide technical mentorship to data engineers
- Set coding standards and peer review practices
Perks/Benefits
- N/A
Skills/Tech-stack
ADLS Gen2 | AWS | Adaptive query execution | Apache Spark | Auto Loader | Azure | Azure Data | Azure Data Factory | Azure DevOps | CI/CD | Caching | Data Factory | Databricks | Databricks Workflows | Delta Lake | Delta Live Tables | Delta Live) | Feature Store | GitHub Actions | Glue | IAM | Kafka | MLflow | Model Serving | Partitioning | PySpark | Python | Query Execution | S3 | Scala | Shuffle Optimization | Structured Streaming | Terraform | Unity Catalog
Education
N/A
Related jobs
-
Manager, Data and Technology - Data Engineer CAD 101K-169KArtificial Intelligence | Azure | Data Architecture | Data Engineering | Data GovernanceFirm-wide closures | Flexible benefit spending account | Flexible work arrangements | Hybrid work | Learning and development daysMid-level Full TimeToronto, ON, CA, M5H 0A913h ago
-
Data Platform DevOps Engineer - Senior Consultant CAD 80K-138KAWS | Access Controls | Alerting | Azure | Azure DevOpsDeloitte Days | Development and Innovation Days | Flexible benefit spending account | Flexible work arrangements | Hybrid work arrangementSenior-level Full TimeToronto, ON, CA, M5H 0A913h ago
-
Active Directory | Apache Spark | Audit Logging | Azure Active Directory | Azure BlobSenior-level Full TimeToronto, ON, CA, M5H 0A913h ago
-
AI Research Engineer CAD 72K-138KAccelerate | Autogen | Benchmarking | DeepSpeed | DevOpsDeloitte Days | Flexible benefit spending account | Flexible work arrangements | Hybrid work structure | Learning daysMid-level Full TimeToronto, ON, CA, M5H 0A914h ago
-
GenAI & Python Specialist - Operate CAD 72K-125KAPI Development | Agent systems | Claude | Clustering | EmbeddingsFlexible work arrangements | Hybrid work arrangement | Learning days | Paid vacation daysMid-level Full TimeToronto, ON, CA, M5H 0A914h ago
-
Data Engineer (MS Fabric / Databricks) - Consultant CAD 58K-102KADLS | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageFlexible work arrangements | Hybrid work structure | Learning development and innovation days | Mental health support benefits | MentoringMid-level Full TimeToronto, ON, CA, M5H 0A914h ago
-
Agile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageFlexible spending account | Flexible work arrangements | Hybrid work arrangement | Learning days | Mental health support benefitsSenior-level Full TimeToronto, ON, CA, M5H 0A914h ago
-
AI Engineer USD 165K-240KAPI Design | AWS | Agentic Workflows | Asynchronous processing | BM25401k enrollment | Gym membership stipend | Health coverage | Hybrid work environment | Paid HolidaysSenior-level Full TimeNew York20h ago
-
Co-op Engineer - AI Software Engineering CAD 58K-104KAgent Orchestration | Architectural refactoring | Automated testing | Autonomous Agents | C++Mid-level Full TimeMarkham, Ontario, Canada23h ago
-
Data Architect (BigQuery / GCP) CAD 90K-130KAirflow | Audit Logging | BigQuery | BigQuery ML | CI/CDHybrid work environment | Professional growth | Supportive work environmentSenior-level Full TimeToronto, ON, Canada R23h ago
-
Lead, Finance Analytics & Enablement AI/ML CAD 400K-500KAmplitude | Automation frameworks | Business Intelligence | Cloud platform | Cost OptimizationCo-working access | HSA | Health insurance | Life insurance | Long-term disabilitySenior-level Full TimeToronto - Remote R1d ago
-
Amplitude | Data Governance | Data Pipelines | Data Reconciliation | Data WarehousingCo-working space access | Health insurance coverage | Health savings account | Life insurance coverage | Long-term disabilitySenior-level Full TimeMontréal - Remote R1d ago
-
Senior Machine Learning Engineer, Menu Personalization CAD 244K-264KData Pipelines | Experimentation | Go | Kafka | KubernetesBox discount | Career growth support | Fitness membership discount | Flexible-hybrid work | Generous vacation and PTOSenior-level Full TimeToronto, Ontario, Canada1d ago
-
Sr. Solutions Engineer CAD 139K-192KAWS | Artificial Intelligence | Azure | Big Data | Cloud ComputingSenior-level Full TimeToronto, Canada1d ago
-
Software Engineer -Performance Dev (StorageGRID) CAD 123K-159KAgile | Amazon S3 | Bamboo | CI/CD | ContainersHealth insurance | Life insurance | Paid time off | Retirement or Pension PlanSenior-level Full TimeVancouver, Canada Office (VANCOUVER)1d ago
-
API Integration | API Security | Authentication | CI/CD | Cloud NetworkingAnnual performance bonus | Career growth | Fully remote workSenior-level Full TimeCanada R1d ago
-
Machine Learning Developer CAD 102KAWS | Angular | Apache Hive | Apache Pig | AzureHealth benefits | Paid time off | Retirement planEntry-level Full TimeRemote - Ontario, Canada R1d ago
-
Senior AI Architect CAD 94K-115KAPI Gateway | AWS Bedrock | AWS Fargate | AWS Lambda | Amazon CloudWatchEmployee stock purchase plan | Extra days off | Flexible work arrangements | Hybrid work model | Paid time off purchaseSenior-level Full TimeMontréal, 2020 Robert-Bourassa, Canada1d ago
-
GenAI platform Advisor CAD 101K-124KAWS | Anomaly Detection | Ansible | Architecture Documentation | AutomationDefined benefit pension plan | Employee stock purchase plan | Extra paid days off | Flexible work arrangements | Hybrid work modelSenior-level Full TimeMontréal, 2020 Robert-Bourassa, Canada1d ago
-
Automation | C# | CAN | CI/CD | Co-simulationAccommodation for disabilities | Hybrid workEntry-level Full TimeMarkham Elevation Centre - Markham Elevation …1d ago
-
AI Engineer- Decision Science CAD 82K-154KAWS | Agentic AI | Anomaly Detection | Apache Spark | Artificial IntelligenceAccident insurance | Health insurance | Life insurance | Retirement savings plan | Tuition reimbursementSenior-level Full TimeBMOPLACE, Canada1d ago
-
Senior Data Engineer CAD 90K-140KAWS | Azure | CI/CD | Data Governance | Data QualityFlexible work life balance options | Training programSenior-level Full TimeRBC WATERPARK PLACE, 88 QUEENS QUAY …1d ago
-
Staff Data/AI Engineer CAD 90K-141KAI Agents | Apache Spark | CI/CD | Cloud Computing | Code reviewCoaching and mentoring | Competitive bonuses | Flexible benefits | Professional development opportunities | Stock optionsSenior-level Full TimeRBC WATERPARK PLACE, 88 QUEENS QUAY …1d ago
-
Data Platform Engineer Analyst CAD 100K-150KAWS | Apache Airflow | DBT | ELT | ETLHealth and wellbeing benefits | Healthcare benefits | Hybrid work model | Prevention and wellness programsMid-level Full TimeToronto, Canada1d ago
-
Agile | Angular | Automated testing | Azure OpenAI | CI/CDEmployee assistance program | Health benefits | Hybrid work environment | Paid time offSenior-level Full TimeMontreal 111, Canada R1d ago