Data Engineer RD-017
Tasks
- Adapt pipelines to external API changes
- Build data transformation workflows
- Configure Delta Lake storage and tables
- Design data ingestion pipelines
- Document pipelines and data dictionaries
- Handle schema evolution and data validation
- Implement medallion architecture bronze/silver/gold
- Implement pipeline reliability monitoring and alerting
- Maintain CI CD deployment pipelines
- Optimize Delta Lake performance
- Orchestrate and schedule Databricks workflows
- Set up deduplication and relevance filtering
Perks/Benefits
Skills/Tech-stack
Alerting | Authentication | Azure | CI/CD | Data Modeling | Data Quality | Data Validation | Databricks | Delta Lake | Delta Live Tables | Delta Live) | Delta Medallion Architecture | Error Handling | External APIs | Git | GitHub Actions | Medallion Architecture | Monitoring | Pagination | Performance Tuning | PySpark | Python | Rate Limiting | Retries | SQL | Schema evolution | Spark | Workflow Orchestration
Education
N/A
Roles
Related jobs
-
Automation | Azure | Azure Data | Azure Data Explorer | CSVCross-functional collaboration | Exposure to large scale security telemetry | Hands on opportunities with automation and infrastructure as code | Professional growth | Remote work opportunityMid-level Full TimeCanada R7h ago
-
API | Agentic AI | Automation | Data Governance | Data ModelingHealthcare benefits | Occasional in person meetings for team or project | Paid time off | Parental leave | Professional development opportunitiesSenior-level Full TimeCanada R11h ago
-
Backend Engineer, AI Engineering: Duo Chat USD 115K-194KAPI Design | GraphQL | Langchain | Langgraph | Language ModelsEmployee stock purchase plan | Equity compensation | Flexible paid time off | Growth and development fund | Parental leaveSenior-level Full TimeRemote, Canada; Remote, United States R15h ago
-
Machine Learning Engineer II, Ads - Response Prediction CAD 154K-162KApache Spark | Deep learning | Foundation Models | Information Retrieval | Language ModelsAnnual refresh grants | Equity grant | Remote workEntry-level Full TimeCanada - Remote (ON, AB, BC, … R16h ago
-
Senior Machine Learning Engineer, Digital Twin Platform CAD 206K-217KAWS | Azure | Cloud infrastructure | Computer Vision | Data PipelinesRemote work flex firstSenior-level Full TimeCanada - Remote (ON, AB, BC, … R17h ago
-
Senior Software Engineer - Data Platform & AI Automation CAD 114K-140KAPIs | AWS | Access Control | Agent workflows | Amazon Web ServicesComprehensive health benefits | Flexible work environment | Generous time off | Global days of service | Meeting-free daysSenior-level Full TimeVancouver, British Columbia, Canada R18h ago
-
Software Engineer, ML Ops USD 123K-154KAWS EC2 | AWS S3 | C plus plus | Dataset versioning | DockerHealth insurance | Onsite work | Paid time offMid-level Full TimeToronto R18h ago
-
Principal AI Engineer BRL 30KAI Studio | AWS | Azure | Azure AI | Azure AI StudioCaregiver leave | Continuous development support program | Employee assistance program | Employee recognition | Flexible paid time offSenior-level Full TimeMississauga R19h ago
-
Senior DevOps Engineer CAD 125K-165KAWS | Access Control | Alerting | Apache Airflow | CI/CDFlexible work options | Life leave days | Professional development days | Vacation time | Volunteer daysSenior-level Full TimeOttawa, ON, Canada R20h ago
-
Senior Embedded Software Engineer CAD 120K-160KAuthentication | Automated testing | Build systems | C++ | Device DriversRemote workSenior-level Full TimeToronto, ON; Mississauga, Ontario; Ottawa, Ontario R20h ago
-
Senior, AI & Data Science CAD 140K-202KCausal Inference | Classification | Clustering | Data Pipelines | Data Quality401k company match | Hybrid flexibility | Learning and development | Medical, dental & vision coverage | Paid parental leaveSenior-level Full TimeMontréal, Quebec, Canada R23h ago
-
Senior Manager, GTM Sales Engineer & AI solutions CAD 136K-218KAPI Integration | API Services | Access Control | Agentic Workflows | AgileSenior-level Full TimeToronto, Ontario, Canada R23h ago
-
API Authentication | API Integration | Alerting | Apache Airflow | BigQueryCareer growth | Company off-sites | Fully remote | Healthcare benefits | Paid time offMid-level Full TimeCanada R1d ago
-
Senior AWS Data Engineer CAD 120K-150KAWS Glue | AWS Glue Data Catalog | AWS Lake | AWS Lake Formation | Access ControlsFlexible workplace | Hybrid work schedule | Referral programSenior-level Full TimeMarkham, ON, Canada R1d ago
-
Machine Learning Engineer Lead CAD 125K-169KAPIs | Azure Machine Learning | CI/CD | Data Quality | DatabricksDefined benefit pension | Extra statutory holidays | Flexible benefits | Health and dental coverage | Remote work with occasional onsite eventsSenior-level Full TimeVancouver, British Columbia, Canada; Toronto, Ontario, … R1d ago
-
[Wattpad] Data Engineer - Maternity Cover CAD 87K-111KAWS DMS | AWS Glue | Amazon DynamoDB | Amazon EMR | Amazon RedshiftHybrid work model | Work from home flexibilityMid-level Contract Full TimeToronto, Ontario R1d ago
-
Senior AI - Computer Vision Engineer CAD 131K-180K3D Computer Vision | BIM | CUDA | Computer Vision | Data pipelineMentorship opportunities | Research and development focus | Technical architecture influence | Work on real world deployed wearable technologySenior-level Full TimeCalgary, Canada R1d ago
-
AI Platform & Machine Learning Engineer CAD 156K-220KAWS | Airflow | Annotation tools | Azure | BIMSenior-level Full TimeCalgary, Canada R1d ago
-
Data Engineering Developer CAD 85K-166KAPI Integration | Apache Spark | Azure Data | Azure Data Factory | Azure FunctionsActivities | Flexible hours | Paid sabbatical | Parental program | TrainingMid-level Full TimeMontreal (EN) R2d ago
-
Senior Data Specialist, Enterprise Fraud Analytics CAD 73K-123KAirflow | BigQuery | CI/CD | Cloud platform | DBTCompany Share Ownership Program | Discounts on products and services | Hybrid work schedule | Incentive program | Paid time off for studySenior-level Full TimeToronto, ONT, Canada R2d ago
-
APIs | Automated rollback | CI/CD | Cloud Computing | Data DriftCollaborative culture | Comprehensive employee benefits | Mentorship | Professional growth | Remote workSenior-level Full TimeCanada R2d ago
-
Staff Machine Learning Engineer CAD 153K-213KA/B | A/B Testing | B testing | Batch inference | CI/CDSenior-level Full TimeHR Toronto - Canada R2d ago
-
Senior Machine Learning Engineer CAD 153K-200KAPI | Airflow | Asynchronous processing | CI/CD | DatabricksSenior-level Full TimeHR Toronto - Canada R2d ago
-
3D Geometry | Adobe Creative | Adobe Creative Cloud | Artificial Intelligence | BlenderFlexible workplace | Mentorship | Paid internship | Remote/hybrid optionsEntry-level Full Time InternshipAMER - Canada - Quebec - … R2d ago
-
Senior Backend Developer, SCM Tech Fulfillment CAD 120K-150KAWS | Airflow | Automated testing | Container Orchestration | DockerCareer development and learning support | Diversity and inclusion initiatives | Fitness membership discount | Generous vacation and PTO | Health and dental benefitsSenior-level Full TimeToronto, Ontario, Canada R2d ago