Data Engineer - Web Scraping
Tasks
- Build data ingestion pipelines
- Clean transform validate datasets
- Design web scraping systems
- Develop automation tools
- Implement data quality checks
- Maintain web scrapers
- Monitor and resolve data pipeline incidents
- Orchestrate scraping workflows
Perks/Benefits
- Collaborative culture
- Continuous learning
- Flexible work environment
- High autonomy
- Professional growth
Skills/Tech-stack
APIs | AWS Lambda | AWS RDS | AWS S3 | AWS SNS | AWS SQS | Apache Airflow | CI/CD | Docker | GitLab CI | GitLab CI/CD | HTML | JavaScript | Jenkins | Kubernetes | Pandas | Postman | Python | SQL | Scrapy | Selenium | XPath
Education
Roles
Related jobs
-
AWS Kinesis | Amazon Web Services | Apache Kafka | Azure | BigtableBonus programs and equity opportunities | Flexible paid time off | Fully remote | Health, dental and vision coverage | Mental health and wellness benefitsSenior-level Full TimeCanada R7h ago
-
GCP Data Engineer CAD 64K-104KApache Flink | Apache Hudi | Apache Iceberg | Apache Kafka | Apache PrestoMid-level Full TimeRemote, Ontario R14h ago
-
Senior-level Full TimeToronto, ON16h ago
-
Senior Analytics Engineer CAD 105K-130KAWS Step Functions | Airbyte | Airflow | Copilot | DBTComprehensive benefits | Paid time off | Professional growth | Supportive workplaceSenior-level Full TimeCanada16h ago
-
Lead Developer, Embedded Engineering CAD 116K-155KARM | Algorithms | Bash | Bootloader | C#Baby bonus and parental leave top up program | Competitive medical and dental benefits | Electric vehicle purchase incentive program | Flexible working arrangements | Home office reimbursementSenior-level Full TimeOakville, Ontario - Canada; Waterloo, Ontario …17h ago
-
Senior Cloud Data Infrastructure Engineer CAD 140K-202KAWS | Amazon EC2 | Apache Kafka | Apache Spark | As-a-ServiceFlexible time off | Flexible work environment | Global gatherings | Healthcare employer contribution | Home office setupSenior-level Full TimeCanada (Remote) R17h ago
-
Senior-level Full TimeMontreal, QC, Canada21h ago
-
Développeur.euse de données IA Sénior CAD 108K-169K.NET | Agile | Apache Flume | Apache Kafka | Apache NiFiMid-level Full TimeMontreal, QC, Canada21h ago
-
Data Engineer - EA Sports FC CAD 96K-133KAWS | Alation | Apache Airflow | Apache Kafka | Apache SparkFree games | Health insurance | Hybrid work | Mental health support | Paid time offMid-level Full Time地点：Vancouver, British Columbia, Canada23h ago
-
Senior Data Engineer CAD 140K-180KAWS Athena | AWS DMS | AWS Glue | AWS S3 | Amazon KinesisFamily obligation days | Flexible work | Health and dental coverage | Hybrid work model | Mental health and wellness supportSenior-level Full TimeMontreal1d ago
-
AI Solutions Deployment Manager, Cloud Applied AI, Telco USD 183K-266KAPI tools | Agentic AI | Architectural Guidance | Cloud Architecture | Cloud platformSenior-level Full TimeWaterloo, ON, Canada; New York, NY, …1d ago
-
Data Engineer CAD 129K-160KApache Airflow | Data Modeling | Database performance | Database performance tuning | ETLAutonomy | Competitive compensation | No micromanagementSenior-level Full TimeToronto, ON, Canada1d ago
-
Airflow | CatBoost | Dask | Data Quality | Feature EngineeringComprehensive health benefits | Dental coverage | Employee stock purchase plan | Flexible spending allowance | Professional growth opportunitiesSenior-level Full TimeCanada1d ago
-
AI Agents | APIs | Cloud infrastructure | Governance | Language ModelsAccess to modern tools and innovation focused environment | Flexible remote work options | Holidays | Medical, dental, and vision coverage | Paid parental leaveSenior-level Full TimeCanada1d ago
-
Lead AI/ML Engineer CAD 120K-170KAPI Gateway | Airflow | Automated testing | BigQuery | CI/CDCareer roadmap | Dental insurance | Medical insurance | PTO | Training and educational opportunitiesSenior-level Full TimeMontreal1d ago
-
Senior AI Developer CAD 141K-170KAWS | Agile | Azure | CI/CD | CachingFlexible hours | Paid sabbatical | Parental program | Training | Wellness and productivity spending accountSenior-level Full TimeMontreal (EN) R1d ago
-
GenAI Software Developer CAD 61K-113KAI Agents | AI Governance | AI orchestration | Agile | Cause analysisAccident insurance | Health insurance | Life insurance | Retirement benefits | Tuition reimbursementMid-level Full TimeGCS, Canada1d ago
-
Staff Embedded Firmware Engineer (Contractor) CAD 140K-170KBluetooth Low Energy | Bootloader optimization | C++ | Cause analysis | DebuggingSenior-level Contract Full TimeCA - British Columbia - Richmond, …1d ago
-
Software and AI Research Coordinator CAD 64K-75KAlgorithms | Computer Vision | Data Structures | Image Processing | Machine LearningMid-level Full TimeUBCV | Gordon B. Shrum Building …1d ago
-
Data Engineering - Data Ingestion & Infrastructure CAD 75K-107KAnsible | BigQuery | Cloud platform | Data Modeling | Data QualityMentorship and career growth | On-call rotationMid-level Full TimeBurnaby, BC, CA1d ago
-
Lead Machine Learning Engineer CAD 156K-251KAWS | Azure | CI/CD | Cloud Computing | Cloud platformAutonomy | Career development support | Learning and development programs | Mentorship and coachingSenior-level Full TimeToronto, Canada R1d ago
-
Principal AI Engineer CAD 214K-344KCI/CD | Container Orchestration | Docker | Fine Tuning | Information RetrievalInteractive career development tools | Learning and development programs | MentorshipSenior-level Full TimeToronto, Canada1d ago
-
Software Developer - Platform (Embedded, C++) CAD 89K-139KARP | C++ | Cellular wireless | DHCP | DNSAgile team environment | Annual compensation bonus programMid-level Full TimeOttawa, ON, Canada1d ago
-
AI Process Automation Engineer CAD 80K-100KAPI Integration | Agile | Application Programming | Application Programming Interface | Artificial IntelligenceAssociate discount | Health and dental benefits | Learning and development programs | Performance bonuses | RRSP/DPSPMid-level Full TimeRichmond Hill, ON, Canada1d ago
-
Senior-level Full TimeToronto, ON, Canada1d ago