Data Engineer - Web Scraping
Tasks
- Build data ingestion pipelines
- Clean transform validate datasets
- Design internal tools and automation frameworks
- Design web scraping systems
- Develop web scrapers for data sources
- Implement data quality control checks
- Process large scale data workflows
- Schedule monitor optimize scraping workflows
- Troubleshoot data pipeline issues
Perks/Benefits
- Autonomy
- Collaborative team
- Continuous learning
- Flexible work environment
- High ownership
- Professional growth
Skills/Tech-stack
APIs | AWS | AWS Lambda | Amazon RDS | Amazon S3 | Amazon SNS | Amazon SQS | Apache Airflow | CI/CD | Docker | GitLab CI | GitLab CI/CD | HTML | JavaScript | Jenkins | Kubernetes | Pandas | Python | SQL | Scrapy | Selenium | XPath
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Senior Data Engineer NOK 949K-1251KAirflow | Apache Spark | Cause analysis | Data Architecture | Data EngineeringAnnual learning budget | Annual leave | Disability insurance | Employee stock options | Enhanced parental leaveSenior-level Full TimeOslo, Norway1d ago
-
Senior Data Engineer GBP 91K-124KAccess Control | Airflow | BigQuery | CI/CD | Census401k employer match | Hackathons | Health and wellness benefits | Learning and development budget | Open source support fundSenior-level Full TimeGermany R2d ago
-
Senior Data Engineer - AI TechEx Data NOK 949K-1188KAccess Controls | Agile | Data Governance | Data Lineage | Data ModelingSenior-level Full TimeBergen, NO3d ago
-
Senior Data Engineer - AI TechEx Data NOK 949K-1188KAccess Control | Batch Processing | Data Governance | Data Lineage | Data ModellingSenior-level Full TimeBergen, NO3d ago
-
Data Engineer - Tieto Tech Consulting (m/f/d) NOK 450K-450KAzure Data | Azure Data Factory | Azure Synapse | Azure Synapse Analytics | Data FactoryCertification support | Course support | Flexible hybrid work model | Learning opportunities | Private health insuranceEntry-level Full TimeFornebu, Norway8d ago
-
Bash | Cloud infrastructure | Data Processing | Docker | GCPAsynchronous culture | Career growth opportunities | Flexible remote work | Friendly work environmentMid-level Full TimeOslo, Norway9d ago
-
Data Platform Engineer NOK 874K-1150KAccess Control | Azure | CI/CD | DBT | DagsterFlexible home office | Insurance | Paid time off | RetirementSenior-level Full TimeBergen, Norway; Odfjell Technology AS12d ago
-
Senior Data Engineer NOK 450K-450KCI/CD | Data Modeling | Data Quality | Data integration | DataOpsFlexible work hours | Professional developmentSenior-level Full TimeFredrikstad, Norway13d ago
-
Machine Learning Engineer - Audio NOK 600K-800KAWS | Audio Processing | C plus plus | Cloud Computing | Cloud platformHybrid work | Relocation supportMid-level Full TimeFornebu, Norway16d ago
-
Senior AI Engineer NOK 949K-1188K.NET | AI Foundry | AI Services | ArgoCD | Azure AIConference access | Employee shares | Flexible location | Flexible working hours | Learning budgetSenior-level Full TimeNorway, Norway R20d ago
-
Teknisk konsulent - utvikler eller data engineer NOK 460K-482K.NET | AWS | Angular | Azure | Azure BicepAccess to company cabins | Flexible working hours | Hackathons | Innovation lab | Insurance coverageEntry-level Full TimeTromsø, Troms, Norway22d ago
-
Data Engineer NOK 450K-450KAWS | Airflow | Azure | C# | CI/CD6 weeks vacation | Certification bonus | Conference attendance | Hybrid work | Internal seminarsSenior-level Full TimeTrondheim, Norway23d ago
-
AI Engineer NOK 450K-450KAPI | Access Management | Agent-based | Agent-based systems | Amazon Web ServicesCompany sports and events | Conference opportunities | Courses and seminars | Extra leave for holidays | Favorable employee share programSenior-level Full TimeSandvika Kjørbo, Norway23d ago
-
Software Engineer - Android Embedded Platform NOK 600K-861KAOSP | Abstraction layer | Android Open Source | Android Open Source Project | BashGrowth and learning opportunities | Inclusive work environment | Ownership and autonomy | Supportive team cultureMid-level Full TimeFornebu, Norway24d ago
-
Mid-level Full TimeNOR-LYSAKER, Norway24d ago
-
Data Engineer NOK 450K-450KAWS | Amazon Kinesis | Apache Spark | Azure | Cloud platformCertification bonus | Equipment provided | Flexible schedule | Hybrid work | Internal seminarsSenior-level Full TimeStavanger, Norway30d ago
-
Senior AI Engineer role (Senior to Principal) NOK 1012K-1250KData Engineering | Generative AI | Google Cloud | Langchain | LanggraphGlobal career opportunities | Hackathons | Meetups | Mentorship | Professional developmentSenior-level Full TimeSentrum, Oslo, Norway1mo ago
-
Mid-level Full TimeOslo, Norway1mo ago
-
ML Engineer NOK 622K-1008KComputer Vision | Data Processing | Deep learning | Language Processing | Machine LearningMid-level Full TimeOslo, Norway1mo ago
-
Data Engineer NOK 450K-450KAzure | Azure Data | Azure Data Factory | Azure Machine Learning | Data FactoryEquity participation | Flexibility | Mentorship | Opportunities for travel | Personal developmentEntry-level Full TimeOslo, Norway1mo ago
-
Mid-level Full TimeKristiansand S, Agder, Norway1mo ago
-
DataOps konsulent - olje og gass NOK 450K-450KCDF | Data Engineering | Data Governance | Data Management | Data Pipelines6 weeks vacation | Company cabins | Knowledge sharing | Pension and insurance plans | SATS membership subsidySenior-level Full TimeStavanger, Rogaland, Norway1mo ago
-
Senior Data Engineer NOK 450K-450KAWS | Automation | Azure | CI/CD | Cloud platformAccess to company cabins | Career development framework | Insurance benefits | Parental leave support | Pension planSenior-level Full TimeOslo, NO1mo ago
-
Senior DevOps Engineer NOK 1474K-2010KAKS | Alerting | Automation | Azure | BashCollaborative team | Flexible work hours | High-impact work | Remote work within EUSenior-level Full TimeOslo, Oslo, Norway R1mo ago