Data Engineer - Web Scraping
Tasks
- Build data ingestion pipelines
- Clean transform validate datasets
- Design automated data collection systems
- Develop internal automation tools
- Develop web scrapers
- Implement data quality checks
- Process scalable maintainable data workflows
- Schedule monitor optimize scraping workflows
- Troubleshoot data pipeline incidents
Perks/Benefits
- Collaborative team culture
- Continuous learning
- Flexible work environment
- High ownership
- Professional growth
- Technology-focused projects
Skills/Tech-stack
APIs | AWS | AWS Lambda | Amazon RDS | Amazon S3 | Amazon SNS | Amazon SQS | Apache Airflow | CI/CD | Docker | GitLab CI | GitLab CI/CD | HTML | JavaScript | Jenkins | Kubernetes | Pandas | Postman | Python | SQL | Scrapy | Selenium | Web Scraping | XPath
Education
Roles
Related jobs
-
AI Engineer - Cloud EUR 62K-93KAI Search | AKS | ARM Templates | Application Insights | Application SecurityDiscount on products and services | Flexible work hours | Home office setup | Hybrid work | Pension contributionsMid-level Full TimeAmsterdam, NH, Netherlands1d ago
-
API Design | APIs | Backend architecture | Command Line | Command-line InterfaceCompany-provided equipment | Continuous learning allowance | Flexible working hours | Fully remote | Health insurance allowanceSenior-level Full TimeNetherlands R1d ago
-
Fullstack TypeScript AI Engineer EUR 45K-62KAzure | Backend Development | Frontend Development | Guardrails | JavaScriptFlexible working hours | Home-office allowance | Hybrid work | Paid time off | Pension planMid-level Full TimeAmsterdam, Noord-Holland, Nederland1d ago
-
Data Engineer EUR 36K-54KAPI | BigQuery | Cloud Run | Cloud Storage | Data Modeling27 days vacation | DISC training | Extra education budget | Flex work in Zwolle and Amsterdam | Free lunchMid-level Full TimeZwolle, Overijssel, Nederland1d ago
-
Data Engineer EUR 46K-65KAWS | Active Directory | Ansible | Apache Airflow | Apache HadoopDevelopment opportunities | Employee assistance program | Flexible working | Hybrid working | Paid time offMid-level Full TimeRidderkerk, ZH, NL1d ago
-
Analog Signal | Analog signal processing | Calibration | Crosstalk mitigation | DeembeddingEntry-level InternshipWageningen, The Netherlands; Wageningen1d ago
-
API Integration | AWS | Access Control | Anonymization | AzureSenior-level Full TimeAmsterdam, North Holland, Netherlands2d ago
-
Senior financieel adviseur analytics (562192) EUR 70K-90KApache Airflow | Artificial Intelligence | Data Analysis | Data Modeling | Data WarehousingCareer development | Flexible working hours | Individual choice budget | OV public transport reimbursement | Paid parental leaveSenior-level Full TimeDen Haag, NL, 2511 CB2d ago
-
Senior-level Full TimeHilversum, Noord-Holland, Nederland2d ago
-
AWS | Bash | Cloud platform | Databricks | DatadogEquity package | Fully remote Europe | Healthcare coverage | Paid parental leave | Relocation assistanceMid-level Full TimeNetherlands R2d ago
-
BI Engineer (Power BI & Microsoft Fabric) EUR 54K-84KAccess Management | Agile | App publishing | CI/CD | Content Lifecycle32 holidays | Company phone | Diversity and inclusion | Excellent pension scheme | Fun workplaceMid-level Full TimeThe Hague, Netherlands2d ago
-
Computational Physics Engineer EUR 67K-77KC++11 | Cloud Computing | Electromagnetic modeling | Fourier optics | High PerformanceSenior-level Full TimeVeldhoven, Building 03, Netherlands2d ago
-
Metrology Machine Learning Engineer EUR 67K-77KC++ | Cloud Computing | Deep learning | Distributed Systems | JuliaOccasional international travelSenior-level Full TimeVeldhoven, Building 03, Netherlands2d ago
-
Senior Data Engineer GBP 91K-124KAccess Control | Airflow | BigQuery | CI/CD | Census401k employer match | Hackathons | Health and wellness benefits | Learning and development budget | Open source support fundSenior-level Full TimeGermany R2d ago
-
AI | API Development | Agentic AI | Backend Services | Batch ProcessingSenior-level Full TimeNetherlands > Almere2d ago
-
Working Student Computer Science EUR 33K-40KAPIs | Analytics | Data integration | Databases | Proof of ConceptFlexible work hours | Paid internship | Pathway to full-time role | Professional developmentEntry-level Part TimeUtrecht2d ago
-
R&D Engineer Experimentele Aerodynamica EUR 45K-62KAeroacoustics | Aerodynamics | Data Analysis | Data Processing | FortranATV days | Coaching | Extra paid parental leave | Health cost reimbursement | Learning budgetEntry-level Full TimeMarknesse, Flevoland, Nederland2d ago
-
Alerting | Amazon Aurora | Amazon Redshift | Batch Processing | CI/CDFully remote workSenior-level Full TimeNetherlands R3d ago
-
Apache Kafka | BIO | Elastic | Grafana | ISO27001Paid parental leave | Study facilities | Travel cost reimbursement public transportEntry-level Full TimeHuis ter Heide (Ut), NL, 3712 …3d ago
-
Senior-level Full TimeUtrecht, NL3d ago
-
Staff/Lead Python Engineer (FastAPI, Orchestration) EUR 82K-112KAsynchronous programming | Cloud platform | Concurrent programming | FastAPI | GenAIAnnual learning and development budget | Co-working budget | Company laptop | Enhanced parental leave | Flexible scheduleSenior-level Full TimeRemote Anywhere R4d ago
-
Data Engineer EUR 72KAzure Data | Azure Data Factory | Azure Databricks | Azure Synapse | Azure Synapse AnalyticsHealth insurance discount | Home working | Home-office allowance | Laptop | Mobility budgetMid-level Full TimeAmsterdam, Noord-Holland, Nederland4d ago
-
Senior Data Platform Infrastructure Developer EUR 68K-84KAlerting | CDC | CI/CD | Databricks | Delta LakeSenior-level Full TimeWeert (Netherlands)5d ago
-
AI Engineer - Cloud EUR 62K-93KAI Search | API Keys | API Management | ARM Templates | Active DirectoryEmployee discounts | Flexible time off | Home office setup budget | Mobile phone and laptop | Pension planSenior-level Full TimeAmsterdam, NH, Netherlands5d ago
-
AI Engineer - Cloud EUR 62K-93KAI Search | AKS | API Keys | ARM Templates | Application InsightsFlexible leave | Home office setup | Personal development budget | Work from homeMid-level Full TimeAmsterdam, NH, Netherlands5d ago