Senior Data Engineer - Web Scraping
Tasks
- Automate web scraping platform improvements
- Build data pipelines
- Clean transform and manipulate datasets
- Design web scrapers
- Develop data products
- Develop web scrapers
- Implement data quality checks
- Ingest scraped data into data warehouses
- Ingest scraped data into databases
- Investigate data incidents
- Maintain Apache Airflow workflows
- Maintain web scrapers
- Orchestrate scheduled workflows
- Resolve operational issues
- Set up alerting systems
- Support data scientists with data tooling
- Validate data accuracy consistency integrity
Perks/Benefits
- Autonomy
- Collaborative team
- Flexible location
- Fully remote
- Innovation focus
- Operational excellence focus
- Ownership
Skills/Tech-stack
APIs | AWS | AWS Lambda | Amazon RDS | Amazon S3 | Amazon SNS | Amazon SQS | Apache Airflow | CI/CD | Docker | GitHub Actions | HTML | JavaScript | Jenkins | Kubernetes | Pandas | Python | SQL | Scrapy | Selenium | Web Scraping | XPath
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Fully remote | Remote flexibility | Remote work autonomySenior-level Contract Full TimeBelgium R1d ago
-
Senior AI Engineer EUR 54K-75KCheckpointing | Concurrent programming | Distributed Systems | Graph Databases | Human-in-the-loopHealth coverage | Hybrid work model | Inclusion and belonging programs | Time offSenior-level Full TimeBrussels, Belgium R1d ago
-
Azure Delta | Azure Delta Lake | CAN | CAN bus | Data EnrichmentMid-level Full TimeBrussels, Belgium R3d ago
-
Apache Airflow | Apache Iceberg | Apache Polaris | ArgoCD | DB2Hybrid workMid-level Full TimeBrussels, Belgium R6d ago
-
Data Analytics Engineer Job ID: JP055023 EUR 65K-65KAPI documentation | Airbyte | Analytics | Apache Spark | BPMNActive knowledge of Dutch and French | Hybrid workMid-level Full TimeBrussels, Belgium R6d ago
-
Data Engineer EUR 47K-72KAWS | Azure | Cloud Computing | Cloud platform | Google CloudCafeteria plan | Dedicated mentor | Flexible hours | Full insurance coverage | Group insurance coverageEntry-level Full TimeEurope, Belgium, Brussels Office - Zaventem … R14d ago
-
Manager Data Engineer EUR 61K-74KAWS | Apache Spark | Azure | Azure Data | Azure Data FactoryBuddy and performance manager | Cafeteria plan benefits | Career development trainings | Charging card | Company carMid-level Full TimeEurope, Belgium, Brussels Office - Zaventem … R14d ago
-
Generative AI Pipeline Engineer (Tech Lead) EUR 75K-85KAPI Integration | Agentic Workflows | Blender | CAD | ComfyUIFitness allowance | Home office support | Medical insurance coverage | Ongoing learning | Paid vacationSenior-level Full TimeBrussels, Remote, BE R15d ago
-
Senior AI Engineer, Unstructured AI EUR 64K-93KAPI Design | Big Data | Context engineering | Data Cataloging | Data PipelinesFlexible benefits | Health coverage | Hybrid work schedule | Inclusion and belonging | Time offSenior-level Full TimeBrussels, Belgium R16d ago
-
Senior Embedded Software Engineer EUR 42K-84KAES | Algorithms | C# | C++ | CI/CDHybrid work schedule | Mentorship | Team collaborationSenior-level Full TimeBelgium-Brussels Office R20d ago
-
Senior Data Engineer (employee or freelance) EUR 60K-79KAirbyte | Bash | CI/CD | DBT | Data GovernanceFlexible work hours | Paid time off | Remote workSenior-level FreelanceBrussels R21d ago
-
Senior Data Engineer EUR 54K-76KApache Airflow | BigQuery | Cloud platform | Containerization | DBTHybrid workSenior-level Full TimeBrussels, Belgium R28d ago
-
Analytics Engineer EUR 52K-78KAWS | Agentic AI | DBT | Dagster | Data ArchitectureEco vouchers | Employee discounts | Group insurance | Hospitalization insurance | LaptopMid-level Full TimeMechelen, Belgium R1mo ago
-
AI Solutions Engineer EUR 55K-75KAI orchestration | Agent Frameworks | Agentic Automation | Evaluation | Language ModelsClient-facing exposure | Hybrid work | Learning and developmentSenior-level Full TimeAntwerp, Belgium R1mo ago
-
Data Engineer EUR 62K-84KAWS | Azure | Azure Synapse | Big Data | CI/CDFlexible working hours | Hybrid work | Mentorship | Mentorship opportunities | OnboardingSenior-level Full TimeMechelen, Vlaams Gewest, Belgium R1mo ago