Data Engineer - Web Scraping
Tasks
- Build data ingestion pipelines
- Build scalable data processing workflows
- Clean transform validate datasets
- Design web scraping systems
- Develop automation frameworks and internal tools
- Develop web scrapers
- Implement data quality checks
- Investigate data pipeline incidents
- Schedule monitor optimize scraping workflows
Perks/Benefits
Skills/Tech-stack
API | AWS | AWS Lambda | Amazon RDS | Amazon S3 | Amazon SNS | Amazon SQS | Apache Airflow | CI/CD | Docker | GitLab CI | GitLab CI/CD | HTML | JavaScript | Jenkins | Kubernetes | Pandas | Python | SQL | Scrapy | Selenium | XPath
Education
Roles
Related jobs
-
Conversational AI Engineer EUR 75K-125KAI orchestration | API Integration | Agentic AI | JSON | JavaScriptMid-level Full TimeRemote, Any Location, ES R2d ago
-
Data Engineer - Digital Change EUR 27K-32KAlteryx | Azure | Azure Data | Azure Data Factory | Azure DevOpsCareer development | Community service | Flexible working hours | Inclusive culture | Sustainability initiativesEntry-level Full TimeLuxembourg, LU2d ago
-
Data Engineer, SCOT Fulfillment Optimization EUR 64K-71KAWS Glue | Data Modeling | Data Monitoring | Data Quality | Data WarehousingSenior-level Full TimeLuxembourg, LUX2d ago
-
AI | Alteryx | Analytics | Automation | Data GovernanceHybrid work model | Learning culture | Performance bonus eligibilityExecutive-level Full TimeLuxembourg, LU R3d ago
-
AI/ML Engineer – GenAI & MLOps (on-site) EUR 45K-60KAI Governance | AWS | Agile | Apache Airflow | AutomationOn-site workMid-level Full TimeLuxembourg, Luxembourg, Luxembourg7d ago
-
AI Engineer (f/m/d) EUR 54K-56KCI/CD | Cloud Platforms | Containerization | Docker | InfrastructureCompany restaurant | Flexible working hours | Health insurance | International mobility | Shuttle bus transportationSenior-level Full TimeLUX.Mamer, Luxembourg8d ago
-
Anomaly Detection | Azure | DVC | Dataiku DSS | DockerOn-site work | Professional development supportMid-level Full TimeLuxembourg, Luxembourg, Luxembourg9d ago
-
Amazon Redshift | Chi squared test | Chi-squared) | Data Modeling | Data pipelineSenior-level Full TimeLuxembourg, LUX11d ago
-
Data Engineer (m/f/d) EUR 42K-50KApache Airflow | Apache Hadoop | Azure Data | Azure Data Factory | Data FactoryEntry-level Full TimeLuxembourg - 2 Rue Gerhard Mercator13d ago
-
Senior Data Engineer Consultant EUR 53K-79KAWS | Apache Kafka | Apache Spark | Azure | Batch ProcessingFlexible working arrangements | International project exposure | Tailored training and developmentSenior-level Full TimeLuxembourg, Luxembourg15d ago
-
Amazon Redshift | Analysis Services | Apache Spark | Apache Spark Pools | AzureFull-time permanent contract | Onsite workSenior-level Full TimeLuxembourg, Luxembourg16d ago
-
Software Developer AI/Machine Learning (f/m/d) EUR 47K-53KData Querying | Git | Linux | Python | SQLCompany restaurant | Flexible working time | Free parking | Insurance | Internal mobilityMid-level Full TimeLUX.Mamer, Luxembourg16d ago
-
AI & Data Analytics Specialist EUR 70K-94KAML | Anomaly Detection | Artificial Intelligence | Data Analysis | Data PipelinesCareer development | Collaboration | Equal access to opportunities | InclusionEntry-level Full TimeLuxembourg, LU, L-244916d ago
-
Senior Data Engineer Consultant EUR 53K-79KApache Kafka | Apache Spark | Azure | Batch Processing | CI/CDFlexible working arrangements | International project exposure | Knowledge sharing environment | Training and development opportunitiesSenior-level Full TimeLuxembourg, Luxembourg23d ago
-
Senior Data Engineer Consultant EUR 53K-79KAWS | Apache Kafka | Apache Spark | Automation | AzureFlexible mobility options | Flexible working arrangements | International project exposure | Professional knowledge sharing | Tailored training and developmentSenior-level Full TimeLuxembourg, Luxembourg28d ago
-
Data Engineer, WW Ops Finance S&A EUR 65K-65KAWS Cloud | AWS Cloud Development Kit | AWS CloudFormation | AWS CloudWatch | AWS CodeDeployMid-level Full TimeLuxembourg, LUX28d ago
-
Senior-level Full TimeLuxembourg, Luxembourg, Luxembourg1mo ago
-
Database & Data Platform Administrator (f/m/d) EUR 54K-66KAudit Logging | Azure Cosmos | Azure Cosmos DB | BI | BashSenior-level Full TimeLuxembourg, LU1mo ago
-
Mid & Senior Data Engineers (m/f) EUR 45K-55KApache Airflow | Apache Iceberg | Apache Spark | Argo | Automated testingOnsite work | Work-life flexibilitySenior-level Full TimeLuxembourg, Luxembourg, Luxembourg1mo ago