Data Engineer - Web Scraping
Tasks
- Build automated web scrapers
- Build data ingestion pipelines
- Clean transform validate datasets
- Develop automation frameworks and internal tools
- Implement data quality control checks
- Investigate and resolve data pipeline incidents
- Process large scale data for business insights
- Schedule monitor optimize scraping workflows
Perks/Benefits
Skills/Tech-stack
APIs | AWS Lambda | AWS RDS | AWS S3 | AWS SNS | AWS SQS | Apache Airflow | CI/CD | Data Warehousing | Data pipeline | Docker | Fiddler | GitLab CI | GitLab CI/CD | HTML | JavaScript | Jenkins | Kubernetes | Pandas | Postman | Python | SQL | Scrapy | Selenium | XPath
Education
Roles
Related jobs
-
Senior Cloud Data Infrastructure Engineer EUR 75K-95KAWS | Apache Kafka | Apache Spark | As-a-Service | AzureEmployer healthcare contributions | Flexible time off | Flexible work environment | Global gatherings | Home office setupSenior-level Full TimeGermany (remote) R15h ago
-
Data Science Engineer (m/w/d) EUR 60K-60KFastAPI | Generative AI | Hugging Face | KNIME | LLM AgentsCorporate discounts | Fitness room | Flexible working hours | Hybrid work | Job bike leasingMid-level Full TimeRegenstauf, Bayern, Deutschland18h ago
-
AWS | Experimentation | Generative AI | Language Models | Language ProcessingBicycle leasing | Development programs | Employee smartphone plan | Good team culture | Interesting ProductsEntry-level Full TimeHamburg20h ago
-
Data & Analytics Engineer (m/f/d) EUR 53K-59KAWS | Amazon Redshift | Avro | BigQuery | CI/CDCareer and training opportunities | Fitness and wellness options | Flex Time | Free access to Langdock | Hybrid workMid-level Full TimeHamburg or Berlin20h ago
-
ABAQUS | ANSYS | CAE | Digimat | FEMFlexible working hours | Job-sharing | Remote workSenior-level Full TimeRenningen, BW, Germany21h ago
-
C plus plus | C# | CT Image Processing | Compliance | Debugging30 days vacation | Cafeteria | Company pension | Company provided training | Corporate benefitsSenior-level Full TimeWiesbaden, HE, Germany1d ago
-
Gazebo | Isaac Sim | Linux | MS Office | NVIDIA IsaacCafeteria student discount | Employee discounts | Flexible work hours | Hybrid work model | International work environmentEntry-level Full Time InternshipHannover, NDS, Germany1d ago
-
Senior Expert Robotics & Automation (m/w/d) EUR 54K-55KAPIs | Automation | Azure | Business case | Business case analysis30 days vacation | Bike leasing | Company pension plan | Employee discounts | Flexible work hoursSenior-level Full TimeKirchdorf/Oberopfingen, DE1d ago
-
Mid-level Full TimeMünchen, DE1d ago
-
3D Perception | Bash | Computer Vision | Deep learning | Image SegmentationMid-level Full TimeMünchen, DE1d ago
-
AI Data Engineer* EUR 53K-53KAI Agents | AI Foundry | API Integration | Azure AI | Azure AI FoundryHome office | Mobile work within GermanySenior-level Full TimeOber-Ramstadt (OBR-01), Germany1d ago
-
APIs | Container Deployment | Data Privacy | Evaluation | Generative AIAdditional leave | Bike leasing | Career development and training | Company pension | Family budgetSenior-level Full TimeKarlsruhe, Deutschland1d ago
-
Senior Machine Learning Engineer EUR 92K-140KCalibration | Computer Vision | Data Drift | Data Pipelines | Deep learningSenior-level Full TimeMunich1d ago
-
AI Security | AWS | Azure | CI/CD | Cloud Native30 days annual leave | Hybrid work | Professional development | Travel flexibilitySenior-level Full TimeDeutschlandweit & Hybrid Work R1d ago
-
Data Science | Git | Machine Learning | Microsoft Office | PandasEntry-level Full TimeWeissach1d ago
-
Data Analyst FlixTrain Operations (m/w/d) EUR 36K-42KDashboards | Data Modeling | Data Warehousing | Data pipeline | GitHealth benefits | Hybrid work model | Learning and development | Mentoring program | Travel benefitsEntry-level Full TimeBerlin1d ago
-
AI Engineer EUR 44K-70KA/B | A/B Testing | API Design | AWS | B testingAdditional annual leave | Discounts | Fitness and wellness memberships | Language apps | Personal development budgetMid-level Full TimeBerlin1d ago
-
Data Engineer (m/f/d) EUR 42K-57KAWS Glue | AWS Lambda | Apache Airflow | Apache Spark | CI/CDCompany pension scheme | Discounted parking or job ticket | Employee assistance programme | Hybrid working model | In-house gymMid-level Full TimeDüsseldorf, Germany1d ago
-
Data Engineer II - (Tech Foundations, Finops and Tech Metrics - 12 month maternity cover) EUR 62K-82KAnomaly Detection | Business Intelligence | Cloud Cost Management | Cost Management | Data ModelingCorporate pension plan | Digital meal vouchers | Educational budget | Employee share purchase plan | Food vouchersMid-level Contract Full TimeBerlin, Germany1d ago
-
Agentic AI Software Engineer EUR 38K-51KAWS | Azure | CI/CD | Docker | FastAPIAccess to certifications | Conferences and Knowledge Sharing | Flexible working model | Hackathons | Remote work optionEntry-level Full TimeMunich, Bayern, Germany1d ago
-
Senior Analytics Engineer - Run & Grow EUR 63K-90KApache Iceberg | Cohort Analysis | DBT | Data Contracts | Data LakehouseCorporate pension scheme | Kita placement assistance | Learning and development budget | Paid time off | Referral bonusSenior-level Full TimeBerlin, Germany1d ago
-
Machine Learning Engineer USD 90K-100K3D Perception | BEV perception | C plus plus | Camera | Computer VisionOn-site beverages | Relocation assistance | Social eventsSenior-level Full TimeBerlin Office1d ago
-
Expedited SVN: Associate Data Engineer (P) EUR 35K-35KAPI Management | Apache Spark | Azure Blob | Azure Blob Storage | Azure DataMid-level Full TimeBerlin, Germany1d ago
-
Data Engineer (m/w/d) | DSIANA EUR 38K-56KAmazon S3 | Data Modeling | Data Warehousing | Data integration | Delta LakeBike leasing | Company pension plan | Family budget | Flexible working hours | Health managementMid-level Full TimeKarlsruhe, Deutschland2d ago
-
Senior Data Engineer (m/f/d) EUR 60K-75KAWS | Azure | Data Lineage | Data Modeling | Data QualityFlexible working hours | Free food | Modern office | Tech conferences | Training budgetSenior-level Full TimeBerlin, BE, Germany2d ago