Senior Data Engineer - Web Scraping
Tasks
- Build data pipelines
- Clean and transform datasets
- Collaborate with stakeholders on data requirements
- Design data products and infrastructure
- Design web scrapers
- Develop automation tools for data pipelines
- Implement data quality checks
- Ingest data into databases and data warehouses
- Maintain web scrapers
- Orchestrate workflows with Apache Airflow
- Set up alerting and incident response
Perks/Benefits
- Autonomy and ownership
- Collaborative team environment
- Fully remote
- Operational excellence culture
- Professional development and innovation
Skills/Tech-stack
APIs | AWS Lambda | AWS RDS | AWS S3 | AWS SNS | AWS SQS | Apache Airflow | CI/CD | Data Warehousing | Database engineering | Docker | GitHub Actions | HTML | JavaScript | Jenkins | Kubernetes | Pandas | Python | SQL | Scrapy | Selenium | XPath
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Benchmarking | Conversational AI | Data Analysis | Embeddings | Evaluation FrameworksAnnual company retreats | Autonomy | Flexible local time zone | Paid annual leave | Referral bonusesSenior-level Full TimeGermany R5h ago
-
(Senior) MLOps Engineer (m/f/d) - REF97172N EUR 70K-75KAWS | Agile | Amazon SageMaker | Apache Airflow | AzureChildcare support | Company pension plan | Discounts | Employee discounts | Flexible working hoursSenior-level Full TimeHannover, NDS, Germany R20h ago
-
Amazon RDS | Amazon Redshift | Apache Airflow | Cause analysis | Data ModelingAnnual company retreats | Flexible working hours | Paid annual leave | Performance incentives | Premium equipmentSenior-level Full TimeGermany R21h ago
-
AWS | Azure | Bash | C plus plus | C#30 days vacation | Health insurance package | Home office flexibility | Remote work from GermanySenior-level Full TimeMunich, Germany R23h ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Collaboration with multidisciplinary teams | Fully remote | Remote work autonomy and flexibilitySenior-level Contract Full TimeGermany R23h ago
-
Senior Consultant Cloud Data Engineering (m/w/d) EUR 68K-95KAWS | Azure | Big Data | Business Intelligence | Data ArchitectureAfterwork events | Christmas party | Corporate benefits platform | Employee Assistance Program (EAP) | External trainingSenior-level Full TimeFrankfurt am Main, Germany R1d ago
-
Senior Data Engineer (Databricks) (m/w/d) EUR 62K-72KApache Spark | Azure Data | Azure Data Factory | CI/CD | ComplianceBicycle leasing | Corporate benefits platform | Deutschlandticket | Employee Assistance Program (EAP) | Employee wellbeing supportSenior-level Full TimeHamburg, Germany R1d ago
-
Senior Data Engineer (Databricks) (m/w/d) EUR 62K-72KAgile | Apache Spark | Azure Data | Azure Data Factory | CI/CD30 days paid vacation | Allowance for mobile workspace | Bahncard | Bike leasing | Corporate benefits platformSenior-level Full TimeStuttgart, Germany R1d ago
-
Senior Data Engineer (Databricks) (m/w/d) EUR 75K-95KApache Spark | Azure Data | Azure Data Factory | CI/CD | DBTCorporate benefits platform | Employee Assistance Program (EAP) | Flexible working hours | Inhouse trainings | MentoringSenior-level Full TimeBerlin, Germany R1d ago
-
AWS | Asynchronous programming | Azure | CI/CD | Data WranglingAsynchronous Working Environment | Career development growth | Hardware provided | Remote-first work model | Work abroad short periodsSenior-level Full TimeGermany R1d ago
-
Senior Data Platform Engineer EUR 78K-98KAmazon Kinesis | Apache Airflow | Apache Spark | DBT | Data GovernanceEquity | Health and wellbeing support | Long-term benefits | SabbaticalSenior-level Full TimeBerlin R1d ago
-
Senior Data Engineer EUR 70K-90KApache Kafka | CI/CD | ClickHouse | Docker | ElasticsearchCoaching and training | Flexible working hours | Learning and development | Remote-first | Six dedicated learning days per yearSenior-level Full TimeGermany /Spain / Romania (Remote) R2d ago
-
APIs | Alerting | Cloud Computing | Data Pipelines | Data PreprocessingCareer development paths | Hybrid work | Remote work | Training opportunitiesEntry-level Full TimeKronberg, Campus Kronberg 1, Germany R2d ago
-
AI Engineer (m/f/d) USD 90K-125KAI Agents | AI Foundry | Azure AI | Azure AI Foundry | Azure DataCompany bonus scheme | Digital learning programs | Employee discounts | Employee shares program | Flexible workingMid-level Full TimeUnterföhring (bei München), DE, 85774 R2d ago
-
Finance Data Engineer (f/m/d) EUR 37K-38KData Governance | Data Modeling | Data Quality | Data integration | Data reportingCareer development opportunities | Childcare support | Flexible work arrangements | Hybrid work model | JobRad bike leasingMid-level Full TimeFrankfurt, DE, 60323 R2d ago
-
AWS | Apache Kafka | Automated testing | CI/CD | Caching100 percent remote work | Collaborative culture | Company offsite weeks | Flexible work location | IT equipment providedSenior-level Full TimeGermany R2d ago
-
Angular | C++ | CI/CD | Code review | CryptoBike leasing | Deutschlandticket subsidy | Fitness and Wellness Subsidy | Flexible working hours | Hybrid work setupSenior-level Full TimeMunich Hybrid (NavVis GmbH) R2d ago
-
Senior Embedded Linux Platform Engineer (F/M/D) EUR 66K-86KAMD64 | ARM | C# | C++ | CI/CDBike leasing | Flexible working hours | Hybrid work setup | Language class support | Paid time offSenior-level Full TimeMunich Onsite (NavVis GmbH) R2d ago
-
(Senior) AI Engineer (all genders) EUR 65K-80KCloud Computing | Containerization | DevOps | Docker | LLMAdditional vacation | Boat access | Businessbike | Company fitness | E-learning reimbursementMid-level Full TimeBremen, Munich, Mannheim, Mainz, Berlin, Remote R2d ago
-
Senior Data Engineer (m/w/d) – Snowflake EUR 60K-84KAWS | Apache Airflow | Azure | Cloud Computing | DagsterCompany events | Deutschlandticket support | Employee bike discount | Extra paid time off | Flexible work hoursMid-level Full TimeMünchen, Full Remote, Kabelsketal OT Großkugel R2d ago
-
Staff Engineer, Data Loop (m/f/d) EUR 56K-84KAutonomous Systems | C++ | CI/CD | Cloud Native | Containerization27 vacation days | Dog-friendly office | Employee discounts | Employer supported pension contribution | Flexible benefits budgetSenior-level Full TimeMunich R2d ago
-
Senior Lead Data Engineer (w/m/d) EUR 109K-138KCloud Computing | Data Compliance | Data Governance | Data Modeling | Data PrivacyChildcare support | Coaching and mentoring | Health checks | Hybrid work flexibility | Learning and development programsSenior-level Full TimeBerlin, Berlin, DE R2d ago
-
Alerting | Data Pipelines | Distributed Systems | GPU Performance | GPU Performance ToolingMid-level Full TimeGermany R3d ago
-
LLM Application Engineer EUR 49K-67KAPIs | Agent Orchestration | Context engineering | Databases | Distributed SystemsMid-level Full TimeGermany R3d ago
-
Agent Orchestration | Backend Development | Best practices | Continuous Monitoring | Data Privacy26 vacation days | Additional volunteer day | E-learning access | Flexible hours | Hybrid workSenior-level Full TimeDEU - Berlin, Germany R3d ago