Data Engineer - Web Scraping
Tasks
- Build automated data collection workflows
- Clean transform validate datasets
- Create automation tools for data pipelines
- Debug data pipeline incidents
- Design web scraping systems
- Develop data ingestion pipelines
- Implement data quality control checks
- Monitor and optimize scraping jobs
- Orchestrate scraping workflows
- Process scalable data sets
Perks/Benefits
- Collaborative team culture
- Continuous learning
- Flexible work environment
- High ownership
- Professional growth
Skills/Tech-stack
API | AWS | AWS Lambda | Amazon RDS | Amazon S3 | Amazon SNS | Amazon SQS | Apache Airflow | CI/CD | Docker | GitLab CI | GitLab CI/CD | HTML | JavaScript | Jenkins | Kubernetes | Pandas | Python | SQL | Scrapy | Selenium | XPath
Education
Roles
Related jobs
-
AWS | Agentic Frameworks | Asynchronous Architecture | Concurrent Systems | Distributed SystemsDevelopment opportunities | International team culture | Social activitiesSenior-level Full TimeTallinn, Estonia2d ago
-
Senior Data Engineer GBP 91K-124KAccess Control | Airflow | BigQuery | CI/CD | Census401k employer match | Hackathons | Health and wellness benefits | Learning and development budget | Open source support fundSenior-level Full TimeGermany R2d ago
-
AI Assisted Development | Claude Code | Code review | Cursor | JavaScriptFlexible hours | Ongoing project based workSenior-level Full TimeMiami2d ago
-
Senior Software Engineer (Data Platform) EUR 53K-80KAWS | Amazon Redshift | Apache Airflow | Apache Flink | Apache KafkaAdditional vacation days | Flexible working hours | Health days | Health insurance | Hybrid workSenior-level Full TimeTallinn, Estonia R3d ago
-
Staff/Lead Python Engineer (FastAPI, Orchestration) EUR 82K-112KAsynchronous programming | Cloud platform | Concurrent programming | FastAPI | GenAIAnnual learning and development budget | Co-working budget | Company laptop | Enhanced parental leave | Flexible scheduleSenior-level Full TimeRemote Anywhere R4d ago
-
Apache Airflow | BigQuery | Cloud platform | DDL | DMLInclusive work environment | Learning opportunities | Mentoring | Supportive team | Wellbeing supportEntry-level Full TimeTartu, Estonia6d ago
-
Applied AI Engineer EUR 66K-79KAppSec | BPMN | Cloud Run | Cloud platform | Code reviewEmployee stock option plan | Flexible working options | Health insurance | Home-office allowance | Parental leaveSenior-level Full TimeRemote Estonia R8d ago
-
Data Engineer (Cards Team) EUR 42K-63KAmazon Redshift | Amazon Web Services | Apache Airflow | Apache Flink | Apache KafkaConference access | Flexible working hours | Health days | Health insurance | Hybrid workMid-level Full TimeTallinn, Estonia R8d ago
-
Bash | Data Ingestion | Data Processing | Docker | GCPAsynchronous work culture | Laid-back atmosphere | Portfolio and LinkedIn sharingMid-level Full TimeTallinn, Estonia9d ago
-
Auto Scaling | CI/CD | CUDA | Container Orchestration | Distributed SystemsEnglish language support | On-call support | Remote workSenior-level Full TimeUkraine R12d ago
-
Data Engineer USD 100K-140KAWS | Airflow | Amazon Redshift | Apache Iceberg | BigQueryAutonomy | Growth opportunities | Occasional travel | Supportive work environment | Work-life balance initiativesMid-level Full TimeEMEA - Remote; Estonia; Malta; Serbia R13d ago
-
Machine Learning Ops and Data Engineer EUR 49K-67KAzure | CI/CD | Data Warehouse | Docker | ETLCollaborative team culture | Continuous improvement culture | Learning and mentorshipMid-level Full TimeTallinn, Estonia14d ago
-
AWS | Agile | Amazon Redshift | Apache Airflow | Apache FlinkAdditional vacation | Discounts on company products and services | Flexible working hours | Health days | Health insuranceSenior-level Full TimeTallinn, Estonia R15d ago
-
Senior Data Engineer (Redshift) EUR 70K-90KAWS Glue | AWS Kinesis | AWS Lambda | AWS RDS | AWS RedshiftSenior-level Full TimeEstonia15d ago
-
Senior-level Full TimeTallinn, Harju County, Estonia21d ago
-
APIs | AWS | Airflow | CI/CD | Distributed SystemsContinuous improvement culture | Hybrid work | MentorshipMid-level Full TimeTallinn, Estonia21d ago
-
Staff Software Engineer - Machine Learning Platform EUR 85K-108KAWS | Apache Airflow | Apache Flink | Apache Kafka | Apache SparkSenior-level Full TimeTallinn, Estonia22d ago
-
Staff Software Engineer - Machine Learning Platform GBP 85K-108KAWS | Apache Airflow | Apache Flink | Apache Kafka | Apache SparkSenior-level Full TimeTallinn, Estonia22d ago
-
Apache Airflow | BigQuery | Cloud platform | DDL | DMLActive lifestyle support | Global scope | Inclusive environment | Learning opportunities | MentoringEntry-level Full TimeTartu, Estonia23d ago
-
AI Solution Engineer EUR 40K-60KAI Evaluation | AI Governance | AI Security | API Design | Agent FrameworksHybrid work | Mentorship | Professional developmentMid-level Full TimeTartu, Estonia28d ago
-
AI Solution Engineer EUR 49K-67KAI Evaluation | AI Security | APIs | Agent Frameworks | DevOpsHybrid work | Mentoring | Work from homeMid-level Full TimeTartu, Estonia1mo ago
-
Software Engineer - Generative AI Platform EUR 66K-80KAWS | Agent Frameworks | Asynchronous Architecture | Concurrent programming | Distributed SystemsDevelopment opportunities | Social activities and eventsSenior-level Full TimeTallinn, Estonia1mo ago
-
AWS | Asynchronous | Concurrent Systems | Distributed Systems | DockerCollaborative team culture | Development opportunities | Social activities and eventsSenior-level Full TimeTallinn, Estonia1mo ago
-
Senior AI/ML Engineer (AI Lead) EUR 67K-93KCI/CD | Data Drift | Data Drift Detection | Data Quality | Drift DetectionBirthday benefits | Employee training and development program | Health compensation | Loyalty benefitsSenior-level Full TimeTallinn, Estonia1mo ago
-
Machine Learning Engineer - Defendec/Reconeyez EUR 60K-78KAgent Frameworks | CometML | Computer Vision | DALI | Data labelingAdditional vacation and health days | Fast-growing business | Innovative company culture | Office in Tallinn | Wellness benefitSenior-level Full TimeTallinn, Harju County, Estonia1mo ago