Data Engineer - Web Scraping
Tasks
- Build data ingestion pipelines
- Clean, transform, and validate datasets
- Design web scraping systems
- Develop and maintain web scrapers
- Develop automation frameworks and internal tools
- Implement data quality control checks
- Investigate data pipeline incidents
- Monitor and optimize scraping workflows
- Orchestrate scraping workflows
- Process large scale data workflows
Perks/Benefits
Skills/Tech-stack
APIs | AWS | AWS Lambda | Amazon RDS | Amazon S3 | Amazon SNS | Amazon SQS | Apache Airflow | CI/CD | Docker | GitLab CI | GitLab CI/CD | HTML | JavaScript | Jenkins | Kubernetes | Pandas | Python | SQL | Scrapy | Selenium | XPath
Education
Roles
Related jobs
-
API | API Design | Backend architecture | Command Line | Command-line InterfaceCompany-provided equipment | Continuous learning allowance | Flexible working hours | Fully remote | Health insurance allowanceSenior-level Full TimeIreland R1d ago
-
Data Engineer EUR 54K-65KAzure Data | Azure Data Factory | Azure Databricks | Azure Pipelines | Data FactoryHybrid work | Paid time offEntry-level Full TimeIE - Dublin - Warrington Plc, …1d ago
-
Lead, Finance Analytics & Enablement AI/ML EUR 50K-61KAmplitude | Business Intelligence | Cloud platform | Data Modeling | Data ScienceFlexdesk Access Pass | Travel stipend | Work from homeSenior-level Full TimeIreland - Remote R2d ago
-
Machine Learning Engineer EUR 69K-100KAPI Integration | Algorithms | Amazon S3 | Apache Spark | Azure DataMid-level Full TimeDublin, Ireland2d ago
-
A/B | A/B Testing | AWS | Alerting | AzureFully remote Europe | Healthcare coverage | Paid parental leave | Professional growth | Relocation assistanceMid-level Full TimeIreland R2d ago
-
Mid-level Full TimeDublin, Ireland2d ago
-
AI engineer II EUR 49K-67KAPIs | Data Preparation | Deep learning | Distributed Systems | ExperimentationMid-level Full TimeDublin, Ireland2d ago
-
AWS | Automated testing | Azure | Data Pipelines | Experiment trackingSenior-level Full TimeDublin, Ireland2d ago
-
APIs | AWS | Automated testing | Azure | Cloud PlatformsSenior-level Full TimeDublin, Ireland2d ago
-
Lead AI engineer EUR 48K-62KAPIs | Agentic Systems | Artificial Intelligence | Data Pipelines | Data PreparationSenior-level Full TimeDublin, Ireland2d ago
-
AWS | Access Control | Alerting | Automated testing | CI/CDSenior-level Full TimeDublin, Ireland2d ago
-
AI Native SW Engineer EUR 48K-62KAgent Orchestration | Anthropic API | Autogen | Braintrust | CI/CDEntry-level Full TimeDublin, 1 Grand Canal Square, Ireland2d ago
-
Forward Deployed Engineer EUR 50K-74KAPI Development | Agentic AI | Anthropic | Artificial Intelligence | CI/CDMid-level Full TimeDublin, 1 Grand Canal Square, Ireland2d ago
-
Cloud Backend Engineer (Database) PLN 220K-338KAWS RDS | Apache Druid | Apache Kafka | Apache Pinot | CassandraSenior-level Full TimeDublin, Ireland; Krakow, Poland2d ago
-
Data Engineer III (Databricks) EUR 64K-101KAWS | CI/CD | Configuration Management | Data Modeling | Data PipelinesCareer growth | Collaboration | Continuous improvementSenior-level Full TimeDublin, Ireland2d ago
-
Senior Data Engineer GBP 91K-124KAccess Control | Airflow | BigQuery | CI/CD | Census401k employer match | Hackathons | Health and wellness benefits | Learning and development budget | Open source support fundSenior-level Full TimeGermany R2d ago
-
Platform Engineer - DevOps & Data Pipelines EUR 65K-77KARM Templates | Access Control | Automated testing | Azure Virtual | Azure Virtual DesktopMid-level Full TimeMullingar, Ireland2d ago
-
AI Engineer EUR 42K-70KAI Observability | Access Control | Anthropic Claude | Azure AI | Azure OpenAIMid-level Full TimeMullingar, Ireland2d ago
-
Principal AI/ML Scientist & Engineer EUR 54K-75KAWS | Agentic Workflows | Apache Kafka | Artificial Intelligence | Batch PipelineCompassionate leave | Flexible hours | Paid leave | Pawternal leave | Performance-based bonusesSenior-level Full TimeIreland, Dublin3d ago
-
Alerting | Amazon Aurora | Amazon Aurora PostgreSQL | Amazon Redshift | Anomaly DetectionContinuous improvement focus | Cross-functional collaboration | Exposure to modern data platforms | Fully remote work | Ownership opportunitiesSenior-level Full TimeIreland R3d ago
-
Senior Data Engineer EUR 71K-125KAWS | Agile | Apache Airflow | Apache Hadoop | Apache KafkaEmployee donations matching | Lunch provided | Pension matching | Private health insurance | Training opportunitiesSenior-level Full TimePrimary location: Dublin, Ireland3d ago
-
Senior-level Full TimeGalway - Ireland3d ago
-
Senior Machine Learning Engineer EUR 64K-85KCost Optimization | Data Pipelines | Deep learning | GPU Computing | JAXSenior-level Full TimeIreland3d ago
-
Member of Technical Staff, Machine Learning EUR 55K-79KData Pipelines | Fine Tuning | GPU Computing | Inference | JAXLearning and development | Rapid iteration culture | Remote-friendlySenior-level Full TimeIreland3d ago
-
Full Stack Engineer, AI systems EUR 54K-75KAPI Architecture | Amazon DynamoDB | Docker | Kubernetes | Language ModelsSenior-level Full TimeIreland3d ago