Data Engineer - Web Scraping
Tasks
- Build web scraping systems
- Clean transform validate datasets
- Develop automation frameworks and internal tools
- Develop data ingestion pipelines
- Implement data quality control checks
- Investigate and resolve data pipeline incidents
- Process scalable data pipelines
- Schedule and monitor scraping workflows
Perks/Benefits
Skills/Tech-stack
API | AWS Lambda | AWS RDS | AWS S3 | AWS SNS | AWS SQS | Apache Airflow | CI/CD | Data Quality | Docker | GitLab CI | GitLab CI/CD | HTML | JavaScript | Jenkins | Kubernetes | Pandas | Postman | Python | SQL | Scrapy | Selenium | XPath
Education
Roles
Related jobs
-
Data Visualization Engineer EUR 45K-60KAPI | Alteryx | Angular | Artificial Intelligence | BlockchainInternational projects | Travel opportunitiesMid-level Full TimePorto, Portugal20h ago
-
AI Engineer (.NET / Azure AI) EUR 25K-65K.NET | API Development | Agentic AI | Angular | AnsibleComfort budget | Dental insurance | Flexible working hours | Health insurance | MentoringMid-level Full TimeCoimbra, Portugal23h ago
-
ML Ops Engineer - Digital Solutions EUR 35K-56KAWS | Agile | Amazon SageMaker | Apache Airflow | AzureAgile collaborative work environment | Continuous learning opportunities | Flexible working model | International work environmentMid-level Full TimeLousado, Braga, Portugal23h ago
-
API Design | Backend architecture | Command Line | Command-line Interface | Database systemsCompany-provided equipment | Continuous learning allowance | Flexible working hours | Fully remote work | Health insurance allowanceSenior-level Full TimePortugal R1d ago
-
GenAI Platform & Operations Trainee EUR 35K-40KAPIs | AWS | Azure | CI/CD | Cloud ComputingCareer mentoring | International travel | Meal allowance | Paid time offEntry-level Full TimePorto, Portugal1d ago
-
Anthropic API | Async Programming | Autogen | CI/CD | Cloud Data15th month salary | Birthday off | Health insurance | Mobility program | Work-life balanceSenior-level Full TimePorto, Portugal R2d ago
-
Adobe Analytics | Adobe Launch | Behavior analytics | Data Quality | Google Ads15th month salary | Birthday off | Health insurance | Mobility program | Work-life balanceMid-level Full TimePorto, Portugal R2d ago
-
Senior/Lead Data Engineer EUR 59K-75KAmazon Redshift | Apache Spark | Azure Data | Azure Data Factory | Azure Data LakeBenefits plan | Hybrid work model | Remote work whenever possible | Work equipment adjusted to roleSenior-level Full TimeLisbon, Portugal2d ago
-
Data Engineer (m/f) EUR 31K-42KApache Kafka | Automation | Azure DevOps | KubernetesCareer development | Collaborative work environment | Competitive benefits package | Continuous training | Professional growthEntry-level Full TimePorto Salvo, Portugal2d ago
-
AI Solutions Engineer EUR 49K-70KAgentic Systems | Data Governance | Data Lineage | Data Quality | Data modelsCollaborative environment | Continuous learning | Experimentation culture | Knowledge sharingMid-level Full TimePorto, Porto, Portugal2d ago
-
AWS | Autoscaling | Azure | Bash | CI/CDFully remote within Europe | Healthcare coverage | Paid parental leave | Professional growth | Relocation assistanceMid-level Full TimePortugal R2d ago
-
Data Analyst & Administrator Trainee EUR 28K-28KDashboard Development | Data Analysis | Data Integrity | Data Visualization | Data cleaningCross-functional team | Flexible working hours | Hybrid work schedule | MentorshipEntry-level Full TimeLisboa, Portugal R2d ago
-
Data Engineer EUR 35K-40K.NET | Data Monitoring | Data Quality | Data Transformation | Data integrationHybrid work modelMid-level Full TimeMatosinhos, Portugal; Matosinhos2d ago
-
Machine Learning Engineer EUR 49K-67KApache Airflow | Automated testing | CI/CD | Data Modeling | Data ValidationCareer growth programs | Health insurance | Hybrid work | Life insurance | Office locationsMid-level Full TimeLisboa, PT2d ago
-
Adobe Analytics | Google Analytics | Google Tag | Google Tag Manager | JavaScriptCareer progression opportunities | Health insurance | Meal card | Remote work options | Special discountsMid-level ContractPortugal2d ago
-
Senior Data Engineer GBP 91K-124KAccess Control | Airflow | BigQuery | CI/CD | Census401k employer match | Hackathons | Health and wellness benefits | Learning and development budget | Open source support fundSenior-level Full TimeGermany R2d ago
-
AI Assisted Development | Claude Code | Code review | Cursor | JavaScriptFlexible hours | Ongoing project based workSenior-level Full TimeMiami2d ago
-
Data Engineering | ETL | IBM DataStage | Python | StambiaElectric bikes | Green transportation budget | Hybrid work policy | Lactation Room | Prayer roomSenior-level Full TimeLisbon, Lisbon, Portugal2d ago
-
Senior AI Engineer - AI Systems EUR 48K-65KAnthropic API | Application engineering | Async | Autogen | CI/CDHealth insurance | Meal allowance | No-term contract | Paid vacation | Referral bonusSenior-level Full TimeLisbon, Portugal R3d ago
-
Digital Analytics Engineer EUR 36K-50KAdobe Analytics | Adobe Launch | Google Ads | Google Analytics | Google TagHealth insurance | Meal allowance | Paid vacation | Referral bonus | Remote workSenior-level Full TimeLisbon, Portugal R3d ago
-
AI Masterminds EUR 54K-75KAWS | Agentic AI | Agile | CI/CD | Data PipelinesAgile environment | Autonomy and collaboration | Extra compensation for after-hours work | Travel opportunitiesSenior-level Full TimeLisboa, Portugal3d ago
-
Forward Deployed AI Engineer EUR 65K-78KAI Agents | API Integration | AWS | Agentic Workflows | AutogenAnnual leave increased every 3 years | Continuous learning | Flexible benefits plan | Hybrid work | Meal allowanceMid-level Full TimeLisbon, Portugal R3d ago
-
Forward Deployed AI Engineer EUR 65K-78KAI Agents | API Integration | Amazon Web Services | Autogen | AzureFlexible benefits plan | Flexible hybrid work model | Meal allowance | Paid time off | Private medical insuranceMid-level Full TimeLisbon, Portugal R3d ago
-
Alerting | Amazon Aurora | Amazon Redshift | CI/CD | DashboardsContinuous improvement culture | Cross-functional collaboration | Exposure to modern data platforms | Fully remote work | Opportunity to influence engineering standardsSenior-level Full TimePortugal R3d ago
-
Azure | Data Drift | Data Modeling | Data Processing | Data QualityFlexible schedule | Freelance service agreement | Remote workMid-level Full TimePortugal - Remote R3d ago