Data Engineer - Web Scraping
Tasks
- Build data ingestion pipelines
- Build scalable data processing workflows
- Clean transform validate large datasets
- Design web scraping systems
- Develop automated data collection workflows
- Develop automation frameworks and internal tools
- Implement data quality control checks
- Investigate data pipeline incidents
- Schedule monitor optimize scraping workflows
Perks/Benefits
- Autonomy
- Collaborative team culture
- Continuous learning
- Flexible environment
- High ownership
- Professional growth
Skills/Tech-stack
API | AWS | AWS Lambda | Amazon RDS | Amazon S3 | Amazon SNS | Amazon SQS | Apache Airflow | CI/CD | Docker | GitLab CI | GitLab CI/CD | HTML | JavaScript | Jenkins | Kubernetes | Pandas | Python | SQL | Scrapy | Selenium | XPath
Education
Roles
Related jobs
-
AWS Glue | Apache Airflow | Apache Kafka | Apache Spark | CassandraCollaborative environment | Continuous learning and knowledge sharing | Fully remote work model | Inclusive workplace open to diverse backgrounds | Professional growth opportunitiesSenior-level Full TimeBrazil R7h ago
-
Agile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCompany-sponsored certifications | Continuing education | Flexible working hours | Paid time off | Remote workSenior-level Full TimePorto Alegre, Brazil R11h ago
-
Agile | Azure | Azure Data | Azure Data Factory | Azure Data LakeCompany-sponsored certifications | Continuing education | Flexible working hours | Paid time off | Remote workSenior-level Full TimeSão Paulo, Brazil R11h ago
-
Agile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCompany-sponsored certifications | Continuing education | Flexible working hours | Paid time off | Remote workSenior-level Full TimeSalvador, Bahia, Brazil R11h ago
-
Agile | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCertification support | Company-sponsored coursework | Conference attendance | Continuing education | Flexible working hoursSenior-level Full TimeRio de Janeiro, Brazil R11h ago
-
Agile | Azure | Azure Data | Azure Data Factory | Azure Data LakeCompany-sponsored certifications | Conference attendance | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeRecife, Pernambuco, Brazil R11h ago
-
Agile | Azure | Azure Data | Azure Data Factory | Azure Data LakeCompany-sponsored certifications | Conference support | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeManaus, Amazonas, Brazil R11h ago
-
Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCompany-sponsored certifications | Conferences | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeGoiânia, Goiás, Brazil R11h ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOpsCompany-sponsored certifications | Company-sponsored coursework | Conferences | Continuing education | Flexible working hoursSenior-level Full TimeFortaleza, Ceará, Brazil R11h ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure DevOpsCompany sponsored certifications exams coursework | Conference opportunities | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeCuritiba, Paraná, Brazil R11h ago
-
Azure | Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake StorageCompany-sponsored certifications | Company-sponsored coursework | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeBrasília, Federal District, Brazil R11h ago
-
Access Controls | Agile | Azure Data | Azure Data Factory | Azure Data LakeCompany-sponsored certifications | Conference attendance | Continuing education | Flexible working hours | Paid time offSenior-level Full TimeBelo Horizonte, Minas Gerais, Brazil R11h ago
-
Azure | Data Lake | Data Modeling | Data Warehousing | DatabricksHybrid workSenior-level Full TimeSao Paulo, SP, Brazil19h ago
-
Contratação de Pessoa Jurídica para Desenvolvimento de Sistema de Coleta de Dados - Vaga Nº 107/2026 BRL 20K-31KAccess Control | Android | Audio capture | Audit Logging | AuthenticationSenior-level ContractHíbrido (Manaus, AM, Brasil) R20h ago
-
Senior-level Full TimeSao Paulo, Brazil21h ago
-
AKS | Airflow | Azure | Azure Event | Azure Event HubsChildcare support | Commuter Transit Benefits | Dental plan | Employee benefits club | Flexible parental leaveMid-level Full TimeSão Paulo, SP22h ago
-
Data Engineer Brazil BRL 120K-132KAmazon Web Services | Amazon Web Services (AWS) | Apache Airflow | Apache Iceberg | Apache SparkMid-level Full TimeRio de Janeiro, RJ, BR22h ago
-
Access Control | Data Governance | Data Modeling | Python | SQLChildcare assistance | Continuous learning opportunities | Dental insurance | Discount club partnerships | Extended paternity leaveMid-level Full TimeBrazil23h ago
-
AWS | Amazon S3 | DBT | Dagster | ETLFully remote | Learning oriented work environment | Opportunity for contract extension | Technical ownership cultureSenior-level Full TimeBrazil R1d ago
-
Anthropic | Benchmarking | CI/CD | Docker | FastAPIAutonomy | Fully remote | PJ contract structureSenior-level Full TimeBrazil R1d ago
-
API documentation | Airflow | Amazon Web Services | Apache Beam | Apache SparkBirthday day off | Dental insurance | Educational support programs | Fitness and wellness partnerships | Fully remote workMid-level Full TimeBrazil R1d ago
-
Azure | CI/CD | Data Architecture | Data Governance | Data LineageChildcare support | Dental insurance | Health insurance | Life insurance | Meal and food allowanceSenior-level Full TimeOSASCO, Brazil; OSASCO, SP, Brazil; OSASCO1d ago
-
Azure | Data Quality | Databricks | Fraud Detection | MLOpsChildcare assistance | Dental insurance | Gym membership | Health insurance | Life insuranceEntry-level Full TimeOSASCO, Brazil; CURITIBA, PR, Brazil; OSASCO, …1d ago
-
Data Engineer III BRL 18K-18KDatabricks | MLOps | Machine Learning | PySpark | PythonChildcare assistance | Dental insurance | Food and meal allowance | Health insurance | Life insuranceSenior-level Full TimeOSASCO, Brazil; CURITIBA, PR, Brazil; OSASCO, …1d ago
-
Analista Data Engineer Junior BRL 19K-19KAzure | C# | C++ | Data Science | Generative AIBirthday off | Career mentoring | Childcare assistance | Dental insurance | Inclusion and wellbeing sessionsEntry-level Full TimeSao Paulo, Torre Paineira, Brazil1d ago