Senior Data Engineer - Web Scraping
Tasks
- Build data pipelines to load databases and data warehouses
- Clean transform and prepare large datasets
- Collaborate with analysts and stakeholders on requirements
- Design build and maintain web scrapers
- Design data products and support data scientists
- Develop automation tools for web scraping platform
- Implement data quality control checks
- Orchestrate scheduled workflows with Apache Airflow
- Set up alerting and investigate data incidents
Perks/Benefits
- Autonomy ownership and trust
- Collaborative team environment
- Continuous innovation
- Fully remote
- Professional development in a technology team
Skills/Tech-stack
APIs | AWS Lambda | AWS RDS | AWS S3 | AWS SNS | AWS SQS | Apache Airflow | CI/CD | Data Warehousing | Docker | Fiddler | GitHub Actions | HTML | JavaScript | Jenkins | Kubernetes | Pandas | Postman | Python | SQL | Scrapy | Selenium | XPath
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
A/B | A/B Testing | APIs | Agentic AI | AlertingEnterprise impact | Fully remote | Knowledge sharing | Mentoring opportunities | Technical growthMid-level Full TimeBrazil R4h ago
-
Backtesting | Classification | Confidence Interval | Credit Risk | Credit risk modelingCareer development | Dental insurance | Emotional wellbeing | Flexible benefits | Flexible scheduleMid-level Full TimeBrazil R6h ago
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Autonomy and delivery ownership | Fully remote | Remote flexibility | Work with large scale digital transformation initiativeSenior-level Contract Full TimeBrazil R23h ago
-
API Integration | BigQuery | Data Analysis | Data Modeling | ELTContinuous learning | Hybrid work | Inclusive workplace | Opportunity for innovation | PJ contract modelMid-level Full TimeBrazil R1d ago
-
AWS | Aggregations | Apache Spark | Asset bundles | AzureAdditional vacation week | Annual team trip | Benefits marketplace credits | Birthday day off | Certification coverageSenior-level Full TimeBrazil R1d ago
-
AWS | Azure | BI | Cause analysis | ComplianceAutonomy | International collaboration | Opportunity to travel | Professional growth and development | Remote workSenior-level Full TimeBrazil R1d ago
-
Airflow | BigQuery | CI/CD | Cloud Composer | Cloud FunctionsCertification incentives | Childcare assistance | Dental insurance | Educational discounts | Fitness benefitsSenior-level Full TimeBrazil R1d ago
-
AWS | AWS Glue | Airbyte | Apache NiFi | AutomationBirthday day off | Childcare assistance | Dental insurance | Employee referral bonuses | Extended parental leaveMid-level Full TimeBrazil R1d ago
-
API | Access Control | Apache Spark | CI/CD | Data GovernanceBirthday day off | Childcare or nanny reimbursement | Christmas allowance | Company life insurance | Day off during childrens birthday monthSenior-level Full TimeBrazil R1d ago
-
AI Agents | APIs | Cloud platform | CrewAI | Data EngineeringCertification support | Childcare assistance | Dental insurance | Education Discounts | Fitness program discountsMid-level Full TimeBrazil R1d ago
-
Sr. Data Engineer BRL 253K-310KAWS | AWS DMS | Amazon AppFlow | Amazon Kinesis | Amazon SNSIn-person team events | International collaboration | Remote-first cultureSenior-level Full TimeRemote - Brazil R1d ago
-
Analytics Engineer (Remote - Brazil) BRL 98K-108KData Governance | Data Modeling | Data Privacy | Data Warehousing | Data integrationEquity package | PTO and sick days unlimited | Remote workMid-level Contract Full TimeFlorianópolis/ Remote R1d ago
-
APIs | Audio Processing | Automation | Computer Vision | Deep learningRemote work | Travel to US for onsite collaborationMid-level Full TimeBrazil (Remote) R1d ago
-
Analista de Dados Sênior BRL 78K-114KData Architecture | Data Modeling | Data Pipelines | Data Visualization | DatabricksSenior-level Full TimeSão Paulo, Brazil R1d ago
-
AI Agents | AI Foundry | AI Search | API Integration | AutogenBenefits club access | Childcare assistance | Dental insurance | Fully remote | Health insuranceSenior-level Full TimeBrazil R1d ago
-
Senior QA Engineer – Data & Analytics EUR 42K-60KAPI Testing | Agile | Anomaly Detection | BI | Data ContractsTraining and mentoring | Travel opportunitiesSenior-level Full TimePortugal / Brazil-%LABEL POSITION TYPE REMOTE … R2d ago
-
Staff Software Engineer, Product USD 85K-125KAWS | Airflow | Confluence | DBT | DatadogFast-moving team | High ownership and autonomy | Professional growth | Work from anywhereSenior-level Full TimeState of São Paulo, Brazil - … R2d ago
-
Grupo QuintoAndar | Staff Data Engineer BRL 144K-144KAWS | Amazon ECS | Apache Airflow | Apache Hadoop | Apache HiveChildcare subsidy | Day off on birthday | Dental plan | Discounts at educational institutions | Employee assistance programSenior-level Full TimeSão Paulo, São Paulo, Brazil R2d ago
-
AI Agent Developer (LLM & Agent Systems) BRL 234K-294KDocument processing | Elasticsearch | Embeddings | Evaluation | LangchainContinuous learning opportunities | Hybrid work | Remote workMid-level Full TimeFlorianópolis, SC, Brazil R2d ago
-
AWS | AWS Glue | AWS Lambda | Amazon Kinesis | Amazon RedshiftContinuous learning | Dental plan | Gym access | Health insurance | Life insuranceSenior-level Full TimeBrazil R3d ago
-
Data integration | Databricks | ELT | ETL | GitCorporate English classes | Fully remote | Learning Development Incentives | Udemy ProgramsMid-level Full TimeBrazil R3d ago
-
Junior Data Engineer (São Paulo, BR) BRL 122K-146KAWS Cloud | AWS Cloud Development Kit | AWS CloudFormation | AWS cloud development | Cloud Development KitCompany-provided laptop | Food allowance | Health insurance reimbursement | Home office setup allowance | Home-office allowanceEntry-level Full TimeSão Paulo, BR R3d ago
-
Forward Deployed Engineer / AI Engineer BR BRL 108K-108KAgile methodology | Architecture Design | Cloud Architecture | Conversational AI | Distributed SystemsClient-facing collaboration | Code review culture | Hybrid work 50 percent | Work with cross-functional teamsSenior-level Full TimeBrasil R3d ago
-
AWS | Airflow | Alerting | Aurora | CI/CDAnnual goal based bonuses | Birthday day off | Childcare assistance | Corporate training platform | Coworking accessSenior-level Full TimeBrazil R4d ago
-
Mid-level Full TimeBrazil - Remote R5d ago