Data Engineer - Web Scraping, Market Intelligence
Tasks
- Build web scraping workflows
- Develop crawl and scrape tools
- Ensure compliance for web crawling
- Extract and ingest website data
- Identify and fix crawl issues
- Monitor and manage scrape data workflows
- Process unstructured data sets
- Scale scraping processes as needed
- Test scraped data accuracy and quality
Perks/Benefits
- Business travel accident insurance
- Educational assistance reimbursement
- Employee assistance program
- Private & life insurance
- Private medical and dental care
- Private pension scheme
- Vision allowance
- Volunteer opportunities
Skills/Tech-stack
Apache Spark | BeautifulSoup | Cloud Computing | HTML | HTTP | JavaScript | Linux | Networking | Pandas | Python | Requests | SQL | Scrapy | Selenium | Version control
Education
Roles
Data Engineer | Engineer | Scraping Engineer | Web Scraping Engineer
Related jobs
-
Data Engineer PLN 153K-247KAWS | Apache Airflow | Azure | Cloud Computing | Cloud platformAgile/Scrum environment | Choice of technical equipment | Flexible schedule | Flexible working hours | Hybrid or remote workMid-level Full TimeKraków, Lesser Poland Voivodeship, Poland - … R23h ago
-
Senior-level Full TimeWarsaw, Poland23h ago
-
Data Engineer PLN 156K-224KAccess Control | Agile | Audit Logging | C# | Data ArchitectureBusiness travel accident insurance | Educational assistance reimbursement | Employee affinity groups | Employee assistance program | Private & life insuranceMid-level Full TimeWarsaw, PL1d ago
-
Senior-level Full TimeWarsaw1d ago
-
Senior AI Engineer, Warsaw PLN 282K-402KCloud Computing | Data Processing | Deep learning | Distributed Systems | Language ModelsCompany pension | Company sick pay | Learning and development budget | Mental health support and therapy | Multisport membershipSenior-level Full TimeWarsaw1d ago
-
Entry-level Full TimeWarsaw, Warsaw, PL1d ago
-
Embedded Software Engineer PLN 146K-204KARM | C# | Linux | RTOSFamily-friendly policies | Flexible working hours | Hybrid work | Medical insuranceMid-level Full TimeKraków, Poland1d ago
-
A/B | A/B Testing | B testing | Cause analysis | DashboardsEquity participation | Fully remote | Healthcare coverage | Home office setup allowance | International company retreatsSenior-level Full TimePoland R1d ago
-
Embedded AI Engineer PLN 282K-400KC++ | Edge AI | FPGA | Langchain | Language ModelsFlexible working hours | Hybrid work | Open Source contribution | Private medical care | Sports cardSenior-level Contract Full TimeWarsaw, Masovian Voivodeship, Poland1d ago
-
Data Engineer PLN 120K-180KAzure | Big Data | CRM | Data Cleansing | Data QualityAdditional paid vacation | Company events | Ergonomic office | Family-friendly activities | Flexible working hoursMid-level Full TimeKatowice, PL, 40-2021d ago
-
Senior Software Engineer PLN 264K-360K.Net Core | ASP.NET | Amazon Web Services | Angular | Azure DevOpsCareer development support | Life insurance | Medical package | Multisport | Provision of equipmentSenior-level Full TimeGdańsk, Pomeranian Voivodeship, PL1d ago
-
Senior Software Engineer PLN 264K-360K.Net Core | AWS | Amazon Web Services | Angular | Automated testingCareer development | Life insurance | Medical package | Multisport | Provision of equipmentSenior-level Full TimeGdańsk, Pomeranian Voivodeship, PL1d ago
-
Senior Software Engineer PLN 258K-360K.Net Core | Amazon Web Services | Angular | Azure DevOps | C#Life insurance | Medical insurance | Multisport equipment | Wellbeing programSenior-level Full TimeGdańsk, Pomeranian Voivodeship, PL1d ago
-
Mid-level Full TimeGdańsk, Pomeranian Voivodeship, PL1d ago
-
Senior-level Full TimeWarsaw, Poland R1d ago
-
C++ | CAN | Debugging | Distributed Systems | EthernetAdditional voluntary benefits | Disability programs | Flexible working hours | Life insurance | Private medical careSenior-level Full TimePL-PDK-WARSAW-ALEJA KRAKOWSKA 110/114 BLD 3014613, Poland1d ago
-
.Net Core | ADLS | API Design | AWS S3 | Access patternsAdditional time off | Company breaks | Flexible work arrangements | Health, dental & vision plans | Long-term incentive programMid-level Full TimeLesser Poland1d ago
-
Data Engineer PLN 216K-267KAPI Integration | Apache Airflow | Bash | BigQuery | CI/CDCafeteria NAIS | Day care | Development programs | Gym and fitness room | Hybrid work modeSenior-level Full TimeWarsaw - Czerska 12, Poland1d ago
-
Senior Engineer (Python, Cloud, AI/GenAI) PLN 216K-355KCI/CD | Celery | Clean Code | Docker | EmbeddingsHybrid work model | Life insurance | Meal allowance | Private medical care | Training opportunitiesSenior-level Full TimeGdańsk, PL1d ago
-
Mid-level Full TimeHomebased - Gdańsk, Poland1d ago
-
Senior Data Engineer PLN 216K-300KAWS | AWS Glue | Amazon EMR | Amazon Kinesis | Amazon RDSFlexibility | In person and online events | Learning and career advancement | Social gatheringsSenior-level Full TimeKrakow2d ago
-
Senior-level Full TimeKraków2d ago
-
Airflow | Amazon Redshift | Amazon S3 | Amazon Web Services | Apache BeamCareer in multinational company | International projects | Language courses | Medical care | Multisport cardMid-level Full TimeWarsaw2d ago
-
Data Engineer / 1 PLN 280K-370KAPI Integration | Autoscaling | Caching | Connection pooling | DjangoCertification funding | Charitable initiatives participation | Conference participation | Inclusive work culture | On site and remote team eventsSenior-level Full TimeWarsaw, Masovian Voivodeship, Poland2d ago
-
Apex | C# | C++ | Case Development | DartContinuous learning | Flexible interview process | Remote workSenior-level Part TimeWarsaw, Poland R2d ago