Senior Data Engineer - Web Scraping
Tasks
- Automate web scraping platform processes
- Build and maintain data pipelines
- Clean transform manipulate large datasets
- Collaborate with analysts and stakeholders on requirements
- Design develop deploy maintain web scrapers
- Design tools applications and automation
- Implement quality control checks
- Ingest data into databases and data warehouses
- Maintain alerting systems and investigate incidents
- Schedule workflows with Apache Airflow
Perks/Benefits
Skills/Tech-stack
APIs | AWS Lambda | AWS S3 | Amazon RDS | Amazon SNS | Amazon SQS | Apache Airflow | CI/CD | Data Pipelines | Data Warehousing | Docker | GitHub Actions | HTML | JavaScript | Jenkins | Kubernetes | Pandas | Python | SQL | Scrapy | Selenium | XPath
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
Azure Data | Azure Data Factory | Azure Data Lake | Azure Data Lake Storage | Azure Data Lake Storage Gen2Fully remote | Remote work flexibilitySenior-level Contract Full TimeSaudi Arabia R1d ago
-
Senior AI Engineer - Professional Services - KSA USD 147K-213KAPI Keys | CI/CD | CrewAI | Docker | FastAPIDental insurance | Employee Assistance Program (EAP) | Flexible time off | Global employee assistance program | Medical insuranceSenior-level Full TimeRemote Saudi Arabia R21d ago
-
Senior AI Backend Engineer - Agent Evaluation & Quality USD 175K-250KAPI Design | CI/CD | Cost Optimization | Experimentation | Function CallingSenior-level Full TimeMakkah, Makkah Province, Saudi Arabia - … R26d ago