ETL / Data Pipeline Specialist
Tasks
- Build monitoring dashboards and alerting
- Build scrapers parsers and ingestion pipelines
- Design scheduling orchestration and error handling for ETL
- Document data lineage and schema changes
- Maintain run logs and error documentation
- Monitor overnight scraper and ingestion runs
- Optimize pipeline performance reliability and cost
- Triaging pipeline failures and apply real time fixes
- Verify data completeness and quality
- Write transformation logic to load data into PostgreSQL
Perks/Benefits
Skills/Tech-stack
AWS EC2 | AWS Lambda | AWS S3 | Apache Airflow | Bash | BeautifulSoup | CloudFormation | Containerization | Cron | Data Ingestion | Data Lineage | Data Parsing | Data Quality | Data Transformation | Docker | Git | GitHub | Headless browser | MongoDB | Playwright | PostgreSQL | Python | SQL | Scrapy | Selenium | Snowflake | Terraform | Web Scraping
Education
N/A
Roles
Data Pipeline Engineer | ETL Engineer | Engineer | Pipeline Engineer
Related jobs
-
SOL-1634 Senior Data Engineer INR 3750K-6250KAWS Glue | AWS Lambda | AWS S3 | Amazon EC2 | Amazon EMRRemote workSenior-level Full TimeRemote R6h ago
-
API Integration | Asynchronous processing | Containerization | Data Pipelines | Deep learningSenior-level ContractAnnapolis, United States R6h ago
-
AWS Glue | AWS Lambda | Amazon Kinesis | Amazon Redshift | Amazon S3Remote workSenior-level Full TimeRemote R7h ago
-
AI INTEGRATION MANAGER -Dayshift, WFH-Philippines AED 120K-120KAPI Integration | Data Pipelines | Data reporting | Database Design | JavaScriptGlobal collaboration | Growth opportunities | Remote-friendlyMid-level Full TimeRemote R7h ago
-
Embedded Software Engineer EUR 57K-76KARM | ARM Cortex | Bitbucket | Bluetooth Low Energy | C ProgrammingEmployee resource groups | Flexible work environment | Hybrid work model | Remote work optionSenior-level Full TimeKallithea, Athens, Greece R9h ago
-
Embedded Software Engineer EUR 57K-76KARM | ARM Cortex | Agile | Bitbucket | Bluetooth Low EnergyEmployee resource groups | Flexible work environment | Hybrid work schedule | Remote work optionSenior-level Full TimeKallithea, Athens, Greece R9h ago
-
AI Engineer (m/f/n) PLN 282K-402KAWS Lambda | Amazon Web Services | Apache Kafka | CI/CD | Cloud platformFlexible work location | International cross functional environment | Remote work option | Technology monitoring involvement | Training opportunitiesSenior-level Full TimeWarszawa, Województwo mazowieckie, Poland R10h ago
-
Senior Data Engineer (f/m/d) EUR 60K-70KAPIs | Azure CLI | Azure Functions | Data mesh | DatabricksFlexible working hours | Gym coverage | Health insurance | Internet reimbursement | Meal allowancesSenior-level Full TimeMadrid, Spain R13h ago
-
Ingénieur / ingénieure Data – IA Générative EUR 60K-70KAWS | AWS Bedrock | Azure DevOps | CI/CD | Copilot StudioDiversity And Inclusion Agreements | Employee participation and profit sharing | Health coverage | Internal mobility | Paid time offSenior-level Full TimeÉchirolles, Auvergne-Rhône-Alpes, France R13h ago
-
Senior Analytics Engineer - Remote AUD 110K-150KAttribution Modeling | CRM | Conversion Tracking | Data Layer | Data PipelinesEmployee share options | Learning and development opportunities | Paternity leave | Remote work | Work from home stipendSenior-level Full TimeSydney, New South Wales 2000, Australia R17h ago
-
Senior Devops Engineer INR 2500K-4000KALB | AWS Config | AWS GuardDuty | AWS Security | AWS Security HubSenior-level Full TimeIndia - Remote R20h ago
-
.NET | Agentic Workflows | CSharp Semantic Kernel | Csharp | DockerCareer growth | Collaborative work environment | State-of-the-art technologySenior-level Full TimePhilippines - Remote R20h ago
-
AI & Analytics Engineer EUR 49K-84KAI Agents | Async Programming | Azure | Blob Storage | Container Apps100 percent remote | Global collaboration | High autonomy | Real-world impactSenior-level Full TimeMAD MA, Spain R20h ago
-
Senior-level Full TimePhilippines - Remote R20h ago
-
Staff Applied ML Engineer AUD 160K-200KA/B | A/B Testing | APIs | AWS | B testingBirthday leave | Continuous learning | Flexible hours | Home office setup allowance | Life insuranceSenior-level Full TimeRemote, Australia R20h ago
-
AI Engineer USD 175K-226KAPI | AWS | Asynchronous Communication | Authentication | AuthorizationHealth benefits | Learning and growth | Professional development | Remote work | Travel opportunitiesSenior-level Full TimeUSA - Remote, United States R20h ago
-
Data Engineer II USD 63K-114KETL | SQL | Snowflake401k | Flexible work schedule | Health insurance | Hybrid work flexibility | Paid time offMid-level Full TimeRemote-MN, United States R20h ago
-
Senior Engineer, Customer Data Operations INR 2520K-3880KAWS | AWS Lambda | Airflow | Amazon EC2 | Amazon S3Access to Headspace | Hybrid work model | Mental health days | Retirement savings | Tuition reimbursementSenior-level Full TimeIndia, Bengaluru, Karnataka R20h ago
-
Senior AI Engineer CAD 120K-160KAWS Lambda | Agile | Amazon EC2 | Amazon RDS | Amazon S3Dental insurance coverage | Health insurance coverage | Hybrid work model | Learning and development resources | Vision insurance coverageSenior-level Full TimeToronto, Ontario R21h ago
-
Senior-level ContractMexico, Remote R22h ago
-
Senior-level ContractUSA, Remote R22h ago
-
Senior AI Engineer - Agentic USD 161K-220KAI Agents | API Development | Autogen | Channel communication | Context engineeringSenior-level Full TimeLehi, Utah, Open to Remote R22h ago
-
Member of Engineering (Post-training) GBP 100K-190KCode generation | Continual Learning | Data Generation | Deep learning | Distributed TrainingCompany equipment | Flexible hours | Fully remote | Health insurance allowance | Home-office allowanceMid-level Full TimeRemote (EMEA/East Coast) R22h ago
-
Staff Software Engineer - Data Platform PLN 275K-381KAPI Design | Alerting | Anomaly Detection | Data Normalization | Data PipelinesEducation stipend | Equipment | Flexible PTO | Health coverage | Life and disability coverageSenior-level Full TimeKrakow, Poland R23h ago
-
Data Science Engineer USD 146K-222KAnomaly Detection | Computer Vision | Cybersecurity Analytics | Data Analysis | Data Visualization401k | Education reimbursement | Flexible schedule | Hybrid work | Relocation assistanceMid-level Full TimeLivermore, CA, United States R23h ago