ETL / Data Pipeline Specialist
Tasks
- Build monitoring dashboards and alerting
- Build scrapers parsers and ingestion pipelines
- Design scheduling orchestration and error handling for ETL
- Document data lineage and schema changes
- Maintain run logs and error documentation
- Monitor overnight scraper and ingestion runs
- Optimize pipeline performance reliability and cost
- Triaging pipeline failures and apply real time fixes
- Verify data completeness and quality
- Write transformation logic to load data into PostgreSQL
Perks/Benefits
Skills/Tech-stack
AWS EC2 | AWS Lambda | AWS S3 | Apache Airflow | Bash | BeautifulSoup | CloudFormation | Containerization | Cron | Data Ingestion | Data Lineage | Data Parsing | Data Quality | Data Transformation | Docker | Git | GitHub | Headless browser | MongoDB | Playwright | PostgreSQL | Python | SQL | Scrapy | Selenium | Snowflake | Terraform | Web Scraping
Education
N/A
Roles
Data Pipeline Engineer | ETL Engineer | Engineer | Pipeline Engineer
Related jobs
-
SOL-1634 Senior Data Engineer INR 3750K-6250KAWS Glue | AWS Lambda | AWS S3 | Amazon EC2 | Amazon EMRRemote workSenior-level Full TimeRemote R7h ago
-
API Integration | Asynchronous processing | Containerization | Data Pipelines | Deep learningSenior-level ContractAnnapolis, United States R8h ago
-
AWS Glue | AWS Lambda | Amazon Kinesis | Amazon Redshift | Amazon S3Remote workSenior-level Full TimeRemote R8h ago
-
AI INTEGRATION MANAGER -Dayshift, WFH-Philippines AED 120K-120KAPI Integration | Data Pipelines | Data reporting | Database Design | JavaScriptGlobal collaboration | Growth opportunities | Remote-friendlyMid-level Full TimeRemote R8h ago
-
Embedded Software Engineer EUR 57K-76KARM | ARM Cortex | Bitbucket | Bluetooth Low Energy | C ProgrammingEmployee resource groups | Flexible work environment | Hybrid work model | Remote work optionSenior-level Full TimeKallithea, Athens, Greece R10h ago
-
Embedded Software Engineer EUR 57K-76KARM | ARM Cortex | Agile | Bitbucket | Bluetooth Low EnergyEmployee resource groups | Flexible work environment | Hybrid work schedule | Remote work optionSenior-level Full TimeKallithea, Athens, Greece R10h ago
-
AI Engineer (m/f/n) PLN 282K-402KAWS Lambda | Amazon Web Services | Apache Kafka | CI/CD | Cloud platformFlexible work location | International cross functional environment | Remote work option | Technology monitoring involvement | Training opportunitiesSenior-level Full TimeWarszawa, Województwo mazowieckie, Poland R12h ago
-
Senior Data Engineer (f/m/d) EUR 60K-70KAPIs | Azure CLI | Azure Functions | Data mesh | DatabricksFlexible working hours | Gym coverage | Health insurance | Internet reimbursement | Meal allowancesSenior-level Full TimeMadrid, Spain R14h ago
-
Ingénieur / ingénieure Data – IA Générative EUR 60K-70KAWS | AWS Bedrock | Azure DevOps | CI/CD | Copilot StudioDiversity And Inclusion Agreements | Employee participation and profit sharing | Health coverage | Internal mobility | Paid time offSenior-level Full TimeÉchirolles, Auvergne-Rhône-Alpes, France R14h ago
-
Senior Analytics Engineer - Remote AUD 110K-150KAttribution Modeling | CRM | Conversion Tracking | Data Layer | Data PipelinesEmployee share options | Learning and development opportunities | Paternity leave | Remote work | Work from home stipendSenior-level Full TimeSydney, New South Wales 2000, Australia R18h ago
-
Senior Devops Engineer INR 2500K-4000KALB | AWS Config | AWS GuardDuty | AWS Security | AWS Security HubSenior-level Full TimeIndia - Remote R22h ago
-
.NET | Agentic Workflows | CSharp Semantic Kernel | Csharp | DockerCareer growth | Collaborative work environment | State-of-the-art technologySenior-level Full TimePhilippines - Remote R22h ago
-
AI & Analytics Engineer EUR 49K-84KAI Agents | Async Programming | Azure | Blob Storage | Container Apps100 percent remote | Global collaboration | High autonomy | Real-world impactSenior-level Full TimeMAD MA, Spain R22h ago
-
Senior-level Full TimePhilippines - Remote R22h ago
-
Staff Applied ML Engineer AUD 160K-200KA/B | A/B Testing | APIs | AWS | B testingBirthday leave | Continuous learning | Flexible hours | Home office setup allowance | Life insuranceSenior-level Full TimeRemote, Australia R22h ago
-
AI Engineer USD 175K-226KAPI | AWS | Asynchronous Communication | Authentication | AuthorizationHealth benefits | Learning and growth | Professional development | Remote work | Travel opportunitiesSenior-level Full TimeUSA - Remote, United States R22h ago
-
Data Engineer II USD 63K-114KETL | SQL | Snowflake401k | Flexible work schedule | Health insurance | Hybrid work flexibility | Paid time offMid-level Full TimeRemote-MN, United States R22h ago
-
Senior Engineer, Customer Data Operations INR 2520K-3880KAWS | AWS Lambda | Airflow | Amazon EC2 | Amazon S3Access to Headspace | Hybrid work model | Mental health days | Retirement savings | Tuition reimbursementSenior-level Full TimeIndia, Bengaluru, Karnataka R22h ago
-
AI & Analytics Engineer EUR 49K-84KAsync Programming | Azure | Blob Storage | Container Apps | Data Drift100 percent remote | Global scale team | High autonomy | Real-world impact | Work from anywhere in Spain or UKSenior-level Full TimeMAD MA, Spain R22h ago
-
Data Engineer EUR 51K-78KCloud Composer | Cloud platform | DBT | Dataflow | GitHub ActionsCompany-Supported Sports Activities | Discounts on Company Services | Extra vacation days | Health insurance | Internal trainingSenior-level Full TimeVilnius, Lithuania R22h ago
-
Agile | Apache Spark | Azure | Azure Data | Azure Data Factory401k match | Company-Funded Pension | Dental insurance | Disability insurance | Education benefitSenior-level Full TimePrudential Tower, 655 Broad Street, Newark, … R22h ago
-
Decision Intelligence Developer PLN 180K-315KAPI Integration | Access Control | Agile | Auditability | By DesignHybrid workMid-level Full TimePoland - Warsaw R22h ago
-
Senior AI Engineer CAD 120K-160KAWS Lambda | Agile | Amazon EC2 | Amazon RDS | Amazon S3Dental insurance coverage | Health insurance coverage | Hybrid work model | Learning and development resources | Vision insurance coverageSenior-level Full TimeToronto, Ontario R22h ago
-
Senior-level ContractMexico, Remote R23h ago
-
Senior-level ContractUSA, Remote R23h ago