ETL / Data Pipeline Specialist
Tasks
- Build monitoring dashboards and alerting
- Build scrapers parsers and ingestion pipelines
- Design scheduling orchestration and error handling for ETL
- Document data lineage and schema changes
- Maintain run logs and error documentation
- Monitor overnight scraper and ingestion runs
- Optimize pipeline performance reliability and cost
- Triaging pipeline failures and apply real time fixes
- Verify data completeness and quality
- Write transformation logic to load data into PostgreSQL
Perks/Benefits
Skills/Tech-stack
AWS EC2 | AWS Lambda | AWS S3 | Apache Airflow | Bash | BeautifulSoup | CloudFormation | Containerization | Cron | Data Ingestion | Data Lineage | Data Parsing | Data Quality | Data Transformation | Docker | Git | GitHub | Headless browser | MongoDB | Playwright | PostgreSQL | Python | SQL | Scrapy | Selenium | Snowflake | Terraform | Web Scraping
Education
N/A
Roles
Data Pipeline Engineer | ETL Engineer | Engineer | Pipeline Engineer
Related jobs
-
Senior Data Engineer USD 135K-165KApache Airflow | Apache Spark | Argo | Cloud Dataproc | Context engineering401k match | Beverages | Dental insurance | Health insurance | Hybrid work availabilitySenior-level Full TimeDenver, CO, US R4h ago
-
Data Science Engineer/Analyst Aerospace (m/w/d) EUR 52K-53KApache Kafka | Apache Spark | Big Data | Data Curation | Data Transformation30 days vacation | Career development opportunities | Company bicycle | Company car | Company fitnessSenior-level Full Timebundesweit, Germany R11h ago
-
Data Pipelines | Debugging | Inference Optimization | Kubernetes | Machine LearningEquity package | Flexible leave options | Inclusive parental leave | Office setup allowance | Remote work within AustraliaSenior-level Full TimeSydney, Australia R13h ago
-
Lead Developer (SQL,Python) INR 2500K-3380KAWS | Cost Optimization | DBT | Data Modeling | Data QualitySenior-level Full TimeVirtual Office (Telangana), India R18h ago
-
Data Engineer GBP 50K-50KAzure | Azure Data | Azure Data Factory | Azure Databricks | Azure DevOpsAnnual leave | Diverse and inclusive culture | Employee assistance programme | Enhanced family-friendly policies | Fully remote and flexible working set upSenior-level Full TimeUnited Kingdom - Remote R18h ago
-
Security Data Engineer USD 136K-270KAPIs | Access Control | Apache Flink | Apache Kafka | Apache SparkMid-level Full TimeUS, CA, Remote, United States R18h ago
-
Senior Deep Learning Engineer PLN 221K-383KCUDA | Diffusion Models | Docker | Inference Server | PyTorchHybrid workSenior-level Full TimeUK, Remote, United Kingdom R18h ago
-
Expert/Principal Artificial Intelligence (AI)/Machine Learning (ML)Engineer, Video (R&D) CHF 128K-192KARM | AWS | Android | C# | C++Hybrid work model | Work from homeSenior-level Full TimeLausanne, Switzerland R18h ago
-
AI Agent | AI agent workflows | AI systems | API Integration | Agent workflowsBonus | Equity | Hybrid work | Leadership growthSenior-level Full TimeNew York, New York; Hybrid R18h ago
-
Principal Software Engineer - AI/ML (Ireland) EUR 95K-135KC++ | CUDA | Distributed Systems | GPU Optimization | Inference RuntimeSenior-level Full TimeRemote Ireland R18h ago
-
API Gateway | C++ | Cilium | Distributed tracing | EnvoyMedical/Dental/Vision insurance | Paid parental leave | Paid time off | Retirement 401k matchSenior-level Full TimeBoston, United States R18h ago
-
Senior Application Security Engineer (SCA/SAST) GBP 80K-102K.NET | Application Security | Application Security Testing | Azure DevOps | C#Senior-level Full TimeUK - Remote, United Kingdom R18h ago
-
1483466 Principal AI Platform Engineer/ AI Infrastructure Engineer(LLMOps / MLOps)- Remote A USD 150K-170KAI Search | AWS | Airflow | Arize Phoenix | AzureFirst shift hours | Fully remote | Monday to Friday scheduleSenior-level Full TimeUnited States R20h ago
-
Agentic AI Engineer USD 187K-253KAPIs | Airflow | Dagster | Debugging | Distributed Systems401k retirement plan company match | Flexible hours | Flexible time off | Monthly wellness benefit | Paid parental leaveSenior-level Full TimeRemote R20h ago
-
Principal Machine Learning Engineer (Remote) USD 50K-95KAWS | AWS CloudWatch | Amazon SageMaker | Apache Spark | Artificial IntelligenceSenior-level Full TimeTürkiye, Remote R21h ago
-
Mid-level Contract FreelanceRemote R22h ago
-
Senior Forward Deployed Engineer (AI/ML) USD 140K-174KBenchmarking | CUDA | Continuous batching | CrewAI | DatabasesConference reimbursement | Employee assistance program | Flexible time off | LinkedIn Learning access | Local Employee MeetupsSenior-level Full TimeSeattle R22h ago
-
Senior Forward Deployed Engineer (AI/ML) USD 140K-174KArtificial Intelligence | CUDA | Continuous batching | CrewAI | DatabasesConference reimbursement | Employee assistance program | Flexible time off | LinkedIn Learning access | Remote workSenior-level Full TimeSan Francisco R22h ago
-
Data Engineer USD 127K-175KAWS | Airbyte | Airflow | Alerting | CI/CD401k contribution | Dental insurance | Health insurance | Life insurance | Long-term disabilitySenior-level Full TimeUnited States - Remote R22h ago
-
Staff Applied AI Engineer, AI Experiences & Agents AUD 240K-250KAPIs | Agent Orchestration | Embeddings | Evaluation | JavaScriptEmployee stock options | Generous time off | Hybrid work option | Parental leave | Remote work optionSenior-level Full TimeMelbourne, VIC, Sydney, NSW R22h ago
-
Staff Engineer - AI Engineer USD 155K-175KERP | Hugging Face | Langchain | MES | ModbusContract extension possible | Remote workSenior-level Full TimeRemote, REMOTE, United States R23h ago
-
Senior Analytics Engineer USD 170K-190KAWS | Automation | Cost-aware query design | DBT | Data Modeling401k | Co working available | Company retreats | Flexible work location | Paid HolidaysSenior-level Full TimeRemote R23h ago
-
Staff Embedded Controls Engineer, Body Controls USD 129K-244KActuator control | Automation frameworks | Automotive Ethernet | Batteries | BazelEmployee resource groups | Flexible family care days | Medical, dental & vision coverage | Paid Holidays | Paid community serviceSenior-level Full TimePalo Alto, CA, United States R1d ago
-
Cloud Database Platform Engineer USD 110K-130KAWS | AWS DMS | AWS SCT | Access Control | Active Directory401k matching | Dental insurance | Flexible time off | Flexible work environment | Hybrid workSenior-level Full TimeHeredia, Heredia, Costa Rica R1d ago
-
API Integration | AWS | Artificial Intelligence | Azure | Cloud ComputingSenior-level Full TimeGermany (Remote) R1d ago