Back-End Data Engineer (Big data/ Cloud)
Tasks
- Build data ingestion pipelines
- Design high-level system architecture
- Develop low latency services
- Implement end to end data feed services
- Lead discovery and design phases
- Migrate legacy Hive pipelines to public cloud
- Orchestrate data workflows
- Troubleshoot production issues
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Airflow | Apache Hive | Apache Spark | BigQuery | Data pipeline | Dataproc | GCP | Low Latency | Low-Latency Systems | Service Implementation | Workflow Orchestration
Education
N/A
Roles
Regions
Countries
States
Cities
Related jobs
-
Embedded Linux | High Throughput | Inter-networking | Low Latency | Network SecuritySenior-level Full TimeSunnyvale, California, USA6h ago
-
Senior-level Full TimeCincinnati, OH, United States11h ago
-
Senior DevOps Engineer ID63545 USD 135K-185KAWS | Apache Airflow | ArgoCD | Azure | BigQueryFlextime | Growth roadmaps | Mentorship | Office work options | Remote work optionsSenior-level Full TimeMiami, United States16h ago
-
BigQuery | Curriculum Development | Data Governance | Data Lakehouse | Data ModelingAsynchronous hiring process | Part-time hoursSenior-level Part TimeBoston, US17h ago
-
Senior Director, AI / Machine Learning Software Engineer USD 136K-300KApache Flink | Apache Spark | CI/CD | Data Lineage | Data PrivacyHealth benefits | Paid leave | Paid volunteer timeSenior-level Full TimeNew York, NY, United States17h ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeDiversity and Inclusion Commitment | Flexible asynchronous culture | Laid-back atmosphere | Portfolio and LinkedIn supportMid-level Full TimeBellevue, WA, USA1d ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous culture | Flexible work environment | Remote-friendlyMid-level Full TimeLos Angeles, CA, USA1d ago
-
Bash | Data Processing | Docker | GCP | LinuxAsynchronous culture | Flexible remote workMid-level Full TimeKirkland, WA, USA1d ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous culture | Flexible remote work | Work on impact driven productsMid-level Full TimeSan Jose, CA, USA1d ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Friendly work environment | Portfolio and LinkedIn submission flexibility | Remote-friendly environmentMid-level Full TimeCupertino, CA, USA1d ago
-
Bash | Cloud infrastructure | Data Ingestion | Data Processing | DockerAsynchronous culture | Flexible management | Friendly work environment | Portfolio supportMid-level Full TimeMadison, WI, USA1d ago
-
Bash | Data Processing | Docker | GCP | LinuxAsynchronous culture | Remote-friendly workMid-level Full TimePittsburgh, PA, USA1d ago
-
Bash | Data Processing | Docker | GCP | LinuxAsynchronous culture | Flexible remote work | Friendly work atmosphere | Opportunity to make impactMid-level Full TimeAlexandria, VA, USA1d ago
-
Bash | Data Ingestion | Data Processing | Docker | GCPAsynchronous culture | Career growth | Collaborative team environment | Remote-friendlyMid-level Full TimeBoise, ID, USA1d ago
-
Bash | Data Processing | Docker | GCP | Large Scale DataAsynchronous culture | Friendly laid-back atmosphere | Hands-off management | Portfolio supportMid-level Full TimeChampaign, IL, USA1d ago
-
Bash | Cloud platform | Data Ingestion | Data Processing | DockerAsynchronous culture | Friendly and laid-back atmosphere | Remote/distributed workMid-level Full TimeOmaha, NE, USA1d ago
-
Bash | Cloud platform | Data Processing | Data pipeline | DockerAsynchronous work culture | Bonus | Equity | Flexible management | Friendly atmosphereMid-level Full TimeSyracuse, NY, USA1d ago
-
Bash | Data Processing | Docker | GCP | Infrastructure as CodeAsynchronous culture | Flexible remote work | Friendly work environment | Opportunity for impactMid-level Full TimeColumbia, SC, USA1d ago
-
Data Platform Engineer USD 205K-316KApache Airflow | Automation | Cloud infrastructure | Cloud platform | DBT401k match | Commuter benefits | Dental insurance | Employee stock options | Health insuranceMid-level Full TimeDenver, CO; New York, NY; San …1d ago
-
Solutions Engineer USD 150K-180KAI | Apache Flink | Apache NiFi | Apache Spark | Applied ScienceContinued Career Development | Employee resource groups | Flexible work from home | Generous paid time off | Paid volunteer timeMid-level Full TimeUS-California-Remote, United States R1d ago
-
Software Engineer - Analytics & Development USD 103K-218KAmazon Web Services | Apache Airflow | Apache Spark | Bash | CI/CDContinuing education | Family support | Financial benefits | Flexible time off | HealthcareSenior-level Full TimeCXN ANNAPOLIS JUNCTION MD, United States1d ago
-
Senior Lead Data Engineer USD 250K-286KAWS | Agile | Amazon Redshift | Apache Kafka | Apache SparkFinancial benefits | Health benefits | Inclusive work environment | Performance incentivesSenior-level Full TimeNew York, NY, United States1d ago
-
Systems Engineer (Tableau Platform) USD 90K-144KAWS | Active Directory | Ansible | Authentication | AzureOn-call rotation | On-site collaboration | Overtime flexibilityMid-level Full TimeBerkeley Heights, New Jersey, United States1d ago
-
Senior Staff Data Engineer USD 155K-262KAWS | Airflow | Apache Flink | Apache Spark | Azure401k match | Accidental death and dismemberment | Dental insurance | Employee assistance program | Equity eligibilitySenior-level Full TimeUnited States, United States1d ago
-
Mid-level Full TimeReston, VA (VA30), United States R1d ago