Senior Data Engineer, Data Lakehouse Infrastructure
Tasks
- Architect data lakehouse on GCP
- Design ETL/ELT pipelines
- Develop streaming and batch data pipelines
- Enable data discovery for governance and observability
- Implement metadata management
- Optimize distributed query engines
- Optimize query performance
- Orchestrate data workflows with Airflow
Perks/Benefits
Skills/Tech-stack
Apache Airflow | Apache Hudi | Apache Iceberg | Apache Spark | Batch Processing | BigQuery | Cloud platform | Composer | Dataflow | Dataproc | Delta Lake | Distributed query | Distributed query engines | ELT | ETL | GCS | Google Cloud | Google Cloud Platform | Kafka | Python | Query Optimization | Query engines | SQL | SQL Query Optimization | SQL query | Snowflake | SparkSQL | Starrocks | Streaming Pipelines | Trino
Education
N/A
Roles
Related jobs
-
Développeur II - IA Vision par ordinateur - MLOps CAD 81K-135KAgile | Apache Spark | Azure | Big Data | CI/CDFlexible schedule | Health insurance | Hybrid telework | Life insurance | Retirement planMid-level Full TimeMontréal, QC, CA4h ago
-
Amazon Redshift | Automated testing | Branching | CI/CD | Cloud DataCollaborative culture | Distributed international team | Equity stock options | Mentorship | OwnershipSenior-level Full TimeCanada R8h ago
-
Algorithm Engineer, Battlespace Awareness USD 126K-167KAgile Development | C++ | Classification | Clustering | ControlsHealth benefits | Paid time off | Recovery BenefitsMid-level Full TimeBroomfield, Colorado, United States; Fort Collins, …10h ago
-
Algorithm Developer, Battlespace Awareness USD 126K-167KAgile | C++ | Clustering | Data Association | DebuggingMid-level Full TimeBroomfield, Colorado, United States; Fort Collins, …10h ago
-
Senior Algorithm Engineer, Battlespace USD 190K-252KAgile | C++ | Classification | Clustering | ControlsSenior-level Full TimeBroomfield, Colorado, United States; Fort Collins, …10h ago
-
Senior Embedded Linux Engineer, C++ USD 169K-216KAnalog electronics | Bare Metal | C Programming | C programming language | C#401k employer match | Cell phone reimbursement | Dinner provided twice weekly | EV charging | Flexible PTOSenior-level Full TimeEl Segundo, California, United States12h ago
-
Embedded Linux Engineer, C++ USD 131K-184KAnalog electronics | Bare Metal | C++ | CAN | CCSDS401k employer match | Cell phone reimbursement | EV charging | Flexible PTO | HSA employer matchEntry-level Full TimeEl Segundo, California, United States12h ago
-
Analytics Engineer USD 130K-175KAmazon Redshift | BigQuery | Business Intelligence | CRM data | CRM data modelingEntry-level Full TimeStockholm R18h ago
-
Lead Embedded Software Engineer USD 211K-286KAerospace Standards | Agile | Bitbucket | C++ | CentOSSenior-level Full TimePhoenix, AZ18h ago
-
Mid-level Full TimeNew Jersey, New Jersey, United States1d ago
-
Sr. Data Engineer USD 61K-94KApache Flink | Apache Kafka | Apache NiFi | Apache Spark | Cassandra401k match | Accidental death and dismemberment insurance | Bonus plan | Commuter benefits | Flexible spending accountsSenior-level Full TimeCordova, TN, United States1d ago
-
Senior-level Full TimeAustin, TX1d ago
-
Full Stack Gen AI Engineer USD 128K-197KAWS Bedrock | AWS Fargate | Amazon ECS | Amazon Web Services | CI/CDSenior-level Full TimeDallas, Texas, United States1d ago
-
Senior Software Engineer, AI Developer Experience USD 200K-400KAlerting | Build systems | CI/CD | Caching | DashboardsDaily lunches & snacks | Disability benefits | Life insurance | Medical/Dental/Vision insurance | Parental leaveSenior-level Full TimeSan Francisco1d ago
-
Machine Learning Engineer USD 129K-220KC++ | Computer Vision | Data Curation | Dataset Preparation | Deep learning401k | Flexible working hours | Free Coffee, Drinks & Snacks | Health insurance | Life insuranceEntry-level Full TimeArlington, Virginia1d ago
-
Machine Learning Engineer USD 129K-190KComputer Vision | Deep learning | Language Processing | Machine Learning | Multi-object Tracking401k | Flexible working hours | Free coffee | Free drinks | Free snacksMid-level Full TimeClifton Park, New York1d ago
-
Senior Data Engineer USD 140K-216KAPI Integration | Airflow | Alerting | Amplitude | BigQuery401k match | Casual dress environment | Development dollars | Disability insurance | Life insuranceSenior-level Full TimeEdmond, OK1d ago
-
Senior Software Engineer, Robotics Platform USD 143K-241KC# | C++ | CUDA | DDS | LinuxHigh ownership team | On-site work scheduleSenior-level Full TimeSan Francisco1d ago
-
01954 - Développeur Big Data Snowflake CAD 73K-108KAWS | AWS Glue | Data Management | Data Modeling | Data TransformationCoaching | Culture subscription reimbursement | Dental insurance | Flexible hours | Health insuranceMid-level Full TimeMontréal, Québec, Canada1d ago
-
Software Engineer IV USD 10KAgile | Apache NiFi | Bash | Configuration Management | Confluence401k contribution | Health allowance | Technology allowance | Technology lab access | Training and education benefitSenior-level Full TimeAnnapolis Junction, MD1d ago
-
Senior-level Full TimeMountain View, CA1d ago
-
Mechatronic/Robotics Engineer or Systems Integration Engineer - Camera and Sensor Calibration USD 132K-200KAR/VR | Bash | C++ | Calibration Techniques | Camera systemsOnsite workMid-level Full TimeMountain View, CA1d ago
-
Space Operations Engineer (Embedded Software) USD 119K-160KARM | C# | C++ | Command and control | Communication ProtocolsMid-level Full TimeSan Francisco, CA1d ago
-
API Development | Ablation Studies | Agent systems | CI/CD | Cloud ComputingDonation matching | Flexible PTO | Health insurance | Lunch stipend | Team building eventsEntry-level InternshipBerkeley, California, United States; San Francisco, …1d ago
-
Embedded Software Engineer – Level 3 or 4 (AHT) USD 114K-171KAerospace Integration | Agile | Analog Input | Bitbucket | Board SupportOnsite work | Relocation assistance | Top secret clearance required | Travel 10 percentMid-level Full TimeVADU01, United States1d ago