Senior Data Engineer (GCP • Python • Iceberg • Delta Lake • Kafka • Snowflake • Databricks)
Tasks
- Align semantic layer models with LookML and KPI catalog
- Build GCS to BigQuery ingestion pipelines
- Build access layer with RBAC and tenant scoped authorization
- Configure Snowflake Horizon external tables
- Design and implement UniForm write layer
- Develop data lineage and dependency tracking
- Develop data pipelines in Python and Spark
- Implement Delta Sharing endpoints for Databricks
- Implement Kafka based CDC ingestion
- Troubleshoot issues across full stack
- Write governed lakehouse tables with Delta Lake and Iceberg UniForm
Perks/Benefits
Skills/Tech-stack
Apache Iceberg | Apache Spark | BigQuery | Change Data Capture | Cloud platform | Data Capture | Data Lakehouse | Data Lineage | Databricks | Databricks Unity | Databricks Unity Catalog | Delta Lake | Delta Sharing | Google Cloud | Google Cloud Platform | Iceberg UniForm | Kafka | Python | RBAC | Snowflake Horizon | Unity Catalog
Education
N/A
Roles
Related jobs
-
Senior Backend Engineer, Experimentation USD 182K-263KA/B | A/B Testing | APIs | Alerting | B testingDental insurance | Health insurance | Mental health benefits | Restricted stock units | Vision insuranceSenior-level Full TimeRemote - US R8h ago
-
Biometrics | C++ | Computer Vision | Machine Learning | Multimodal LearningOffice three days per week | Relocation assistanceMid-level Full TimeSan Francisco R8h ago
-
Software Engineer - Backend Platform USD 140K-205KAPI Design | AWS | Cloud platform | Databases | Distributed SystemsComprehensive health coverage | Coworking stipend | Equity | Flexible savings accounts | Remote workMid-level Full TimePalo Alto, California, United States R9h ago
-
Backend Engineer, AI Engineering: Duo Chat USD 115K-194KAPI Design | GraphQL | Langchain | Langgraph | Language ModelsEmployee stock purchase plan | Equity compensation | Flexible paid time off | Growth and development fund | Parental leaveSenior-level Full TimeRemote, Canada; Remote, United States R11h ago
-
Machine Learning Engineer USD 210K-240KA/B | A/B Testing | B testing | Data Pipelines | Data Processing401k plan | Commuter benefits | E Bike Transportation | Generous PTO | Learning and development stipendMid-level Full TimeSan Francisco, CA (Hybrid) R11h ago
-
Data Engineer USD 130K-150KApache Spark | Azure | Azure DevOps | CI/CD | Data ContractsFree dog walking services | Free on-site gym | Free parking | Hybrid remote work | Medical/Dental/Vision insuranceMid-level Full TimeSeattle, United States R12h ago
-
Data Engineer II USD 130K-150KAWS | Airflow | CI/CD | CloudWatch | Containerization401k matching | Employee equity plan | Fertility and adoption reimbursement | Flexible time off | Health insuranceMid-level Full TimeUnited States R12h ago
-
Senior Algorithms and Simulation Software Engineer USD 95K-145KC++ | CI/CD | Control Systems | Git | Hardware-in-the-loopSenior-level Full TimeDevens, MA R13h ago
-
APIs | Anomaly Detection | Classification Pipeline | Evaluation | HuggingFace Transformers100 percent remote | BYOD | Eastern Time overlap | USD paymentSenior-level Full TimeRemote R14h ago
-
Staff Software Engineer (L4) Data Platform USD 171K-213KAWS | Apache Hadoop | Apache Hudi | Apache Kafka | Apache Spark401k retirement account | Health care insurance | Paid parental leave | Paid personal time off | Paid sick timeSenior-level Full TimeRemote - US R14h ago
-
Staff Machine Learning Engineer USD 176K-308KA2A | Asyncio | Embeddings | FastAPI | Java401k match | ESPP | Family leave | Flexible spending accounts | Flexible time offSenior-level Full TimeSanta Clara, California, United States R15h ago
-
Senior-level Full TimeRemote or VA, VA, United States R16h ago
-
Associate Data Engineer USD 60K-60KAWS | Apache Spark | Azure | Data Analysis | Data MappingComprehensive health coverage | Equity | Growth-focused environment | Home office setup allowance | Monthly cell phone allowanceMid-level Full TimeRemote US R16h ago
-
Data Engineer USD 150K-200KAWS EMR | Amazon S3 | Docker | Elasticsearch | Grafana401k match | Open PTO | Remote first flexible scheduleSenior-level Full TimeUnited States R16h ago
-
Automated Speech Recognition | CI/CD | CUDA | Computer Vision | Containerization401k matching | Company events | Dental coverage | Flexible work schedules | Internet reimbursementSenior-level Full TimeAnnapolis Junction, MD R17h ago
-
Practice Architect AI/ML USD 136K-204KAWS | Agent Orchestration | Agentic Workflows | AutoGen Orchestration | Autogen401k retirement plan | Employee assistance program | Health spending account | Life insurance | Long-term disabilitySenior-level Full TimeHanover, Maryland, USA R17h ago
-
4G | 5G | Automation | Network Analysis | Network TroubleshootingEducational assistance | Paid Holidays | Paid vacation | Sick leave | Stock purchase planMid-level Full TimePlano,Texas,United States R18h ago
-
ArgoCD | CSI | Cluster API | Distributed Systems | FluxCompany outings | Conference attendance | Hackathons | Happy hours | Professional developmentSenior-level Full TimeRemote, USA, United States R18h ago
-
Senior Data Engineer - Revenue Data Platform USD 192K-240KApache Airflow | Apache Iceberg | Apache Spark | Billing | Cause analysisSenior-level Full TimeBoston, Massachusetts, USA; New York, New … R18h ago
-
Data Engineer USD 124K-124KAWS | Apache Spark | DBT Cloud | DBT Core | JSON401k | Employee assistance program | Health savings account | Life insurance | Medical, dental, and visionMid-level ContractWestbrook, Maine, USA R20h ago
-
Senior Solution Engineer, Financial Services USD 165K-216KCloud Computing | Data Lake | Data Warehouse | Data fabric | Data meshSenior-level Full TimeUS-NY-New York City-Remote R21h ago
-
Sr Data Engineer_ITDE1I USD 102K-170KAPI Integration | AWS | Automation | Azure | CI/CD401k match | Dental insurance | Disability insurance | Hybrid work policy | Life insuranceSenior-level Full TimeHOUSTON, US, 77056 R1d ago
-
Sr. Automation Engineer Robotics Software USD 115K-144KAutomation | Computer Vision | Databases | FastAPI | HMIDental insurance | Health insurance | Paid time off | Retirement plan | Sick leaveSenior-level Full TimeAnkeny, Iowa, US R1d ago
-
Mid-level Full TimeTel Aviv-Yafo, Tel Aviv District, IL R1d ago
-
Software Engineer, Data Platform Team USD 152K-179KApache Airflow | Data APIs | Data Catalog | Data Lineage | Data Modeling401k matching | Medical, dental, and vision | Quarterly in person surges | Remote-first work environmentMid-level Full TimeRemote - USA R1d ago