Senior Data Engineer, Off
Tasks
- Build and operate batch and streaming pipelines
- Build and publish semantic layer data models for governed BI endpoints
- Capture lineage for analytical outcomes
- Define and manage data product interfaces for consumption
- Design implement and optimize Iceberg tables partitions and metadata
- Implement data validation data quality and reconciliation checks
- Integrate reference and application data sources into curated datasets
- Optimize pipeline and query performance with partitioning archiving and purging
- Standardize deployment with CI CD and productionize jobs
- Support on call major incidents and perform backfills
Perks/Benefits
- Employee networks
- Flexible work/life support
- Inclusive development opportunities
- Paid volunteer days
Skills/Tech-stack
APIs | AWS | Apache Iceberg | Apache Spark | BI | CI/CD | Cloud platform | Data Governance | Data Lineage | Data Modeling | Data Quality | Data Validation | Databricks | Google Cloud | Google Cloud Platform | Informatica | Java | Microsoft Azure | Python | SQL | Schema evolution | Semantic layer | Snowflake
Education
N/A
Roles
Related jobs
-
Data Analysist / Digital Consultant CNY 304K-399KAPI Integration | AWS Data | AWS Data Lake | Amazon Web Services | Apache SparkFlexible working hours | Inclusive work environment | Work-life balanceMid-level Full TimeShanghai, Asia, China9h ago
-
Principal Perception Engineer CNY 380K-540K3D Geometry | BEV Transformation | C++ | CNN | Camera CalibrationSenior-level Full Time5-8F TOWER C, 788 JINZHONG ROAD, …21h ago
-
Senior Perception Engineer CNY 360K-540K3D Geometry | BEV Transformation | C++ | Camera Calibration | Computer VisionSenior-level Full Time5-8F TOWER C, 788 JINZHONG ROAD, …21h ago
-
Senior Data Engineer, Off CNY 260K-366KAWS | Apache Iceberg | Apache Spark | Batch Processing | CI/CDEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeHangzhou, China21h ago
-
AWS | Apache Iceberg | Apache Spark | Automated testing | AzureEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeHangzhou, China21h ago
-
Data Platform Achitecture - Senior Lead, AVP CNY 397K-520KAWS | Access Control | Apache Iceberg | Audit Logging | CI/CDEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeHangzhou, China21h ago
-
Senior Data Platform Infrastructure Engineer, Off CNY 397K-540KAWS | AWS CloudFormation | Access Management | Alerting | Audit LoggingEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeHangzhou, China21h ago
-
AWS | Access Control | Apache Iceberg | Authentication | AzureEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeHangzhou, China21h ago
-
Senior-level Full TimeChina, Shanghai21h ago
-
Senior-level Full Time北京、苏州1d ago
-
Senior-level Full Time上海、苏州、北京、广州、深圳1d ago
-
Mid-level Full Time北京、上海、苏州1d ago
-
Senior-level Full Time上海、苏州、北京、深圳1d ago
-
Deep Learning Planning R&D Engineer CNY 180K-300KAlgorithms | Autonomous Driving | C++ | Computer Vision | Data StructuresMid-level Full Time上海、苏州、北京、深圳1d ago
-
Mid-level Full Time北京、深圳、苏州、上海1d ago
-
Senior-level Full Time北京、上海1d ago
-
Mid-level Full Time北京、上海、苏州1d ago
-
Mid-level Full Time北京、深圳、苏州、上海1d ago
-
Entry-level Full Time深圳1d ago
-
Senior-level Full Time上海1d ago
-
Applied AI Engineer - Silicon Co-Design Group CNY 300K-480KAgent Framework | Autogen | C# | C++ | CrewAISenior-level Full TimeChina, Shanghai1d ago
-
Data Analysis Engineer CNY 25K-37KArtificial Intelligence | Dashboard Development | Data Analysis | Data Visualization | Data cleaningEntry-level Full TimeGuangzhou, China1d ago
-
Senior-level Full TimeChina, Shanghai1d ago
-
Senior Machine Learning Engineer II CNY 240K-480KAPI Integration | AWS | Agent Frameworks | Apache Spark | Artificial IntelligenceAnnual Medical Checkup | Birthday gifts | Family care leave | Festival Gifts | Flexible benefitsSenior-level Full TimeChina-Shanghai (Tianshan-W-Rd)1d ago
-
Senior Machine Learning Engineer II CNY 240K-480KAPI Integration | AWS | Agent Framework | Azure DevOps | CI/CDAnnual Medical Checkup | Family care leave | Flexible benefits | Life insurance | Long service awardSenior-level Full TimeChina-Shanghai (Tianshan-W-Rd)1d ago