大数据开发工程师
Tasks
- Build daily batch data pipelines
- Create data development coding standards
- Create data warehouse architecture standards
- Define warehouse layers for write and query performance
- Design data warehouse dimensional models
- Develop data service interfaces for BI and applications
- Develop real time stream processing jobs
- Implement data validation rules and automated checks
- Ingest data from files databases logs and message queues
- Maintain data assets directory with business and technical metadata
- Monitor data production timeliness
- Optimize job scheduling pipelines and SQL performance
- Write ETL with Hive and Spark SQL
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Flink | Apache Hive | Apache Spark | DataWorks | Dimensional Modeling | DolphinScheduler | ETL | Java | Kettle | Python | SQL | Shell | Sqoop
Education
Roles
Related jobs
-
Senior-level Full TimeShanghai, China7h ago
-
Automation & Robotics Engineer CNY 360K-600KActuator integration | Analog Signal | Artificial Intelligence | C++ | CADSenior-level Full TimeHangzhou, China18h ago
-
Data Analytics & Management, AVP CNY 300K-420KAgentic AI | Alteryx | Anomaly Detection | Classification | Code DevelopmentFlexible work/life support | Inclusive development opportunities | Paid volunteer days | Vibrant employee networksExecutive-level Full TimeHangzhou, China18h ago
-
Senior-level Full Time上海、北京22h ago
-
Entry-level Internship上海23h ago
-
Mid-level Full Time上海23h ago
-
Mid-level Full Time上海23h ago
-
Senior-level Full Time上海23h ago
-
Mid-level Full Time北京、上海23h ago
-
Entry-level Full Time上海23h ago
-
Entry-level Full Time北京、上海23h ago
-
Mid-level Full Time上海23h ago
-
Mid-level Full Time上海、北京23h ago
-
Mid-level Full Time上海23h ago
-
Senior-level Full Time上海23h ago
-
Senior-level Full Time上海 R23h ago
-
Mid-level Full Time上海 R23h ago
-
Senior-level Full Time上海 R23h ago
-
loco-manipulation(高动态物体交互方向) CNY 180K-360KBody Dynamics | C++ | Contact dynamics | Curriculum learning | Domain RandomizationMid-level Full Time上海23h ago
-
Data Infra高级研发工程师(AI与具身智能数据平台方向) CNY 144K-240KAsynchronous processing | Automated testing | CI/CD | CRD | Cloud NativeSenior-level Full Time上海 R23h ago
-
Mid-level Full Time北京、上海23h ago
-
Senior-level Full Time上海23h ago
-
Ego-Centric视觉感知算法工程师 CNY 180K-300K2D Segmentation | 3D Gaussian Splatting | 3D Hand Pose | 3D Reconstruction | 6D PoseMid-level Full Time上海、北京 R23h ago
-
Vr算法专家 CNY 144K-240KAHRS | Algorithm Deployment Optimization | Algorithm deployment | C++ | Deployment optimizationSenior-level Full Time上海23h ago
-
Senior-level Full Time上海23h ago