大数据开发工程师
Tasks
- Build data asset catalog
- Build offline data pipelines
- Create data development coding standards
- Define warehouse layer architecture
- Design data warehouse dimensional models
- Develop ETL jobs with Hive and Spark SQL
- Develop data quality validation jobs
- Develop real time stream processing jobs
- Establish data warehouse architecture principles
- Generate data service interfaces
- Improve query performance
- Integrate data sources with Flink and Spark
- Monitor data production timeliness
- Optimize SQL performance
- Optimize jobs
- Optimize scheduling pipelines
- Write data to big data platforms
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Flink | Apache Hive | Apache Kettle | Apache Spark | Apache Spark SQL | Data Processing | Data Warehouse | DataWorks | Dimensional Modeling | Distributed Computing | Distributed Storage | DolphinScheduler | ETL | Java | Job Scheduling | Python | Real Time | Real-time Data | Real-time Data Processing | SQL | SQL Optimization | Shell | Spark SQL | Sqoop
Education
Bachelor of Arts | Bachelor of Engineering | Bachelor of Science
Roles
Big Data Developer | Data Developer | Data Engineer | Developer | Engineer
Related jobs
-
Senior-level Full Time深圳6h ago
-
Mid-level Full Time深圳6h ago
-
【27届校招】嵌入式camera系统培训生 CNY 25K-37KBrightness Consistency | C# | C++ | Color correction | Computer VisionEntry-level Full Time广州7h ago
-
Entry-level Full Time北京、上海7h ago
-
AI Foundation Model方向实习生 CNY 25K-37KAlgorithms | C plus plus | C# | Caffe | Computer VisionInternshipEntry-level Internship北京、上海7h ago
-
AI Foundation Model方向实习生 CNY 25K-37KAlgorithms | C# | C++ | Caffe | Computer VisionInternship opportunityEntry-level Internship北京、上海8h ago
-
Entry-level Full Time广州8h ago
-
Entry-level Full Time北京、上海8h ago
-
Data Engineer China Hub CNY 360K-600KAWS | Azure | CI/CD | Data Governance | Data LakeBarrier-free facilities | Cafeteria | Company doctor | Employee discounts | Employee eventsSenior-level Full TimeBeijing, CN1d ago
-
Senior Deep Learning Test Development Engineer, SDET CNY 360K-600KAI Agents | Automation | Bash | Bug Fixing | BullseyeSenior-level Full TimeChina, Shanghai1d ago
-
Machine Learning Engineer CNY 248K-315KAndroid | C++ | Concurrency optimization | Edge Computing | Embedded SystemsMid-level Full TimeShenzhen, Guangdong, China1d ago
-
Sr. Optimization Software Engineer CNY 360K-600KAI machine learning | API Development | Azure | Backend architecture | C#Senior-level Full TimeLOC3254: No.3239 Shenjiang Road, Shanghai, Pudong …1d ago
-
API | Accelerated computing | Agentic AI | Artificial Intelligence | CUDASenior-level Full TimeChina, Beijing1d ago
-
Senior AI Agent Test Developer, SDET CNY 360K-600KArtificial Intelligence | Defect Detection | Docker | Fault injection | Flaky test detectionSenior-level Full TimeChina, Shanghai1d ago
-
Entry-level Full TimeSuzhou, Jiangsu, China1d ago
-
Senior-level Full TimeGuangzhou, Guangdong, China2d ago
-
R&D Engineer CNY 28K-50KAI Agents | Computer Vision | Deep learning | Industrial Automation | LangchainEntry-level Full TimeShanghai, Shanghai, China2d ago
-
Edge AI Software Architect CNY 200K-300KC# | C++ | Embedded Systems | Machine Learning | MicrocontrollersSenior-level Full TimeBeijing, China2d ago
-
Data Analytics & Management, Off CNY 300K-380KAWS | Access Control | Alerting | Authentication | AzureEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysMid-level Full TimeHangzhou, China2d ago
-
Technical Consultant – Machine Learning CNY 240K-330KAWS | Amazon SageMaker | Azure | CI/CD | Data PipelinesMid-level Full TimeDalian Office, China2d ago
-
Senior Product Software Engineer CNY 120K-180KAPIs | Agile | Analysis | Documentation | Problem SolvingOnsite interview requiredSenior-level Full TimeCHN - Beijing, China2d ago
-
Ai算法实习生（振动与力学方向） CNY 25K-37KAPI Integration | Convolutional Neural Network | Keras | Langchain | Neural NetworkEntry-level Internship深圳2d ago
-
Associate Director, Data and Analytics Specialist CNY 240K-360KAgile | Ansible | Apache Spark | Bamboo | BitbucketMid-level Full TimeXi'an, Shaanxi, China2d ago
-
Data Analytics Engineer - Shenzhen CNY 38K-50KData Lineage | Data Modeling | Data Quality | Data Transformation | Data WarehousingMid-level Full TimeShenzhen2d ago
-
Analyst, Data Science CNY 144K-240KApplication Integration | Debugging | Documentation | Java | JavaScriptSenior-level Full TimeCN-M Plaza, China3d ago