数据管线高级工程师
CNY 240K-480K (estimate) Senior-level Full Time
Tasks
- Build data cleaning labeling quality inspection tools
- Build data visualization capabilities
- Build distributed data processing system
- Build distributed storage and table management
- Collaborate with model teams to deliver data solutions
- Design data models
- Design data pipeline core workflow
- Design high throughput low latency pipelines
- Develop data lake ingest workflow
- Develop data mining toolchain
- Enable data lineage tracking
- Expose data as data services
- Implement data version control
- Implement log tagging and data collection
- Manage data synchronization and data standardization
- Manage memory and I O performance
- Manage metadata and fast search
- Optimize end to end data collection cleaning transformation performance
- Perform offline data processing
- Perform real-time data processing
- Provide unified data access across teams
- Resolve large scale data transmission bottlenecks
- Support algorithm teams with error case localization
- Troubleshoot distributed system performance issues
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Iceberg | Caching | Columnar Storage | Data Lake | Data Lakehouse | Data Lineage | Data Mining | Data Modeling | Data Quality | Data Services | Data Standardization | Data Versioning | Data Visualization | Data cleaning | Data labeling | Distributed Computing | Distributed Systems | Distributed messaging | Docker | ETL | Git | Go | I/O | I/O Optimization | Java | Kafka | Kubernetes | Log Collection | Memory Management | Metadata Management | MongoDB | MySQL | NoSQL | Offline processing | Partitioning | Performance optimization | PostgreSQL | Pulsar | Python | Query engines | RabbitMQ | Real Time | Real-time Processing | Redis | Relational databases | Snapshots | Stream processing | Time processing
Education
Related jobs
-
大模型算法实习生 CNY 36K-37KDeep learning | DeepSpeed | Distributed Training | Java | Machine LearningCollaborative team | Large NLP dataset access | Long term internship support | Technical mentorship | Technical resourcesEntry-level Internship北京、上海17h ago
-
Mid-level Full Time北京17h ago
-
Entry-level Full Time北京、广州、上海18h ago
-
Entry-level Internship深圳18h ago
-
Computer Vision | Computer Vision Analytics | Data Annotation | Data Engineering | Data PipelinesSenior-level Full TimeSuzhou, China1d ago
-
Sr. Embedded Software Engineer-2 CNY 360K-600KBACnet | Buildroot | C# | Continuous integration | CybersecurityCompetitive benefits plans | Flexible time off | Paid parental leave | Vacation and holiday leaveSenior-level Full TimeXi'an, China1d ago
-
Sr. Embedded Software Engineer-1 CNY 360K-600KBACnet | Build Automation | Buildroot | C Programming | C#Flexible time off | Holiday leave | Paid parental leave | Vacation leaveSenior-level Full TimeXi'an, China1d ago
-
Senior Embeded QA Engineer CNY 120K-180KAPI Testing | Agile | Alarms | Ant | BACnetFlexible time off | Paid parental leave | Vacation and holiday leaveSenior-level Full TimeXi'an, China1d ago
-
Deep Learning Compiler CI/Infrastructure Engineer CNY 160K-240KAI Agents | Agent workflows | Artifact management | Automated triage | AutomationGenerous benefits packageSenior-level Full TimeChina, Shanghai R1d ago
-
Senior MLOps Manager CNY 360K-600KCause analysis | Continuous Improvement | Dashboards | Data Operations | Data QualitySenior-level Full TimeChina, Shanghai1d ago
-
Applied AI Engineer CNY 300K-399KA/B | A/B Testing | API Integration | Analytics | AnthropicCareer growth | Fully remote | Global team collaboration | High ownership culture | Learning and development budgetMid-level Full TimeChina R2d ago
-
Lead AI Engineer (AI Systems & Automation) CNY 360K-600KAlerting | Anthropic | Distributed Systems | Docker | EmbeddingsFully remote | High ownership culture | Learning and development budgetSenior-level Full TimeChina R2d ago
-
Data Analysis Engineer-AI CNY 156K-240KAlgorithms | Amazon Web Services | Apache Spark | Artificial Intelligence | Big DataSenior-level Full TimeCQM01 - CQM01, Chongqing Software Park, …2d ago
-
Data Analysis Engineer CNY 300K-420KAlgorithms | Amazon Web Services | Apache Spark | Artificial Intelligence | Big DataMid-level Full TimeCQM01 - CQM01, Chongqing Software Park, …2d ago
-
Mid-level Full TimeShanghai, Shanghai, China3d ago
-
Entry-level Internship深圳3d ago
-
Entry-level Full Time上海3d ago
-
AI Feedback | Deep learning | Direct Preference Optimization | Fine Tuning | Human FeedbackMid-level Full Time上海3d ago
-
Senior-level Full Time上海、武汉、北京3d ago
-
算法工程师-大模型数据方向 CNY 240K-360KApache Spark | Clustering | Data Augmentation | Data Deduplication | Data GovernanceSenior-level Full Time上海3d ago
-
数据开发工程师（Ai知识方向） CNY 180K-300KContent processing | Data Governance | ETL | Elasticsearch | Information ArchitectureFull-time employmentMid-level Full Time上海3d ago
-
Mid-level Full Time上海3d ago
-
Senior-level Full Time上海3d ago
-
Senior-level Full Time上海3d ago
-
大模型算法工程师（开放域对话） CNY 180K-300KA/B | A/B Testing | Agentic reinforcement learning | B testing | DeepSpeedMid-level Internship上海、北京3d ago