分布式计算与存储软件工程师
Tasks
- Build data loading tools
- Collaborate with algorithm teams for requirements delivery
- Create unified Dataset management system
- Design efficient storage partitioning compression vectorized reads
- Develop DataLoader SDK
- Implement caching prefetching
- Integrate multi source heterogeneous data
- Optimize parallel data loading pipeline
- Process data conversion parsing formatting
- Support distributed data processing performance troubleshooting
- Tune Linux file system and network I O performance
Perks/Benefits
- N/A
Skills/Tech-stack
Airflow | Apache Flink | Apache Ray | Apache Spark | C++ | Containerization | Golang | Hadoop | Hive | Hugging Face | Hugging Face Datasets | Java | Kubeflow | Kubernetes | Linux | Microservices | MongoDB | MySQL | NFS | NVIDIA DALI | ORC | Object storage | Parquet | Petastorm | PostgreSQL | Prometheus | PyTorch | Python | Redis | TensorFlow
Education
Roles
Related jobs
-
Mid-level Full TimeZhuhai, Guangdong14h ago
-
Computer Vision | Computer Vision Analytics | Data Annotation | Data Engineering | Data PipelinesSenior-level Full TimeSuzhou, China18h ago
-
Sr. Embedded Software Engineer-1 CNY 360K-600KBACnet | Build Automation | Buildroot | C Programming | C#Flexible time off | Holiday leave | Paid parental leave | Vacation leaveSenior-level Full TimeXi'an, China1d ago
-
Senior Embeded QA Engineer CNY 120K-180KAPI Testing | Agile | Alarms | Ant | BACnetFlexible time off | Paid parental leave | Vacation and holiday leaveSenior-level Full TimeXi'an, China1d ago
-
Senior-level Full TimeChina, Shanghai1d ago
-
Deep Learning Compiler CI/Infrastructure Engineer CNY 160K-240KAI Agents | Agent workflows | Artifact management | Automated triage | AutomationGenerous benefits packageSenior-level Full TimeChina, Shanghai R1d ago
-
Applied AI Engineer CNY 300K-399KA/B | A/B Testing | API Integration | Analytics | AnthropicCareer growth | Fully remote | Global team collaboration | High ownership culture | Learning and development budgetMid-level Full TimeChina R1d ago
-
Lead AI Engineer (AI Systems & Automation) CNY 360K-600KAlerting | Anthropic | Distributed Systems | Docker | EmbeddingsFully remote | High ownership culture | Learning and development budgetSenior-level Full TimeChina R1d ago
-
Mid-level Full TimeShanghai, Shanghai, China3d ago
-
Entry-level Internship深圳3d ago
-
Entry-level Full Time上海3d ago
-
Entry-level Full Time上海3d ago
-
AI Feedback | Deep learning | Direct Preference Optimization | Fine Tuning | Human FeedbackMid-level Full Time上海3d ago
-
Senior-level Full Time上海、武汉、北京3d ago
-
算法工程师-大模型数据方向 CNY 240K-360KApache Spark | Clustering | Data Augmentation | Data Deduplication | Data GovernanceSenior-level Full Time上海3d ago
-
数据开发工程师（Ai知识方向） CNY 180K-300KContent processing | Data Governance | ETL | Elasticsearch | Information ArchitectureFull-time employmentMid-level Full Time上海3d ago
-
Mid-level Full Time上海3d ago
-
Senior-level Full Time上海3d ago
-
Senior-level Full Time上海3d ago
-
大模型算法工程师（开放域对话） CNY 180K-300KA/B | A/B Testing | Agentic reinforcement learning | B testing | DeepSpeedMid-level Internship上海、北京3d ago
-
Mid-level Full Time上海3d ago
-
大语言模型后训练/Agentic算法工程师 CNY 180K-360KDistributed Training | Function Calling | GRPO | Human Feedback | JSONEntry-level Full Time上海、北京3d ago
-
Senior-level Full Time上海3d ago
-
Embedded Software Eng. CNY 180K-300KARM | ASPICE | Automotive Software | Automotive Software Development | C#Mid-level Full TimeWuhu, CN3d ago
-
Senior Software Engineer Control Software Embedded CNY 120K-180KAgile Development | Algorithms | Bug Fixing | CAN | Code Version ControlSenior-level Full TimeSuzhou - Industrial Park, China4d ago