分布式计算与存储软件工程师
Tasks
- Build data loading tools
- Collaborate with algorithm teams for requirements delivery
- Create unified Dataset management system
- Design efficient storage partitioning compression vectorized reads
- Develop DataLoader SDK
- Implement caching prefetching
- Integrate multi source heterogeneous data
- Optimize parallel data loading pipeline
- Process data conversion parsing formatting
- Support distributed data processing performance troubleshooting
- Tune Linux file system and network I O performance
Perks/Benefits
- N/A
Skills/Tech-stack
Airflow | Apache Flink | Apache Ray | Apache Spark | C++ | Containerization | Golang | Hadoop | Hive | Hugging Face | Hugging Face Datasets | Java | Kubeflow | Kubernetes | Linux | Microservices | MongoDB | MySQL | NFS | NVIDIA DALI | ORC | Object storage | Parquet | Petastorm | PostgreSQL | Prometheus | PyTorch | Python | Redis | TensorFlow
Education
Roles
Related jobs
-
Entry-level Full Time上海6h ago
-
Mid-level Full Time广州 R6h ago
-
Mid-level Full Time深圳、上海、北京、中国香港6h ago
-
机器学习工程师 – 模型推理优化 CNY 180K-300KModel Distillation | Model Pruning | Model Quantization | Model Sparsity | ONNXEntry-level Full Time北京6h ago
-
Mid-level Full Time深圳、上海、北京、中国香港6h ago
-
Ai 多模态软件工程师(数据飞轮方向) CNY 180K-300KBatch Processing | Data Processing | Feature extraction | Language Models | Large Language ModelsCareer growth | Large-scale project experience | Learning opportunities | Team collaborationMid-level Full Time广州、北京6h ago
-
Mid-level Full Time深圳、上海、北京、中国香港7h ago
-
Entry-level Full Time深圳、北京、上海7h ago
-
Entry-level Full Time深圳、北京、上海7h ago
-
大语言模型后训练算法工程师 CNY 240K-480KDistributed Training | Docker | Fine Tuning | Human Feedback | KubernetesMid-level Full Time深圳、上海7h ago
-
Senior-level Full Time广州7h ago
-
数据平台开发工程师 CNY 180K-360KCode Refactoring | Data Governance | Data Lake | Data Modeling | Data WarehouseMid-level Full Time广州7h ago
-
Senior-level Full Time上海、深圳7h ago
-
Senior Consultant Specialist (RAG Backend Developer) CNY 144K-240KA/B | A/B Testing | ABAC | Audit Logging | B testingSenior-level Full TimeGuangzhou, Guangdong, China14h ago
-
AWQ | AWS | Batching | CPU architecture | CUDASenior-level Full TimeGuangzhou, Guangdong, China17h ago
-
Sr. AI Process Engineer, Seller Compliance CNY 360K-600KAWS | CI/CD | Code review | Data Pipelines | DocumentationSenior-level Full TimeShanghai, CHN1d ago
-
Senior Manufacturing AI Engineer – Machine Learning CNY 144K-240KClustering | Docker | Hypothesis Testing | Kubernetes | LightGBMSenior-level Full TimeChina Jiangmen1d ago
-
Senior Data Engineer (Smart Manufacturing) CNY 144K-240KApache Airflow | ClickHouse | Clustering Algorithms | Data Governance | Data ModelingDiversity and equity workplace | Global team | Inclusive work environmentSenior-level Full TimeChina Jiangmen1d ago
-
Entry-level Internship上海1d ago
-
具身智能 / Vla / Wam 算法工程师 CNY 180K-360KC plus plus | Camera Calibration | Coordinate transformations | Data Quality | Data labelingEntry-level Full Time上海1d ago
-
软件工程师 - pytorch训练框架国产芯片适配 CNY 240K-480KCUDA | GPU Architecture | GPU Programming | PyTorch | PythonMid-level Full Time北京1d ago
-
Mid-level Full TimeGuangzhou, Guangdong, China1d ago
-
Senior Consultant Specialist CNY 160K-240KApache Airflow | Apache Beam | Apache Spark | Cloud Composer | Cloud DataflowSenior-level Full TimeXi'an, Shaanxi, China1d ago
-
R&D – Embedded Display Software Development Engineer CNY 180K-300KAndroid | Android Display Stack | C# | C++ | Device DriversMid-level Full TimeShenzhen, Guangdong, China2d ago
-
R&D – Embedded Audio Software Development Engineer CNY 180K-300KALSA | Android | Audio HAL | C# | C++Mid-level Full TimeShenzhen, Guangdong, China2d ago