分布式计算与存储软件工程师
Tasks
- Build data loading tools
- Collaborate with algorithm teams to deliver solutions
- Design partition compression and vectorized reads
- Develop DataLoader SDK
- Implement dataset management system
- Implement distributed data processing
- Integrate multi source heterogeneous data
- Optimize NFS and object storage performance
- Optimize parallel data reading and caching
- Support data augmentation
- Troubleshoot performance issues
- Tune Linux file system and network I O
Perks/Benefits
- N/A
Skills/Tech-stack
Airflow | Apache Ray | C++ | Data loading | Docker | Faster Data Loading | Flink | Golang | Hadoop | Hive | Hugging Face | Hugging Face Datasets | Java | Kubeflow | Kubernetes | Linux | Microservices | MongoDB | MySQL | NVIDIA DALI | ORC | Parquet | Petastorm | PostgreSQL | Prometheus | PyTorch | Python | Redis | Spark | TensorFlow
Education
Related jobs
-
Mid-level Full Time上海3h ago
-
Mid-level Full Time深圳、上海3h ago
-
Mid-level Full Time北京3h ago
-
机器人大模型软件工程师 CNY 180K-300KAgent systems | Asynchronous programming | C++ | Context Scheduling | Conversation ManagementMid-level Full Time深圳3h ago
-
Mid-level Full Time广州3h ago
-
驾舱一体专家/高级专家/总监 CNY 240K-480KComputer Vision | Data Generation | Data Preprocessing | Data cleaning | Deep learningSenior-level Full Time北京、上海、深圳、广州3h ago
-
Entry-level Full Time深圳、上海、北京4h ago
-
Mid-level Full Time深圳、上海、北京、中国香港4h ago
-
Data Visualization | Data alignment | Data labeling | Data pipeline | Dataset integrationMid-level Full Time深圳、上海4h ago
-
Software Engineer (All Levels) – 大模型与智能机器人系统 CNY 240K-480KC++ | CUDA | DDS | GPU memory | GPU memory managementEntry-level Full Time广州、深圳4h ago
-
Entry-level Full Time InternshipBeijing4h ago
-
多模态大模型算法工程师(Vlm / 自动驾驶方向) CNY 180K-264KAgent modeling | Autonomous Driving | Autoregressive modeling | BEV | Behavior ModelingEntry-level Full Time北京、苏州4h ago
-
Java服务端研发工程师 - 商业化 CNY 180K-420KData Analysis | Data Attribution | Distributed Computing | Doris | FlinkMid-level Full Time北京6h ago
-
Mid-level Full Time北京6h ago
-
大模型算法专家 CNY 240K-480KAgentic RL | Language Models | Large Language Models | Linux | Multimodal GenerationSenior-level Full Time北京7h ago
-
Java开发工程师(大数据方向) CNY 180K-420KApache Flink | Apache Spark | Data platform | Data platform architecture | IOMid-level Full Time武汉7h ago
-
Entry-level Full Time北京8h ago
-
Senior-level Full Time北京8h ago
-
Data Modeling | Data Visualization | Deep learning | Machine Learning | PyTorchMid-level Full TimeChina-Shanghai (Tianshan-W-Rd)1d ago
-
(Senior) Manager-Advanced Analytics CNY 240K-480KAWS | Agile | Azure | Causal Inference | Cloud ComputingSenior-level Full TimeChina1d ago
-
(Senior) Manager-Advanced Analytics CNY 240K-480KAWS | Agile | Azure | Causal Inference | Cloud ComputingSenior-level Full TimeChina1d ago
-
Senior-level Full Time上海1d ago
-
【算法】多模态/大模型算法专家(上海) CNY 240K-480KAgent Framework | C plus plus | Computer Vision | Language Processing | LinuxSenior-level Full Time上海1d ago
-
Java技术专家(搜推模型样本数据平台方向) CNY 300K-480KApache Flink | Apache HBase | Apache Kafka | Apache Spark | C plus plusSenior-level Full Time北京、上海1d ago
-
【算法】计算机视觉算法专家(上海) CNY 240K-480KAnomaly Detection | C++ | Computer Vision | Deep learning | Fine-grained classificationSenior-level Full Time上海1d ago