分布式计算与存储软件工程师
Tasks
- Build data loading tools
- Collaborate with algorithm teams for requirements delivery
- Create unified Dataset management system
- Design efficient storage partitioning compression vectorized reads
- Develop DataLoader SDK
- Implement caching prefetching
- Integrate multi source heterogeneous data
- Optimize parallel data loading pipeline
- Process data conversion parsing formatting
- Support distributed data processing performance troubleshooting
- Tune Linux file system and network I O performance
Perks/Benefits
- N/A
Skills/Tech-stack
Airflow | Apache Flink | Apache Ray | Apache Spark | C++ | Containerization | Golang | Hadoop | Hive | Hugging Face | Hugging Face Datasets | Java | Kubeflow | Kubernetes | Linux | Microservices | MongoDB | MySQL | NFS | NVIDIA DALI | ORC | Object storage | Parquet | Petastorm | PostgreSQL | Prometheus | PyTorch | Python | Redis | TensorFlow
Education
Roles
Related jobs
-
Mid-level Full TimeHangzhou14h ago
-
Mid-level Full Time深圳15h ago
-
【27届实习】云原生Ai平台研发工程师-杭州 CNY 37K-37KArgo Workflow | Computer networks | Containers | Data Structures | GoEntry-level Internship杭州15h ago
-
【27届实习】数据挖掘工程师 CNY 25K-37KData Structures | Deep learning | Distributed machine learning | Go | JavaFull-time conversion opportunityEntry-level Internship Temporary上海15h ago
-
【27届实习】Dba(数据库工程师) CNY 37K-37KAutomation | Database Architecture | Distributed Systems | MongoDB | MySQLFull-time conversion opportunityEntry-level Internship上海15h ago
-
Mid-level Full Time东莞16h ago
-
Ai算法工程师 CNY 180K-300KConvolutional Neural Networks | Data Mining | Data Warehouse | Data labeling | Deep learningMid-level Full Time东莞16h ago
-
Senior-level Full TimeChina2d ago
-
Senior-level Full TimeChina2d ago
-
Bash | Data Ingestion | Data Processing | Docker | GCPAsynchronous work culture | Friendly laid-back atmosphereMid-level Full TimeShanghai, China2d ago
-
Computer Graphics | Computer Vision | CoreML | Deep learning | Diffusion ModelsSenior-level Full TimeBeijing, Beijing, China2d ago
-
CUDA | DeepSpeed | Distributed Training | FSDP | Gradient CheckpointingEntry-level Full TimeBeijing, Beijing, China2d ago
-
Senior-level Full TimeBeijing, China2d ago
-
AI Computing Software Development Engineer, TensorRT CNY 144K-240KArtificial Intelligence | C# | C++ | Debugging | Deep learningSenior-level Full TimeChina, Shanghai2d ago
-
Entry-level Internship深圳2d ago
-
Entry-level InternshipBeijing,Beijing,China3d ago
-
(Sr) Cloud & Data Engineer CNY 192K-240KAWS | Automation | CI/CD | Container Security | Data ModelingMid-level Full TimeBeijing, Beijing, CN3d ago
-
[Growth Engineering] Staff Back-end Engineer I CNY 144K-240KAnomaly Detection | CI/CD | Containerization | FastAPI | Graph DatabaseSenior-level Full TimeShanghai, China3d ago
-
[Growth Engineering] Staff Back-end Engineer I CNY 144K-240KCI/CD | Containerization | FastAPI | Graph Database | Inference ServerSenior-level Full TimeShanghai, China3d ago
-
Deep Learning Performance Architect CNY 152K-240KComputer Architecture | Deep learning | Inference | JAX | Language ModelsSenior-level Full TimeChina, Shanghai3d ago
-
C# | C++ | Computer Vision | Debugging | Deep learningSenior-level Full TimeChina, Shanghai3d ago
-
Entry-level Internship Part TimeShanghai - Daning Main Blg, China3d ago
-
大数据开发(数据挖掘、数据测试、java) CNY 25K-37KApache Kafka | Apache Spark | Data Mining | Data Modeling | Data WarehousingMid-level Full Time保定3d ago
-
Entry-level Full Time广州3d ago
-
Senior-level Full Time上海、北京3d ago