数据管线高级工程师
Tasks
- Build data management platform from ingestion to training
- Build data visualization
- Build distributed high throughput low latency systems
- Collaborate with model teams to deliver business requirements
- Design and build data pipeline core workflow
- Design data models
- Develop data access and collaboration workflows
- Develop data annotation quality checks tools
- Develop data cleaning tools
- Develop data mining toolchain
- Enable fast data retrieval
- Implement data lineage tracking
- Implement data version control
- Implement metadata management
- Manage memory and I O optimization
- Optimize distributed data processing performance
- Perform data cleaning and standardization
- Perform data collection and synchronization
- Perform offline data processing
- Perform real-time data processing
- Service data APIs
- Solve large scale data transfer bottlenecks
- Support data collection and logging
- Support unified data ingestion across teams
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Iceberg | Data Lineage | Data Processing | Data Versioning | Distributed Systems | Docker | ETL | Go | Java | Kafka | Kubernetes | Lance | Metadata Management | MongoDB | MySQL | PostgreSQL | Pulsar | Python | RabbitMQ | Real Time | Real-time Data | Real-time Data Processing | Redis | SQL
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Related jobs
-
Ai算法工程师-汽车专项-实习 CNY 25K-37KAutoml | C# | C++ | Computer Vision | Data ProcessingInternship | Mentorship | Real-world projectsEntry-level Internship南京5h ago
-
None Full Time深圳1d ago
-
Mid-level Full Time深圳1d ago
-
Apache Spark | Batch Processing | Big Data | Cloud Architecture | Cloud DataSenior-level Full TimeShenzhen, Guangdong Province, China1d ago
-
Senior-level Full TimeShanghai, China1d ago
-
数据平台工程师 CNY 180K-300KAWS | Azure | CI/CD | CloudFormation | Data GovernanceFlexible work arrangements | In-person collaborationMid-level Full TimeSHC01 - DXC Shanghai Campus Phase …1d ago
-
Sr. Consultant - Data Scientist CNY 360K-540KAgile | Computer Vision | Containerization | Data Governance | Data ScienceEmployee assistance program | Mindfulness programs | On demand digital course library | Personalized wellbeing programs | Volunteer matching programSenior-level Full TimeChina Shanghai (Hongmei)1d ago
-
Sr. Application Engineer CNY 360K-600KAutomated Workflows | C# | Cross-Functional Collaboration | Cross-functional | Data AnalysisSenior-level Full TimeChina - Beijing - Building 102, …1d ago
-
Mid-level Full Time北京 R2d ago
-
Miclaw-端云协同调度专家 (Hybrid AI Architect) CNY 240K-480K5G | Cloud API | Consistency protocols | Data Compression | Data PrivacyHybrid workSenior-level Full Time北京 R2d ago
-
Mid-level Full Time武汉2d ago
-
Entry-level Full Time北京 R2d ago
-
Entry-level Full Time北京 R2d ago
-
Mid-level Full Time北京 R2d ago
-
具身世界模型训练INFRA工程师 - XiaomiRobotics CNY 180K-360KDeep learning | DeepSpeed | Distributed Training | Fault Tolerance | KubernetesMid-level Full Time北京2d ago
-
具身智能算法工程师-模型 CNY 500K-500KActor-critic | Deep learning | Distributed Training | Implicit Q Learning | Inference accelerationMid-level Full Time北京 R2d ago
-
Entry-level Full Time北京2d ago
-
AI基础设施研发工程师（Sandbox / 容器化)-MiMo CNY 180K-420KAppArmor | Argo Workflows | CPU resource scheduling | Cgroup | ContainerdMid-level Full Time北京 R2d ago
-
大模型 Infra 研发实习生（Agentic RL 方向） CNY 25K-37KAsynchronous programming | Concurrency | Distributed Systems | Docker | GitEntry-level Internship深圳2d ago
-
Ai应用工程师（提效方向 0-1） CNY 50K-50KAI Programming | AI Programming Tools | API Integration | JavaScript | Language ProcessingEngineering resource support | Hands-on product development | Model and compute support | Real world usageEntry-level Internship深圳2d ago
-
大模型 Infra 研发实习生（Agentic RL 方向） CNY 25K-37KAlerting | Asynchronous programming | Concurrency | Data Retrieval | Data StorageEntry-level Internship深圳2d ago
-
Entry-level Full Time北京2d ago
-
Entry-level Internship北京2d ago
-
大语言模型后训练/Agentic算法工程师 CNY 180K-360KAgentic RL | DAPO | Distributed Training | Function Calling | GRPOEntry-level Full Time上海、北京2d ago
-
大模型 Infra 研发实习生（Agentic RL 方向） CNY 25K-37KAlerting | Asynchronous programming | Concurrency | Data pipeline | Distributed SystemsEntry-level Internship深圳2d ago