Data Engineer
Shanghai, CN, 201203
CNY 360K-600K (estimate) Senior-level Full Time
Tasks
- Automate ingestion validation
- Build real time and batch ingestion
- Conduct root cause analysis
- Deploy production ready data systems
- Design scalable data pipelines
- Develop metadata pipelines for profiling and lineage
- Develop production grade data code
- Enforce data quality controls
- Establish monitoring and alerting
- Evaluate AI ML readiness of data assets
- Implement anonymization and PII controls
- Implement unit and integration testing
- Maintain data pipeline architecture
- Map technical fields to business terminology
- Perform code reviews
- Troubleshoot data issues and reliability incidents
Perks/Benefits
- N/A
Skills/Tech-stack
API Integration | Airflow | Alerting | Anonymization | Apache Spark | CI/CD | Dagster | Data Contracts | Data Governance | Data Modeling | Data redaction | Delta Lake | Elasticsearch | Git | Integration Testing | Kafka | Kubeflow Pipelines | Logging | Metrics | Observability | PII handling | Pandas | Polling | PostgreSQL | Prefect | Push based ingestion | PySpark | Pytest | Python | SQL | Schema evolution | Unit Testing | Workflow Orchestration
Education
Roles
Related jobs
-
Research MLE (Training Optimization）大模型训练优化工程师 CNY 144K-240KC++ | CUDA | DeepSpeed | Distributed Training | FSDPSenior-level Full TimeBeijing, Beijing, China6h ago
-
Senior-level Full TimeCN-OCG International Center, Cheng Du, China12h ago
-
Associate Director, Data and Analytics CNY 240K-360KAPI Testing | AWS | CI/CD | Cause analysis | Cloud TestingMid-level Full TimeGuangzhou, GD, CN, 5106201d ago
-
Sr. Associate Director, Data and Analytics CNY 240K-360KAmazon Web Services | Angular | CI/CD | Java | JenkinsMid-level Full TimeGuangzhou, GD, CN, 5106201d ago
-
Ai算法工程师 CNY 144K-240KBig Data | Big data processing | Computer Vision | Data Analysis | Data ProcessingEntry-level Full Time深圳1d ago
-
【校招实习】Ai算法工程师 CNY 25K-37KComputer Vision | Data Analysis | Deep learning | Distributed Computing | Feature EngineeringInternship | Research and development experienceEntry-level Internship深圳1d ago
-
Ai算法暑期实习生（Llm/强化学习） CNY 37K-37KDPO | Deep learning | Fine Tuning | LLM Fine-tuning | Language ModelsAt least 2 months internship duration | Beijing internship location | Full-time internshipEntry-level Internship北京2d ago
-
Ai算法暑期实习生（Llm/强化学习） CNY 37K-37KDPO | Deep learning | Language Models | Large Language Models | Machine LearningFull-time internship | Internship 2 months or moreEntry-level Internship北京2d ago
-
视觉感知算法工程师 - 榫卯计划 CNY 96K-120K4D reconstruction | C++ | Camera | Computer Vision | Computer Vision DatasetsEntry-level Full Time深圳2d ago
-
数据开发工程师 CNY 144K-240KAPI Development | Airbyte | Apache Flink | Apache Kafka | Apache SparkAI tool subscription | API quota | Cloud credits | Fast execution | Flat organizationSenior-level Full Time深圳2d ago
-
Ai算法暑期实习生（Llm/强化学习） CNY 37K-37KDPO | Deep learning | Experimentation | Information Retrieval | Language ModelsContinuous internship 2 months or more | Full-time internship | Location BeijingEntry-level Internship北京2d ago
-
Ai 应用工程师实习生 CNY 36K-37KLangchain | Language Models | Large Language Models | Prompt engineering | PyTorchEntry-level Internship北京2d ago
-
Applied AI Engineer (Agents)-算法2组 CNY 144K-240KApp Development | Context engineering | Data Engineering | Evaluation | Fine TuningMid-level Full Time深圳、厦门2d ago
-
Mid-level Full TimeChina - Hong Kong3d ago
-
CUDA | DeepSpeed | Diffusion Models | Distributed Training | FSDPSenior-level Full TimeBeijing, Beijing, China3d ago
-
LLM Agent Research Scientist 大模型智能体研究科学家 CNY 144K-240KAgent systems | Benchmarking | Cloud Computing | DeepSpeed | GPU ComputingInternship opportunity | Open to all experience levelsEntry-level Full TimeBeijing, Beijing, China3d ago
-
Sr. Associate Director, Data and Analytics CNY 240K-360KAgile | Alerting | Automated testing | CI/CD | Data ArchitectureMid-level Full TimeGuangzhou, GD, CN, 5106203d ago
-
DevOps Engineer CNY 240K-360KAWS | Alibaba Cloud | Ansible | Bamboo | CI/CDProfessional development opportunities | Smart casual dress code | Supportive work environmentSenior-level Full TimeShenzhen, Guangdong Sheng, China3d ago
-
Senior-level Full TimeChina, Shanghai3d ago
-
Senior AI Engineer CNY 144K-240KCheckpointing | Distributed Training | Efficient Fine Tuning | Fine Tuning | GPU workload optimizationSenior-level Full TimeBeijing Yizhuang, China3d ago
-
Senior-level Full TimeCHN - Guangzhou - EMS, China3d ago
-
IT Developer Engineer (Phython+AI) CNY 300K-420KArtificial Intelligence | Code review | Debugging | Machine Learning | Production SupportFlexible work hours | In-person collaborationMid-level Full TimeCN004 - Shanghai, China (CN004)3d ago
-
Entry-level Full TimeShanghai, China3d ago
-
Entry-level Internship广州3d ago
-
Mid-level Full Time深圳3d ago