数据管线高级工程师
Tasks
- Build data cleaning labeling quality tools
- Build distributed data lakehouse management platform
- Build high throughput low latency distributed systems
- Build offline and real time processing
- Collaborate with machine learning teams to deliver data for model iteration
- Create data visualizations
- Design data models
- Design data pipeline end to end
- Develop data mining tools
- Enable data lineage tracking
- Implement data collection and synchronization
- Implement data version control
- Implement streaming data pipelines
- Manage metadata and fast search
- Optimize distributed data processing performance
- Perform data cleaning and standardization
- Provide data services
Perks/Benefits
- N/A
Skills/Tech-stack
Apache Iceberg | Caching | Columnar Storage | Data Lakehouse | Data Lineage | Data Mining | Data Modeling | Data Quality | Data Versioning | Data cleaning | Data labeling | Distributed Systems | Distributed file systems | Docker | ETL | File systems | Git | Go | I/O | I/O Optimization | Java | Kafka | Kubernetes | Log instrumentation | Memory Management | Metadata Management | MongoDB | MySQL | Performance optimization | PostgreSQL | Pulsar | Python | RabbitMQ | Real Time | Real-time Processing | Redis | Snapshot Isolation | Stream processing | Table Partitioning | Time processing
Education
Bachelor of Engineering | Bachelor of Science | Master of Science
Roles
Related jobs
-
资深机器视觉专家 CNY 180K-300KComputer Vision | Deep learning | Docker | Image Segmentation | Image classificationMid-level Full Time北京、上海9h ago
-
Entry-level Full Time北京9h ago
-
Senior-level Full Time常州9h ago
-
Hr智能化（共享服务） CNY 192K-300KAgent Framework | JavaScript | Language Model | Large Language Model | Low CodeMid-level Full Time北京9h ago
-
Mid-level Full Time北京9h ago
-
AI 平台高级 SRE 工程师 / DevOps 工程师 CNY 144K-240KAutomation | CUDA | Capacity Management | Containerd | Device pluginSenior-level Full Time北京9h ago
-
Senior-level Full Time北京9h ago
-
Mid-level Full Time北京9h ago
-
大模型规划算法工程师- 北京 CNY 240K-480KAgent architecture | Agent systems | Context Management | Deep learning | Dialogue UnderstandingMid-level Full Time北京9h ago
-
大数据Ai测试工程师 CNY 144K-240KAPI Testing | Apache Airflow | Data Drift | Data Drift Detection | Data GovernanceMid-level Full Time上海10h ago
-
Senior AI Engineer CNY 120K-180KAPI Integration | AWS SageMaker | Java | JavaScript | LLM integrationSenior-level Full TimeChengdu, China17h ago
-
Senior Data Engineer, Retail Business Intelligence CNY 360K-510KAWS | Alibaba Cloud | CSS | Cloud Computing | Data ModelingSenior-level Full TimeShanghai20h ago
-
Senior Data Engineer, Retail Business Intelligence CNY 336K-510KAWS | Alibaba Cloud | CSS | Cloud Computing | Data LakeSenior-level Full TimeShanghai22h ago
-
Automation Excellence Senior Specialist CNY 360K-600K.NET | AI Document Intelligence | AI Governance | Access Management | Azure AISenior-level Full TimeCNCGD02 - Chengdu - 1-4 Floor, …1d ago
-
Data Analytics & Management, SrAssc CNY 510K-510KAPIs | AWS | Apache Iceberg | Apache Spark | AzureEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysMid-level Full TimeHangzhou, China1d ago
-
Data Analytics & Management, SrAssc CNY 510K-510KAWS | Apache Iceberg | Apache Spark | CI/CD | Data LineageEmployee networks | Flexible work/life support | Paid volunteer daysMid-level Full TimeHangzhou, China1d ago
-
Data Engineer-AVP CNY 300K-420KAWS | Access Control | Apache Spark | Authentication | AzureEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysExecutive-level Full TimeHangzhou, China1d ago
-
Data Platform Engineer officer CNY 336K-510KAWS | Apache Iceberg | Azure | CI/CD | Data CatalogEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeHangzhou, China1d ago
-
Data Platform Engineer officer CNY 336K-510KAWS | Apache Iceberg | Apache Spark | Azure | CI/CDEmployee networks | Flexible work/life support | Inclusive development opportunities | Paid volunteer daysSenior-level Full TimeHangzhou, China1d ago
-
ASIC | Agent Frameworks | Automation | Bus Fabric | CADMid-level Full TimeChina, Shanghai1d ago
-
Data Engineer大数据工程师 CNY 144K-240KAWS Glue | Amazon S3 | Anomaly Detection | Apache Iceberg | Apache SparkSenior-level Full TimeHangzhou1d ago
-
API Integration | Agent Design | Agent evaluation | Context engineering | Data PipelinesEntry-level Full Time深圳1d ago
-
Entry-level Full Time InternshipShenzhen, Changsha1d ago
-
Entry-level Full Time InternshipChangsha, Shenzhen1d ago
-
Agent Design | Agent evaluation | Context engineering | Data Pipelines | LLM APIEntry-level Full Time InternshipShenzhen1d ago