Lead AI Data Engineer
Tasks
- Build embedding and RAG pipelines
- Build production data pipelines at scale
- Create feature ready datasets for ML
- Design distributed data system architecture
- Design ingestion ETL and ELT for Delta Lake and Iceberg
- Drive data governance and risk remediation
- Implement data quality checks and failure routing
- Ingest batch and streaming data
- Integrate heterogeneous data sources
- Manage team project assignments and deadlines
- Mentor and hire data engineering talent
- Own end to end data delivery and schema changes
- Process unstructured data for AI consumption
- Support vector stores and retrieval quality
- Tune Spark jobs for performance
Perks/Benefits
- Annual leave
- Continuous learning culture
- Digital wellbeing platform
- Employee assistance program
- Flexible working options
- Life insurance
- Medical insurance
- Parental/maternity leave
- Resilience development courses
- Sabbatical
- Volunteering leave
Skills/Tech-stack
Amazon S3 | Apache Iceberg | Apache Spark | Azure Data | Azure Data Lake | Azure Data Lake Storage | Data Governance | Data Lake Storage | Data Quality | Databricks | Delta | Delta Lake | Distributed Systems | ELT | ETL | Embeddings | Flink | Iceberg | Kafka | Kinesis | Minio | Parquet | Pulsar | Python | RAG | SQL | Streaming Data | System design | Unity Catalog | Vector Databases
Education
Roles
AI | AI Data Engineer | Data Engineer | Data Engineering | Data Engineering Manager | Engineer | Engineering Manager | Manager
Related jobs
-
Data Platform Engineer CNY 277K-370KAWS | Apache Airflow | CI/CD | Data Modeling | Data PrivacyEmployee assistance programme | Flexible working | Learning and development | Life insurance | Medical insuranceSenior-level Full TimeGuangzhou, CN2d ago
-
AI Data Engineer- CNY 300K-420KAmazon S3 | Apache Spark | Azure Data | Azure Data Lake | Azure Data Lake StorageAnnual leave | Continuous learning culture | Digital wellbeing platform | Employee assistance programme | Flexible benefitsMid-level Full TimeGuangzhou, CN2d ago
-
Amazon CloudWatch | Amazon EKS | Ansible | Apache Airflow | Apache SparkSenior-level Full TimeShanghai2d ago
-
Apache Doris | Apache Doris Operations | As-a-Service | Automation | Backup and RestoreMid-level Full TimeShanghai2d ago
-
Apache Doris | Automation | Backup | CSI | Cloud ComputingSenior-level Full TimeShanghai2d ago
-
Apache Doris | Backup and Restore | CSI | CockroachDB | CouchbaseSenior-level Full TimeShanghai2d ago
-
Apache Flink | Apache Kafka | Apache Spark | CI/CD | JavaSenior-level Full TimeShanghai2d ago
-
Apache Doris | Backup and Restore | CSI | Cloud automation | CockroachDBSenior-level Full TimeShanghai2d ago
-
Apache Doris | Automation | Backup and Restore | CSI | CockroachDBSenior-level Full TimeShanghai2d ago
-
AWS | Alibaba Cloud | Automation | Backup and Restore | CSISenior-level Full TimeShanghai2d ago
-
Backup and Restore | CSI | CockroachDB | Disaster Recovery | GoSenior-level Full TimeShanghai2d ago
-
Bash | Cloud platform | Data Processing | Docker | Google CloudAsynchronous culture | Competitive compensation | Friendly work environment | Remote-friendlyMid-level Full TimeShanghai, China2d ago
-
Mid-level Full Time北京2d ago
-
资深机器视觉专家 CNY 180K-300KComputer Vision | Deep learning | Docker | Image Segmentation | Image classificationMid-level Full Time北京、上海2d ago
-
Entry-level Full Time北京2d ago
-
Senior-level Full Time常州2d ago
-
Hr智能化（共享服务） CNY 192K-300KAgent Framework | JavaScript | Language Model | Large Language Model | Low CodeMid-level Full Time北京2d ago
-
Mid-level Full Time北京2d ago
-
Senior-level Full Time北京2d ago
-
Mid-level Full Time北京2d ago
-
大模型规划算法工程师- 北京 CNY 240K-480KAgent architecture | Agent systems | Context Management | Deep learning | Dialogue UnderstandingMid-level Full Time北京2d ago
-
Senior-level Full Time InternshipGuangzhou2d ago
-
大数据Ai测试工程师 CNY 144K-240KAPI Testing | Apache Airflow | Data Drift | Data Drift Detection | Data GovernanceMid-level Full Time上海2d ago
-
Senior AI Engineer CNY 120K-180KAPI Integration | AWS SageMaker | Java | JavaScript | LLM integrationSenior-level Full TimeChengdu, China3d ago
-
Senior Data Engineer, Retail Business Intelligence CNY 360K-510KAWS | Alibaba Cloud | CSS | Cloud Computing | Data ModelingSenior-level Full TimeShanghai3d ago