Big Data Engineer
Tasks
- Architect and administer big data platforms
- Build and manage Spark ETL frameworks
- Build end to end data pipelines with infrastructure support
- Design and deploy machine learning pipelines
- Develop and maintain data lake and data warehouse platforms
- Orchestrate data pipelines with Airflow
- Support Trino and Presto queries
Perks/Benefits
Skills/Tech-stack
Agile | Amazon EMR | Amazon S3 | Apache Airflow | Apache Hadoop | Apache Hive | Apache Kafka | Apache Spark | Continuous Delivery | Databricks | DevOps | Docker | H2O | Java | KNIME | Kubernetes | Microservices | Presto | Python | Scala | Trino
Education
N/A
Roles
Related jobs
-
Entry-level Full Time合肥23h ago
-
None Full Time上海23h ago
-
Apache Flink | Apache Iceberg | Apache Spark | CI/CD | DockerMid-level Full TimeShanghai R23h ago
-
Apache Flink | Apache Iceberg | Apache Spark | CI/CD | DatabasesMid-level Full TimeShanghai R23h ago
-
Mid-level Full Time北京23h ago
-
None Full Time上海23h ago
-
None Full Time北京、上海23h ago
-
Entry-level Full Time上海23h ago
-
Ai算法工程师 CNY 144K-240KDeep learning | Feature Engineering | Hadoop | Hive | Human-Computer InteractionEntry-level Full Time深圳1d ago
-
【中台】后端开发专家（Ai 方向） CNY 144K-240KAsynchronous Queue | Caching | Code review | Distributed Systems | Function CallingSenior-level Full Time上海1d ago
-
AI Infra工程师-算法研发方向 CNY 180K-300KC++ | CI/CD | CUDA | Data parallelism | Distributed TrainingMentorshipMid-level Full Time北京、上海1d ago
-
Mid-level Full Time北京、上海1d ago
-
Mid-level Full Time北京、上海1d ago
-
Ai技术专家 CNY 144K-240KAPI Development | Access Control | Agent systems | Code Development | Content SafetySenior-level Full Time上海1d ago
-
Associate Director, Data and Analytics Specialist CNY 510K-510KAgentic AI | Authentication | Authorization | CI/CD | Data StoresMid-level Full TimeGuangzhou, Guangdong, China1d ago
-
Senior AI Data Infrastructure Engineer CNY 144K-240KAlerting | CI/CD | Caching | Cloud Object Storage | Data Lakes401k plan | Comprehensive office equipment | Flexible work environment | Health, dental, vision insurance | Remote work optionsSenior-level Full TimeShanghai R1d ago
-
Senior Software Engineer(IC3 Data Platform) CNY 480K-540KAgent workflows | Airflow | Apache Flink | C# | C++Senior-level Full TimeChina, Jiangsu, Suzhou; China, Beijing, Beijing; …1d ago
-
Data Services Engineer CNY 160K-264KAWS | Alerting | Alibaba Cloud | Amazon Aurora | Amazon ElastiCacheSenior-level Full TimeShanghai1d ago
-
Senior Machine Learning Engineer CNY 360K-510KBandits | C++ | Deep learning | Distributed Systems | Graphical ModelsSenior-level Full TimeShanghai1d ago
-
Associate Director, Data and Analytics Specialist CNY 510K-510KAPI Development | Agentic AI | Authentication | Authorization | CI/CDMid-level Full TimeGuangzhou, GD, CN, 5106201d ago
-
Clustering | Convolutional Neural Network | Decision Tree | Deep learning | Language ModelsMid-level Full TimeAIA AI Tech ED (Shanghai) Hongkou, …1d ago
-
Apache Flink | Apache Iceberg | Apache Spark | CI/CD | DockerEntry-level Full TimeShanghai R1d ago
-
CI/CD | Docker | Flink | Go | IcebergEntry-level Full TimeShanghai R1d ago
-
Apache Flink | Apache Iceberg | Apache Spark | CI/CD | DockerSenior-level Full TimeShanghai R1d ago
-
高级数据工程师（座舱Ai） CNY 144K-240KApache Airflow | Apache Flink | Apache Hive | Apache Kafka | Apache SparkSenior-level Full Time北京1d ago