Data Engineer - Commodities
London - 62 Buckingham Gate
Millennium
Millennium is a global, diversified alternative investment firm with the mission to deliver high-quality returns for our investors.Millennium is a top tier global hedge fund with a strong commitment to leveraging innovations in technology and data science to solve complex problems for the business. We are assembling a strong Commodity Technology team to build our next generation in-house commodity technology platform and associated ecosystem of tools, applications and systems. Commodity Technology provides a dynamic and fast-paced environment with excellent growth opportunities and projects involving cutting edge technologies.
Responsibilities
- Develop data and analytics platform using standard tools for batch and stream processing, container management, observability, and persistence
- Work closely with traders, quants, and other technologists globally to understand system development requirements
- Develop software solutions in an agile fashion using modern software development practices e.g. comprehensive testing, version control practices, CI/CD, etc.
- Test and deploy software solutions in an automated fashion using CI/CD practices
- Ensure DevOps style management and operation of DEV/UAT/PROD deployment environments
Mandatory Requirements
- 5+ years’ of professional experience with Python software development
- Experience with Airflow, event streaming (e.g., Kafka) and processing (e.g., Flink), big data table formats like Iceberg or Delta Lake, and Kubernetes
- Working knowledge of traditional and cloud SQL databases
Preferred Requirements
- Familiarity with quantitative finance and electronic trading concepts.
- Experience using AWS technologies such as S3, Athena, SQS, Batch, Lambda
- Knowledge of the various Python dependency management and packaging tools (e.g., Poetry) and data quality, observability, and testing tools (e.g., Great Expectations, Soda)
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile Airflow Athena AWS Big Data CI/CD Data quality DevOps Finance Flink Kafka Kubernetes Lambda Python SQL Streaming Testing
Perks/benefits: Career development
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.