Senior Data Engineer

About BestEx Research

BestEx Research is a financial technology and research firm specializing in building sophisticated execution algorithms and transaction cost modeling tools servicing multiple asset classes. The firm provides its services to performance-demanding hedge funds, CTAs, asset managers, and banks through a traditional electronic broker and in a broker-neutral Software as a Service (SaaS) model.

BestEx Research's mission is to become the leader in automation and measurement of execution across asset classes globally and significantly reduce transaction costs for our clients.

Its cloud-based platform, Algo Management System (AMS), is the first end-to-end algorithmic trading solution for equities, futures, and foreign exchange that delivers an entire ecosystem around execution algorithms, including transaction cost analysis (TCA), an algo customization tool called Strategy Studio, a trading dashboard, and pre-trade analytics in a single platform. The platform is currently live for US equity and global futures trading.

BestEx Research is disrupting a $100 billion industry by challenging the status quo of stale, black-box solutions from banks and offering next-generation execution algorithms that combine performance improvement with transparency and customization. BestEx Research uses leading-edge technology to support its low-latency, highly scalable research and trading systems with its back end in C++, research libraries in C++/Python and R, and web-based technologies for delivering its front-end platforms.

Visit bestexresearch.com for more information about our mission, products, research, and services.

Why work at BestEx Research?

If you want to help investors reduce their trading costs in an incredibly complex market structure and help create and shape an industry-defining product, then this is an excellent opportunity to join an incredibly talented team of 20+. BestEx Research has almost zero turnover, top notch clients, zero bureaucracy, and a flat organizational structure. It is a true meritocracy, set in a collaborative environment, where every employee is working on extraordinarily interesting projects. Our pay scale and benefits are comparable to top-tier firms in our industry, either in the form of cash compensation or a hybrid cash and equity compensation plan. Employees at BestEx Research have exposure to much more variety in the projects they complete and opportunities for growth than in similar roles at other firms.

Location: Bengaluru, India

Our beautiful Bangalore office is conveniently located in Outer Ring Rd, Bangalore. Amenities include modern work spaces, free parking, recreational games, a fitness center, bike storage, wellness room, and free meals.

Primary Job Responsibilities:

Senior Data Engineer in the Data Engineering team contributes to the design and management of execution algorithms for global equities, futures, and foreign exchange with the aim of continuously minimizing trading costs for our clients. BestEx Research data engineers will learn about algorithmic trading, data management, performance measurement, and market microstructure, globally. The data engineering team is nested within a talented team of experts and practitioners in the field. Our internal research environment includes tick-by-tick market data, historical back testing simulators, instrument-level daily analytics and normalized order and execution data to facilitate new research. Day-to-day responsibilities of this role may include but are not limited to:

• ETL Pipeline Design & Implementation - Design, implement and optimize robust ETL pipelines for processing large volumes of financial market data, including tick-by-tick data, ensuring data quality and integrity.
• Data Infrastructure & Automation - Design and maintain databases (schemas, design, functions) and automate daily BAU tasks, ensuring smooth data workflows and managing data dependencies effectively. Investigate issues and implement solutions to maintain data integrity.
• System Design - Architect scalable and efficient data systems to handle high-volume, high-frequency financial data, with a focus on performance optimization and reliability.
• Analytics and Reporting Tools - Develop and optimize production-level code for new reports and analyses, ensuring they integrate seamlessly into both the post-trade product and pre-trade analytics tools.
• Data Management & Processing - Work with large financial data sets, applying efficient data wrangling techniques to ensure clean, structured, and ready-to-use data for analysis. Investigate issues and perform custom analysis for special market events or days, ensuring the highest level of accuracy and performance.
• Documentation - Document methodologies, system architecture, and data flows clearly for a range of internal and external audiences, ensuring transparency and reproducibility.
• Cross-Collaboration - Work closely with Quants and Product teams, contributing in data collection, model development, evaluation, and continuous improvement.

Requirements

• Highly motivated, willing to take ownership of work
• Collaborative mindset with strong independent problem-solving abilities
• Minimum 3+ years of experience in designing and implementing ETL pipelines
• Strong experience with data orchestration tools such as Airflow or Dagster
• Proven experience in system design for large-scale data processing
• Experience maintaining and managing large databases and raw tick-by-tick data
• Experience with Bitbucket, Git, Confluence, and Jira
• Bachelor's degree in Engineering or Master's degree in computer science, data science, statistics, or other quantitative fields
• Strong command of PostgreSQL and proficiency in Python or R
• Experience with building API backends is valued but not required
• Excellent communication skills and ability to articulate complex technical concepts to colleagues and clients