Data Engineer
Delhi
HighLevel
HighLevel is the all-in-one sales & marketing platform that agencies can white-label and resell to their clients!Our Website - https://www.gohighlevel.com/YouTube Channel - https://www.youtube.com/channel/UCXFiV4qDX5ipE-DQcsm1j4gBlog Post - https://blog.gohighlevel.com/general-atlantic-joins-highlevel/
Our Customers:HighLevel serves a diverse customer base, including over 60K agencies & entrepreneurs and 500K businesses globally. Our customers range from small and medium-sized businesses to enterprises, spanning various industries and sectors.
Scale at HighLevel:We operate at scale, managing over 40 billion API hits and 120 billion events monthly, with more than 500 micro-services in production. Our systems handle 200+ terabytes of application data and 6 petabytes of storage.
About the Role:We are seeking a talented and motivated data engineer to join our team who will be responsible for designing, developing, and maintaining our data infrastructure and developing backend systems and solutions that support real-time data processing, large-scale event-driven architectures, and integrations with various data systems. This role involves collaborating with cross-functional teams to ensure data reliability, scalability, and performance. The candidate will work closely with data scientists, analysts and software engineers to ensure efficient data flow and storage, enabling data-driven decision-making across the organisation.
Responsibilities:
- Software Engineering Excellence: Write clean, efficient, and maintainable code using JavaScript or Python while adhering to best practices and design patterns
- Design, Build, and Maintain Systems: Develop robust software solutions and implement RESTful APIs that handle high volumes of data in real-time, leveraging message queues (Google Cloud Pub/Sub, Kafka, RabbitMQ) and event-driven architectures
- Data Pipeline Development: Design, develop and maintain data pipelines (ETL/ELT) to process structured and unstructured data from various sources
- Data Storage & Warehousing: Build and optimize databases, data lakes and data warehouses (e.g. Snowflake) for high-performance querying
- Data Integration: Work with APIs, batch and streaming data sources to ingest and transform data
- Performance Optimization: Optimize queries, indexing and partitioning for efficient data retrieval
- Collaboration: Work with data analysts, data scientists, software developers and product teams to understand requirements and deliver scalable solutions
- Monitoring & Debugging: Set up logging, monitoring, and alerting to ensure data pipelines run reliably
- Ownership & Problem-Solving: Proactively identify issues or bottlenecks and propose innovative solutions to address them
Requirements:
- 3+ years of experience in software development
- Education: Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field
- Strong Problem-Solving Skills: Ability to debug and optimize data processing workflows
- Programming Fundamentals: Solid understanding of data structures, algorithms, and software design patterns
- Software Engineering Experience: Demonstrated experience (SDE II/III level) in designing, developing, and delivering software solutions using modern languages and frameworks (Node.js, JavaScript, Python, TypeScript, SQL, Scala or Java)
- ETL Tools & Frameworks: Experience with Airflow, dbt, Apache Spark, Kafka, Flink or similar technologies
- Cloud Platforms: Hands-on experience with GCP (Pub/Sub, Dataflow, Cloud Storage) or AWS (S3, Glue, Redshift)
- Databases & Warehousing: Strong experience with PostgreSQL, MySQL, Snowflake, and NoSQL databases (MongoDB, Firestore, ES)
- Version Control & CI/CD: Familiarity with Git, Jenkins, Docker, Kubernetes, and CI/CD pipelines for deployment
- Communication: Excellent verbal and written communication skills, with the ability to work effectively in a collaborative environment
- Experience with data visualization tools (e.g. Superset, Tableau), Terraform, IaC, ML/AI data pipelines and devops practices are a plus
#LI-Remote #LI-NJ1
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Airflow APIs Architecture AWS CI/CD Computer Science Dataflow Data pipelines Data visualization dbt DevOps Docker ELT Engineering ETL Flink GCP Git Google Cloud Java JavaScript Jenkins Kafka Kubernetes Machine Learning MongoDB MySQL Node.js NoSQL Pipelines PostgreSQL Python RabbitMQ Redshift Scala Snowflake Spark SQL Streaming Superset Tableau Terraform TypeScript Unstructured data
Perks/benefits: Startup environment Team events
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.