Data Engineer
Bangalore, India
OneByZero
OneByZero offers leading AI and Machine Learning services, specializing in AI research, development, and implementation across the Asia-Pacific Region.- Design and implement scalable ETL/ELT pipelines.
- Automate data workflows and ensure fault-tolerant execution.
- Support real-time data ingestion and streaming.
- The ability to design, implement, and optimize large-scale data and analytics solutions on Data Engineering
- Development of scripts using Python for loading, extracting, and transforming data.
- Assist with production issues in data warehouses like loading data, transformations, and translations
- Set up and manage data lakes and warehouses.
- Optimize data storage for performance and cost.
- Ensure high-performance data access during AI/ML training.
- Implement data security best practices (encryption, access control).
- Ensure data governance, compliance, and privacy.
Requirements
Basic Qualifications- 3+ years of experience in data engineering, ETL/ELT pipeline development, and data management.
- Proven hands-on experience with large-scale data processing, real-time data streaming, and data warehousing.
- Strong understanding of data modeling, data architecture, and database design.
- Hands-on experience with data partitioning, indexing, and query optimization.
- Proficiency in Python, Java, or Scala for data manipulation and automation.
- Experience with Apache Hadoop, Spark, or similar big data frameworks on AWS
- Understanding of enterprise data management concepts (Data Ingestion, Data Lake, Data Warehouse, Data Engineering, Data Sharing, Data Applications)
- Working knowledge of software engineering best practices
- Able to add unit test cases of the developed workload, pipelines & workflow.
- Should be able to develop and deploy the CI/CD pipelines
- Hands on experience working on Glue, Apache4
- Experience in GenAI tools and framework
- Implementing AWS services in a variety of distributed computing, enterprise environments
- Proficiency with at least one the languages such as C++, Java, Scala or Python
- Should know database concepts and working knowledge of SQL, data modelling.
Benefits
We provide a competitive salary and benefits package, a vibrant work environment, and numerous opportunities for professional growth. You'll have the opportunity to work with a team of industry experts on exciting projects that transform businesses and create significant value. Join us to revolutionize the way companies leverage technology for digital transformation.OnebyZero is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Architecture AWS Big Data CI/CD Data governance Data management Data warehouse Data Warehousing ELT Engineering ETL Generative AI Hadoop Java Machine Learning Pipelines Privacy Python Scala Security Spark SQL Streaming
Perks/benefits: Career development Competitive pay
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.