Account Manager - Data Management

We are seeking an experienced Data Engineer proficient in Python, SQL, and Google Cloud Platform (GCP). The ideal candidate will have a solid understanding of data engineering best practices, data pipeline development, and cloud-based infrastructure. The role involves designing and implementing scalable data pipelines, ensuring data quality, and enabling efficient data processing to support analytics and machine learning models

Job Description:

Key Responsibilities:

Data Pipeline Development:

• Design, build, and maintain scalable data pipelines to efficiently process and transform large volumes of structured and unstructured data.
• Work with Python to develop ETL scripts and integrate data from various sources into cloud-based storage solutions (such as Google Cloud Storage or BigQuery).
• Ensure data quality, integrity, and consistency by implementing automated data validation and testing processes.

SQL & Database Management:

• Develop complex SQL queries for data extraction, transformation, and loading (ETL) to support business intelligence and analytics efforts.
• Optimize and manage cloud-based relational databases and data warehouses, such as BigQuery, ensuring performance, scalability, and cost efficiency.

Cloud Infrastructure (GCP):

• Design and manage data storage solutions using Google Cloud Platform tools, including BigQuery, Google Cloud Storage, and Dataflow.
• Implement cloud-based solutions for data ingestion, transformation, and storage, ensuring that data processing workflows are highly available and fault-tolerant.
• Work with GCP services like Dataflow, Cloud Pub/Sub, and Cloud Functions to build and manage real-time and batch data pipelines.

Collaboration with Data Scientists and Analysts:

• Work closely with data scientists and business intelligence teams to ensure the availability and accessibility of the data needed for advanced analytics, reporting, and machine learning.
• Provide clean, transformed datasets for use in predictive modeling, data analytics, and reporting.

Performance Optimization:

• Monitor and optimize the performance of data pipelines and cloud storage solutions.
• Implement best practices for managing data in the cloud, including performance tuning, cost optimization, and data governance.

Required Skills and Qualifications:

• Experience: 3+ years of experience in data engineering, with a focus on Python, SQL, and cloud-based platforms.
• Python: Strong experience with Python for building ETL processes, automation scripts, and data transformations.
• SQL: Advanced knowledge of SQL for querying and managing large datasets, working with relational databases, and optimizing queries.
• GCP: Proficiency with Google Cloud Platform (GCP), particularly BigQuery, Google Cloud Storage, Dataflow, Cloud Pub/Sub, and other GCP data services.
• Data Pipeline Development: Experience in designing and implementing end-to-end data pipelines that process and transform data for analytics and reporting.
• Data Warehousing: Experience working with cloud-based data warehousing solutions like BigQuery and integrating them with various data sources.
• Version Control: Familiarity with version control systems such as Git for code management and collaboration.

Preferred Qualifications:

• ETL Tools: Experience with ETL tools (e.g., Apache Airflow, Talend, or similar) for managing data workflows.
• Machine Learning: Knowledge of machine learning frameworks such as TensorFlow or PyTorch, especially for data pre-processing tasks.
• Data Governance: Understanding of data governance and best practices for data security, privacy, and compliance.
• Other Cloud Platforms: Familiarity with other cloud platforms (e.g., AWS, Azure) for cross-platform compatibility.

Location:

Chennai

Brand:

Paragon

Time Type:

Full time

Contract Type:

Permanent