Data Engineer

•  4+ years of Data Engineering experience.
• IBM DataStage/Cloud Pak Developer who has end-to-end ETL design/development/deployment experience on multiple projects.
• Ensure business requirements are met, validates correct information boundaries, integrity, security, accuracy and use of information, coordinates cross-application database changes.
• The Candidate should have over 6 years of Experience in IBM DataStage/Cloud Pak tools.
• The Candidate should be Self-driven solution provider  With Problem solving skills.
• The Candidate should have experience in SQL writing using DB2, Big Query.
• The Candidate should have exposure to most of the Database connectors/file /processing stages and applications pack stages.
• Experience in FTP/SFTP and Scheduling tools like Astronomer / Airflow.
• Knowledge in Terraform,Tekton pipeline - bucket, cloud ,BigQuery, Secret manager
• Knowledge of Data warehousing schemas/modelling and usage of Agile Methodology.
• Debug data quality issues by analysing the upstream sources and provide guidance to data integration resolutions.
• Maintain and constantly refine the Production failures and escalations; be able to capture, articulate and create knowledge base of Production ETL functioning and potential points of failures.
• Experience in Data modelling and solutioning (design and implementation) for medium to large scale products

Ø Responsible for gathering business requirements and design ETL systems to meet application needs

Ø Work closely with stakeholders to gather requirements and provide technical guidance on the migration process

Ø Collaborate with the data engineering teams to understand the existing JCL/ETL jobs and their dependencies.

Ø Design and implement migration strategies for moving ETL to CloudPak/GCP native services.

Ø Develop, test, and deploy data pipelines and workflows, ensuring scalability, reliability, and performance

Ø Troubleshoot and resolve any production issues or challenges that arise during the migration

Ø Deliver product features as part of roadmap and business needs

Ø Co-create test plan and lead execution and code deployment. Lead performance tuning to optimize the job execution.

Ø Responsible for validating, run, scheduling and monitoring jobs using CloudPak/DS and Atronomer.

Ø Responsible for debugging Ticketing issues in the production phase

Ø BE/BTech in CS/IT or equivalent degree in relevant domain

Ø Proven experience in data engineering [IBM DataStage or equivalent Cloud ETL preferred]

Ø Strong knowledge of ETL processes, data integration and data management

Ø Proficiency in programming languages such as SQL, Python or Java.

Ø Familiarity with cloud computing platforms and data storage solutions.