Apps Dev Intmd Programmer Analyst - C11 - Data Engineering (Pyspark, HDFS , Python and Oracle) - PUNE

Data Engineer:

The Data Engineer is responsible for building Data Engineering Solutions using next generation data techniques. The individual will be working directly with product owners, customers and technologists to deliver data products/solutions in a collaborative and agile environment.

Responsibilities:

• Responsible for design and development of big data solutions. Partner with domain experts, product managers, analyst, and data scientists to PySpark and Python
• Work with data scientist to build Client pipelines using heterogeneous sources and provide engineering services for data science applications
• Ensure automation through CI/CD across platforms both in cloud and on-premises
• Define needs around maintainability, testability, performance, security, quality and usability for data platform
• Drive implementation, consistent patterns, reusable components, and coding standards for data engineering processes
• Convert Talend based pipelines into languages like PySpark, Python to execute on Hadoop and non-Hadoop ecosystems
• Tune Big data applications on Hadoop and non-Hadoop platforms for optimal performance
• Evaluate new IT developments and evolving business requirements and recommend appropriate systems alternatives and/or enhancements to current systems by analyzing business processes, systems and industry standards.
• Applies in-depth understanding of how data analytics collectively integrate within the sub-function as well as coordinates and contributes to the objectives of the entire function.
• Produces detailed analysis of issues where the best course of action is not evident from the information available, but actions must be recommended/taken.
• Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency.

Qualifications:

• 4 -8 years of total IT experience
• 4+ years of relevant experience with Pyspark and Python
• Experience on designing and developing Data Pipelines for Data Ingestion or Transformation using Python.
• Experience with Spark programming (pyspark or Python )
• Hands-on experience with Python/Pyspark and basic libraries for machine learning is required;
• Exposure to containerization and related technologies (e.g. Docker, Kubernetes)
• Exposure to aspects of DevOps (source control, continuous integration, deployments, etc.)
• Can-do attitude on solving complex business problems, good interpersonal and teamwork skills
• Possess team management experience and have led a team of data engineers and analysts.
• Experience in Oracle performance tuning, SQL, Autosys and basic Unix scripting.

Education:

• Bachelor’s degree/University degree or equivalent experience

This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.

------------------------------------------------------

Job Family Group:

Technology

------------------------------------------------------

Job Family:

Applications Development

------------------------------------------------------

Time Type:

Full time

------------------------------------------------------

Most Relevant Skills

Please see the requirements listed above.

------------------------------------------------------

Other Relevant Skills

For complementary skills, please see above and/or contact the recruiter.

------------------------------------------------------

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

 

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View Citi’s EEO Policy Statement and the Know Your Rights poster.