Lead Data Engineer (Committer/Contributor - Spark, Kafka, Kubernetes, Flink, Druid, Yugabyte, Pinot, Starrocks, temporal)

Company Description

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.

Job Description

The candidate must have:

• Lead and participate in the design and implementation of our Analytics infrastructure to align with the vision of Visa Data Platform Infrastructure.
• Design and implement technical solutions to support the ecosystem of Analytics Infrastructure.
• Responsible for the implementation of ongoing Hadoop infrastructure including monitoring, tuning and troubleshooting.
• Design, implement and maintain security, data capacity, node forecasting and planning.
• Provide hardware architectural guidance, plan and estimate cluster capacity, and create roadmaps for the Hadoop cluster deployment.
• Actively participate in all phases of software development lifecycle: analysis, technical design, planning, development, testing/CICD, release, post production/escalation support
• Work on a highly scalable, available and resilient multi-tenant Platforms that can host large scale and highly critical applications.
• Innovate and Re-pivot Problems with Models and Higher Dimension Concepts to  Support Current and Potential Use Case(s).
• Develop large scale multi-tenant software components on the Platform in an Agile based methodology to provide self-service capabilities.
• Collaborating with partners from business and technology organizations, develop key deliverables for Data Platform Strategy - Scalability, optimization, operations, availability, roadmap.
• Build partnerships with other technology teams to drive efficiency in their use of platforms and services that PaaS provide.
• Expertise in applying the appropriate software engineering patterns to build robust and scalable systems.
• Ability to act as a Tech lead, guide and mentor junior members in the scrum team.  Stays current with new and evolving technologies.
• Take ownership of Platform/PaaS architecture and drive it to the next level of effectiveness to support current and long term requirements.
• Builds strong commitment within the team to support the appropriate team priorities.

This is a hybrid position. Expectation of days in office will be confirmed by your Hiring Manager.

Qualifications

Bachelor's degree in Computer Science, or related technical discipline.
With 12+ years of software development experience in building PaaS or Application frameworks.
Good to be as contributor/committer in one of the Bigdata technologies – Spark,, Kafka, Kubernetes, Flink, Druid,Yugabyte, Pinot, Starrocks,t emporal.
Proficiency in engineering practices and writing high quality code, with expertise in either one of Java or Python or Go.
Experience in building REST services and API’s following best practices of service abstractions, Micro-services and Orchestration frameworks.
Experience building generic services and instrumentation of libraries for wide use across an organization / company.
Experience in Agile methodology and CICD - tool Integration, automation, configuration management in GIT.
Excellent verbal and written communication and presentation skills, analytical and problem-solving skills
Appreciation of diverse LOBs(Line of business) and their platform needs combined with deep technical expertise are required for the role.

Additional Information

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.