Staff Data Automation Engineer - 7.5 Yrs , Java , Spark , Scala , Automation

Company Description

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.

Job Description

New Payment Flows (NPF) division’s charter is to capture new sources of money movement through card and non-card flows, including Visa Business Solutions, Government Solutions and Visa Direct which presents an enormous growth opportunity. Our team brings payment solutions and associated services to clients around the globe.  Our global clients and partners deploy our solutions to serve the needs of Small Businesses, Middle Market Clients, Large Corporate Clients, Multi Nationals and Governments.  

The Visa Business Solutions (VBS) and Visa Government Solutions (VGS) team is a world-class technology organization experiencing tremendous, double-digit growth as we expand products into new payment flows and continue to grow our core card solutions. This is an incredibly exciting team to join as we expand globally.  

Responsibilities:

• Design and implement comprehensive test automation frameworks for Spark and Hadoop batch applications

• Architect scalable, maintainable automation solutions for complex data processing pipelines

• Develop testing strategies that ensure data quality, performance, and reliability

• Create technical documentation for automation frameworks and testing approaches

• Develop automated tests for batch data processing applications using appropriate frameworks and languages

• Build and maintain REST API test automation suites for data services and integrations

• Implement data validation mechanisms to verify data transformations across pipelines

• Create reusable test components that can be leveraged across multiple projects

• Design and implement performance testing for data processing workloads

• Identify and implement test coverage metrics to measure effectiveness of testing

• Analyze test results and provide actionable insights to development teams

• Drive root cause analysis for complex data processing failures

• Continuously improve automation frameworks to increase efficiency and coverage

• Research and evaluate new testing tools and methodologies for data engineering applications

• Introduce innovative approaches to test complex distributed data processing systems

• Develop custom testing tools when commercial or open-source solutions are insufficient

• Stay current with advances in big data technologies and testing approaches

• Collaborate with data engineers, architects, and product teams to understand testing requirements

• Provide technical leadership for automation initiatives across data engineering projects

This is a hybrid position. Expectation of days in office will be confirmed by your Hiring Manager.

Qualifications

Basic Qualifications:
5+ years of relevant work experience with a Bachelor’s Degree or at least 2 years of work experience with an Advanced degree (e.g. Masters, MBA, JD, MD) or 0 years of work experience with a PhD, OR 8+ years of relevant work experience.

Preferred Qualifications:
5+ years of relevant work experience with a Bachelor’s Degree or at least 2 years of work experience with an Advanced degree (e.g. Masters, MBA, JD, MD) or 0 years of work experience with a PhD, OR 8+ years of relevant work experience.
• 7+ years of experience in data engineering and test automation
• Expert-level knowledge of Apache Spark and Hadoop ecosystems
• Advanced proficiency in Python, Scala, or Java for developing test automation frameworks
• Extensive experience with REST API testing using tools like Postman, RestAssured, or similar frameworks
• Strong understanding of ETL processes and data validation techniques
• Experience with CI/CD pipelines (Jenkins, GitLab CI, GitHub Actions)
• Advanced proficiency in SQL and experience with various database technologies
• Proven experience automating tests for batch processing applications in distributed environments
• Experience with performance testing of big data applications
• Ability to design and implement data quality validation frameworks
• Experience with mocking services and test data generation for complex data pipelines
• Knowledge of test metrics collection and reporting
• Strong analytical and problem-solving abilities
• Experience mentoring junior engineers and leading technical initiatives
• Excellent communication skills to collaborate with cross-functional teams
• Ability to translate complex technical concepts to non-technical stakeholders
• Self-motivated with a focus on continuous improvement

Additional Information

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.