Senior/Lead Quality Engineer - AI Platform

To get the best candidate experience, please consider applying for a maximum of 3 roles within 12 months to ensure you are not duplicating efforts.

Job Category

Software Engineering

Job Details

About Salesforce

We’re Salesforce, the Customer Company, inspiring the future of business with AI+ Data +CRM. Leading with our core values, we help companies across every industry blaze new trails and connect with customers in a whole new way. And, we empower you to be a Trailblazer, too — driving your performance and career growth, charting new paths, and improving the state of the world. If you believe in business as the greatest platform for change and in companies doing well and doing good – you’ve come to the right place.

About the organization:

Einstein products & platform democratize AI and transform the way our Salesforce Ohana builds trusted machine learning and AI products - in days instead of months. It augments the Salesforce Platform with the ability to easily create, deploy, and manage Generative AI and Predictive AI applications across all clouds. We achieve this vision by providing unified, configuration-driven, and fully orchestrated machine learning APIs, customer-facing declarative interfaces and various microservices for the entire machine learning lifecycle including Data, Training, Predictions/scoring, Orchestration, Model Management, Model Storage, Experimentation etc.

We are already producing over a billion predictions per day, Training 1000s of models per day along with 10s of different Large Language models, serving thousands of customers. We are enabling customers' usage of leading large language models (LLMs), both internally and externally developed, so they can leverage it in their Salesforce use cases. Along with the power of Data Cloud, this platform provides customers an unparalleled advantage for quickly integrating AI in their applications and processes.
 

About the team:

Join the AI Platform Quality Engineering team, and become a specialist on Salesforce's AI Platform! You'll get to work with latest technology in the AI space (including generative AI), and collaborate with the team and cloud to identify and run quality initiatives to support massive scale planned for this year. We are a small, friendly team that has been working together for 3+ years - outside of quality, we focus on volunteering and shared interests such as gardening!
 

Job description summary:

As a Quality Engineer for the Salesforce AI platform, you will be responsible for designing and driving the implementation of comprehensive test plans/quality strategies. You will collaborate closely with engineering teams to ensure the successful execution of tests and maintain high FIT test pass rates in various environments. You’ll learn about the E2E workings of the platform and write integration tests to cover critical areas for traditional and generative AI flows. You’ll have the opportunity to enhance our test frameworks, work on test environment stability, and develop automation strategies.

Skills needed for role:

• A related 4-year technical degree required

• Test Gap Analysis & Coverage: Experience analyzing coverage gaps and writing E2E and integration tests, partnering with teams to drive service level test coverage

• Technical Expertise: Technical excellence and leadership required to learn a new platform and drive end to end test strategy for it, partnering with Quality Engineering team, Hawking scrum teams and customer cloud Q3s to implement.

• Cross Team Collaboration: Ability to work with stakeholders (internal customers) from multiple organizations that leverage our central platform for custom applications. Collaborate with them to determine key shared usage patterns to prioritize test coverage of, as well as provide guidance to these customers on how to proactively test their application’s integration with the platform.

• Experience with Java/Python

• Experience with test runners and configuration

• Extensive experience debugging systems

• 6+ years of QE experience and 2+ years of experience leading projects and partnering/leading jr members of the team.

Nice to have skillsets (or skillsets expect to develop in this role):

• Understanding of LLM Model hosting with Sagemaker

• Understanding LLM Finetuning and Inferencing

• Testing integrations with Cohere, Anthropic, Dolly, Google, etc. as well as internal models

• Sagemaker GroundTruth integration for data labeling, Bedrock for model serving

• Experience with Azure Open AI

What the team does:

• Daily management of FIT tests pass rates in Test, App-Dev, Stage & Prod environments - bugs created/assigned if failures not being addressed, building test result dashboards and simplifying feedback/debugging process for engineers

• Integration testing for hawking, generative AI flows

• Building/enhancing test frameworks for Hawking platform

• CUJs (Critical User Journey test) for cross-cloud integration and apps

• Configure tooling/runners for FIT tests, used by scrum teams

• Configure various test environments such as setting up core test env ↔ hawking test env connectivity

• Manage test coverage (code coverage, endpoint coverage, service coverage, etc.) goals are met for the ML foundations

• Automation strategies and framework for certain end-to-end scenarios

• Identify risks with releases, deployments, etc.

#LI-Y

Accommodations

If you require assistance due to a disability applying for open positions please submit a request via this Accommodations Request Form.

Posting Statement

At Salesforce we believe that the business of business is to improve the state of our world. Each of us has a responsibility to drive Equality in our communities and workplaces. We are committed to creating a workforce that reflects society through inclusive programs and initiatives such as equal pay, employee resource groups, inclusive benefits, and more. Learn more about Equality at www.equality.com and explore our company benefits at www.salesforcebenefits.com.

Salesforce is an Equal Employment Opportunity and Affirmative Action Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender perception or identity, national origin, age, marital status, protected veteran status, or disability status. Salesforce does not accept unsolicited headhunter and agency resumes. Salesforce will not pay any third-party agency or company that does not have a signed agreement with Salesforce.

Salesforce welcomes all.

Pursuant to the San Francisco Fair Chance Ordinance and the Los Angeles Fair Chance Initiative for Hiring, Salesforce will consider for employment qualified applicants with arrest and conviction records.

For California-based roles, the base salary hiring range for this position is $165,600 to $265,200.

Compensation offered will be determined by factors such as location, level, job-related knowledge, skills, and experience. Certain roles may be eligible for incentive compensation, equity, benefits. More details about our company benefits can be found at the following link: https://www.salesforcebenefits.com.