AIML - Senior Software Engineer, ML Product Evaluation Engineer

Summary

Posted: Oct 22, 2024
Weekly Hours: 37.5
Role Number:200574361

Do you want to play a part in building groundbreaking technology for Apple Intelligence, working on large-scale systems, natural language processing, big data, and artificial intelligence? Join our team and help improve the user experience for Siri and other AI/ML-powered products, impacting millions of people across the globe! As a Senior Software Engineer, you will contribute to high-quality release of innovative features across multiple platforms, by leading evaluation efforts through end-to-end functional test automation and ML models evaluation. You will play a critical role in shaping the future of Siri and Apple Intelligence, by defining test&eval strategies for LLM-powered products. Join our team to push the boundaries of AI technology and enhance how users interact with intelligent systems.

Description

As a Senior Software Engineer, you will lead the charge in developing an approach that seamlessly combines functional testing and machine learning evaluation to deliver an exceptional user experience. You will take ownership of ensuring the quality and reliability of features in large-scale distributed systems integrated with machine learning models. We will develop and execute test strategies that guarantee both functional stability and ML model performance in real-world applications. Collaborating with software developers, data scientists, and product managers, you will play a key role in delivering high-quality solutions used by millions worldwide. If you're passionate about functional testing, ML model evaluation, and seamless integration across complex platforms, this role is for you! In this role, you will: * Lead the design and execution of test plans for features across various platforms. * Automate functional testing using tools like Swift, XCTest, and ensure cross-platform integration. * Test and evaluate on-device ML models and Large Language Models (LLMs) for accuracy, performance, and stability. * Create datasets and conduct model performance evaluations to ensure ML models meet required standards. * Configure and manage CI/CD pipelines using tools like GitHub, TeamCity, or Jenkins. * Debug complex issues by analyzing logs and collaborating with developers to resolve root causes efficiently. * Provide detailed test reports, highlight risks, and ensure issues are addressed before product release. * Collaborate with data scientists and ML engineers to validate model deployment and performance in production environments.

Minimum Qualifications

• 7+ years of experience in functional testing and test automation with tools like Swift, XCTest, or equivalent.
• 7+ years of experience in testing distributed systems across various platforms, ensuring reliable performance and integration.
• Extensive experience with on-device ML model testing and evaluating LLMs in production environments.
• Strong experience in creating datasets for model evaluation and conducting performance benchmarks.
• Proficiency in configuring and maintaining CI/CD pipelines using tools such as GitHub, TeamCity, Jenkins, or similar platforms.
• Advanced debugging skills, including log stream analysis and issue reproduction.
• Familiarity with ML model testing tools like TensorFlow, PyTorch, or MLflow.
• Proactive and creative mindset with a can-do attitude and strong focus on delivering high-quality results.

Preferred Qualifications

• Experience in testing large-scale distributed systems with machine learning components.
• Experience validating the performance and scalability of machine learning models in a production setting.
• Ability to lead and influence testing initiatives in fast-paced, dynamic environments.
• Bachelor’s degree in Computer Science, Engineering, or a related field. A Master’s degree is a plus.