AIML - Senior Software Engineer, ML Product Evaluation Engineer
Yokohama, Kanagawa-ken, Japan
Apple
We’re a diverse collective of thinkers and doers, continually reimagining what’s possible to help us all do what we love in new ways.Summary
Posted: Oct 22, 2024Weekly Hours: 37.5
Role Number:200574361
Do you want to play a part in building groundbreaking technology for Apple Intelligence, working on large-scale systems, natural language processing, big data, and artificial intelligence? Join our team and help improve the user experience for Siri and other AI/ML-powered products, impacting millions of people across the globe! As a Senior Software Engineer, you will contribute to high-quality release of innovative features across multiple platforms, by leading evaluation efforts through end-to-end functional test automation and ML models evaluation. You will play a critical role in shaping the future of Siri and Apple Intelligence, by defining test&eval strategies for LLM-powered products. Join our team to push the boundaries of AI technology and enhance how users interact with intelligent systems.
Description
As a Senior Software Engineer, you will lead the charge in developing an approach that seamlessly combines functional testing and machine learning evaluation to deliver an exceptional user experience. You will take ownership of ensuring the quality and reliability of features in large-scale distributed systems integrated with machine learning models. We will develop and execute test strategies that guarantee both functional stability and ML model performance in real-world applications. Collaborating with software developers, data scientists, and product managers, you will play a key role in delivering high-quality solutions used by millions worldwide. If you're passionate about functional testing, ML model evaluation, and seamless integration across complex platforms, this role is for you! In this role, you will: * Lead the design and execution of test plans for features across various platforms. * Automate functional testing using tools like Swift, XCTest, and ensure cross-platform integration. * Test and evaluate on-device ML models and Large Language Models (LLMs) for accuracy, performance, and stability. * Create datasets and conduct model performance evaluations to ensure ML models meet required standards. * Configure and manage CI/CD pipelines using tools like GitHub, TeamCity, or Jenkins. * Debug complex issues by analyzing logs and collaborating with developers to resolve root causes efficiently. * Provide detailed test reports, highlight risks, and ensure issues are addressed before product release. * Collaborate with data scientists and ML engineers to validate model deployment and performance in production environments.
Minimum Qualifications
- 7+ years of experience in functional testing and test automation with tools like Swift, XCTest, or equivalent.
- 7+ years of experience in testing distributed systems across various platforms, ensuring reliable performance and integration.
- Extensive experience with on-device ML model testing and evaluating LLMs in production environments.
- Strong experience in creating datasets for model evaluation and conducting performance benchmarks.
- Proficiency in configuring and maintaining CI/CD pipelines using tools such as GitHub, TeamCity, Jenkins, or similar platforms.
- Advanced debugging skills, including log stream analysis and issue reproduction.
- Familiarity with ML model testing tools like TensorFlow, PyTorch, or MLflow.
- Proactive and creative mindset with a can-do attitude and strong focus on delivering high-quality results.
Preferred Qualifications
- Experience in testing large-scale distributed systems with machine learning components.
- Experience validating the performance and scalability of machine learning models in a production setting.
- Ability to lead and influence testing initiatives in fast-paced, dynamic environments.
- Bachelor’s degree in Computer Science, Engineering, or a related field. A Master’s degree is a plus.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Job stats:
1
0
0
Tags: Big Data CI/CD Computer Science Distributed Systems Engineering GitHub Jenkins LLMs Machine Learning MLFlow ML models Model deployment NLP Pipelines PyTorch Swift TensorFlow Testing
Region:
Asia/Pacific
Country:
Japan
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.
Business Intelligence Developer jobsData Engineer II jobsPrincipal Data Scientist jobsBI Developer jobsPrincipal Data Engineer jobsStaff Data Scientist jobsCopywriter - Freelance AI Tutor jobsData Manager jobsData Scientist II jobsJunior Data Analyst jobsData Science Manager jobsResearch Scientist jobsBusiness Data Analyst jobsLead Data Analyst jobsSr. Data Scientist jobsData Science Intern jobsBI Analyst jobsSr Data Engineer jobsSenior Artificial Intelligence/Machine Learning Engineer - Remote, Latin America jobsJunior Data Scientist jobsJunior Data Engineer jobsSenior AI Engineer jobsData Engineer III jobsSoftware Engineer, Machine Learning jobsData Analyst Intern jobs
Snowflake jobsLinux jobsEconomics jobsHadoop jobsPhysics jobsOpen Source jobsRDBMS jobsJavaScript jobsComputer Vision jobsKafka jobsAirflow jobsScala jobsMLOps jobsBanking jobsNoSQL jobsData Warehousing jobsData warehouse jobsKPIs jobsSAS jobsGoogle Cloud jobsPostgreSQL jobsOracle jobsClassification jobsGitHub jobsScrum jobs
Scikit-learn jobsR&D jobsCX jobsStreaming jobsTerraform jobsData Mining jobsPandas jobsLooker jobsDistributed Systems jobsIndustrial jobsJira jobsPySpark jobsBigQuery jobsRobotics jobsJenkins jobsRedshift jobsReact jobsMySQL jobsMicroservices jobsMatlab jobsdbt jobsUnstructured data jobsData strategy jobsE-commerce jobsNumPy jobs