Large Language Model Evaluation Engineer
Singapore
PatSnap
Patsnap empowers IP and R&D teams with advanced AI to get better answers and make faster decisions. Increase IP productivity by 75% while reducing R&D wastage by 25%.
About PatSnapPatsnap empowers IP and R&D teams by providing better answers, so they can makefaster decisions with more confidence. Founded in 2007, Patsnap is the global leaderin AI-powered IP and R&D intelligence. Our domain-specific LLM, trained on ourextensive proprietary innovation data, coupled with Hiro, our AI assistant, deliversactionable insights that increase productivity for IP tasks by 75% and reduce R&Dwastage by 25%. IP and R&D teams collaborate better with a user-friendly platformacross the entire innovation lifecycle. Over 15,000 companies trust Patsnap toinnovate faster with AI, including NASA, Tesla, PayPal, Sanofi, Dow Chemical, andWilson Sonsini.
About the RoleWe are seeking a highly skilled and innovative Large Model Evaluation andDevelopment Engineer to join our cutting-edge AI team. In this role, you will beresponsible for designing and implementing evaluation frameworks andmethodologies for large-scale models, ensuring they meet the highest qualitystandards. You will collaborate closely with AI researchers, data scientists, andproduct managers to drive advancements in AI technologies. This position offers theopportunity to work on groundbreaking projects that push the boundaries of AI andmachine learning.
About the RoleWe are seeking a highly skilled and innovative Large Model Evaluation andDevelopment Engineer to join our cutting-edge AI team. In this role, you will beresponsible for designing and implementing evaluation frameworks andmethodologies for large-scale models, ensuring they meet the highest qualitystandards. You will collaborate closely with AI researchers, data scientists, andproduct managers to drive advancements in AI technologies. This position offers theopportunity to work on groundbreaking projects that push the boundaries of AI andmachine learning.
Key Responsibilities
- Design and implement comprehensive evaluation frameworks for large-scale AI models, including language models and vision models.
- Develop and maintain high-quality datasets for model evaluation and benchmarking.
- Create and apply metrics to assess model performance, robustness, and quality.
- Collaborate with cross-functional teams to integrate evaluation results into the development cycle, providing actionable insights for model improvement.
- Develop tools and platforms for automated model evaluation and reporting.
- Stay updated with the latest research and trends in AI model evaluation and incorporate best practices into our processes.
- Improve the efficiency of the evaluation process, including data collection and annotation strategies.
Desired Qualifications
- Bachelor’s or Master’s degree in Computer Science, Artificial Intelligence, or a related field.
- Proficiency in programming languages such as Python and database languages like SQL, familiarity with Linux operating systems, and strong programming skills.
- Experience with data manipulation and analysis using libraries such as NumPy and Pandas.
- In-depth understanding and application of common language or vision model evaluation metrics such as Cross-entropy, BPC/BPW, Perplexity, and ROUGE.
- Deep understanding of Bert and GPT-related principles, including key technologies like prompt, finetuning, and transformer.
- Familiarity with large model architectures and training processes.
- Strong insight into the development of large model technologies and the ability to learn and practice.
- Excellent problem-solving and communication skills, with the ability to work collaboratively in a team environment.
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Job stats:
2
1
0
Categories:
Engineering Jobs
NLP Jobs
Tags: Architecture BERT Computer Science GPT Linux LLMs NumPy Pandas Python R R&D Research SQL
Perks/benefits: Career development Team events
Region:
Asia/Pacific
Country:
Singapore
More jobs like this
Explore more career opportunities
Find even more open roles below ordered by popularity of job title or skills/products/technologies used.
Principal Data Scientist jobsBI Developer jobsData Scientist II jobsStaff Data Scientist jobsPrincipal Data Engineer jobsData Manager jobsJunior Data Analyst jobsData Science Manager jobsResearch Scientist jobsBusiness Data Analyst jobsLead Data Analyst jobsSenior AI Engineer jobsData Engineer III jobsSr. Data Scientist jobsData Science Intern jobsData Specialist jobsJunior Data Engineer jobsSenior Data Scientist, Performance Marketing jobsSoftware Engineer, Machine Learning jobsData Analyst Intern jobsSr Data Engineer jobsBI Analyst jobsSoftware Engineer II jobsData Analyst II jobsData Engineering Manager jobs
Snowflake jobsLinux jobsEconomics jobsHadoop jobsJavaScript jobsOpen Source jobsPhysics jobsComputer Vision jobsMLOps jobsAirflow jobsKafka jobsRDBMS jobsBanking jobsNoSQL jobsGoogle Cloud jobsData Warehousing jobsScala jobsR&D jobsKPIs jobsData warehouse jobsGitHub jobsScikit-learn jobsOracle jobsPostgreSQL jobsCX jobs
Classification jobsStreaming jobsSAS jobsTerraform jobsLooker jobsScrum jobsDistributed Systems jobsPandas jobsData Mining jobsPySpark jobsBigQuery jobsRobotics jobsJenkins jobsJira jobsIndustrial jobsRedshift jobsReact jobsdbt jobsUnstructured data jobsMicroservices jobsData strategy jobsE-commerce jobsMySQL jobsMatlab jobsNumPy jobs