Senior Service Development Manager - Evaluation

Senior Service Development Manager - Evaluation 

About Our Team 

As part of our team, you will manage the operational process for your own portfolio of academic journals while collaborating with a large team of Journal Managers. You will be a member of a collaborative and inclusive team that encourages autonomy and personal development, while working together to achieve our department's objectives and key results. You will contribute to a supportive team culture by sharing knowledge, ideas, and best practices. 

We operate a hybrid working policy, with office attendance requested in our centrally located Exeter office one day per week. The office is open and available most days for staff who prefer to come in. 

About the Role 

The Service Development, Evaluation Manager (AI/LLM Focus) is responsible for optimizing and evaluating service processes, with a specialized focus on services leveraging Generative AI (GenAI), Large Language Models (LLMs), and associated AI tools. This role requires expertise in service operations as well as an understanding of AI technologies to ensure high-quality AI-driven service delivery. The manager will collaborate with AI development teams, operations, and customer-facing teams to enhance service efficiency and drive innovation using AI. 

Responsibilities 

AI/LLM-Based Service Evaluation and Improvement: 

• Evaluate and assess AI-driven service processes, focusing on the use of Generative AI and LLMs for customer support, content generation, and other service functions. 

• Analyze the performance of AI systems and suggest enhancements to improve accuracy, efficiency, and user satisfaction. 

• Lead continuous improvement initiatives to optimize service quality, customer satisfaction, and operational efficiency. 

• Collaborate with AI development teams to improve model performance, including tuning, prompt engineering, and data refinement for better outputs. 

Subject Matter Expertise in Generative AI and LLMs: 

• Serve as the in-house expert for technical or operational service-related queries on the Generative AI and Large Language Models in service delivery processes. 

• Conduct knowledge-sharing sessions and training workshops to enhance service capabilities. 

• Stay up-to-date with advancements in AI technologies, including new LLMs, evaluation methods, and AI governance practices. 

• Act as a point of contact for escalated service issues, ensuring timely resolution of complex problems. 

AI Tool Evaluation and Selection: 

• Assess various AI evaluation tools and platforms, such as model monitoring systems, accuracy measurement tools, and error detection frameworks. 

• Define criteria for evaluating AI model performance, including metrics for language model accuracy, generative quality, and user satisfaction. 

• Collaborate with the AI engineering team to integrate evaluation tools that can continuously monitor model output quality and provide insights for further improvement. 

• Recommend and implement best practices for using AI safely and ethically, ensuring compliance with industry standards and company policies. 

Collaboration with AI and Operations Teams: 

• Work closely with AI engineers, data scientists, and service operations teams to align service strategies with AI capabilities. 

• Provide feedback to teams on refining LLM outputs based on customer needs and service requirements. 

• Facilitate cross-functional communication, ensuring that AI-driven service goals are clearly understood and properly implemented. 

• Act as the escalation point for AI-related service issues, collaborating with AI teams to resolve complex issues quickly. 

Documentation and Reporting of AI-Driven Processes: 

• Create and maintain documentation for AI/LLM service processes, outlining best practices for using AI tools, handling escalations, and improving outputs. 

• Track service performance through key metrics such as response accuracy, speed, user engagement, and overall model effectiveness. 

• Generate reports for stakeholders on the performance of AI systems, including analysis of failures, successes, and recommendations for future improvements. 

• Maintain a knowledge base on AI/LLM usage within the service department, ensuring up-to-date resources for team members. 

Requirements 

• Bachelor’s degree in a relevant field (e.g., AI, Computer Science, Data Science, or Business with a focus on AI) or equivalent work experience. 

• Proven experience as a subject matter expert in service delivery with a focus on AI and LLM-driven services. 

• Strong project management skills. 

• Knowledge of Generative AI technologies, including hands-on experience with popular Large Language Models (e.g., GPT, LLaMA, PaLM). 

• Familiarity with AI evaluation tools and techniques for assessing model accuracy and quality. 

• Strong analytical, problem-solving, and communication skills, with the ability to work cross-functionally. 

Preferred Skills: 

• Certifications in AI/ML technologies. 

• Experience with prompt engineering. 

• Familiarity with data analysis tools such as Python, R, or LangChain for LLM applications. 

Key Competencies: 

• AI Expertise: Deep understanding of AI and LLM technologies, with the ability to translate this knowledge into practical service applications. 

• Innovation: Ability to identify new ways AI can improve service processes and customer satisfaction. 

• Collaboration: Strong cross-functional collaboration skills, particularly with AI, data science, and service operations teams. 

• Ethics and Governance: Awareness of AI ethics, data privacy, and the responsible use of AI in service environments. 

• Problem Solving: Ability to address complex AI-related service issues quickly and efficiently, ensuring continuous service improvement. 

Work in a way that works for you 

We promote a healthy work/life balance across the organization. We offer an appealing working prospect for our people. With numerous wellbeing initiatives, shared parental leave, study assistance, and sabbaticals, we will help you meet your immediate responsibilities and your long-term goals. 

Working for You 

We understand that your well-being and happiness are essential to a successful career. Here are some benefits we offer: 

• Comprehensive Health Insurance covering you and your immediate family. 

• Enhanced Health Insurance Options with competitive rates. 

• Group Life Insurance ensuring financial security for your loved ones. 

• Group Accident Insurance offering extra protection. 

• Flexible Working Arrangements to achieve a balanced work-life schedule. 

• Employee Assistance Program providing support for personal and work-related challenges. 

• Medical Screening prioritizing your well-being. 

• Modern Family Benefits including maternity, paternity, and adoption support. 

• Long-Service Awards recognizing dedication and commitment. 

• Celebratory New Baby Gift. 

• Various Paid Time Off options including Casual Leave, Sick Leave, Privilege Leave, Compassionate Leave, Special Sick Leave, and Gazetted Public Holidays. 

About the Business 

A global leader in information and analytics, we help researchers and healthcare professionals advance science and improve health outcomes for the benefit of society. Building on our publishing heritage, we combine quality information and vast data sets with analytics to support visionary science and research, health education, and interactive learning, as well as exceptional healthcare and clinical practice. At Elsevier, your work contributes to the world's grand challenges and a more sustainable future. We harness innovative technologies to support science and healthcare to partner for a better world. 

-----------------------------------------------------------------------

Elsevier is an equal opportunity employer: qualified applicants are considered for and treated during employment without regard to race, color, creed, religion, sex, national origin, citizenship status, disability status, protected veteran status, age, marital status, sexual orientation, gender identity, genetic information, or any other characteristic protected by law. We are committed to providing a fair and accessible hiring process. If you have a disability or other need that requires accommodation or adjustment, please let us know by completing our Applicant Request Support Form: https://forms.office.com/r/eVgFxjLmAK , or please contact 1-855-833-5120.

Please read our Candidate Privacy Policy.