Legal AI Prompter / Evaluator

Legal AI Prompter / Evaluator

Role Summary
We are seeking a dynamic SME Prompter / Evaluator with a Juris Doctor (JD) degree and a strong interest in emerging technologies. This role is crucial in our mission to pioneer innovative solutions within the legal industry, specifically leveraging Large Language Models (LLMs) and other advanced technologies to meet the needs of legal professionals. 

In this role, you will be responsible for both the assessment of new product opportunities and the hands-on development of LLM-driven solutions. You will play a key role in crafting and refining prompts, developing methodologies for evaluating and grading LLM outputs, and integrating these technologies into our suite of legal products. The ideal candidate is a legal professional with a passion for technology, who can creatively and effectively translate legal expertise into innovative solutions. 

About the Role

Responsibilities in this role as Legal AI Prompter / Evaluator will include: 

Product Opportunity Assessment: 

• Collaborate with cross-functional teams to evaluate the feasibility and potential impact of new product ideas and enhancements. 

• Stay informed about the latest developments in legal technology, particularly in the area of Large Language Models (LLMs), and apply this knowledge to guide product strategy. 

LLM Prompt Development: 

• Create, test, and refine prompts for LLMs, ensuring they deliver accurate, legally sound, and contextually appropriate responses. 

• Develop and iterate on LLM-driven experiences that enhance productivity and delight our customers. 

• Collaborate with engineers and data scientists to integrate LLMs into multi-step systems, assessing their performance and making necessary modifications. 

Evaluation: 

• Develop and implement evaluation schemes to test model outputs. 

• Create and implement automated testing methods to ensure quality and consistency over large datasets and time periods. 

Stakeholder Collaboration: 

• Engage with legal professionals, industry leaders, and internal stakeholders to gather insights and validate product concepts. 

• Liaise with various teams to understand requirements and ensure that products meet the needs of legal professionals across different sectors. 

• Represent the company at industry conferences, webinars, and other events to promote our technology solutions and gather feedback. 

About You

You're a fit for the role of Legal AI Prompter / Evaluator if you have:

• Juris Doctor (JD) and 2+ years of practice experience 

• Strong interest in and understanding of emerging technologies, particularly in the context of the legal profession. 

• Experience with Large Language Models (LLMs) or other AI technologies is highly desirable. 

• Excellent communication and collaboration skills, with the ability to work effectively across departments. 

• Ability to think creatively and strategically about the future of legal technology. 

• Experience using large language models through code (e.g., OpenAI, Anthropic, Cohere, Llama, Mistral). 

#LI-DS4

What's in it For You?

You will join our inclusive culture of world-class talent, where we are committed to your personal and professional growth through:

• Wellbeing: Comprehensive benefit plans; flexible and supportive benefits for work-life balance: flexible vacation, two company-wide Mental Health Days Off; work from another location for up to a total of 8 weeks in a year, 4 of those weeks can be out of the country and the remaining in the country, Headspace app subscription; retirement, savings, tuition reimbursement, and employee incentive programs; resources for mental, physical, and financial wellbeing.

• Culture: Globally recognized and award-winning reputation for equality, diversity and inclusion, flexibility, work-life balance, and more.

• Learning & Development: LinkedIn Learning access; internal Talent Marketplace with opportunities to work on projects cross-company; Ten Thousand Coffees Thomson Reuters café networking.

• Social Impact: Ten employee-driven Business Resource Groups; two paid volunteer days annually; Environmental, Social and Governance (ESG) initiatives for local and global impact.

• Purpose Driven Work: We have a superpower that we’ve never talked about with as much pride as we should – we are one of the only companies on the planet that helps its customers pursue justice, truth and transparency. Together, with the professionals and institutions we serve, we help uphold the rule of law, turn the wheels of commerce, catch bad actors, report the facts, and provide trusted, unbiased information to people all over the world.

In the United States, Thomson Reuters offers a comprehensive benefits package to our employees. Our benefit package includes market competitive health, dental, vision, disability, and life insurance programs, as well as a competitive 401k plan with company match. In addition, Thomson Reuters offers market leading work life benefits with competitive vacation, sick and safe paid time off, paid holidays (including two company mental health days off), parental leave, sabbatical leave. These benefits meet or exceeds the requirements of paid time off in accordance with any applicable state or municipal laws. Finally, Thomson Reuters offers the following additional benefits: optional hospital, accident and sickness insurance paid 100% by the employee; optional life and AD&D insurance paid 100% by the employee; Flexible Spending and Health Savings Accounts; fitness reimbursement; access to Employee Assistance Program; Group Legal Identity Theft Protection benefit paid 100% by employee; access to 529 Plan; commuter benefits; Adoption & Surrogacy Assistance; Tuition Reimbursement; and access to Employee Stock Purchase Plan.

Thomson Reuters complies with local laws that require upfront disclosure of the expected pay range for a position. The location(s) for this role include one or more of the following states: CA, CO, CT, IL, NV, TX, WA and/or remote. The base compensation range for the role in any of those locations is $91,700 - $170,300. The base compensation range in other locations may vary. This role may also be eligible for an Annual Bonus based on a combination of enterprise and individual performance. Base pay is positioned within the range based on several factors including an individual’s knowledge, skills and experience with consideration given to internal equity. Base pay is one part of a comprehensive Total Reward program which also includes flexible and supportive benefits and other wellbeing programs.

This job posting will close 11/08/2024.

Do you want to be part of a team helping re-invent the way knowledge professionals work? How about a team that works every day to create a more transparent, just and inclusive future? At Thomson Reuters, we’ve been doing just that for almost 160 years. Our industry-leading products and services include highly specialized information-enabled software and tools for legal, tax, accounting and compliance professionals combined with the world’s most global news services – Reuters. We help these professionals do their jobs better, creating more time for them to focus on the things that matter most: advising, advocating, negotiating, governing and informing.

We are powered by the talents of 26,000 employees across more than 70 countries, where everyone has a chance to contribute and grow professionally in flexible work environments that celebrate diversity and inclusion. At a time when objectivity, accuracy, fairness and transparency are under attack, we consider it our duty to pursue them. Sound exciting? Join us and help shape the industries that move society forward. 

Accessibility 

As a global business, we rely on diversity of culture and thought to deliver on our goals. To ensure we can do that, we seek talented, qualified employees in all our operations around the world regardless of race, color, sex/gender, including pregnancy, gender identity and expression, national origin, religion, sexual orientation, disability, age, marital status, citizen status, veteran status, or any other protected classification under applicable law. Thomson Reuters is proud to be an Equal Employment Opportunity/Affirmative Action Employer providing a drug-free workplace.

We also make reasonable accommodations for qualified individuals with disabilities and for sincerely held religious beliefs in accordance with applicable law.

Protect yourself from fraudulent job postings click here to know more.

More information about Thomson Reuters can be found on https://thomsonreuters.com.