Software Engineer II

The Bing Metrics Team has a unique opportunity to join Bing Search, a global search engine powering billions of searches daily, as a Software Engineer II.
 
The Bing Metrics team is looking for passionate full stack developers and data scientists to work on a new generation of metrics and quality control for the entire Bing search landscape. The team ensures that Bing shows high-quality, error-free, and authoritative results using a variety of different approaches. We routinely query petabytes of user activity data to uncover potential issues in user interactions with the search engine. We build complex pipelines including crowd judging and leverage the power of large language models (LLMs) to verify our suspicions. LLMs allow us to evaluate the quality of search results at multiple levels: query, answer, whole page and generate insights for the teams who are responsible for this experience.

As a part of an international and distributed team you will be responsible for identifying issues and implementing search quality metrics within Bing Search. The job provides you with the opportunity to work with multiple teams across the entire Bing organization (>80 different teams) and greatly influence search engine relevance and search result quality. We are an established core team in Bing with very high visibility and impact.

We are looking for a talented engineer who is detail oriented, with a passion to work with large scale computing, loves to design complex data pipelines built on top of LLM models, create new tools for running multi-step prompts to evaluate search engine quality and generate actionable insights for teams. If your blood boils when you see bad search results and you wish you could do something about them, this is the job for you!

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Responsibilities

• Build tools and pipelines with Bing Logs using Big Data platforms.
• Design and implement E2E pipelines (from data collection, evaluation and result display).
• Design and implement tools for LLM models, engineer prompts for textual and multi-model LLMs for data processing and insight generation.
• Design and implement creative visualization for your results.

Qualifications

Required Qualifications: 

• Bachelor's Degree in Computer Science or related technical field AND 2+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • OR equivalent experience.  
• 2+ years of experience in writing automation code with Chromium and Puppeteer.
• 2+ years of experience in developing solutions on Azure, utilizing Functions, WebJobs, Cloud Services, Azure Database, and Queues.

Preferred Qualifications:

• Bachelor's Degree in Computer Science or related technical field AND 5+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • OR equivalent experience.
• 3 years of experience with modern web development.
• 3 years of experience with SQL, T-SQL.
• 3 years of experience with HTML, JavaScript, jQuery.
• Experience in testing and relevance evaluation
• Experience or deep interest in Large-Language Models (ChatGPT)
• Experience in (big) data and data analytics.
• Interest in designing dashboards for data visualization in novel ways.  

Software Engineering IC3 - The typical base pay range for this role across the U.S. is USD $98,300 - $193,200 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $127,200 - $208,800 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay

Microsoft will accept applications for the role until November 25, 2024.  

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances.  We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form.

Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.

#Bing