Member of Technical Staff, Evaluation

Boson AI is an early-stage startup building large language tools for everyone to use. Our founders (Alex Smola,Mu Li), and a team of Deep Learning, Optimization, NLP, AutoML and Statistics scientists and engineers are working on high quality generative AI models for language and beyond.
We are seeking research scientists and engineers to join our team full-time in our Santa Clara office. As part of your role, you will work on implementing and training deep neural networks, understanding and interpreting model behavior and aligning models to human values. The ideal candidate will possess a strong background in machine learning, and have motivations for developing state-of-the-art models towards AGI. 
We encourage you to apply even if you do not believe you meet every single qualification. As long as you are motivated to learn and join the development of foundation models, we’d love to chat.

Responsibilities:

• Design and run evaluations to measure model’s capabilities.
• Write efficient and clean code to build evaluation pipeline.
• Share your findings to help model development and data annotation guidelines.

You may be a good fit if you have:

• Experience in prompt engineering or other ways to interact with large language models.
• Experience in data analysis, familiar with data processing and visualization tools.

Strong candidates may also have:

• Proficiency in at least one deep learning framework, such as PyTorch.
• Think out of box, can find solutions to ambiguously scoped problems.
• Ability to summarize results, clearly communicate the observations in your work.
• Participated in research projects on model evaluation or related topics.
• Experience in training/finetuning large language or multimodal models.