The Senior Applied Scientist will play a key role in the development and evaluation of LLM and vLLM models, with a focus on creating innovative solutions to real-world problems. This role requires a deep understanding of machine learning techniques, including foundation model evaluation, traditional NLP/CV metrics, LLMaaJ techniques, human evaluation, confidence estimation, agentive application evaluation, and RAG application evaluation. The successful candidate will have experience in conducting in-depth research, producing production-ready code, and collaborating with cross-functional teams to integrate evaluation capabilities into various applications and products. A strong background in programming languages, such as Python, and experience with machine learning frameworks, such as TensorFlow or PyTorch, is also required.

The successful candidate will be responsible for conducting in-depth research on foundation model evaluation, producing production-ready code for handoff to engineering counterparts, and helping to build and mentor a high-performing team of scientists and engineers. They will work closely with cross-functional teams to integrate evaluation capabilities into various applications and products, identify new opportunities for evaluation, and explore emerging technologies. The candidate will also be responsible for staying up-to-date with industry trends and advancements in evaluation, and for applying this knowledge to drive innovation and improvement in the team's work. This will involve designing and executing experiments, researching new algorithms, and finding new ways of optimizing risk, profitability, and customer experience. A PhD in Computer Science, Mathematics, Statistics, Physics, Linguistics, or a related field, with a dissertation or thesis centered on machine learning techniques, is preferred, although a Master's or Bachelor's degree with relevant experience will also be considered.

Career Level - IC4