New Paper Published in Nature by the Team

We are pleased to announce that our recent paper on foundation metrics for evaluating LLM-powered healthcare chatbots has been published in #Nature npj Digital Medicine.

How can trust be built between end-users and interactive healthcare conversational models?

In this paper, we present and discuss:

1) A detailed set of evaluation metrics specifically tailored for assessing healthcare chatbots from an end-user perspective.
2) An evaluation framework designed to facilitate the implementation of an end-to-end and standardized approach for the metrics’ evaluation.
3) The challenges associated with defining and implementing these evaluation metrics, highlighting factors such as the target audience, evaluation methods, and prompt techniques integral to this process.

Please find more details here: https://lnkd.in/gGQv9Zqe

This work was accomplished in collaboration with teams from UC Irvine, Stanford University, National Institute of Standards and Technology (NIST), UCI-Institute for Future Health, University of Pittsburgh, and University of Toronto.