Actionable user feedbacks for LLM applications

LLM-based use cases are getting introduced and implemented within digital products every day. Yet, revolutionary as these new tools might be, product teams must stay grounded to actual user requirements and measure their effectiveness within the boundaries of product management best practices. This means observing user behaviour, A/B testing to validate product assumptions, monitoring and assessing user experience.

But how can traditional product analytics work with the non deterministic nature of LLM outputs?

In this talk we’ll explore how open-source project Langfuse can help correlating user actions or explicit feedbacks to LLM generations, so that the team can quickly A/B test different versions of prompting and measure what works best for the final users.

Speaker Bio:

In love with technology and innovation since university, he has transitioned from software engineering to product management a few years ago. Co-founded xtream, a boutique of digital products and AI, where he helped scale-ups and corporates build better solutions for their businesses.