OpenWebUI and Langfuse integration for feedback and comment collection

[HEI204]

I am using OpenWebUI and Langfuse for the model monitoring. May I ask about is it support the feedback mechasim in OpenWebUI?

Seems the feedback raised in the chat inside the OpenWebUI did not reflect in Langfuse trace

[dosubot[bot]]

OpenWebUI does not natively send feedback or comments to Langfuse—only trace and usage data are integrated by default. That's why feedback you submit in OpenWebUI chats doesn't show up in Langfuse traces.

Langfuse does support collecting user feedback and comments via its public API endpoints and SDKs. To make feedback from OpenWebUI appear in Langfuse, you would need to extend OpenWebUI to call Langfuse's feedback APIs when a user submits feedback. For example, you can use the LangfuseWeb SDK in your UI to send feedback like this:

import { LangfuseWeb } from "langfuse";

export function UserFeedbackComponent({ traceId }) {
  const langfuseWeb = new LangfuseWeb({ publicKey: "YOUR_LANGFUSE_PUBLIC_KEY" });

  const handleUserFeedback = async (value) =>
    await langfuseWeb.score({
      traceId,
      name: "user_feedback",
      value, // e.g. 1 for thumbs up, 0 for thumbs down
    });

  return (
    <div>
      <button onClick={() => handleUserFeedback(1)}>👍</button>
      <button onClick={() => handleUserFeedback(0)}>👎</button>
    </div>
  );
}

You can also submit feedback or comments directly via the Langfuse API (POST /api/public/scores for feedback, POST /api/public/comments for comments) by sending the relevant traceId and feedback value.

If you want to enable this, you'll need to add a hook or middleware in OpenWebUI that triggers these API calls when feedback is submitted. There are no official or community-documented examples of this integration yet, so it would require some custom development. Let me know if you want more detailed implementation guidance or have questions about the API or SDK usage!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[jannikmaierhoefer]

Hi @HEI204, dosubot is correct - this is not natively supported with the Langfuse pipeline, but it can be custom-built.

[errd]

@dosu I'm working on implementing this, could you give me more details on a hook implementation?

[dosubot[bot]]

To implement a feedback hook in OpenWebUI that sends user feedback to Langfuse, you’ll want to trigger a call to the Langfuse feedback API (or use the SDK) whenever a user submits feedback (like a thumbs up/down or comment). The key is to ensure you have the correct traceId for the conversation or message you want to associate the feedback with.

Here’s a typical approach:

1. When you create or receive a trace in OpenWebUI (e.g., when a chat session starts or a message is sent), store the Langfuse traceId alongside your conversation data.
2. In your feedback UI component or handler (wherever the user submits feedback), call the Langfuse API or SDK with the traceId, feedback value, and any optional comment.

Example using the Langfuse JS SDK in a React component:

import { LangfuseWeb } from "langfuse";

const langfuse = new LangfuseWeb({ publicKey: "YOUR_LANGFUSE_PUBLIC_KEY" });

function FeedbackButtons({ traceId }) {
  const handleFeedback = async (value) => {
    await langfuse.score({
      traceId,
      name: "user_feedback",
      value, // 1 for thumbs up, 0 for thumbs down
    });
  };

  return (
    <>
      <button onClick={() => handleFeedback(1)}>👍</button>
      <button onClick={() => handleFeedback(0)}>👎</button>
    </>
  );
}

If you want to send comments, use the POST /api/public/comments endpoint with the traceId and comment content.

Make sure your hook or handler has access to the traceId (from your conversation/session state) and that your Langfuse API key or public key is configured securely. The Langfuse backend will handle authentication and associate the feedback with the correct trace as long as the traceId is valid and belongs to your project. For more on the API and payloads, see the Langfuse feedback docs.

If you need to use direct API calls instead of the SDK, POST to /api/public/scores with a JSON body like:

{
  "traceId": "your-trace-id",
  "name": "user_feedback",
  "value": 1,
  "comment": "Great answer!"
}

and include your project API key in the Authorization header.

Let me know if you want more details on a specific part of the integration!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}