Cost Tracking for Input cache writes

[Agilan3996]

Does Langfuse detects the cost for input cache write as well?

I couldn't find relevant documentation on that.

I can see that it tracks separate cost for input cache reads, but not sure whether it track cost for input cache writes.

Also, I can see that it detects number of input cache write tokens and read tokens.

Can someone clarify on this?

[dosubot[bot]]

Hello @Agilan3996! I'm Dosu, here to help you with any questions or issues you might have. While you're waiting for a human maintainer, I'll do my best to assist you!

Yes, Langfuse does support cost tracking for input cache writes. According to the changelog from December 20, 2024, Langfuse tracks costs for all usage types, including cached tokens ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[Agilan3996]

@dosu I could see cache_read is mentioned, but not cache_write in the documentation or changelog. Does cache_write also included? And cache writes tend to be costlier than normal input tokens, Is the cost calculation being done accordingly?

[dosubot[bot]]

Yes, Langfuse does include cost tracking for input cache writes. The cost calculation for input cache writes is different from normal input tokens. The formula used is:

[ \text{cost} = ((\text{input} - \text{input_cache_read}) \times \text{input_token_cost}) + (\text{input_cache_read} \times \text{cached_input_token_cost}) + (\text{output} \times \text{output_token_cost}) ]

This means that cached input tokens are subtracted from the total input tokens, and the remaining tokens are calculated at the regular input token cost, while cached input tokens are calculated at a reduced cached input token cost ^[1].

Additionally, the cost for input cache writes is indeed higher than the cost for cache reads. For example, for the model "claude-3.5-sonnet-latest," the cost for input cache creation is specified as (3.75 \times 10^{-6}), whereas the cost for reading from the input cache is (0.3 \times 10^{-6}) ^[2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[Agilan3996]

@dosu Where can I find this details in documentation or is there any way I can confirm this calculation in self hosted langfuse?

[dosubot[bot]]

The details regarding the cost calculation for input cache writes are not explicitly documented in the Langfuse documentation. However, you can confirm the calculation in a self-hosted Langfuse setup by examining the codebase. Specifically, the calculateUsageCosts method in the IngestionService class handles the cost calculation for input cache writes. This method uses the input price specified in the database to calculate costs ^[1].

Additionally, the default-model-prices.json file in the codebase specifies the costs for input cache creation, which you can review to understand the pricing for different models ^[2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[Haknt]

I don’t see anything related to cache_write_input_tokens (or input cache write costs) being included explicitly.
• How are input cache writes modeled in this formula?
• Is there a separate term or field for cache_write_input_tokens in the cost calculation or pricing model?
• If so, what would the full cost formula look like when we also include cache writes?

I just want to make sure I’m accounting correctly for both cache reads and cache writes in our own cost calculations.