Tracking Indexing and Querying Costs in GraphRAG

[silhouettehustler]

Is there a way to keep track of the costs associated with indexing and querying. Is there a tool or method that can be integrated with GraphRAG to monitor and manage these costs effectively?

[paulsp94]

Hey @silhouettehustler I normally use Helicone for it and started a QA thread if there is any way to simply integrate it.
#655

[silhouettehustler]

I've not used Helicone, but it doesn't seem much different to what's already there in the OpenAI Console?

[colegottdank]

Hi, looking into this.

[DanielSchuhmacher]

related to #440

[sebastianschramm]

@silhouettehustler I have implemented a simple openai request tracker: https://github.com/sebastianschramm/openai-cost-tracker

Update: just added a CLI wrapper, so now you can install my openai-cost-tracker and then just run:

track-costs graphrag.index --root foo (for indexing)

or

track-costs graphrag.query --root foo --method local "My query" (for querying)

Just call cost_tracker.init_tracker() in front of the index script (take a look at the readme for how to do that for the indexing phase: https://github.com/sebastianschramm/openai-cost-tracker/blob/main/README.md#in-code-usage). Once enabled, it will log all openai requests to file. And the repo offers a "display-costs" command to retrieve the costs per model from all requests recorded in one log file.