Skip to content

Slide 06 — Expand Cached Token explanation (what it is, how it forms) #3

Description

@KardelRuveyda

Feedback

Slide 06 mentions Cached Token but doesn't really explain 'what it is, how it gets created, why it costs less'.

Problem: If the listener is hearing 'cached token' for the first time, the rest of the formula stays in the air.

Information to add

  • Definition: Instead of re-sending the same prompt context to the model, the previously processed part is reused from cache.
  • How it forms: Prevents re-generating tokens — especially useful for long, repeating system / context prompts.
  • Why it's cheaper: Lower inference cost on the provider side → reflected back to us as a lower credit rate.
  • Practical effect: Credit usage drops noticeably on follow-up questions in the same file.

Suggestion

  • Add a small diagram (Full token vs Cached token).
  • One-line formula: Cached token = no regeneration = lower credit rate.

File

  • src/components/sections/AiCreditsSection.tsx (taxi-meter / formula section).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions