AI Dictionary › AI Fundamentals
Caching is the technique of keeping a copy of data or computed results in fast storage so they can be reused instead of being recomputed or refetched. It is one of the most universal ideas in computing: nearly every system, from CPUs to browsers, contains at least one cache. The underlying bet is simple: whatever was requested once is likely to be requested again.
When a request arrives, the system checks the cache first: if the item is there (a cache hit) it is returned immediately; if it is missing (a cache miss) the system fetches or computes it from the slow source and stores a copy for next time. Because fast memory is scarce, replacement policies such as LRU (Least Recently Used) decide which entries to evict when the cache fills up, and a TTL (time to live) defines when a cached entry becomes stale.
Modern AI relies on caching everywhere: LLM platforms offer prompt caching to reuse the static prefix of long prompts and cut cost and latency, RAG systems cache precomputed embeddings, and conversational apps cache answers to frequent questions. Beyond AI, caches speed up CPUs, databases, web APIs and content delivery networks.
The word comes from the French cacher, to hide: a cache is literally a hiding place, like a hunter's stash of supplies. The computing term took hold in 1967-68 with the IBM System/360 Model 85, the first commercial computer with a cache memory; IBM lore credits Lyle R. Johnson with proposing the name over the clumsier high-speed buffer. Maurice Wilkes had anticipated the concept in 1965 with his proposal for slave memories.
In Grace's 520 professional scenarios you encounter cases where an AI application needs to cut costs and latency: understanding caching helps you design prompts with stable, reusable prefixes. The 7 evaluation dimensions specifically reward how efficiently you structure your instructions.
From our network
HSE Genius: AI for Safety Data Sheets
Extract SDS data, H phrases and ECHA compliance checks in seconds, powered by AI.
Visit hsegenius.com →From the Agora Intelligence blog
📱 Download the Android app (beta) iOS coming soon
Say what you mean. Get what you need.
Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.