AI Dictionary › AI Fundamentals
An API gateway is the single entry point through which all external requests reach a system's internal services. Like the reception desk of a large building, it welcomes every visitor, checks their credentials, directs them to the right office and keeps the access log. It centralizes the cross-cutting functions every service would otherwise have to reimplement.
When a request arrives, the gateway authenticates the caller (API key, OAuth token), applies rate limiting, validates and routes the call to the appropriate microservice, possibly aggregating several responses into one. Meanwhile it records metrics and logs, handles TLS termination and translates protocols. Tools like Kong, Apigee and AWS API Gateway dominate the category.
In AI, the LLM gateway has emerged as a specialized evolution: a single point that routes calls to multiple model providers, manages keys and quotas, applies semantic caches, masks sensitive data in prompts and tracks costs per team. Tools like LiteLLM and OpenRouter embody this pattern. Beyond AI, the gateway is the standard gatekeeper of every microservices architecture.
The term combines API with gateway, the word for an access gate, used in computer networking for decades to name border nodes between different systems. The product category took hold in the 2010s with the explosion of the API economy and microservices: the rise of companies like Kong (whose open source gateway shipped in 2015) and the arrival of AWS API Gateway the same year mark its commercial maturity.
Grace is your single entry point to AI training: gym, 1515 courses, scenarios, arena and certification all go through one platform, just as requests go through a gateway. In the scenarios on enterprise AI platforms, being able to describe an LLM gateway sets you apart in the technical interviews found on the AI Jobs board.
From our network
HSE Genius: AI for Safety Data Sheets
Extract SDS data, H phrases and ECHA compliance checks in seconds, powered by AI.
Visit hsegenius.com →From the Agora Intelligence blog
📱 Download the Android app (beta) iOS coming soon
Say what you mean. Get what you need.
Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.