AI Dictionary › AI Fundamentals

Reward Function

Funzione di ricompensa

The reward function is the rule that, in reinforcement learning, assigns a number to every outcome the agent produces: positive if the outcome is desired, negative or zero otherwise. The agent does not know the goal itself; it knows only the reward, and learns the strategy that maximizes it over time. Everything the designer wants has to pass through that number.

Definition

It is the most delicate point of the whole paradigm, because translating an intention into a number is almost always an approximation: "win the race" becomes "collect points along the course", "be helpful" becomes "earn the rater's preference". Every gap between the intention and the number is room for reward hacking. In language models the reward function is itself a model, trained on human preferences, with its own imprecisions.

The operating rule: write the reward function together with the list of ways it could be earned without being deserved. If the list is long, the function needs rewriting before anything is trained.

Related terms

• Reinforcement Learning
• RLHF
• Reward Hacking
• Goodhart's Law
• Gym (Training Environment)

More in AI Fundamentals

• LLM (Large Language Model)
• Token
• Context Window
• Temperature
• AI Hallucination
• Fine-tuning
• RAG (Retrieval Augmented Generation)
• Embedding

Put it into practice

• Real scenario: apply this concept with an instant AI verdict.
• Daily gym: short drills to train dictionary concepts.
• Playoff game: challenge other players, elimination-style.

Say what you mean. Get what you need.

Grace Certified, the AI coach that trains and certifies your prompt engineering, by Agora Intelligence.