What Is Gradient Descent?

Gradient descent is how a model improves little by little, step by step, like carefully walking down a hill to find the lowest point of a valley.

Imagine it like this: Picture yourself blindfolded on a hill, trying to reach the bottom of a valley. You can't see anything, but you can feel which direction slopes down under your feet. So you take a small step in that direction, feel again, take another small step… Until you reach the bottom!

Examples:

• Used in almost every neural network to slowly reduce its errors
• Used to refine the best straight line in linear regression

So? Gradient descent is the "practice, adjust, practice again" engine that makes models smarter over time, one tiny step at a time.