The gradient of the model's performance surface represents the derivatives of the model's performance with regard to its parameters. A positive gradient indicates the direction of the steepest uphill. A negative gradient indicates the direction of the steepest downhill. Moving the model's parameters in these two directions affects the model's performance most strongly.
| \( \theta \) | This symbol represents the parameters of the model |
| \( \nabla \) | This symbol represents the gradient of a function. |
| \( R \) | This symbol denotes the risk of a model. |
Given the risk function of a model parametrized by \( \htmlClass{sdt-0000000083}{\theta} \), \( \htmlClass{sdt-0000000062}{R} \)(\( \htmlClass{sdt-0000000083}{\theta} \)), we use the definition of the gradient operator, \( \htmlClass{sdt-0000000093}{\nabla} \), to define the gradient of the risk, often called performance surface:
\[\htmlClass{sdt-0000000093}{\nabla} \htmlClass{sdt-0000000062}{R}(\htmlClass{sdt-0000000083}{\theta})=(\frac{\delta \htmlClass{sdt-0000000062}{R}}{\delta \htmlClass{sdt-0000000083}{\theta}_1}, ..., \frac{\delta \htmlClass{sdt-0000000062}{R}}{\delta \htmlClass{sdt-0000000083}{\theta}_D})\]