Understanding Gradient Descent

MACHINE LEARNING

A simple implementation in Python

The algorithm starts at a random point on the function and moves in the negative direction of the gradient of the function to reach the local or global minima.

In supervised learning, models have cost functions. Cost functions are used to estimate how badly models are performing. In other words, cost functions are a measure of how badly the model is performing in terms of its ability to estimate the relation between X and y. This is typically the distance between the actual and the predicted values.

This is the basic idea. More on it at the end.

We move along the slope of the hill which gives us the maximum descent at each step .i.e along the negative direction of the tangent at any given point. But how much? We define a learning rate for this.

Hence, we can update our position after each step as follows:

Related posts:

Technological advances are being used in numerous aspects of daily life. Communications, industry, travel and household appliances are a mere tip of the iceberg where technology has become…

Chicken Tikka Masala is often the first dish many try when exploring Indian chicken recipes. Welcome to the most velvety, tasty, mediumly spiced, weekend night, must-have-it chicken tikka masala…

A person who wishes to engage in the import, production, or trading of telecom equipment in India must first obtain TEC Certification. Once the equipment has undergone the Mandatory Testing &…