How does stochastic gradient descent work?

Related questions

1. What is cost function gradient descent?
2. What is gradient descent?
3. What is stochastic gradient descent used for?
4. Does linear regression use gradient descent?
5. Can stochastic gradient descent be parallelized?
6. Does gradient descent always converge?
7. Who invented stochastic gradient descent?
8. Is gradient descent greedy?
9. Is gradient descent a convex function?
10. What is stochastic gradient descent vs gradient descent?

Tags

1. #computational
2. #descent
3. #important
4. #gradient
5. #iteration
6. #hence
7. #training
8. #indexed
9. #based
10. #scales
11. #where
12. #function
13. #highlight
14. #linearly
15. #instance