Tag: gradients

All the articles with the tag "gradients".

• Softmax to the Max

  Published:Dec 30, 2024 |  at 10:02 AM |  in 10 min read

  A deep dive into the softmax, logsoftmax, and logsumexp functions, their gradients, and how they relate to each other.

• AutoDiff Puzzles

  Published:Dec 20, 2024 |  at 05:32 AM |  in 19 min read

  My solutions to srush's AutoDiff Puzzles. This is useful as a quick refresher for computing gradients.