Auto-regression & diffusion: concepts, differences
Resources: Why Does Diffusion Work Better than Auto-Regression?
Tensors
arrays of scalars
Chain rule (calculus)
Intuitively, the chain rule states that knowing the instantaneous rate of change of z relative to y and that of y relative to x allows one to calculate the instantaneous rate of change of z relative to x as the product of the two rates of change.
As put by George F. Simmons: “If a car travels twice as fast as a bicycle and the bicycle is four times as fast as a walking man, then the car travels 2 × 4 = 8 times as fast as the man.”
Neuron
Multi-Layer Perceptron (MLP)
Underscores in Python (_ and __): Instance
In Python, __self__ is used to refer to the current instance of the class within instance methods. It is explicitly passed as the first parameter to instance methods.
Resources:
Weights of a Neural Network
Weights are not inputted data. Leaf nodes that will affect the loss function. Get iterated using the gradient information.
Need more.
Biases in the context of a Neural Network
So, weights and biases are...
Here will be the content.
Cost Function
Computation Graph
(mentioned in autograd)
Resources:
Sample computation graph
Semantic Search
Vector Embeddings
A bunch of numbers capturing the essence of our texts. Numerical representations (basically vector representations!) of our texts to map different data types to points in a multidimensional space, with similar data points positioned near each other. These representations assist machines to understand and process our data more effectively.
