Matrix Multiplication Rules and Dimensions
Unlike addition, matrix multiplication is not entry-wise. It is a more complex operation where rows are combined with columns. For this to work, the dimensions must match like interlocking gears.
The Inner-Dimension Rule
The number of columns in the first matrix must equal the number of rows in the second. If Matrix $A$ is $(2 \times 3)$, Matrix $B$ must have 3 rows (e.g., $3 \times 4$).
Resulting Dimensions
If you multiply $A(m \times n)$ by $B(n \times p)$, the result is a matrix $C$ with dimensions $(m \times p)$.
Dot-Product View
Every element in the result is calculated as a dot product of a row from $A$ and a column from $B$.
The Engine of Layers
This operation is the heartbeat of a neural network layer. The matrix represents thousands of neurons, and multiplication calculates all their activations in a single split-second step.