Vector Operations in Python

Vectors are the language of AI. Word embeddings, image features, and user preferences are all represented as vectors. The three operations you will use most — magnitude, dot product, and cosine similarity — are all single-line calls in NumPy.

Magnitude (The L2 Norm)

The magnitude (or Euclidean norm) of a vector $\mathbf{v}$ is $\|\mathbf{v}\| = \sqrt{\sum v_i^2}$. It measures the "length" of the vector in space. In AI, it's used to measure the size of gradients, distances between points, and for normalizing vectors.

Computing Norms

<pre><code class="language-python">import numpy as np v = np.array([3.0, 4.0]) # Manual (educational) mag_manual = np.sqrt(np.sum(v ** 2)) # 5.0 # Professional (use this) mag = np.linalg.norm(v) # 5.0 # L1 norm (Manhattan distance) l1 = np.linalg.norm(v, ord=1) # 7.0 </pre>

The Dot Product

The dot product $\mathbf{a} \cdot \mathbf{b} = \sum a_i b_i$ measures how much two vectors point in the same direction. It is positive if they align, zero if perpendicular, negative if opposite. Every neuron in a neural network computes a dot product between its inputs and its weights.

Dot Product in NumPy

<pre><code class="language-python">a = np.array([1, 2, 3]) b = np.array([4, 5, 6]) dot = np.dot(a, b) # 1*4 + 2*5 + 3*6 = 32 # Or equivalently: dot = a @ b # 32 </pre>

Cosine Similarity

Cosine similarity removes the effect of magnitude and measures only the angle between two vectors: $$\cos(\theta) = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{a}\| \|\mathbf{b}\|}$$

It returns a value between -1 (completely opposite) and 1 (identical direction). It is the standard metric for comparing text embeddings — two sentences with similar meaning will have a cosine similarity close to 1.

Implementing Cosine Similarity

<pre><code class="language-python">def cosine_similarity(a, b): return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)) v1 = np.array([1, 2, 3]) v2 = np.array([1, 2, 3]) # identical v3 = np.array([-1, -2, -3]) # opposite print(cosine_similarity(v1, v2)) # 1.0 print(cosine_similarity(v1, v3)) # -1.0 </pre>