The inner product of two complex vectors u⃗,v⃗∈Cn\vec u, \vec v \in \mathbb{C}^n is defined as

u⃗Hv⃗=[uˉ1,uˉ2,...,uˉn][v1v2...vn]=uˉ1v1+uˉ2v2...+uˉnvn. \vec u^H \vec v = [ \bar u_1, \bar u_2, ..., \bar u_n ] \begin{bmatrix} v_1 \\ v_2 \\ . \\ . \\ . \\ v_n \end{bmatrix} = \bar u_1 v_1 + \bar u_2 v_2 ... + \bar u_n v_n.

It is tempting to think the inner product should be u⃗Tv⃗\vec u^T \vec v, as the one for real vectors. Why do we take the conjugate of u⃗\vec u? To answer this, we need to study the geometric meaning of inner product. We begin from the inner product in Rn\mathbb{R}^n.

Inner Product in Rn\mathbb{R}^n

The inner product is closely related to the concept of "length". Generally, we express the length of a vector in term of inner product. Here we take a reverse approach. We consider the length of a vector as a more fundamental concept. Once the length of a vector is defined, it leads naturally the definition of inner product.

For a vector u⃗∈Rn\vec u \in \mathbb{R}^n, we define its length as

∣∣u⃗∣∣=∑i∈[n]ui2. ||\vec u || = \sqrt{ \sum_{i \in [n] } u_i^2 }.

Given two vector u⃗,v⃗∈Rn\vec u, \vec v \in \mathbb{R}^n, the definition of the inner product of u⃗Tv⃗\vec u^T \vec v stems from the following problem:

Find a vector in the subspace generated by u⃗\vec u, such that it distance to v⃗\vec v is minimized.

The subspace generated by u⃗\vec u is the line that passes through u⃗\vec u and can be represented as

{tu⃗:t∈R}. \{ t \vec u : t \in \mathbb{R} \}.

The problem gives rise to the optimization problem:

min⁡t∈R∣∣v⃗−tu⃗∣∣2. \min_{ t \in \mathbb{R} } ||\vec v - t \vec u||^2.

Define

y≐∣∣v⃗−tu⃗∣∣2=∣∣u⃗∣∣2t2−2(v⃗Tu⃗)t+∣∣v⃗∣∣2. y \doteq || \vec v - t \vec u||^2 = || \vec u ||^2 t^2 - 2 ( \vec v^T \vec u) t + || \vec v||^2.

By differentiating and setting the derivative equal to zero, we get

y′=2∣∣u⃗∣∣2t−2(v⃗Tu⃗)=0  ⟹  t=v⃗Tu⃗∣∣u⃗∣∣2. y' = 2 || \vec u ||^2 t - 2 (\vec v^T \vec u) = 0 \implies t = \frac{ \vec v^T \vec u }{ || \vec u ||^2 }.

The inner product appears exactly in the numerator. Its meaning is clear now: it is related to the magnitude of the closest vector in the subspace spanned by u⃗\vec u. In case that u⃗\vec u is a unit vector, t=v⃗Tu⃗t = \vec v^T \vec u is exactly the length of this closest vector.

This also sheds light on the meaning of orthogonality:

If u⃗\vec u is orthogonal to v⃗\vec v, then the best approximation (measured by ∣∣⋅∣∣|| \cdot ||) of v⃗\vec v in the subspace spanned by u⃗\vec u is the zero vector.

Inner Product in Cn\mathbb{C}^n

To get the desired expression of inner product Cn\mathbb{C}^n, we first extend the definition of length in Rn\mathbb{R}^n to Cn\mathbb{C}^n. For u⃗∈Cn\vec u \in \mathbb{C}^n, define

∣∣u⃗∣∣=∑i∈[n]∣ui∣2. || \vec u || = \sqrt{ \sum_{i \in [n]} |u_i|^2 }.

Given u⃗,v⃗∈Cn\vec u, \vec v \in \mathbb{C}^n, we consider the similar problem:

Find a vector in the subspace generated by u⃗\vec u, such that it distance to v⃗\vec v is minimized.

However, this time the subspace generated by u⃗\vec u is given by

{(tr+iti)u⃗:tr,ti∈R}. \{ (t^r + i t^i ) \vec u: t^r, t^i \in \mathbb{R} \}.

Let u⃗r∈Rn\vec u^r \in \mathbb{R}^n be the real part of u⃗\vec u and u⃗i∈Rn\vec u^i \in \mathbb{R}^n be its imaginary part. Then u⃗=u⃗r+iu⃗i\vec u = \vec u^r + i \vec u^i. Similarly, we can decompose v⃗\vec v into v⃗=v⃗r+iv⃗i\vec v = \vec v^r + i \vec v^i.

We want to minimize

y≐∣∣(tr+iti)u⃗−v⃗∣∣2=∣∣(tr+iti)(u⃗r+iu⃗i)−v⃗r−iv⃗i∣∣2=∣∣tru⃗r−tiu⃗i−v⃗r+i(tiu⃗r+tru⃗i−v⃗i)∣∣2=∣∣tru⃗r−tiu⃗i−v⃗r∣∣2+∣∣tiu⃗r+tru⃗i−v⃗i∣∣2 \begin{aligned} y &\doteq ||(t^r + i t^i ) \vec u - \vec v||^2 \\ &= ||(t^r + i t^i ) (\vec u^r + i \vec u^i) - \vec v^r - i \vec v^i ||^2 \\ &= || t^r \vec u^r - t^i \vec u^i - \vec v^r + i ( t^i \vec u^r + t^r \vec u^i - \vec v^i ) ||^2 \\ &= || t^r \vec u^r - t^i \vec u^i - \vec v^r ||^2 + || t^i \vec u^r + t^r \vec u^i - \vec v^i ||^2 \\ \end{aligned}

Taking the derivative with respect to trt^r, we get

∂y∂tr=2∣∣u⃗r∣∣2tr−2(u⃗r⋅u⃗i)ti−2(u⃗r⋅v⃗r)+2∣∣u⃗i∣∣2tr+2(u⃗r⋅u⃗i)ti−2(u⃗i⋅v⃗i) \frac{\partial y}{ \partial t^r } = 2 ||\vec u^r||^2 t^r - 2 ( \vec u^r \cdot \vec u^i) t^i - 2 ( \vec u^r \cdot \vec v^r) + 2 ||\vec u^i||^2 t^r + 2 (\vec u^r \cdot \vec u^i ) t^i - 2 ( \vec u^i \cdot \vec v^i )

where u⃗r⋅u⃗i\vec u^r \cdot \vec u^i denotes their dot product. Setting the derivative to zero, we have

tr=u⃗r⋅v⃗r+u⃗i⋅v⃗i∣∣u⃗r∣∣2+∣∣u⃗i∣∣2=u⃗r⋅v⃗r+u⃗i⋅v⃗i∣∣u⃗∣∣2. t^r = \frac{ \vec u^r \cdot \vec v^r + \vec u^i \cdot \vec v^i }{ ||\vec u^r||^2 + ||\vec u^i||^2 } = \frac{ \vec u^r \cdot \vec v^r + \vec u^i \cdot \vec v^i }{ ||\vec u||^2 }.

Also,

∂y∂ti=2∣∣u⃗i∣∣2ti−2(u⃗r⋅u⃗i)tr+2(u⃗i⋅v⃗r)+2∣∣u⃗r∣∣2ti+2(u⃗r⋅u⃗i)tr−2(u⃗r⋅v⃗i). \frac{\partial y}{ \partial t^i } = 2 ||\vec u^i||^2 t^i - 2 ( \vec u^r \cdot \vec u^i) t^r + 2 ( \vec u^i \cdot \vec v^r) + 2 ||\vec u^r||^2 t^i + 2 (\vec u^r \cdot \vec u^i ) t^r - 2 ( \vec u^r \cdot \vec v^i ).

Setting the derivative to zero, we obtain

ti=−u⃗i⋅v⃗r+u⃗r⋅v⃗i∣∣u⃗r∣∣2+∣∣u⃗i∣∣2=−u⃗i⋅v⃗r+u⃗r⋅v⃗i∣∣u⃗∣∣2. t^i = \frac{ -\vec u^i \cdot \vec v^r + \vec u^r \cdot \vec v^i }{ ||\vec u^r||^2 + ||\vec u^i||^2 } = \frac{ -\vec u^i \cdot \vec v^r + \vec u^r \cdot \vec v^i }{ ||\vec u||^2 }.

On the other hand, the Hermitian inner product is

(u⃗r+iu⃗i)‾⋅(v⃗r+iv⃗i)=(u⃗r⋅v⃗r+u⃗i⋅v⃗i)+i(−u⃗i⋅v⃗r+u⃗r⋅v⃗i). \overline{(\vec u^r + i \vec u^i)} \cdot (\vec v^r + i \vec v^i ) = (\vec u^r \cdot \vec v^r + \vec u^i \cdot \vec v^i) + i ( -\vec u^i \cdot \vec v^r + \vec u^r \cdot \vec v^i ).

Therefore,

tr+iti=1∣∣u⃗∣∣2u⃗Hv⃗. t^r + i t^i = \frac{1}{||\vec u||^2} \vec u^H \vec v.

□\square