Exploring the Cauchy-Schwarz Inequality and Its Applications
1 Introduction
The fundamental Cauchy-Schwarz inequality forms essential connections among vectors, functions, and random variables across diverse fields. This blog provides a concise overview of the inequality and its widespread applications
2 Cauchy-Schwarz inequality
2.1 Definition in inner product space
Within an inner product space, the Cauchy-Schwarz inequality asserts that for any vectors \(\boldsymbol{u}\) and \(\boldsymbol{v}\):
\[ |\langle\boldsymbol{u}, \boldsymbol{v}\rangle| \leq\| \boldsymbol{u}\|\|\boldsymbol{v}\| . \]
Here, \(\langle\cdot, \cdot\rangle\) represents the inner product, and \(\| \boldsymbol u \|\) is the induced norm defined by
\[ \|\boldsymbol{u}\|_2:=\sqrt{\langle\boldsymbol{u}, \boldsymbol{u}\rangle}, \]
The equality holds only when \(\boldsymbol{u}\) and \(\boldsymbol{v}\) are linearly dependent.
The norm and inner product relate through the condition \(\|\boldsymbol{u}\|_2^2=\langle\boldsymbol{u}, \boldsymbol{u}\rangle\), where \(\langle\boldsymbol{u}, \boldsymbol{u}\rangle\) is always a non-negative real number.
2.2 Special Case in \(\mathbb{R}^{n}\)
In the Euclidean space \(\mathbb{R}^n\), we utilize the standard inner product (dot product):
\[ \boldsymbol{u} \cdot \boldsymbol{v}=\sum_{i=1}^n u_i v_i=u_1 v_1+u_2 v_2+\cdots+u_n v_n \]
where \(\boldsymbol{u} = [u_1, u_2, \ldots,u_n]\) and \(\boldsymbol{v} = [v_1, v_2, \ldots,v_n]\).
Then the Cauchy-Schwarz inequality becomes:
\[ \begin{equation}\begin{aligned}|\langle\boldsymbol{u}, \boldsymbol{v}\rangle|^2 \leq& \| \boldsymbol{u}\|^2\|\boldsymbol{v}\|^2 ,\\ \left(\sum_{i=1}^n u_i v_i\right)^2 \leq&\left(\sum_{i=1}^n u_i^2\right)\left(\sum_{i=1}^n v_i^2\right).\end{aligned} \end{equation} \]
The proof, grounded in quadratic polynomial concepts from elementary algebra, centers around the non-negativity of a certain quadratic term.
\[ 0 \leq\left(u_1 x+v_1\right)^2+\cdots+\left(u_n x+v_n\right)^2=\left(\sum_i u_i^2\right) x^2+2\left(\sum_i u_i v_i\right) x+\sum_i v_i^2 . \]
Since it is nonnegative, it has at most one real root for \(x\), hence its discriminant is less than or equal to zero. That is,
\[ \begin{align*}2^2\left(\sum_i u_i v_i\right)^2-4\left(\sum_i u_i^2\right)\left(\sum_i v_i^2\right) \leq& 0\\ \left(\sum_i u_i v_i\right)^2-\left(\sum_i u_i^2\right)\left(\sum_i v_i^2\right) \leq& 0,\\ \Rightarrow \left(\sum_{i=1}^n u_i v_i\right)^2 \leq\left(\sum_{i=1}^n u_i^2\right)\left(\sum_{i=1}^n v_i^2\right)&.\end{align*} \]
2.3 Special Case in \(\mathbb{L}^2\)
For the inner product space of square-integrable complex-valued functions, the inequality becomes:
\[ \left|\int_{\mathbb{R}^n} f(x) {g(x)} d x\right|^2 \leq \int_{\mathbb{R}^n}|f(x)|^2 d x \int_{\mathbb{R}^n}|g(x)|^2 d x . \]
The Hölder inequality generalizes this result.
There is a proof for this inequality when \(n=1\) similar to the previous proof. It consider a non-negative quadratic term of
\[ \forall t \in \mathbb{R}: 0 \leq(t f(x)+g(x))^2 \]
3 Application on Analysis: Triangle inequality
\[ \begin{equation} \|\boldsymbol{u}+\boldsymbol{v}\| \leq\|\boldsymbol{u}\|+\|\boldsymbol{v}\| \text {. }\end{equation} \]
The triangle inequality is a direct consequence of the Cauchy–Schwarz inequality in any inner product space:
\[ \begin{array}{rlr}\|\boldsymbol{u}+\boldsymbol{v}\|^{2}= & \langle\boldsymbol{u}+\boldsymbol{v},\boldsymbol{u}+\boldsymbol{v}\rangle \\= & \|\boldsymbol{u}\|^{2}+\langle\boldsymbol{u},\boldsymbol{v}\rangle+\langle\boldsymbol{v},\boldsymbol{u}\rangle+\|\boldsymbol{v}\|^{2}\quad\\ & \text{ where }\langle\boldsymbol{v},\boldsymbol{u}\rangle=\overline{\langle\mathbf{u},\boldsymbol{v}\rangle}\\= & \|\boldsymbol{u}\|^{2}+2\operatorname{Re}\langle\boldsymbol{u},\boldsymbol{v}\rangle+\|\boldsymbol{v}\|^{2}\\ & \operatorname{Re}\text{ denotes the real part}\\\leq & \|\boldsymbol{u}\|^{2}+2|\langle\boldsymbol{u},\boldsymbol{v}\rangle|+\|\boldsymbol{v}\|^{2}\\\leq & \|\boldsymbol{u}\|^{2}+2\|\boldsymbol{u}\|\|\boldsymbol{v}\|+\|\boldsymbol{v}\|^{2}\\ & \text{using Cauchy-Schwarz inequality } & \text{ }\\= & (\|\boldsymbol{u}\|+\|\boldsymbol{v}\|)^{2}.\end{array} \]
Taking the square root on both sides yields the triangle inequality.
4 Application in Probability theory: Covariance inequality
Let \(X\) and \(Y\) be random variables, then the covariance inequality is given by:
\[ \operatorname{Cov}(X,Y)^{2}\leq\operatorname{Var}(X)\operatorname{Var}(Y) \]
To prove this, we can define an inner product on the set of random variables using the expectation of their product,
\[ \langle X, Y\rangle:=\mathrm{E}(X Y) \]
the Cauchy-Schwarz inequality becomes
\[ |\mathrm{E}(X Y)|^2 \leq \mathrm{E}\left(X^2\right) \mathrm{E}\left(Y^2\right) . \]
We can prove the covariance inequality using this Cauchy-Schwarz inequality.
Let \(\mu=\mathrm{E}(X)\) and \(\nu=\mathrm{E}(Y)\),
\[ \begin{aligned}|\operatorname{Cov}(X,Y)|^{2}= & |\mathrm{E}((X-\mu)(Y-\nu))|^{2} \\= & |\langle X-\mu,Y-\nu\rangle|^{2}\\ & \text{(Def of inner product)} \\ \leq & \langle X-\mu,X-\mu\rangle\langle Y-\nu,Y-\nu\rangle\\ & (\text{Cauchy-Schwarz inequality}) \\= & \mathrm{E}\left((X-\mu)^{2}\right)\mathrm{E}\left((Y-\nu)^{2}\right)\\= & \operatorname{Var}(X)\operatorname{Var}(Y),\end{aligned} \]
5 Reference
https://en.wikipedia.org/wiki/Cauchy–Schwarz_inequality
https://en.wikipedia.org/wiki/Hölder's_inequality
https://proofwiki.org/wiki/Cauchy-Bunyakovsky-Schwarz_Inequality/Definite_Integrals