Chapter 7 Symmetric Matrices and Quadratic Forms

What is a singular value decomposition?

...

Theorem 1: State and Prove This tells us something about the eigenvectors of symmetric matrices

Eigenvectors from different eigenspaces of a symmetric matrix are orthogonal (pg. 397)

Theorem 5: Quadratic forms and eigenvalues

If A is an nxn symmetric matrix, then the quadradic form x^TAx is positive definite iff the eigenvalues of A are all positive negative definite iff the eigenvalues of A are all negative indefinite iff A has both positive and negative eigenvalues

Redo problem 26

pg. 409

Pictures for the classes of quadratic forms

(a) positive definite (b) is positive semidefinite (c) indefinite (d) negative definite (pg. 407)

Suppose a matrix A is orthogonally diagonalizable, what is the transpose of A?

A is the transpose of itself (pg. 398)

What is a symmetric matrix?

A matrix that's equal to it's transpose. All symmetric matrices are square (pg. 397)

What does it mean for a matrix to be orthogonally diagonalizable?

A nxn matrix is orthogonally diagonalizable if it has n linearly independent, orthogonal eigenvectors (pg. 398)

Classify quadratic forms Define positive definite, negative definite, indefinite, positive semidefinit, and negative semidefinite

A quadratic form is positive definite if it is greater than zero for all x≠0 A quadratic form is negative definite if it is less than zero for x≠0 A quadratic form is indefinitie if it assumes both positive and negative values. A quadratic form is positive semidefinite if it is greater than or equal to zero for all x. A quadratic form is negative semidefinite if it is less than or equal to zero for all x. (pg. 407)

What is a quadratic form on R^n?

A quadratic form on R^n is a function Q defined on R^n where at each vector x, the value of the function is given by x^TAx where A is some symmetric matrix. We call A the matrix of the quadratic form (pg. 403).

What is an orthogonal matrix?

An orthogonal matrix is a square matrix with orthonormal columns whose transpose is its inverse. A square invertible matrix U such that U^-1=U^T.

Explain the three parts to a singular value decomposition

Any mxn matrix A can be decomposed into three matrices A=U∑V^T, where U is an mxm orthogonal matrix. The first r columns of U are Av1,..., Avr, where r is the rank of A. ∑ is an mxn "diagonal" matrix V^T is an nxn orthogonal matrix. The columns of V are the unit eigenvectors of A^TA obtained from the diagonalization of A^TA

What is a spectral decomposition?

If an nxn matrix A is orthogonally diagonalizable (i.e. A= PDP^T where the columns of P are mutually orthogonal), then A can be factored to show it's eigenvalues and eigenvectors (pg. 400)

What is a change of variable?

If x represents a variable in R^n then the equation of the form x=Py is a change of variable where P is an invertible matrix and y is a new variable in R^n (pg. 404)

What are left singular vectors? What are right singular vectors?

In a singular value decomposition of a mxn matrix A where A=U∑V^T, the left singular values of A are the columns of U and the right singular values of A are the columns of V (rows of V^t) (pg. 419)

How do we ensure that our quadradic form on R^n has no cross terms?

Make the quadradic form x^TAx such that A is diagonal (see example 1 pg. 403)

In a singular value decomposition for an mxn matrix A, where A= U∑V^T, explain what ∑ is

Please see the picture, D is an rxr diagonal matrix where r does not exceed the smaller of m and n. We obtain the entries of D by finding the first r singular values of A, σ1,...,σ3>0. Σ is an mxn matrix (SAME SIZE AS A) with a diagonal matrix D in its upper left corner and zeros everywhere else. The entries of D are the nonzero singular values of A.

Theorem 9: This tells us something about the set {Av1,...,Avr}

Suppose {v1,...,vn} is an orthonormal basis for R^n consisting of eigenvectors of A^TA arranged so that the corresponding eigenvalues satisfy λ1≥...≥λn, and suppose A has r nonzero singular values. Then the set {Av1,...,Avr} is an orthogonal basis for the column space of A, and rankA=r (pg. 418)

Theorem 6: State This theorem tells us how the eigenvalues of the matrix A in the quadratic form Q(x)=x^TAx are related to the maximum and minimum values of Q on the constraint x^Tx≤1

The maximum of Q on the constraint is the biggest eigenvalue of A and the minimum of Q on the constraint is the smallest eigenvalue of A. To achieve the maximum, plug in a unit eigenvector corresponding to the biggest eigenvalue of A. To achieve the minimum, plug in a unit eigenvector corresponding to the smallest eigenvalue of A (pg. 412)

Theorem 7: This tells us how to find the maximum value of a quadratic form subject to the restraints x^Tx=1 and x^Tu1=0 (u1 is the unit eigenvector corresponding to the greatest eigenvalue of A)

The maximum value of a quadratic form with constraints x^Tx=1 and x^Tu1=0 is the second greatest eigenvalue of A. This maximum is attained by plugging in the unit eigenvector corresponding to the second greatest eigenvalue of A (pg. 413)

What are the singular values of a matrix A?

The square roots of the eigenvalues of the matrix A^TA. They also correspond to to the lengths of the images Avi, vi is a unit eigenvector of A^TA (pg. 418)

T/F: If A is symmetric, then A is automatically orthogonally diagonalizable

This is true by theorem 2

Theorem 2: Under what circumstances is a matrix orthogonally diagonalizable?

This theorem tells us that all symmetric matrices are orthogonally diagonalizable and vice versa.

Theorem 10: The singular decomposition theorem

This theorem tells us that any mxn matrix A can be decomposed into the product of three matrices with certain properties (pg. 419)

Theorem 4: The principle axes theorem. State and then explain what principle axes are

This theorem tells us that for any nxn symmetric matrix there is an orthogonal change of variable x=Py such that the quadratic form x^TAx is transformed into y^TDy, where D is a diagonal matrix (containing the eigenvalues of A), resulting in no cross product terms. The columns of P are called the principle axes of the quadradic form x^TAx (the columns of P are orthonormal eigenvectors of A that correspond to the eigenvalues of A) (pg. 405) If this is confusing see example 4

Theorem 3: Four properties of a Symmetric matrix

nxn symmetric matrices have 1) n real eigenvalues, counting multiplicities 2) the dimension of the eigenspace for each eigenvalue equals the multiplicity of the eigenvalue as a root of the characteristic equation 3) The eigenspaces are mutually orthogonal 4) are orthogonality diagonalizable (i.e. A=PDP^T) (pg. 398)

Notation for the maximum or minimum of a quadratic form given the constraint x^Tx≤1

pg. 411

Prove theorem 6

pg. 412