Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
February 5, 2019
1 Proof Methods
2 Linear Algebra
3 Geometry
4 Calculus
Proof Methods
Proofs
Theorem
Let a, b be real numbers. Then a = b if and only if for all > 0, ∣a − b∣ <
Theorem
n(n+1)
Let n ∈ N, then ∑ni=1 i = 2
Linear Algebra
Linear Combination
α ⋅ v1 + β ⋅ v2 ∈ V, ∀v1 , v2 ∈ V, α, β ∈ F (1)
Linear Dependency
implies that αk = 0, k = 1 . . . , n.
Definition (Span)
The set of all linear combinations of a list of vectors {ak } is called the
span of {ak }
n
span[a1 , . . . , an ] = { ∑ αk ak ∶ αk ∈ R}
k=1
Matrix Rank
▶ Nice, but is there a formula for testing if the matrix has full rank? Is
there a scalar quantity that measures the independency of columns?
∎ For square matrices (m = n), the answer is yes! It is called the
determinant of the matrix.
Asst. Prof. N. Kemal Ure (ITU) Lecture 2 February 5, 2019 13 / 57
Linear Algebra
Determinant
det[a1 , . . . , 0, . . . , an ] = 0.
Norm Properties
▶ There are many other vector norms. Actually any function that
satisfies the properties above is a norm.
1
∎ p-norm: ∥x∥p = (∣x1 ∣p + ∣x2 ∣p + ⋅ ⋅ ⋅ + ∣xn ∣p ) p
∎ p = 2, the Euclidean norm
Linear Transformations
L(ax) = aL(x), a ∈ R
L(x + y) = L(x) + L(y)
Similarity Transformations
Eigenvalues
▶ Although the number of eigenvalues is finite, that this is not the case
for eigenvectors..
∎ If v is an eigenvector, so is cv.
∎ So there are actually an infinite number of eigenvectors
Eigenvectors as a Basis
Symmetric Matrices
Orthogonal Subspaces
V ⊥ = {x ∈ Rn ∶ v T x = 0, v ∈ V }
Quadratic Forms
f (x) = xT Qx,
Matrix Norms
Induced Norms
Geometry
Line Segments
Hyperplanes
Definition (Hyperplane)
Let u ∈ Rn and v ∈ R. An hyperplane is the set of points of the form
{x ∈ Rn ∶ ⟨u, x⟩ = v}
Linear Varieties
{x ∈ Rn ∶ Ax = b},
Convex Sets
βΘ = {x ∶ x = βv, v ∈ Θ}
is also convex.
2 If Θ1 and Θ2 are convex, then the set
Θ1 + Θ2 = {x ∶ x = v1 + v2 , v1 ∈ Θ1 , v2 ∈ Θ2 }
is also convex.
3 If Θi is a collection of convex sets, then ∩i Θi is also convex.
Neighborhoods
Definition (Neighborhood)
Let > 0. An −neighborhood of a
point x ∈ Rn is the set
N (x) = {y ∈ Rn ∶ ∥y − x∥ < }
Calculus
Limit of a sequence
k ≥ K Ô⇒ ∣xk − x∣ <
Subsequences
Theorem (Bolzano-Weierstrass)
Every bounded sequence contains a convergent subsequence.
Matrix Limits
lim ∥Ak − A∥ = 0
k→∞
Differentiability
A(x) = L(x) + y
The Hessian
Differentiation Rules
▶ D(y T Ax) = y T A.
▶ D(xT Ax) = xT (A + AT ) if n = m.
▶ D(y T x) = y T .
▶ D(xT x) = 2xT
Level Set
Taylor’s Theorem
Order Symbols
∥f (x)∥
∥x∥ < δ Ô⇒ ≤K
∣g(x)∣
∎ We say that f (x) = o(g(x)) if ∥f (x)∥ goes to zero faster than ∣g(x)∣,
∥f (x)∥
lim =0
x→0 ∣g(x)∣
∎ φ(∥x − x0 ∥) = f (x)
∎ Since φ is a single variable function, we can apply Taylor’s Theorem!
▶ If f ∈ C 3 :
1
f (x) = f (x0 )+Df (x0 )(x−x0 )+ (x−x0 )⊺ D2 f (x−x0 )+O(∥x−x0 ∥3 )
2
Summary
▶ Next:
∎ Basics of optimization theory.