Problem Set 1

January 7, 2014
1.

(a) Prove that for rectangular matrices B, D, and invertible matrices A, C

A(A + BCD)1 = I B(C1 + DA1 B)1 DA1 .
(b) Using the above result, prove that
(A1 + BT C1 B)1 BT C1 = ABT (BABT + C)1 .
(c) Find an expression for (A + xyT )1 in terms of A1 , x and y for invertible
matrix A and (column) vectors x and y.

2. Let i (A) denote an eigenvalue of A

(a) i (AB) = i (BA) where A, B are not necessarily square.
(b) If A = AT , find the following in terms of i (A)
i.
ii.
iii.
iv.

Tr(Ap ).
i (I + cA).
i (A cI).
i (A1 ).

3. The notation A 0 denotes the fact that A is positive definite. Prove the following
results for A 0:
(a) A1 0

(b) [A]ii 0 for all i, where [A]ii denotes the i-th diagonal entry of A.
(c) For any B, rank(BABT ) = rank(B).

(d) If B is full row-rank, then BABT 0.

(e) If A  0, then for any matrix X, XAXT = 0 AX = 0.

4. Prove the following for the operator norm (with a = b, and all vector norms are
k.ka ).
(a) kIka,a = 1

(b) kAka,a = supkxk=1 kAxka

(c) kAka,a = supkxk6=0

kAxk
.
kxk

(d) Let AT A  0, then kAk22,2 = maxi i (AT A)

(e) kAk1,1 = maxj

i |[A]i,j |

(f) kAxk kAka,a kxk

(g) kABka,a kAka,a kBka,a

5. Consider X = {x Rn |Ax = b} for A Rmn and b Rm with b R(A).
Prove that the following two statements are equivalent (i.e., (a) (b) and (b)
(a))
(a) cT x = d for all x X

(b) There exists v Rm such that c = AT v and d = bT v.

6. Prove that
(a) ku + vk22 = kuk22 + kvk22 if and only if uT v = 0.

(b) 2 < a, b > +2 < x, y >=< a + x, b + y > + < a x, b y >

(c) kxk1 n kxk2

(d) kxk1 kxk2 kxk

7. Consider A Rmn where m n and rank of A is n. Suppose there exists B such

that BA = I and that AB = BT AT .
(a) Let X = AB. Show that
i. X2 = X
ii. Tr(X) = rank(X)
(b) Find an expression for B in terms of A.
(c) For any two vectors b Rn and c Rm , prove that kAb ck2
kABc ck2 .

EE609

Assignment 2

January 8, 2015

Weight: 1% if submitted by Jan 15, 2014.

Assignment will not be graded. No need to submit practice problems.

1 Assignment Problems
1. Show that the set {x Rn |kx x1 k2 kx x2 k2 } is actually a half space. and express it
in canonical form.
2. Show that the set {x Rn |kxk1 2} is a polyhedra and express it as an intersection of half
spaces (and hyperplanes, if required).
3. Show that the set {x R2++ |x1 x2 1} is convex and express it as an intersection of infinite
number of half-spaces.
4. Is the following set convex: {x Rn |x 0, xT y 1 for all y with kyk2 = 1}? Show that
the set {x Rn |xT y 1 for all y with y C} is always convex, even if C is not convex.
5. Consider the two-dimensional positive semidefinite cone S2+ defined as
{X = [ xy yz ] |x, y, z R, X  0}

(1)

Show that it can equivalently be expressed as {x, y, z R|x 0, z 0, xz y 2 }.

2 Practice Problems
6. Show that a set is affine if and only if its intersection with any line is affine.
7. What is the minimum distance between two parallel halfspaces {x Rn |aT x b1 } and
{x Rn |aT x b2 } (Hint: it depends on where the origin is)?
, b, and b, is the following true?
8. Under what conditions on a, a
{x Rn |aT x b} {x Rn |
aT x b}

(2)

9. For m < n, let b Rm and A Rmn be full row rank. Show that any affine set {x
Rn |Ax = b} can be expressed in the form {Cu + v|u Rm }. For example, the set
{x R2 |x1 + x2 = 1} can be expressed as {[u 1 u]T |u R}
10. What is the affine hull of the set {x R3 |x21 + x22 = 1, x3 = 1}.
11. Is the following set affine: {x Rn |kx x1 k1 kx x2 k1 }?
12. Given two vectors y, z Rn , consider the set S{a,b} = {x Rn |x = ay + bz}. Show that
the following set is a polyhedra, and find its boundaries.
[
[
S{a,b}
(3)
a[1,1] b[1,1]

13. Show that the set {x Rn |kxk 1} is a polyhedra and express it as an intersection of half
spaces (and hyperplanes, if required).

EE609

Assignment 2, Page 2 of 2

January 8, 2015

14. Show that the set of all doubly stochastic matrices is convex polyhedral in Rnn . A doubly
stochastic matrix is a square matrix with nonnegative entries with the property that the sum of
entries in every row and column is exactly 1.
15. What kind of set is C = {x R2 |p(0) = 1, |p(t)| 1, for 1 t 2} where p(t) =
x1 + x2 t?
16. Given , consider the set S = {x Rn |kx ak2 kx bk2 } for a 6= b. Show that S is
a halfspace for = 1, convex for < 1. Give an example to prove that S can be non-convex
for > 1.
17. Find the separating hyperplane between the two sets C = {x R2 |x2 0} and D = {x
R2+ |x1 x2 1}.
18. Express the following norm balls as intersection half-spaces (a) {x Rn |kxk2 1}, (b)
{x Rn |kxk1 1}, and (c) {x Rn |kxk 1}.
19. Which of the following sets are convex (provide proof or counterexample)
(a) {x R2 |x21 + 2ix1 x2 + i2 x22 1 i = 1, 2, . . . , 10}
(b) {x R2 |x21 + ix1 x2 + i2 x22 1 i = 1, 2, . . . , 10}
(c) {x Rn | mini xi = 1}
(d)
20. Show that the perspective function transformation, P(C) := {x/t|[xT t]T C} preserves
convexity.
21. Show that the set {x Rn |(Ax + b)T (Ax + b) (cT x + d)2 , cT x + d > 0} is a convex
cone.

EE609

Assignment 3

January 16, 2015

Weight: 1% if submitted by Jan 22, 2014.

Assignment will not be graded. No need to submit practice problems.

1 Assignment Problems
1. Starting from Jensens inequality, show that x y 1 x + (1 )y.
2. Consider a differentiable function f : R R with R+ dom f , and its running average,
defined as
Z
1 x
f (t)dt
(1)
F (x) =
x 0
with dom F = R++ . Show that F (x) is convex if f (x) is convex.
P
3. Show that the harmonic mean f (x) = ( ni=1 1/xi )1 is concave.

4. Define x[j] as the j-th largest component of x; e.g., x[1] = maxi {xi }, and x[n] =Pmini {xi }.
Given any non-negative numbers 1 2 . . . r 0, prove that f (x) = ri=1 i x[i]
is convex.

n
5. Prove
Pn the reverse Jensens inequality for a convex f with dom f = R , i > 0 and 1
i=2 i = 1

f (1 x1 2 x2 . . . n xn ) 1 f (x1 ) 2 f (x2 ) . . . n f (xn )

(2)

2 Practice Problems
6. Consider an increasing and convex function f : R R with dom f = {a, b}. Let the function
g : R R denote its inverse, i.e., g(f (x)) = x for a x b and dom g = {f (a), f (b)}.
Show that g is a concave function.
7. Show that the running average of the non-differentiable function f (x) is also convex. Since f
is not differentiable, you must use the zeroth order condition to prove the convexity of F .
8. Give an example of a function f (x) whose epigraph is (a) half-space, (b) norm cone, and (c)
polyhedron.
9. Give an example of a concave positive function (f (x) > 0) with domain Rn .
10. Show that the function f (x) is convex if and only if the function f (a + tb) is convex for all
a + tb dom f and t.
P
11. Show that the harmonic function: f (x) = ( ni=1 xai )1/a for a < 1, and a 6= 0 is concave.
P
12. Show that the entropy function H(x) = ni=1 xi log(xi ) with dom H = {x Rn++ |1T x =
1} is concave.
13. Determine if the following functions with dom f = R2++ are convex or concave
(a) f (x1 , x2 ) = x1 x2
(b) f (x1 , x2 ) = x1 /x2
(c) f (x1 , x2 ) = x21 /x2

EE609

Assignment 3, Page 2 of 2

January 16, 2015

(d) f (x1 , x2 ) = x1 x21 for 0 1.

(e) f (x1 , x2 = x1 x2
P
14. Show that the harmonic mean f (x) = ( ni=1 1/xi )1 is concave.

15. Show that the function f (x, t) = xT x/t on dom f = Rn R++ is convex in (x, t).
16. Show that the function f (x, s, t) = log(st xT x) is convex on dom f = {(x, s, t)
Rn+2 |st > xT x, s, t > 0}.
17. Show that the function f (x, t) = kxk33 /t2 is convex on {(x, t)|t > 0}.
18. Show that the function f (x) =

kAx+bk
cT x+d

is convex on {x|cT x + d > 0}.

19. Prove the following for non-differentiable functions f and g.

(a) If f and g are convex, both nondecreasing (or nonincreasing), and positive functions on
an interval, then f g is convex.
(b) If f , g are concave, positive, with one nondecreasing and the other nonincreasing, then
f g is concave.
(c) If f is convex, nondecreasing, and positive, and g is concave, nonincreasing, and positive,
then f /g is convex.
(d) If f is nonnegative and convex, and g is positive and concave, then f 2 /g is convex.
20. A function f (x) is log-concave if log(f (x)) is a concave function on {x|f (x) > 0}. Show
that the Gaussian cdf F (x) is log-concave for all x.
21. Show that the maximum of a convex function f over the box B := {xl x xu } is achieved
at one of its 2n vertices.
22. Using the following intermediate steps, show that the function f (x, t) = log(t2 xT x) over
dom f = {(x, t) Rn+1 |t > kxk2 } is convex.
(a) Show that t xT x/t is convex over dom f .

(b) Show that log(t xT x/t) is convex over dom f .

23. Show that the following function is convex on {x|kxk2 1}
f (x) =

kAx bk22
1 kxk22

(3)

24. Show that the weighted geometric mean

f (x) =

n
Y

xk k

(4)

i=1

with dom f = Rn++ is concave for k 0 and

25. Show that the Huber function
f (x) =
is convex on Rn .

Pn

i=1 k

kxk22 /2
kxk22 1/2

= 1.

kxk2 1
kxk2 > 1

(5)

EE609

Assignment 4

January 26, 2015

No submission required.

1 Assignment Problems
1. What is the solution of the following linear program
min cT x

(1)

s. t. 0 xi 1

i = 1, . . . , n.

(2)

2. Consider the following linear program

min cT x

(3)

s. t. Ax b

(4)

where A is square and full rank.

(a) When is the problem infeasible?
(b) When is the problem unbounded below?
(c) When does the problem have a finite solution, and what is it?
3. Show that any linear programming problem can be expressed as
min cT x

(5)

s. t. Ax = b

(6)

xi 0

i = 1, . . . , n

(7)

4. Solve the following quadratic optimization problem

min cT x

(8)

s. t. kxk2 1

(9)

5. Consider the sinusoidal measurement model in Gaussian noise described as,

y(n) = d + c cos(2f0 n) + s sin(2f0 n) + w(n), 0 n N 1,

(10)



where w(n) is additive white Gaussian noise and E |w(n)|2 = n2 . Answer the questions
that follow
(a) Formulate the LS estimation problem for parameters d , c , s .
(b) Derive the LS estimator of d , c , s with a suitable approximation for large N .

EE609

Assignment 4, Page 2 of 3

January 26, 2015

2 Practice Problems
6. Consider the following linear program
min cT x

(11)

s. t. Ax = b

(12)

(a) When is the problem infeasible?

(b) When is the problem unbounded below?
(c) When does the problem have a finite solution, and what is it?
7. Consider the following linear program
min cT x

(13)

s. t. a x b

(14)

where a 6= 0.
(a) When is the problem infeasible?
(b) When is the problem unbounded below?
(c) When does the problem have a finite solution, and what is it?
8. Given that ci 0 for i = 1, 2, . . . , n, solve the following problem
min cT x

(15)

s. t. i xi bi

i = 1, . . . , n

(16)

where i bi for all i = 1, 2, . . . , n.

9. What is the solution of the following linear program
min cT x
n
X
xi = 2
s. t.

(17)
(18)

i=1

0 xi 1

i = 1, . . . , n

(19)

10. What is the solution of the following linear program

min cT x
n
X
xi = 1
s. t.

(20)
(21)

i=1

xi 0

i = 1, . . . , n

(22)

11. Consider the two sets:

S1 = {v1 , v2 , . . . , vp }

(23)

S2 = {u1 , u2 , . . . , uq }

(24)

EE609

Assignment 4, Page 3 of 3

January 26, 2015

Formulate the linear optimization problem to determine the separating hyperplane, i.e., find
a Rn and b R such that
aT v i b

i = 1, . . . , p

(25)

aT u i b

i = 1, . . . , q

(26)

Ensure that your problem excludes the trivial solution a = b = 0.

12. Consider the following quadratic optimization problem
min cT x

(27)

s. t. x Ax 1

(28)

Let i for i = 1, 2, . . . n be the real eigenvalues of a symmetric matrix A. What is the solution
of this problem for the following cases.
(a) If i > 0 for i = 1, 2, . . . n.
(b) If 1 < 0, while i > 0 for i = 2, . . . n.
(c) If 1 = 0, while i > 0 for i = 2, . . . n.
13. Solve the following optimization problem for A 0,
min cT x

(29)
T

s. t. (x xc ) A(x xc ) 1

(30)

14. Solve the following optimization problem for A 0,

min xT Ax
s. t.

kxk22

(31)
(32)

Hint: Given that the eigenvalue decomposition A = UUT , use the change of variable
y = UT x.