Solving a system of equations

1
Solving a system of equations
1 x1 3 x2 6 x3 3 2 x1 4 x2 7 x3 2 3
x1 5 x2 8 x3 3
2
Solving a system of equations
1 x1 3 x2 6 x3 3 2 x1 4 x2 7 x3 2 3
x1 5 x2 8 x3 3
1
-2
1
0 2
no solution
3
Solving a system of equations
a11 x1 a12 x2 a13 x3 b1 a21 x1 a22 x2
a23 x3 b2 a31 x1 a32 x2 a33 x3 b3
A x b has a solution iff rank
(A) rank (A b)
4
Solving a system of equations
A x b has a solution iff
rank (A) rank (A b)
rank (A) lt rank (A b) then ? y such that
yT A 0 and yT b ? 0
5
Solving a system of equations
A x b has a solution iff
rank (A) rank (A b)
rank (A) lt rank (A b) then ? y such that
yT A 0 and yT b ? 0
y is a witness that A x b does not have a
solution
6
Solving a system of equations
yT A 0 and yT b ? 0
y is a witness that A x b does not have a
solution
A x b ? yT (A x) yT b ? (yT A) x yT b ? 0
yT b ? a contradiction
7
Solving a system of equations
Theorem If Axb doesnt have solution then ?
y such that yT A 0 and yT b ? 0
8
Solving a system of equations
1 x1 3 x2 6 x3 3 2 x1 4 x2 7 x3 2 3
x1 5 x2 8 x3 1
9
Solving a system of equations
1 x1 3 x2 6 x3 3 2 x1 4 x2 7 x3 2 3
x1 5 x2 8 x3 1
x1 -3 x2 2 x3 0
10
Back to linear programming
1 x1 3 x2 6 x3 3 2 x1 4 x2 7 x3 2 3
x1 5 x2 8 x3 1
x1 ? 0 x2 ? 0 x3 ? 0
I.e., we want a non-negative solution.
11
Back to linear programming
1 x1 3 x2 6 x3 3 2 x1 4 x2 7 x3 2 3
x1 5 x2 8 x3 1
0
-1
1
x1 x2 x3 -1
x1 ? 0 x2 ? 0 x3 ? 0
I.e., we want a non-negative solution.
12
Back to linear programming
yTA ? 0 and yTb lt 0
y is a witness that A x b, x ? 0 does not have
a solution
A x b ? yT(A x) yTb ? (yTA) x yTb ?
non-negative negative, a contradiction
13
Back to linear programming
Theorem (Farkas) If Axb, x ? 0 doesnt have
a solution then ? y such that yTA ? 0 and yTb lt
0
Theorem If Axb doesnt have solution then ?
y such that yTA 0 and yTb ? 0
14
Idea of the proof
Theorem (Farkas) If Axb, x ? 0 doesnt have
a solution then ? y such that yTA ? 0 and yTb lt
0
Ax, x ? 0
b
15
Idea of the proof
Sconvex, b not in S ? c such that (? x? S)cT x
gt cT b
Theorem (Farkas) If Axb, x ? 0 doesnt have
a solution then ? y such that yTA ? 0 and yTb lt
0
cTx
Ax, x ? 0
b
separating hyperplane
16
Theorem (Farkas) If Axb, x ? 0 doesnt have
a solution ? ? y such that yTA ? 0 and yTb lt 0
Duality
max cT x Axb x ? 0
min yT b yT A ? cT
17
? ? and non-negativity
a1 x1 ... an xn ? b
a1 x1 ... an xn b y, y ? 0
a1 x1 ... an xn y b, y ? 0
18
? ?
a1 x1 ... an xn b
a1 x1 ... an xn ? b a1 x1 ... an xn
? b
a1 x1 ... an xn ? b -a1 x1 - ... -
an xn ? -b
19
Duality
max c1Tx1c2Tx2 c3Tx3 c4Tx4 A1 x1b1
A2 x2 ? b2 A3 x3 b3 A4 x4 ? b4
x1 ? 0,x2 ? 0
min y1Tb1y2Tb2y3Tb3 y4Tb4 y1T A1 ? c1T
y2T A2 ? c2T y3T A3 c3T y4T A4
c4T y2? 0, y4? 0
20
Solving linear programs
Simplex (Danzig, 40s) Ellipsoid (Khachiyan,
80s) Interior point (Karmakar, 80s)