Download Adequacy of Solutions - Numerical Analysis - Solved Exam and more Exams Mathematical Methods for Numerical Analysis and Optimization in PDF only on Docsity!
Chapter 04.
Adequacy of Solutions
After reading this chapter, you should be able to:
- know the difference between ill-conditioned and well-conditioned systems of
equations,
- define the norm of a matrix, and
- relate the norm of a matrix and of its inverse to the ill or well conditioning of the
matrix, that is, how much trust can you having in the solution of the matrix.
What do you mean by ill-conditioned and well-conditioned system of equations?
A system of equations is considered to be well-conditioned if a small change in the
coefficient matrix or a small change in the right hand side results in a small change in the
solution vector.
A system of equations is considered to be ill-conditioned if a small change in the
coefficient matrix or a small change in the right hand side results in a large change in the
solution vector.
Example 1
Is this system of equations well-conditioned?
y
x
Solution
The solution to the above set of equations is
y
x
Make a small change in the right hand side vector of the equations
04.09.2 Chapter 04.
y
x
gives
y
x
Make a small change in the coefficient matrix of the equations
y
x
gives
y
x
This last systems of equation “looks” ill-conditioned because a small change in the
coefficient matrix or the right hand side resulted in a large change in the solution vector.
Example 2
Is this system of equations well-conditioned?
y
x
Solution
The solution to the above equations is
y
x
Make a small change in the right hand side vector of the equations.
y
x
gives
y
x
Make a small change in the coefficient matrix of the equations.
y
x
gives
04.09.4 Chapter 04.
Solution
3
max
j
aij i
A
max 10 7 0 , 3 2. 099 6 , 5 1 5
max 10 7 0 , 3 2. 099 6 , 5 1 5
max 17 , 11. 099 , 11
How is the norm related to the conditioning of the matrix?
Let us start answering this question using an example. Go back to the ill-conditioned
system of equations,
y
x
that gives the solution as
y
x
Denoting the above set of equations as
A X C
X
C
Making a small change in the right hand side,
y
x
gives
y
x
Denoting the above set of equations by
A X ' C '
right hand side vector is found by
C C ' C
and the change in the solution vector is found by
X X ' X
then
Adequacy of Solution 04.09.
C
and
X
then
0. 001
C
X
The relative change in the norm of the solution vector is
X
X
The relative change in the norm of the right hand side vector is
4
- 250 10
C
C
See the small relative change of
4
- 250 10
in the right hand side vector results in a
large relative change in the solution vector as 2.9995.
In fact, the ratio between the relative change in the norm of the solution vector and the
relative change in the norm of the right hand side vector is
4
- 250 10
C C
X X
Let us now go back to the well-conditioned system of equations.
y
x
gives
Adequacy of Solution 04.09.
X
X
4 5 10
The relative change in the norm of the right hand side vector is
C
C
4
- 429 10
See the small relative change the right hand side vector of
4
- 429 10
results in the
small relative change in the solution vector of
4 5 10
.
In fact, the ratio between the relative change in the norm of the solution vector and the
relative change in the norm of the right hand side vector is
4 1429 10
4 5 10
/
/
. C C
X X
What are some of the properties of norms?
- For a matrix [ A ] , A 0
- For a matrix^ [ A ]^ and a scalar k,^ kA^ k A
- For two matrices [ A ] and [ B ] of same order, A B A B
- For two matrices [ A ] and [ B ] that can be multiplied as [ A ] [ B ], AB A B
Is there a general relationship that exists between X / X and C / C or
between X / X and A / A? If so, it could help us identify well-conditioned and
ill conditioned system of equations.
If there is such a relationship, will it help us quantify the conditioning of the matrix?
That is, will it tell us how many significant digits we could trust in the solution of a
system of simultaneous linear equations?
There is a relationship that exists between
04.09.8 Chapter 04.
C
C
X
X
and
and between
A
A
X
X
and
These relationships are
C
C
A A
X X
X
and
A
A
A A
X
X
The above two inequalities show that the relative change in the norm of the right hand
side vector or the coefficient matrix can be amplified by as much as
1 A A.
This number
1 A A is called the condition number of the matrix and coupled with the
machine epsilon, we can quantify the accuracy of the solution of [ A ] [ X ] [ C ].
Prove for
[ A ] [ X ][ C ]
that
A
A
A A
X X
X
Proof
Let
A X C (1)
Then if [ A ] is changed to A ' , the [ X ]will change to X ', such
that
A ' X ' C (2)
From Equations (1) and (2),
A X A ' X '
Denoting change in [ A ] and [ X ]matrices as A and X , respectively
A A ' A
X X ' X
04.09.10 Chapter 04.
2000 1000
A
A
1
A
1 Cond A A A
Assuming single precision with 24 bits used in the mantissa for real numbers, the
machine epsilon is
6
124
mach
6 ( ) 35990 0. 119209 10
Cond A mach
2
- 4290 10
Comparing it with
m
- 5 10 2
- 5 10 0. 4290 10
m
m 2
So two significant digits are at least correct in the solution vector.
Example 5
How many significant digits can I trust in the solution of the following system of
equations?
y
x
Solution
For
A
it can be shown
(^)
2 1
A
Then
5
A ,
Adequacy of Solution 04.09.
1
A.
1 Cond (A) A A
Assuming single precision with 24 bits used in the mantissa for real numbers, the
machine epsilon
124 2
mach
6
- 119209 10
6 ( ) 25 0. 119209 10
Cond A mach
5
- 2980 10
Comparing it with
m
- 5 10 5
- 5 10 0. 2980 10
m
m 5
So five significant digits are at least correct in the solution vector.
Key Terms:
Ill-Conditioned matrix
Well-Conditioned matrix
Norm
Condition Number
Machine Epsilon
Significant Digits
