Download Gaussian Elimination - Numerical Analysis - Solved Exam and more Exams Mathematical Methods for Numerical Analysis and Optimization in PDF only on Docsity!
Chapter 04.
Gaussian Elimination
After reading this chapter, you should be able to:
- solve a set of simultaneous linear equations using Naïve Gauss elimination,
- learn the pitfalls of the Naïve Gauss elimination method,
- understand the effect of round-off error when solving a set of linear equations withthe Naïve Gauss elimination method,
- learn how to modify the Naïve Gauss elimination method to the Gaussian elimination with partial pivoting method to avoid pitfalls of the former method,
- find the determinant of a square matrix using Gaussian elimination, and
- understand the relationship between the determinant of a coefficient matrix and thesolution of simultaneous linear equations.
How is a set of equations solved numerically? One of the most popular techniques for solving simultaneous linear equations is the Gaussian elimination method. The approach is designed to solve a general set of n equations and n unknowns a 11 (^) x 1 + a 12 x 2 + a 13 x 3 +... + a 1 n xn = b 1 a 21 (^) x 1 + a 22 x 2 + a 23 x 3 +... + a 2 n xn = b 2
.. .. .. an (^) 1 x 1 + an 2 x 2 + an 3 x 3 +...+ annxn = b n Gaussian elimination consists of two steps
- Forward Elimination of Unknowns: In this step, the unknown is eliminated in each equation starting with the first equation.equation and one unknown in each equation. This way, the equations are reduced to one
- Back Substitution: In this step, starting from the last equation, each of the unknowns is found. Forward Elimination of Unknowns: In the first step of forward elimination, the first unknown, x 1 is eliminated from all rows below the first row. The first equation is selected as the pivot equation to eliminate (^) x 1. So,
04.06.2 Chapter 04.
to eliminate x 1 in the second equation, one divides the first equation by a 11 (hence called the pivot element) and then multiplies it by a 21. This is the same as multiplying the first equation by a (^) 21 / a 11 to give a 21 (^) x 1 + aa 1121 a 12 x 2 +... + aa 1121 a 1 n xn = aa 1121 b 1 Now, this equation can be subtracted from the second equation to give
^ a^22^^ − aa 1121 a^12 x^2 +...^ +^ a^2 n^ − aa 1121 a^1 n xn = b^2 − aa 1121 b^1 or
where a^22^ ′^ x^2 +...^ + a^2 ′ n^ xn = b^2 ′
an an aa a n
a a aa a
2 2 1121 1
22 22 1121 12
′ = −
This procedure of eliminating x 1 , is now repeated for the third equation to the n th equation to reduce the set of equations as a 11 (^) x 1 + a 12 x 2 + a 13 x 3 +... + a 1 n xn = b 1 a 22 ′ x 2 + a 23 ′ x 3 +... + a 2 ′ n xn = b 2 ′ a 32 ′ x 2 + a 33 ′ x 3 +... + a 3 ′ n xn = b 3 ′
... ... ...
This is the end of the first step of forward elimination. Now for the second step of forward an^ ′^^^2 x^2 + an ′^3 x^3 +...+ ann ′ xn = b^ n ′ elimination, we start with the second equation as the pivot equation and a 22 ′ as the pivot element. So, to eliminate x (^) 2 in the third equation, one divides the second equation by a 22 ′ (the pivot element) and then multiply it by (^) a 32 ′. This is the same as multiplying the second equation by (^) a 32 ′ (^) / a 22 ′ and subtracting it from the third equation. This makes the coefficient of x 2 zero in the third equation. The same procedure is now repeated for the fourth equation till the n thequation to give a 11 (^) x 1 + a 12 x 2 + a 13 x 3 +... + a 1 n xn = b 1 a 22 ′ x 2 + a 23 ′ x 3 +... + a 2 ′ n xn = b 2 ′ a. 33 ′ ′ x 3 +... +. a 3 ′′ n xn = b 3 ′′
.. .. an ′′^ 3 x 3 +...+ ann ′′ xn = b n ′′
04.06.4 Chapter 04.
3
2
1 a
a
a
Find the values of a 1 (^) , a 2 , and a 3 using the Naïve Gauss elimination method. Find the velocity at t = 6 , 7. 5 , 9 , 11 seconds.
Solution Forward Elimination of Unknowns Since there are three equations, there will be two steps of forward elimination of unknowns. First step Divide Row 1 by 25 and then multiply it by 64, that is, multiply Row 1 by 64/25 = 2.56.
( [ 25 5 1 ] [ 106. 8 ]) × 2. 56 gives Row 1 as
[ 64 12. 8 2. 56 ] [ 273. 408 ]
Subtract the result from Row 2
[ ] [ ]
[ ] [ ]
to get the resulting equations as
3
2
1 a
a
a
Divide Row 1 by 25 and then multiply it by 144, that is, multiply Row 1 by 144/25 = 5.76.
( [ 25 5 1 ] [ 106. 8 ]) × 5. 76 gives Row 1 as
[ 144 28. 8 5. 76 ] [ 615. 168 ]
Subtract the result from Row 3
[ ] [ ]
[ ] [ ]
to get the resulting equations as
3
2
1 a
a
a
Second step We now divide Row 2 by –4.8 and then multiply by –16.8, that is, multiply Row 2 by − 16.8/ −4.8= 3.5.
( [ 0 − 4. 8 − 1. 56 ] [− 96. 208 ]) × 3. 5 gives Row 2 as
[ 0 − 16. 8 − 5. 46 ] [ − 336. 728 ]
Subtract the result from Row 3
Gaussian Elimination 04.06.
[ ] [ ]
[ ] [ ]
to get the resulting equations as
3
2
1 a
a
a
Back substitution From the third equation
- 7 a 3 = 0. 76 07
a =. = 1. Substituting the value of a 3 in the second equation, − 4. 8 a 2 − 1. 56 a 3 =− 96. 208
- 8
2 96.^2081.^563
a = − + a
=− + ×
Substituting the value of a 2 and a 3 in the first equation, 25 a 1 + 5 a 2 + a 3 = 106. 8 25 a 1 =^106.^8 −^5 a^2 −^ a^3 25
=^106.^8 −^5 ×^19.^6905 −^1.^08571
Hence the solution vector is
3
2
1 a
a
a
The polynomial that passes through the three data points is then
v ( ) t = a 1 t^2 + a 2 t + a 3
= 0. 290472 t^2 + 19. 6905 t + 1. 08571 , 5 ≤ t ≤ 12 Since we want to find the velocity at t = 6 , 7. 5 , 9 and 11 seconds, we could simply substitute
each value of t in v ( ) t = 0. 290472 t^2 + 19. 6905 t + 1. 08571 and find the corresponding
velocity. For example, at t = 6
Gaussian Elimination 04.06.
Subtract the result from Row 2
[ ] [ ]
[ ] [ ]
to get the resulting equations as
3
2
1 x
x
x
Divide Row 1 by 20 and then multiply it by 5, that is, multiply Row 1 by 5 / 20 = 0. 25
( [ 20 15 10 ] [ 45 ]) × 0. 25 gives Row 1 as
[ 5 3. 75 2. 5 ] [ 11. 25 ] Subtract the result from Row 3
[ ] [ ]
[ ] [ ]
to get the resulting equations as
3
2
1 x
x
x
Second step Now for the second step of forward elimination, we will use Row 2 as the pivot equation and eliminate Row 3: Column 2. Divide Row 2 by 0.001 and then multiply it by –2.75, that is, multiply Row 2 by − 2. 75 / 0. 001 =− 2750.
( [ 0 0. 001 8. 5 ] [ 8. 501 ]) × − 2750 gives Row 2 as
[ 0 − 2. 75 − 23375 ] [− 23377. 75 ]
Rewriting within 6 significant digits with chopping
[ 0 − 2. 75 − 23375 ] [− 23377. 7 ]
Subtract the result from Row 3
[ ] [ ]
[ ] [ ]
Rewriting within 6 significant digits with chopping
[ 0 0 23375. 5 ] [ − 23375. 4 ]
to get the resulting equations as
3
2
1 x
x
x
This is the end of the forward elimination steps.
04.06.8 Chapter 04.
Back substitution We can now solve the above equations by back substitution. From the third equation,
- 5 x 3 = 23375. 4
- 5 x 3 =^23375.^4 = 0. 999995 Substituting the value of x (^) 3 in the second equation
- 001 x 2 + 8. 5 x 3 = 8. 501
2 8.^5018.^53
= −^ ×
x x
=^8.^501 −^8.^49995
=^0.^00105
Substituting the value of x (^) 3 and x 2 in the first equation, 20 x 1 + 15 x 2 + 10 x 3 = 45 20 x 1 =^45 −^15 x^2 −^10^ x^3 20
=^45 −^15 ×^1.^05 −^10 ×^0.^999995
Hence the solution is
3
2
1 [ ] x
x
x X
Compare this with the exact solution of
04.06.10 Chapter 04.
So it is important to consider that the possibility of division by zero can occur at the beginning of any step of forward elimination. Round-off error: The Naïve Gauss elimination method is prone to round-off errors. This is true when there are large numbers of equations as errors propagate. Also, if there is subtraction of numbers from each other, it may create large errors. See the example below. Example 3 Remember Example 2 where we used Naïve Gauss elimination to solve 20 x 1 + 15 x 2 + 10 x 3 = 45 − 3 x 1 − 2. 249 x 2 + 7 x 3 = 1. 751 5 x 1 + x 2 + 3 x 3 = 9 using six significant digits with chopping in your calculations? Repeat the problem, but now use five significant digits with chopping in your calculations. Solution Writing in the matrix form
3
2
1 x
x
x
Forward Elimination of Unknowns First step Divide Row 1 by 20 and then multiply it by –3, that is, multiply Row 1 by − 3 / 20 =− 0. 15.
( [ 20 15 10 ] [ 45 ]) × − 0. 15 gives Row 1 as
[ − 3 − 2. 25 − 1. 5 ] [ − 6. 75 ]
Subtract the result from Row 2
[ ] [ ]
[ ] [ ]
to get the resulting equations as
3
2
1 x
x
x
Divide Row 1 by 20 and then multiply it by 5, that is, multiply Row 1 by 5 / 20 = 0. 25.
( [ 20 15 10 ] [ 45 ]) × 0. 25 gives Row 1 as
[ 5 3. 75 2. 5 ] [ 11. 25 ] Subtract the result from Row 3
[ ] [ ]
[ ] [ ]
Gaussian Elimination 04.06.
to get the resulting equations as
3
2
1 x
x
x
Second step Now for the second step of forward elimination, we will use Row 2 as the pivot equation andeliminate Row 3: Column 2. Divide Row 2 by 0.001 and then multiply it by –2.75, that is, multiply Row 2 by − 2. 75 / 0. 001 =− 2750.
( [ 0 0. 001 8. 5 ] [ 8. 501 ]) × − 2750 gives Row 2 as
[ 0 − 2. 75 − 23375 ] [− 23377. 75 ]
Rewriting within 5 significant digits with chopping
[ 0 − 2. 75 − 23375 ] [− 23377 ]
Subtract the result from Row 3
[ ] [ ]
[ ] [ ]
Rewriting within 6 significant digits with chopping
[ 0 0 23375 ] [ − 23374 ]
to get the resulting equations as
3
2
1 x
x
x
This is the end of the forward elimination steps. Back substitution We can now solve the above equations by back substitution. From the third equation, 23375 x 3 = 23374 23375 x 3 =^23374 = 0. 99995 Substituting the value of x (^) 3 in the second equation
- 001 x 2 + 8. 5 x 3 = 8. 501
2 8.^5018.^53
= −^ ×
x x
=^ −
Gaussian Elimination 04.06.
How does Gaussian elimination with partial pivoting differ from Naïve Gauss elimination? The two methods are the same, except in the beginning of each step of forward elimination, a row switching is done based on the following criterion. If there are n equations, then there areone finds the maximum of n − 1 forward elimination steps. At the beginning of the k th step of forward elimination,
a kk , a (^) k + 1 , k , …………, ank Then if the maximum of these values is a (^) pk in the p th row, k ≤ p ≤ n , then switch rows p and The other steps of forward elimination are the same as the Naïve Gauss elimination method. k. The back substitution steps stay exactly the same as the Naïve Gauss elimination method. Example 4 In the previous two examples, we used Naïve Gauss elimination to solve 20 x 1 + 15 x 2 + 10 x 3 = 45 − 3 x 1 − 2. 249 x 2 + 7 x 3 = 1. 751 5 x 1 + x 2 + 3 x 3 = 9 using five and six significant digits with chopping in the calculations. Using five significant digits with chopping, the solution found was
[ ]
3
2
1 x
x
x X
This is different from the exact solution of
[ ]
3
2
1 x
x
x X
Find the solution using Gaussian elimination with partial pivoting using five significant digits with chopping in your calculations. Solution
3
2
1 x
x
x
04.06.14 Chapter 04.
Forward Elimination of Unknowns Now for the first step of forward elimination, the absolute value of the first column elementsbelow Row 1 is
20 , − 3 , 5 or 20, 3, 5 So the largest absolute value is in the Row 1. So as per Gaussian elimination with partial pivoting, the switch is between Row 1 and Row 1 to give
3
2
1 x
x
x
Divide Row 1 by 20 and then multiply it by –3, that is, multiply Row 1 by − 3 / 20 =− 0. 15.
( [ 20 15 10 ] [ 45 ]) × − 0. 15 gives Row 1 as
[ − 3 − 2. 25 − 1. 5 ] [ − 6. 75 ]
Subtract the result from Row 2
[ ] [ ]
[ ] [ ]
to get the resulting equations as
3
2
1 x
x
x
Divide Row 1 by 20 and then multiply it by 5, that is, multiply Row 1 by 5 / 20 = 0. 25.
( [ 20 15 10 ] [ 45 ]) × 0. 25 gives Row 1 as
[ 5 3. 75 2. 5 ] [ 11. 25 ] Subtract the result from Row 3
[ ] [ ]
[ ] [ ]
to get the resulting equations as
3
2
1 x
x
x
This is the end of the first step of forward elimination. Now for the second step of forward elimination, the absolute value of the second column elements below Row 1 is
- 001 , − 2. 75 or 0.001, 2.
04.06.16 Chapter 04.
20
=^45 −^15 ×^1 −^10 ×^1
=^ −
=^20
So the solution is=^1
[ ]
3
2
1 x
x
x X
This, in fact, is the exact solution.fully removed. By coincidence only, in this case, the round-off error is
Can we use Naïve Gauss elimination methods to find the determinant of a square matrix? One of the more efficient ways to find the determinant of a square matrix is by takingadvantage of the following two theorems on a determinant of matrices coupled with Naïve Gauss elimination.
Theorem 1: Let [ A ] be a n × n matrix. Then, if [ B ] is a n × n matrix that results from adding or subtracting a multiple of one row to another row, then det( A ) = det( B )(The same is true for column operations also).
Theorem 2: Let [ A ] be a n × n matrix that is upper triangular, lower triangular or diagonal, then det( A )= a 11 × a 22 ×...× aii ×...× a nn
n i 1 aii This implies that if we apply the forward elimination steps of the Naïve Gauss eliminationmethod, the determinant of the matrix stays the same according to Theorem 1. Then since at the end of the forward elimination steps, the resulting matrix is upper triangular, the determinant will be given by Theorem 2.
Gaussian Elimination 04.06.
Example 5 Find the determinant of
[ A ]
Solution Remember in Example 1, we conducted the steps of forward elimination of unknowns using the Naïve Gauss elimination method on (^) [ A ] to give
[ ]
B
According to Theorem 2 det( A ) =det( B ) = 25 ×(− 4. 8 )× 0. 7 =− 84. 00 What if I cannot find the determinant of the matrix using the Naïve Gauss elimination method, for example, if I get division by zero problems during the Naïve Gauss elimination method? Well, you can apply Gaussian elimination with partial pivoting. However, the determinant of the resulting upper triangular matrix may differ by a sign. The following theorem applies in addition to the previous two to find the determinant of a square matrix. Theorem 3: Let (^) [ A ] be a (^) n × n matrix. Then, if (^) [ B ] is a matrix that results from switching one row with another row, then det( B ) = −det( A ).
Example 6 Find the determinant of
[ A ]
Solution The end of the forward elimination steps of Gaussian elimination with partial pivoting, we would obtain
[ B ]
det (^ B )^ = 10 × 2. 5 × 6. 002
