ABSTRACT
The main goal of this manuscript is to introduce a discrete dynamical system defined by symmetric matrices and a real parameter. By construction, we rediscovery the Power Iteration Method from the Projected Gradient Method. Convergence of the discrete dynamical system solution is established. Finally, we consider two applications, the first one consists in find a solution of non linear equation problem and the other one consists in verifies the optimality conditions when we solve quadratic optimization problems over linear equality constraints.
Keywords:
Projected gradient type-method; Power Iteration Method; Symmetric matrix; Discrete Dynamical System
1 INTRODUCTION
Discrete dynamical system appears as a tool in order to understand differential equations from numerically view point (for more details, see Galob (20073 GALOB O. 2007. Discrete Dynamical Systems. Springer Verlag, Berlin Heidelberg.) and chapter 6 in Loneli & Rumbos (20037 LONELI OHE & RUMBOS PIB. 2003. Métodos dinámicos en Economia. Otra busqueda del tiempo perdido. International Thompson Editores, Mexico.)). The classical model, in finite dimensional space, is as follows:
where is an operator and Ω is a subset of the domain of the operator F (the domain of F is denoted by ). According to the literature, the equation 1 is not exclusive for differential equations, for example it appears in order to find fixed points for contractive operators (remember, F is contractive if , with and ). For details about contractive operators, see classical books in functional analysis or general topology or fixed point theorems as for instance Brezis (19831 BREZIS H. 1983. Analyse fonctionnelle - Théorie et applications. Masson, Paris.), Istrǎţescu (19815 ISTRǍŢESCU VI. 1981. Fixed point theory. D. Reidel Publishing Co., Dordrecht.), Kelley (19556 KELLEY JL. 1955. General Topology. Van Nostrand Co.. Princeton.). Other example is the autoregressive model (for more details see Shumway & Stoffer (20179 SHUMWAY RH & STOFFER DS. 2017. Time series analysis and its applications. Springer International Publishing AG.)).
Given a symmetric matrix A and a real number λ such that . We consider the following operator defined by
where I is the identity matrix, and is the eigenvalue set of matrix A.Note that, in this setting, is a non singular matrix. So, operator Tλ is well defined.
The focus of this manuscript is the operator defined by the equation 2, which is very interesting, because:
-
when, either or A is the null matrix, the operator Tλ is the identity. Here, every is a fixed point of Tλ.
-
when has a dominant eigenvalue (i.e there exists an eigenvalue α* such that for all eigenvalue ), the operator T was used in the famous Power Iteration Method introduced by R. Von Mises and H. Pollaczek-Geiringer in 19298 MISES RV & POLLACZEK-GEIRINGER H. 1929. Praktische Verfahren der Gleichungsauflösung. ZAMM - Zeitschrift für Angewandte Mathematik und Mechanik, 9: 152-164. (see Mises & Pollaczek-Geiringer (1929)).
-
when , where and n is the size of is a strong monotone operator. Moreover, each eigenvector of A belonging to S is a fixed point of T (we prove it in section 2).
1.1 The Power Iteration Method
In order to understand the Power Iteration Method, consider a function called “Rayleigh quotient” which is defined, as follows, for each
If x is an eigenvector, then (i.e. r(x) is the corresponding eigenvalue of x). Suppose that is a set of eigenvectors of B which is a basis of and . Taking a vector with , we have that .
Then,
and so
Here, converges to v 1, because . The Power Iteration method is elegant, simple and can be stated as follows
but, convergence is only guaranteed if the following two assumptions hold:
-
Non singular matrix B has an eigenvalue that is strictly greater in absolute value than its other eigenvalues.
-
The starting vector x0 has a nonzero component in direction of an eigenvector associated with the dominant eigenvalue.
The reader can verify that for (matrix B is nonsingular and symmetric), the sequences generated by Power Iteration Method diverge for any stated point x (different to any eigenvector of B), because does not have a dominant eigenvalue, but is positive definite (all its eigenvalues are strictly positive) and it has a dominant eigenvalue.
1.2 The Projected Gradient Method
The Projected Gradient method was introduced by Goldstein (for more detail see Goldstein (19644 GOLDSTEIN AA. 1964. Convex programmin in Hilbert space. Bulletin of The American Mathematical Society, 70: 709-710.)) for solving the following differentiable optimization problem
where is differentiable in each point of a nonempty closed subset C of .
The essence of the Projected Gradient method is based on two facts:
-
First order Necessary Optimality condition: If is a solution of P, then (Here is the tangent cone of C at , for more details see Crouzeix et al. (20112 CROUZEIX J, KERAGEL A & SOSA W. 2011. Programación Matematica diferenciable. Universidad Nacional de Ingenieria.))
-
Orthogonal projection: If u is an orthogonal projection of v over C, then (in short ).
Now, given a symmetric matrix A, we know that all eigenvalues of A are real numbers and we can consider n eigenvectors of matrix A as a basis of . Moreover if we consider problem (P) with defined by and , where F is a subspace of generated by eigenvectors of matrix A. The optimal value is an eigenvalue of A and any maximizer is a normalized eigenvector associated to the optimal value.
Here, if , then is an hyperplane in F defined by normal vector and contain vector (from now on ). So, the necessary condition is reduced to , which is equivalent to and fixed, which is also equivalent to fixed.
So,
If we define , then both matrices A and B have the same eigenvectors. Moreover, δ is an eigenvalue of A and u an associate eigenvector () if and only if if and only if if and only if is an eigenvalue of B and u an associate eigenvector to it. In the next section we introduce an easy result which establishes that for each symmetric matrix A and each such that for all eigenvalue δ of A, we have that is a Symmetric Positive Definite (SPD) matrix. It implies that , for all eigenvalue δ of A.
In the section 2, we introduce a discrete dynamical system defined by symmetric matrices and a real parameter λ. We show that, under some conditions on the parameter λ, any sequence generated by the discrete dynamical system converges to a fixed point of the operator which define the discrete dynamical system. Moreover, there is an equivalence between the fixed point of the operator and the eigenvector of the symmetric matrix.
In section 3 we consider two applications, the first one consists in find a solution for the non linear equation problem and the second one consists in verifies the optimality conditions when we solve quadratic optimization problems over linear equality constraints.
2 A DISCRETE DYNAMICAL SYSTEM
We start this section with two elementary results.
Lemma 1. Let A be a no null symmetric matrix with size n and . If with , then .
Proof. Take and consider an eigenvector such that . Then,
So, we have that
But (follows directly applying optimality conditions). It implies that, . And so the statement follows. □
Note that the eigenvalue set σ(A) of A exists, but its elements are unknown explicitly in the previous Lemma.
Lemma 2. If A is a symmetric matrix and , then for all with , then matrix is a SPD matrix. Moreover, for each we have that and .
Proof. If , then
But, . So, . Then, the statement follows. □
From now, for each non null symmetric matrix A, define the following operator by
where is such that and .
Theorem 1.Let A be a non null symmetric matrix andsuch that. A vector x∗ is a fixed point of Tλ if and only if there existssuch thatand.
Proof. If x∗ is a fixed point of . So, and . Let and let be an eigenvector set of B such that . Here, is a basis of , then , where . Note that , because . On the other hand . Since is a basis of , then . It implies that . So, . Finally,
The statement follows from Lemma 2, taking .
If there exists such that and , then . The statement follows because . □
Now, we are able to find the solution of the discrete dynamical system, for each non null symmetric matrix A and , where and .
Initial step Given a non null symmetric matrix A.
Iterative step Calculate:
The following result establishes that the sequence generated by the discrete dynamical system (the solution of the discrete dynamical system) is asymptotically stable for any starting point .
Theorem 2.Let A be a non null symmetric matrix. For each, the sequence {xk} generated by the discrete scheme converges to an eigenvector of A belonging to S and the sequenceconverges to its respective eigenvalue.
Proof. From Lemma 1, we have that B is a SPD matrix. It implies that for all and so is well defined for all . Let be the eigenvalue set of B and a respective eigenvector set. Without loss of generality consider . Since is an orthonormal basis of , then . Here, , where . Taking and , then . It implies that . Note that for any we have that . It implies that, the sequence {xk} converges to . It is easy to verify that the cluster point is a normalized eigenvector of B associated to an eigenvalue λj. Since A and B have the same eigenvectors set, then the statement follows. □
3 APPLICATIONS
In this section we consider two applications.
3.1 The Non Linear Equation Problem
This problem consists in find a feasible point of a nonlinear equation defined by a function (here, f is twice differentiable on ) and . The Mathematical Model is:
Take , the representation of Taylor around x is
Taking with , we have that
Using this approach, the problem consists in find a direction d such that the function defined by has at least one real roots.
Definition 1. Given be a function twice differentiable, let . A vector is called a feasible direction for the problem (NLEP), if the function h defined by has at least one real root.
We point out, in the case that f be a linear function. Here for . In
this case, . Note that is a feasible direction. Moreover, is the orthogonal projection of x over the hyperplane , where .
Lemma 3. Let be a quadratic function and . The following statement follows:
-
The matrix is symmetric.
-
If , then the problem (NLEP) has no solution.
-
If , then the problem (NLEP) has at least one solution. Moreover, any eigenvector associated to positive eigenvector, is a feasible direction.
-
If d is an eigenvector associated to null eigenvalue and , then d is not a feasible direction when .
-
If d is an eigenvector associated to null eigenvalue and , then d is a feasible direction.
Proof. If f is a quadratic function, then
So, the equation has solution if the discriminant
All items follows because the discriminant need to be non negative, in order to find real roots of the quadratic equation. □
3.2 Linearly Constrained Quadratic Programming Problems
This problem can be formulated as follows:
without loss of generality Q is a non null symmetric matrix, P is a non null matrix, and .
For the next result, we use the following notation: is the eigenvalue set of matrix PT P, an eigenvector set and is the subspace generated by . By convention .
Lemma 4. If P is a non null matrix with size , then
Proof. If , then PT P is non singular and then and so . If not, take , then and so PT. It implies that h is an eigenvector of PT P and so . Conversely, if , then . Hence, PT. Then and so . □
Now, consider a matrix V such that is the orthogonal projection over . Matrix V can be calculated as follows: Apply our scheme and obtain and such that . Then , where . The following result is important, because we can verify the condition , verifying that VT QV is semi definite positive (i. e. all its eigenvalues of VQV are nonnegative real values).
Corollary 1. if and only if VQV is symmetric semi definite positive.
For the next result, consider and defined by .
Theorem 3. The problem LCQPP has a solution if and only if
where and . Moreover, if , then is a solution of LCQPP.
Proof. If is a solution of LCQP, then the KKT optimality conditions imply that and . So for . The first order necessary optimality
condition tells us that (because the tangent cone of L in the point is equal to kernel of P, denoted by ). So, for any and and so .
Now, if and , then taking we claim that is a solution of LCQP. Indeed, implies that and . Defining , we need to show that for all x such that .
Note that for we have that . So . It implies that and the claim follows. □
4 NUMERICAL EXPERIMENTS
Here, we show numerical experiments using a program code written in SciLab software. Of course, this program code is very simple and developed by an amateur in Computer Science (Wilfredo Sosa).
The following numerical experiment concern to verify optimality condition when we solve linearly constrained quadratic programming problems. If
Running our program code in SciLab, we obtain the spectral set and the respective eigenvector set is
By definition
Again applying our program code to we have that
It implies that, the vector
is solution of , and so is solution of LCQP ().
Also, we applied our scheme for find eigenvalues and eigenvectors of symmetric matrixes. We simulate symmetric matrices and then calculate their eigenvalues and eigenvectors using our program code. We build symmetric matrices as follows: Given each two matrices (data), the first one D is a diagonal matrix and the second one V is an unitary matrix (i.e. ), then we define , here diagonal entries of D are the eigenvalues of A and the column vectors of V are eigenvectors of A. We generate V using Gram-Schimidt process.
-
The first matrix was built with 10 eigenvalues equal to -30; 10 eigenvalues equal to zero; and 10 eigenvalues equal to 30.
-
The second matrix was built with 20 eigenvalues equal to -2000; 20 eigenvalues equal to zero; and 20 eigenvalues equal to 2000.
-
The next matrix was built with eigenvalues following the rule , starting with until and .
-
The next matrix was built with 20 eigenvalues equal to -3000; 10 eigenvalues equal to -3; 20 eigenvalues equal to zero; 10 eigenvalues equal to 3; and 20 eigenvalues equal to 3000.
-
The next matrix was built with 50 eigenvalues equal to -100 and 50 eigenvalues equal to 100.
Of course, our scheme finds all eigenvalues and a eigenvector set. Unfortunately, the Power Method Iteration does not run for generated matrices, because the first four matrices has null eigenvalues and the last one has as absolute value of all eigenvalues equal to 100 (it is not dominant eigenvalue matrix).
ACKNOWLEDGEMENTS
Wilfredo Sosa was partially supported by Fundação de Apoio à Pesquisa do Distrito Federal (FAP-DF) [grant 0193.001695/2017 and 00193.00002100/2018-51 and CNPq [Grants 302074/2012-0 and 471168/2013-0]. Part of this research was carried out during visits to IMPA.
References
-
1BREZIS H. 1983. Analyse fonctionnelle - Théorie et applications. Masson, Paris.
-
2CROUZEIX J, KERAGEL A & SOSA W. 2011. Programación Matematica diferenciable. Universidad Nacional de Ingenieria.
-
3GALOB O. 2007. Discrete Dynamical Systems. Springer Verlag, Berlin Heidelberg.
-
4GOLDSTEIN AA. 1964. Convex programmin in Hilbert space. Bulletin of The American Mathematical Society, 70: 709-710.
-
5ISTRǍŢESCU VI. 1981. Fixed point theory. D. Reidel Publishing Co., Dordrecht.
-
6KELLEY JL. 1955. General Topology. Van Nostrand Co.. Princeton.
-
7LONELI OHE & RUMBOS PIB. 2003. Métodos dinámicos en Economia. Otra busqueda del tiempo perdido. International Thompson Editores, Mexico.
-
8MISES RV & POLLACZEK-GEIRINGER H. 1929. Praktische Verfahren der Gleichungsauflösung. ZAMM - Zeitschrift für Angewandte Mathematik und Mechanik, 9: 152-164.
-
9SHUMWAY RH & STOFFER DS. 2017. Time series analysis and its applications. Springer International Publishing AG.
-
1
Mathematics Subject Classification (2000): 15A18, 90C52, 90C20
APPENDIX
In this section we present a program code of our scheme written in SciLab Software. Of course, this program code is very simple and we do not use numerical strategies in order to reduce the time of compilation or reduce the accumulation of errors. For theses reason, we do not compare our program code with others in the literature, because it is not our subject. Criterions for the program code are the following:
1. Try to find a great eigenvalue in absolute value, for do it we find j such that .
2. For build a matrix B, we consider and .
3. If , then, we consider matrix A as a null matrix.
4. The error to find an eigenvalue will be less to 10−16.
The following function calculate an eigenvector of a symmetric matrix A.
The main part of the code is the following. Of course, it is necessarily read a matrix DC.
Publication Dates
-
Publication in this collection
2 Dec 2019 -
Date of issue
Sep-Dec 2019
History
-
Received
28 Nov 2018 -
Accepted
11 Sept 2019