Convergence Properties of Dynamic String Averaging Projection Methods in the Presence of Perturbations

Assuming that the absence of perturbations guarantees weak or strong convergence to a common fixed point, we study the behavior of perturbed products of an infinite family of nonexpansive operators. Our main result indicates that the convergence rate of unperturbed products is essentially preserved in the presence of perturbations. This, in particular, applies to the linear convergence rate of dynamic string averaging projection methods, which we establish here as well. Moreover, we show how this result can be applied to the superiorization methodology.


Introduction
For a given common fixed point problem, that is, find x ∈ ∞ k=0 Fix T k , we consider an iterative scheme of the following form: x 0 ∈ H and x k+1 := T k x k , (1.1) where each T k : H → H, k = 0, 1, 2, . . ., is a nonexpansive operator and H is a Hilbert space. We recall that T : H → H is nonexpansive if T x−T y ≤ x −y for all x, y ∈ H. Scheme (1.1) is called perturbation resilient if under the assumption that it generates a sequence converging to a solution of the problem, it follows that any sequence {y k } ∞ k=0 satisfying y 0 ∈ H and y k+1 − T k y k ≤ e k , (1.2) where {e k } ∞ k=0 is a suitable sequence of errors, also converges to a (possibly different) solution of the problem.
Perturbation resilience of iterative projection methods was first studied in [21], where the sequence of errors is assumed to be summable. In [12], both weak and strong convergence properties of infinite products of nonexpansive operators in the presence of summable perturbations are considered. As an example of a result on perturbation resilience with respect to weak and strong convergence, we recall a variant of [12,Theorems 3.2 and 5.2] in the Hilbert space setting.
Theorem 1 For every k = 0, 1, 2, . . ., let T k : H → H be nonexpansive and assume that C ⊆ k Fix T k is nonempty, closed, and convex. Let {e k } ∞ k=0 ⊆ [0, ∞) be a given summable sequence of errors and let {x k } ∞ k=0 ⊆ H be an inexact trajectory such that x k+1 − T k x k ≤ e k (1.3) holds for each k = 0, 1, 2, . . .. Assume that for all i = 0, 1, 2, . . ., there is an x ∞ i ∈ C such that T k · · · T i x i x ∞ i . (1.4) Then there is a point x ∞ ∈ C such that x k x ∞ . If, in addition, for all i = 0, 1, 2, . . ., Clearly, assumptions (1.4) and (1.5) fit into the perturbation resilience paradigm. The interpretation of these conditions is that if at some point, say i, we interrupt the perturbed process {x k } ∞ k=0 and begin exact computations starting from x i , then our method will converge. We note here that originally Theorems 3.2 and 5.2 of [11] were formulated with stronger conditions in complete metric and Banach spaces, respectively. Nevertheless, the proofs remain to be true if one assumes only (1.4) and (1. The assumption of summable errors seems to be common for a variety of iterative schemes. For example, in [35], perturbation resilience with respect to summable errors of an iterative scheme for finding zeros of an accretive operator is considered. On the other hand, summability of errors is a basic assumption for the quasi-Fejér monotone sequences which constitute another important tool in the study of numerical robustness; see, for example, [22,23]. Besides taking into account computational errors, the main reason for the interest in perturbation resilience is the superiorization methodology. Roughly speaking, the idea behind this methodology is to use perturbation resilience of an iterative method to introduce perturbations which steer the sequence towards a limit which not only solves the original problem but should also be superior with respect to the solution obtained without perturbations. In this context, iterative schemes of the form x 0 ∈ H and x k+1 := T k x k − β k v k , (1.6) are considered where the sequence {β k } ∞ k=0 ⊂ (0, ∞) is summable and the steering sequence {v k } ∞ k=0 is bounded. In practice, each v k is somehow related to the subgradient of a certain convex, continuous function φ : H → R, for example, v k ∈ ∂φ(x k ), whereas "superior" is interpreted as a smaller value of φ. For an introduction to superiorization, we refer the interested reader to [15]. Applications of the superiorization methodology include optimization, see, for example, [17], and image reconstruction, see, for example, [16,28,31,32,42].
String-averaging projection methods have been introduced in [18] for solving the convex feasability problem, that is, given closed and convex sets C i , i = 1, . . . , M, such that C := M i=1 C i is nonempty, find a point x ∞ ∈ C. We denote by P C i the metric projection onto C i , that is, the mapping which maps a point x ∈ H to the unique point in C i with minimal distance to x. For n = 1, . . . , N, let J n = (j n 1 , . . . , j n |J n | ) be a finite ordered subset of {1, . . . , M} called a string. In addition, let ω n ∈ (0, 1), n = 1, . . . , N, satisfy N n=1 ω n = 1. The string-averaging projection method for these data is the iterative method defined by where x 0 is an arbitrary initial point and j ∈J P C j := P C j l . . . P C j 1 for a string J = (j 1 , . . . , j l ). In [18], it is shown that any sequence {x k } ∞ k=0 generated by the above method converges to a point x ∞ ∈ C ⊆ R n . In addition to metric projections, in [18] also, relaxed metric projections and Bregman projections are considered.
In [19], a modification of method (1.7) is introduced. Instead of applying the same operator at each iterative step, both the strings and the weights may be different at each step. In more detail, the dynamic string-averaging (SA) projection method is defined by where (1.9) and for all k = 0, 1, 2, . . ., J k n ⊆ I is a string and ω k n ∈ (0, 1) with N k n=1 ω k n = 1. Under the assumption that the family {C 1 , . . . , C M } is boundedly regular and that the control is 1-intermittent, that is, at all steps k, each index i ∈ {1, . . . , M} appears in at least one of the strings J k n , the authors of [19] show that any sequence generated by the dynamic string-averaging projection method converges in norm to an element x ∞ ∈ C ⊆ H. In addition, they prove that the superiorized version has the same convergence properties.
A static version of the superiorized SA projection method appeared for the first time in [10] although it relies heavily on [11]. Many variants of the SA methods with operators more general than the metric projection can be found in the literature. For example, static SA methods based on the averaged operators in Hilbert space can be found in [12,Section 6], whereas their dynamic variants (with s-intermittent control, s ≥ 1) appeared in [1] and [6,Corollary 5.18]. In [39], a general framework for the study of a modular SA method based on cutters and firmly nonexpansive operators in Hilbert spaces is introduced and convergence properties, including perturbation resilience and superiorization, are studied. In particular, if the family of the fixed point sets is boundedly regular and the operators satisfy certain regularity assumptions, [39,Theorem 4.5] shows perturbation resilience for the weak and strong convergence of these methods under summable perturbations.
In all of the above-mentioned results, it is shown that weak or strong convergence holds true and that in some cases, the type of convergence can be preserved under summable perturbations. The focus of the present article is on preservation of the convergence rate. We are interested, in particular, in the case of linear convergence, which is known to be the case for some variants of the SA projection methods such as cyclic and simultaneous projection methods; see, for example, [2, 5, 7, 8, 25-27, 30, 33, 34, 36]. We emphasize that to the best of our knowledge, linear convergence rates for static/dynamic SA projection methods were unknown till now. We recall that a sequence {x k } ∞ k=0 in H converges linearly to a point x ∞ ∈ H if there are constants c > 0 and 0 < q < 1 such that for all k = 0, 1, 2, . . .. Our paper is structured as follows. In Section 2, we introduce some notation and, for the convenience of the reader, we present the concepts and properties we use in the other sections of this article. In Section 3, we consider the convergence properties of the dynamic string-averaging projection method and investigate their relation with the regularity of the family {C 1 , . . . , C M }. In particular, we prove linear convergence provided the family {C 1 , . . . , C M } is boundedly linearly regular. In Section 4, we provide general results regarding the preservation of the convergence rate for infinite products of nonexpansive operators in the presence of summable perturbations. We specialize our main result to the preservation of linear convergence rates and to the case of superiorization. In Section 5, we combine the results of Sections 3 and 4 to discuss the behavior of dynamic string-averaging methods in the presence of summable perturbations. As a particular case, we consider the superiorized dynamic string-averaging projection method.

Preliminaries
In this paper, H always denotes a real Hilbert space. For a sequence x k ∞ k=0 in H and a point x ∞ ∈ H, we use the notation to indicate that x k ∞ k=0 converges to x ∞ weakly and in norm, respectively. Given a nonempty, closed, and convex set C ⊆ H, we denote by P C : H → H the metric projection onto C, that is, the operator which maps x ∈ H to the unique point in C closest to x. The operator P C is well defined for such sets C and it is not difficult to see that it is nonexpansive; see, for example, [ the distance of x to C. In addition, for ε > 0, we define 3) The following lemma provides a connection between the metric projection onto a nonempty, closed, and convex set C and the metric projection onto the enlarged set C ε . The statement of this lemma can be found in [6, Proposition 28.10] without a proof. We provide a proof here for the convenience of the reader.

Lemma 2
Let C ⊆ H be nonempty, closed, and convex, ε ≥ 0 and let x ∈ H \ C ε . Then Consequently, The inequality d(x, C) ≤ d(x, C ε ) + ε follows from the properties of the Hausdorff distance. For the convenience of the reader, we present this argument. For all points y ∈ C ε , we have and hence In order to show the reverse inequality, note that x − y > ε for all y ∈ C. We set z = P C x, where P C is the nearest point projection onto C. This implies that z − x = d(x, C) and ε x−z < 1. We define a new point z ε by and get On the other hand, we get which finishes the proof.
The point y = P C x is characterized by: y ∈ C and for all z ∈ C; see, for example, [ for all z ∈ C. As a generalization of this property, for ρ ≥ 0, an operator T : for all x ∈ H and all z ∈ Fix T , where Fix T denotes the set of all fixed points of T . The equivalence of (2.11) and (2.12) has been extended to ρ-strongly quasi-nonexpansive operators and α-relaxed cutters; see, for example, [13, Theorem 2.1.39]. For the convenience of the reader, we now recall a few important properties of ρ-strongly quasi-nonexpansive operators.
where min i ρ i > 0. Then the following statements hold true: where Q 0 := Id and Q i : Given a nonempty, closed, and convex set holds for all z ∈ C and k = 0, 1, 2, . . ..

Theorem 5
Let C ⊆ H be nonempty, closed, and convex, and let {x k } ∞ k=0 be Fejér monotone with respect to C.
holds for all x ∈ S and some constant κ S > 0.
We say that the family C is boundedly (linearly) regular if it is (κ S -linearly) regular over every bounded subset S ⊆ H. We say that C is (linearly) regular if it is (κ Slinearly) regular over S = H.
Note that the constant κ S always satisfies Example 7 Let C i ⊆ H and C be as in Definition 6. Then the following statements hold: For a prototypical convergence result combining properties of Fejér monotone sequences with regularity properties of sets, see, for example, [4, Theorem 2.10]. For more information regarding regular families of sets, see [5,40,41].

String-averaging projection methods
In this section, we present sufficient conditions for weak, strong, and linear convergence of dynamic string-averaging projection methods depending on the regularity of the constraint sets.
. . , M}, be closed and convex, and assume C := i∈I C i = ∅. Let U be the string-averaging projection operator defined by where J n ⊆ I is a string (finite ordered subset) and ω n ∈ (0, 1), N n=1 ω n = 1. Moreover, assume that I = J 1 ∪ . . . ∪ J N . Then the following statements hold true: (iii) For all x ∈ H, z ∈ C, and i ∈ I , we have where ω := min 1≤n≤N ω n .
Proof Note that (i) follows from the fact that both compositions and convex combinations of nonexpansive operators are nonexpansive. Statement (ii) follows immediately from Theorem 3. In order to show (iii), let x ∈ H and z ∈ C := i∈I C i . Using the convexity of the function x → x 2 and Theorem 4, we get where m n := |J n |, J n := (i 1 , . . . , i m n ), Q n l := l j =1 P C i j , and Q n 0 := Id. Note that in the above estimates, we also use the facts that C ⊆ j ∈J n C j and N n=1 ω n = 1. From the above inequality, we deduce that where ω = min 1≤n≤N ω n , because Given x ∈ S and i ∈ I , choose n ∈ {1, . . . , N} and 1 ≤ p ≤ m n = |J n |, so that i ∈ J n and Q n p = P C i Q n p−1 . Then we may conclude that by using the triangle inequality, the convexity of the function t → t 2 , and the fact that m = max 1≤n≤N m n . Combining (3.8) with (3.5), we obtain which finishes the proof of (ii).
holds for all x ∈ S, which proves the claim in (iv).
. . , M}, be closed and convex, and assume C := i∈I C i = ∅. Let the sequence x k ∞ k=0 be defined by the dynamic string averaging projection method, that is, x 0 ∈ H and for every k = 0, 1, 2, . . ., x k+1 := T k x k , where

11)
J k n ⊆ I is a string and ω k n ∈ (0, 1), N k n=1 ω k n = 1. Moreover, let I k := J k 1 ∪. . .∪J k N k . Assume that ω k n ≥ ω for some ω > 0 and that m := sup k max 1≤n≤N k |J k n | < ∞. Moreover, assume that there is an integer s ≥ 1 such that for each k = 0, 1, 2, . . ., Then the following statements hold true: (3.12) Moreover, for each k = 0, 1, 2, . . . , we have the following error bound: Proof It is not difficult to see that statements (i) and (ii) are special cases of [ x 0 , r). Therefore, it suffices to show statement (iii). We divide the proof into three steps.
Step 1. We first show that the inequality holds for every ν ∈ I , k ∈ N, and z ∈ C.
Fix ν ∈ I and z ∈ C. Given k ∈ N, we choose l k ∈ {ks, . . . , (k + 1)s} to be the smallest index so that ν ∈ I l k = N l k n=1 J l k n and n ∈ {1, . . . , N l k } to be the smallest index such that ν ∈ J l k n , and set Since the mapping d(·, C ν ) is nonexpansive, we have and hence (3.17) where the second inequality follows from the choice of i k . Squaring this inequality, we obtain by the Cauchy-Schwarz inequality, (3.2), and since l k − ks + 1 ≤ (k + 1)s − ks = s. Note that Theorem 3 implies that T p−1 is 1 m -strongly quasi-nonexpansive and hence (3.19) which, when combined with (3.18), leads to , (3.20) where the last inequality holds since the sequence {x k } ∞ k=0 is Fejér monotone with respect to C.
Step 2. Setting z = P C x ks in (3.14) and using we may deduce that Since ν ∈ I is chosen arbitrarily, we may use the κ r -linear regularity of (3.24) Applying Theorem 5 (ii), we see that the subsequence {x kp } ∞ k=0 converges linearly to a point x ∞ ∈ C. In other words, for the point x ∞ ∈ C, we get (3.25) Finally, Theorem 5 (iii) implies that the whole sequence {x k } ∞ k=0 converges linearly too. More precisely, it implies that the sequence converges linearly with the constants as in (3.12).
Step 3. Since x k → x ∞ , we can use Theorem 5 (i) and the fact that {C i | i ∈ I } is κ r -linearly regular to obtain On the other hand, using C ⊆ C i , x ∞ ∈ C and Step 2, we get which finishes the proof of the error bound (3.13).

Linear convergence of the cyclic and simultaneous projection methods
In this section, we specialize Theorem 9 (iii) to cyclic and simultaneous projection methods applied to general closed and convex sets. Results related to closed linear subspaces are discussed in Section 3.2. To this end, assume that the family {C i | i ∈ I } is κ r -linearly regular over B P C x 0 , r . Consider first the cyclic projection method Consequently, by Theorem 9 (iii), (3.29) The cyclic projection method oftentimes is considered in an equivalent formulation involving composition of projections, that is, (3.31) On the other hand, the latter variant of the cyclic projection method is a particular case of the string-averaging projection method with one string of length m = M. Consequently, by using Theorem 9 (iii) with s = 1, we recover formula (3.31). The first linear convergence result for general closed and convex sets and the remotestset projection method can be found in [30,Theorem 1]. It is not difficult to see that in the case of M = 2, the remotest-set projection method coincides with the cyclic projection. In addition, it was also reported in [30], although without a proof, that the cyclic projection method may converge linearly even for M > 2. A similar derivation of q r can also be made for the simultaneous projection method In this case, the lower bound for the convex combination coefficients ω = 1 M , m = 1 and s = 1. Consequently, we obtain the same formula as in (3.31). The first result on the linear convergence rate of iteration (3.32) in the case of general closed and convex sets is due to Pierra [36, Theorem 1.1], who established it in the equivalent product space setting using [30,Theorem 1].
In Table 1, we compare several examples of the convergence rates related to the cyclic and simultaneous projection methods, which can be found in the literature. It is worth mentioning that in [5,7,9,27], the formula for q r was not given explicitly Table 1 Examples of q r for cyclic and simultaneous projection methods (PM) for which one has a linear convergence rate of the form x k − x ∞ ≤ c r q k r . The estimates attributed to [5,7,9,27] were deduced from the proofs by using Theorem 5 in the statement of the theorem and, for the purpose of this paper, it was derived from the proof by using Theorem 5. On the other hand, in [5,7,9], as well as in this manuscript, the authors deal with more general schemes than cyclic and simultaneous projection methods, which might result in weaker estimates. Due to the equivalence of methods (3.28) and (3.30), we focus only on the latter case.

Closed linear subspaces
Assume that each C i ⊆ H, i ∈ I is a closed linear subspace. In this case, the limit point x ∞ = P C x 0 ; see, for example, [6,Proposition 5.9]. Note that without any loss of generality, we can assume that the family {C i | i ∈ I } is κ-linearly regular over H in Theorem 9. Indeed, we have the following proposition.  (3.34) where the inequality follows from the fact that r x − P C x 0 / x − P C x 0 ∈ B(P C x 0 , r).

Perturbation resilience and superiorization
In this section, we consider the question of perturbation resilience of infinite products of nonexpansive operators. We show that the rate of convergence, both for weak as well as for strong convergence, is essentially preserved under summable perturbations. These results are applicable, in particular, to the string-averaging projection methods discussed in Section 3. We present this connection in more detail in Section 5.
Theorem 11 (Perturbation resilience) For every k = 0, 1, 2, . . ., let T k : H → H be nonexpansive and assume that C ⊆ k Fix T k is nonempty, closed, and convex. Let {e k } ∞ k=0 ⊆ [0, ∞) be a given summable sequence of errors and let {x k } ∞ k=0 ⊆ H be an inexact trajectory such that holds for every k = 0, 1, 2, . . .. Assume that for all i = 0, 1, 2, . . ., there is an Then there is a point holds for all y ∈ H and all i = 0, 1, 2, . . .. If, in addition, for all i = 0, 1, 2, . . ., Proof We claim that for every integer k ≥ i ≥ 0, we have where x k+1 i := T k . . . T i x i and x i i = x i . Note that by the definition of x k i , this inequality is true for every integer k = i ≥ 0 with the right-hand side equal to zero. Now fix i and assume that k > i. Thus, by the triangle inequality, (4.1), and the nonexpansivity of T k , we have which, by induction, yields (4.6). Now we can apply Theorem 1 to get x k x ∞ . Therefore, for any y ∈ H, we may take the limit as k → ∞ in (4.8) and using (4.2), we arrive at Consequently, x ∞ i x ∞ . Note that (4.9) implies that x ∞ i → x ∞ as well. Indeed, we have (4.10) and by letting i → ∞, we get strong convergence. Moreover, by the triangle inequality, (4.6) and (4.9), we have the following estimate: which proves (4.3). Now assume that x k i → x ∞ i . Since convergence in norm implies weak convergence, we may use (4.11) to obtain for all y ∈ H with y ≤ 1 and hence which proves x k → x ∞ and (4.5).

Remark 12
Note that the proof of Theorem 11 also works in the case of Banach spaces if we replace y ∈ H by a bounded linear functional. Hence, all the results in this section are also true in the setting of Banach spaces. Since in this article we are only interested in operators on Hilbert spaces, we formulate the theorem only for this case.
Indeed, if we use the term "weak convergence rate" in the sense that for every y ∈ H there is a mapping r y : N → (0, ∞) such that | y, x k − x ∞ | ≤ r y (k), (4.14) then Theorem 11 can be interpreted as follows: both for weak and strong convergence, the convergence rate is essentially preserved in the presence of summable perturbations. Note that since d(x k+1 , C) ≤ x k+1 − x ∞ , by (4.5) and Lemma 2, we may also deduce estimates concerning the distance of the sequence to the set C ε . Indeed, given ε > 0, there is i ε ≥ 0 such that trivially holds true, or if x k+1 ∈ C ε , we may then obtain the inequality by using Lemma 2. in H, we define for x 0 ∈ H the sequence x k ∞ k=0 by Assume that for every i = 0, 1, 2, . . ., there is Then there is a point holds for all y ∈ H and all i = 0, 1, 2, . . .. If, in addition, for all i = 0, 1, 2, . . ., Proof By the nonexpansivity of T k , we have x k+1 − T k x k ≤ β k v k . Thus, it suffices to substitute e k := β k v k and apply Theorem 11.

Remark 15
As we have already mentioned in the introduction, the steering sequence {v k } ∞ k=0 is related to a subgradient of some convex, continuous function φ : H → R. For example, one can use v k ∈ ∂φ(x k ). A variant of (4.17) can be considered, where v k is replaced by a linear combination, that is, the sequence of all v k,n is bounded, L k ≤ L for a fixed integer L, and the coefficients satisfy the summability condition we can rewrite (4.22) in the form of (4.17), where v k ≤ max 1≤n≤L k v k,n .
Example 16 (Preservation of the linear rate) Let us consider the basic method x k+1 = T k x k , where T k : H → H are nonexpansive such that ∅ = C ⊆ Fix T k . Assume that this method converges linearly, that is, for given starting points x i ∈ H, i = 0, 1, 2, . . . , there are c i ∈ (0, ∞), q i ∈ (0, 1) and x ∞ i ∈ C such that for every k ≥ i, (4.25) We show that this linear rate can be preserved, first, by adding perturbations and, second, by considering a superiorized version of the basic iterative method.
(a) (Perturbation resilience) To this end, let {x k } ∞ k=0 be a perturbed trajectory of the basic method with summable perturbations {e k } ∞ k=0 ⊆ [0, ∞), that is, T k x k − x k+1 ≤ e k for every k = 0, 1, 2, . . .; compare with Theorem 11. Then we have the following estimate: (4.26) In particular, for every ε > 0, there is an index i ε ≥ 0 such that the inequality Note that in some cases, we have more information on the subset C, for example, (4.29) Proof For every integer k ≥ i ≥ 0, let x k+1 i := T k . . . T i x i and x i i := x i . By Theorem 11, {x k } ∞ k=0 is convergent and thus bounded. Note that {x k i } ∞ k=i is Fejér monotone with respect to C. Therefore, for every integer k ≥ i and z ∈ C, we have which proves that x k i ∞ k=i ⊆ B(0, R) for every i = 0, 1, 2, . . . , and some R > 0. Now we show that estimate (4.29) holds true. Since the sequence x k i ∞ k=i converges to some x ∞ i ∈ C, by Theorem 5 (i), we have for every integer k ≥ i ≥ 0. Moreover, by (4.6), Thus, by (4.5), (4.31), linear regularity over B(0, R), and the definition of the metric projection, we get which by the triangle inequality, (4.32), and again by the definition of the metric projection is less than or equal to 2κ max

String-averaging projection methods revisited
In this section, we revisit string-averaging projection methods and combine the results of the previous sections in order to obtain results regarding the convergence rate of dynamic string-averaging projection methods in the presence of perturbations. As a corollary, we consider the convergence rate of superiorized dynamic string averaging methods. (i) For every i = 0, 1, 2, . . . , there is x ∞ i ∈ C such that T k · · · T i x i k x ∞ i → i x ∞ for some x ∞ ∈ C. Moreover, for all y ∈ H, we have y, x k+1 − x ∞ ≤ y, T k · · · T i x i − x ∞ i + 2 y ∞ k=i e k .
(5.2) (ii) If the family {C i | i ∈ I } is boundedly regular, then for every i = 0, 1, 2, . . . , there is x ∞ i ∈ C such that T k · · · T i x i → k x ∞ i → i x ∞ for some point x ∞ ∈ C. Moreover, we have x 0 ∈ H and x k+1 = T k (x k − β k v k ), (5.5) where {β k } ∞ k=0 ⊆ [0, ∞) is summable and {v k } ∞ k=0 ⊆ H is bounded. Then the following statements hold true: (i) for every i = 0, 1, 2, . . . , there is x ∞ i ∈ C such that T k · · · T i x i k x ∞ i → i x ∞ for some x ∞ ∈ C. Moreover, for all y ∈ H, we have