On the O ( 1 / k ) Convergence Rate of He ’ s Alternating Directions Method for a Kind of Structured Variational Inequality Problem

The alternating directions method for a kind of structured variational inequality problem (He, 2001) is an attractive method for structured monotone variational inequality problems. In each iteration, the subproblems are convex quadratic minimization problem with simple constraints and a well-conditioned system of nonlinear equations that can be efficiently solved using classical methods. Researchers have recently described the convergence rate of projection and contraction methods for variational inequality problems and the original ADM and its linearized variant. Motivated and inspired by research into the convergence rate of these methods, we provide a simple proof to show the O(1/k) convergence rate of alternating directions methods for structured monotone variational inequality problems (He, 2001).


Introduction
Let f be a continuous and monotone mapping from I R n onto itself and let K ⊂ I R m be a nonempty closed convex cone with vertex at the origin.Consider the variational inequality problem VI(S , f ) (He, 2001) where In fact, by attaching a Lagrange multiplier vector y ∈ Y to the constraint Ax − b ∈ K, we can obtain the structured variational inequality VI(Ω, F) (He, 2001): where and Y is a proper subset of I R m .
Throughout we assume that the solution set of VI(Ω, F), denoted by Ω * , is nonempty.Let ∥ • ∥ denote the Euclidean norm and P K (y) the projection of y to the closed convex set K, that is, The alternating directions method (ADM) is a classical method for solving variational inequality problems first introduced in the mid-1970s by Gabay and Mercier, and Glowinski and Marrocco, although similar ideas emerged in the mid-1950s (Douglas & Rachford, 1956, Peaceman & Rachford 1955).Much development of the ADM has been carried out in the areas of variational inequality problems and convex programming to yield simple single-processor algorithms (Boyd , Parikh, Chu, Peleato & Eckstein, 2010, Eckstein& Bertsekas, 1992, Gabay & Mercier, 1976, Gabay, 1983, Glowinski, 1984, He, 2001, He, Liao, Han & Yang, 2002, Peaceman & Rachford, 1955, Tao & Yuan, 2011) .
To solve the structured variational inequality VI(Ω, F), the ADM of Gabay and Mercier produces the new iterate u k+1 = (x k+1 , y k+1 ) T from a given u k = (x k , y k ) T ∈ Ω via the following steps (Gabay & Mercier, 1976, Gabay, 1983): Step 1. Find x k+1 such that Step 2. The multipliers y k+1 are updated according to  4) is also a variational inequality problem.Naturally, it is difficult to solve the structured variational inequality VI(Ω, F) using the original ADM.
He recently provided a modified ADM for VI(Ω, F) that only solves a convex quadratic minimization problem with simple constraints and a system of strongly monotonic nonlinear equations in each iteration (He, 2001).This approach is obviously easier than solving (3) and ( 4) in many situations.Subsequently, we review the He's ADM and related results (He, 2001).
The He's ADM (He, 2001): Given γ ∈ (0, 2), and Step 2. If ∥ f (x k ) − A T y k ∥ > ϵ, update x k+1 according to Subproblem ( 5) is a symmetric linear variational inequality and it can be solved according to the convex quadratic minimization problem min{ 1 2 y T My + q T y|y ∈ Y}, 5) is equivalent to the symmetric linear complementarity problem y ≥ 0, AA T y + q ≥ 0, y T (AA T y + q) = 0, which can be solved by the pivotal Lemke method (Lemke, 1965).Numerical experiments showed that (5) can be solved within a finite number of steps (He, 2001).Moreover, (6) can be solved by some triangular decomposition to I + ∇ f (x) while solving x + f (x) − c k = 0 using Newton methods (Eckstein& Bertsekas, 1992), where In fact, f (x) is a monotone operator and then x + f (x) is a strictly monotone operator, so ( 6) is a well-conditioned system of nonlinear equations.Naturally, the subproblems in the He's ADM are easier than those in the classical ADM.
Nesterov introduced a method for solving a convex programming problem and provided a simple proof of the O(1/k 2 ) convergence rate of the method (Nesterov, 1983), which is derived from the Nemirovski non-relaxational minimizing sequence.Nemirovski subsequently showed that the extragradient method has the O(1/t) convergence rate for variational inequalities with Lipschitz continuous monotone operators (Nemirovski, 2005).He recently described the convergence rate of projection and contraction methods under a unified conceptual framework (He, 2011).It is worth noting that the O(1/t) convergence rate of the ADM is given in the literature (He, 2012, Tao & Yuan, 2011).Much research has been devoted to this topic (Han & Yuan, 2014, Li, 2014, Shen, Chen & Cai, 2013, Jia, Guo & Cai, 2013, Yuan, 2012).Motivated and inspired by the encouraging progress for the convergence rate, we consider the O(1/k) convergence rate of He's ADM.
The remainder of the paper is organized as follows.Section 2 presents some basic definitions and fundamental lemmas that are useful for the convergence rate analysis.In Section 3, the convergence rate of He's ADM (He, 2001) is proved.Finally, conclusions are drawn in Section 4.

Preliminaries
In this section, we provide some important lemmas for subsequent analysis.The proofs are widely available in the literature (Facchinei & Pang, 2003, He, 2001).
Definition 1 Let f be a continuous mapping from I R n onto itself.Then f (x) is a monotone operator if and only if In fact, if f (x) is a monotone operator, then According to the relationship between a variational inequality and a fixed point problem, we can easily obtain the following lemma.
In fact, the solution of structured variational inequality VI(F, Ω)(2) is the zero point of ) .
Subsequently, we recall some previous results (He, 2001) for proof of the convergence rate.
Lemma 2 (He, 2001, Remark 1) For a given u k = (x k , y k ) T ∈ I R n × Y, let {u k } be produced by the He's ADM.Then we have By Lemma , taking ∥ f (x k ) − A T y k ∥ ≤ ϵ as a stopping criterion is reasonable.
Lemma 3 (He, 2001, Proposition 2) For a given u * = (x * , y * ) T ∈ Ω * , 0 < γ < 2, let {u k } be produced by the He's ADM.Then we have The most important properties of ∥ f (x k ) − A T y k ∥ 2 are summarized in the following.For integrity of the proof, we provide a proof of the following lemma.Moreover, for convenience, we denote x k , y k , x k+1 , and y k+1 by x, y, x, and ỹ, respectively.
Lemma 4 (He, 2001, Proposition 4) Let u = (x, y) T be the sequence produced by the He's ADM, 0 < γ < 2. Then we have Proof.By a series of computations, we can obtain Hence, we need only to show that According to (5), we have Adding ( 7) and ( 8), we get Note that (A T ỹ − A T y) T (x − x) can be changed to According to (9), ( 10), ( 11) and ( 12), we obtain Hence the lemma is proved.

Convergence Rate
In this section, we analyze the O(1/k) convergence rate of He's ADM.
In fact, when we take ∥e(u k )∥ 2 ≤ ϵ or ∥ f (x k ) − A T y k ∥ 2 ≤ ϵ as the stopping criterion, by Lemma 1 and Lemma 2 and Theorem 1, we obtain the O(1/k) convergence rate of He's ADM.

Conclusions
We proved the O(1/k) convergence rate of He's ADM.This result can be considered an improvement and refinement of previous results (He, 2001).