Rules for Identifying the Initial Design Points for Use in the Quick Convergent Inflow Algorithm

The starting point of search is an important factor in optimal design construction as a poor starting point may require a longer time before convergence is reached. Hence the location of the initial design points for use in the Quick Convergent Inflow Algorithm on segmented regions is examined with the aim of developing useful criteria for identifying the initial design points. Proportional allocation of design points to go into the initial design measures is proposed. The allocation of 100% vertex points, 100% boundary points and 100% boundary points as well as the allocation of 50% vertex and 50% boundary points, 50% vertex and 50% interior points and 50% interior and 50% boundary points are investigated. Results show that a combination of design points comprising of 50% vertex points and 50% interior points or 50% vertex points and 50% boundary points forms helpful rules in identifying the initial design points for use in the Quick Convergent Inflow Algorithm. With these combinations, a moderate number of iterations needed to reach the required optimal or near-optimal solution is maintained.


Introduction
Finding an optimal experimental design is seen as one of the most important topics in the context of experimental design.Any effective search scheme usually seeks to maximize performance.In locating the optimizer of an objective function using line equations, best decisions should be made on the design of the experiments, the starting point of search, the direction of search and the step-length.As an important aspect in optimal design construction, a poor initial design may require a longer time before convergence is reached.We consider, in this paper, the location of the initial design points for use in the Quick Convergent Inflow Algorithm (QCIA) introduced by Iwundu and Odiakosa (2013).In particular, we propose helpful rules for identifying the initial design points for use in the QCIA.
The Quick Convergent Inflow Algorithm as a line search algorithm is based on experimental design principles and has been used in solving linear programming problems.The use of experimental design methods to solve constrained and unconstrained optimization problems is established in the literature.Onukogu (1997) introduced the Minimum Variance line search algorithm, that is based on minimum variance properties, for locating the optimizers of unconstrained surfaces.Umoren (1999) considered the Maximum Norm Exchange Algorithm (MNEA), an experimental design technique for solving constrained optimization problems.The iterative method exchanges a design point having maximum norm with the end point of the r th iteration.The initial design measure for the Maximum Norm Exchange Algorithm consists of design points selected from the boundary of the design region.Other works that apply experimental design principles for solving optimization problems include Umoren and Etukudo (2010), Osita and Iwundu (2013), Ekezie and Nzenwa (2013), Chigbu and Ukaegbu (2013), Ekezie etal. (2013), Iwundu and Hezekiah (2014), Iwundu and Ebong (2014) and Iwundu and Ndiyo (2015).
The Quick Convergent Inflow Algorithm of Osita and Iwundu (2013) used in solving Linear Programming (LP) problems moves in the direction of minimum variance and relies on adding the point reached by the line equation, at each iteration, to an existing design.The addition of such point(s) would usually guarantee convergence of the algorithm to an approximate optimum.Iwundu and Hezekiah (2014) examined the effect of the QCIA on segmented regions and observed that although the QCIA is effective in solving linear programming problems, it is possible that by its stopping rule, the algorithm may, for some problems, converge only locally.Also, the algorithm may not always reach the optimum of the linear objective function for the design size, N, bounded by p < N < ½ p(p+1) + 1 for a p-parameter model.Hence, for such problems the Quick Convergent Inflow Algorithm fails to converge at the required optimum.As a consequence, Iwundu and Hezekiah (2014) presented a method that attempts to overcome the possible convergence of the QCIA at a non-global optimum.The method is based on variance exchange of a design point in the design measure at a current iteration, having minimum (or maximum) variance of prediction with the point reached by the line equation in a maximization (or minimization) problem.As applied to some LP problems, the modification of Iwundu and Hezekiah (2014) could require fairly long iterative moves to reach the optimum.This may be as a result of choosing a poor starting design.
As a further observation on the QCIA, Iwundu and Ebong (2014) noted that problem occurs with the QCIA when the point reached by the line equation does not satisfy the linear inequality constraints and hence cannot be used as an admissible point of the experimental design.At this instance, the QCIA cannot successfully converge at the required optimum.To circumvent this problem, Iwundu and Ebong proposed a modification to the basic algorithm by considering the addition of a point of optimum predictive variance (from among a set of candidate points from the feasible region that satisfy the constraint equations) to an existing design.As shown theoretically, a point of maximum predictive variance when added to an existing design leads to the maximizer of the response function in a maximization problem.Similarly, the addition of a point of minimum predictive variance to an existing design leads to the minimizer of the response function in a minimization problem.The idea behind the Modified Quick Convergent Inflow Algorithm (MQCIA) of Iwundu and Ebong (2014) stemmed from the fact, as established by Atkinson and Donev (1992), that relationship often exists between the experimental design and the variance of predicted response in optimal design construction.Subsequently, by improving an existing experimental design the optimizer of the response function is approached.For any sequential search that seeks to arrive at the optimun, there is the need to start the search optimally.Atkinson and Donev (1992) and a number of other researchers have discussed on techniques for constructing starting designs for use in search algorithms when selection of design points is paramount.Some of the techniques rely on random selection of design points and some are guided by specific rules.For example, Iwundu (2008) observed the importance of starting experimental designs and thus offered an improvement on the starting design for use in variance exchange algorithms.The starting designs were constructed on the basis of distances of design points from the centre of the design regions.Iwundu (2010) stipulated rules for obtaining the starting design points for use in the combinatorial algorithm of Onukogu and Iwundu (2008).The rules were based on permutation-invariance of design points.Iwundu and Abolaji (2014) applied the permutation-invariance technique in constructing the starting design for use in the variance exchange algorithm in constructing D-optimal designs and compared its performance with the commonly used random selection method.
The aim of this research is to develop useful criteria for obtaining the starting design for use in the Quick Convergent Inflow Algorithm.Particularly, it seeks to address where to select design points that makes up the initial experimental design for use in the algorithm.The motivation for this research centers around the behaviour of the QCIA when design points are chosen only from the boundary of the feasible region.In most examined cases, it failed to converge at the required optimum.

Methodology
The fundamental algorithm used in this work is due to Odiakosa and Iwundu (2013).However, we employ in particular the segmentation algorithm of Iwundu and Hezekiah (2014) whose abridged steps are outlined in section 2.1.

The Algorithmic Framework of the QCIA on Segmented Regions
The Quick Convergent Inflow Algorithm on segmented regions is defined by the following steps; i. Partition the feasible region into s segments.
ii. Select N support points from each of the s segments and hence make up the design measures  1 ,  2 , …,  s where p < N < ½ p(p+1)+1 and p represents the number of model parameters.
iii.Compute, at each iteration, the starting point of search x r

Starting Point of Search
The starting point of search is obtained from the initial design measure, ξ 1 , ξ 2 , …, ξ s , define on the s segments of the design region.For the purpose of this work s=2.Hence, the starting point is obtained using ξ 1 and ξ 2 , where ξ 1 and ξ 2 are defined, respectively, as and where N 1 and N 2 are the respective design sizes for ξ 1 and ξ 2 and are such that x sj is the j th design (support) point obtained from the s th segment and defined as a function of the model.The design points to go into the initial design measures shall be obtained from proportional allocation of vertex, interior and boundary points as outlined in section 2.4.For the illustrations used in this work, the number of model parameters p, equals the number of variates k.Hence p = k.
In the first segment Similarly, In the second segment The starting point of search, x 0 , is hence the arithmetic mean of x 11 , x 12 , …, x 1N 1 , x 21 , x 22 , … , x 2N 2 .
Other components of the line equation are as in Iwundu and Hezekial (2014).

Stopping Rule
In a minimization problem, let  1 In establishing rules for selecting initial design points for use in the Quick Convergent Inflow Algorithm on segmented regions, we employ the following proportional allocation of design points to go into the design; (i) 100% vertex points (ii) 100% boundary points (iii) 100% interior points (iv) 50% vertex points and 50% boundary points (v) 50% vertex points and 50% interior points (vi) 50% boundary points and 50% interior points The remaining task is as in the algorithmic framework outlined in section 2.1.The consideration of proportional allocation of design points stems from experiences in the use of the QCIA.

Numerical Illustrations
We present results on the behaviours of vertex, boundary and interior points for the Quick Convergent Inflow Algorithm (QCIA) in obtaining optimal solutions for linear programming problems on segmented regions.

Illustration 1
We consider the maximization of To solve the problem using the Quick Convergent Inflow Algorithm, we consider proportional allocation of support points to make up the initial design.

100% Vertex Points
Here the starting design is made up of design points which satisfy the constraints and are obtained only from the vertices of the design region.
Using 2 segments defined by we form the initial design measures These measures together with the model form the design matrices X 1 and X 2 respectively as The associated information matrices are, respectively, (.) T represents transpose.
The inverse of each information matrix is, respectively, 3 ) The matrices of coefficient of convex combination of the dispersion matrices are, respectively, H 1 = diag (0.1936, 0.4737) H 2 = diag (0.8064, 0.5263) The average information matrix is given by: M 1 (ξ N ) = ( 15.9065 2.0644 2.0644 2.2300 ) The inverse of the average information matrix is The response vector of the i th row of the average information matrix is Hence the direction vector is By normalizing of the vector d 1 , we have The optimal starting point is 8333 ) The optimal step-length using the four constraints is ρ 1 * =1.1492.
With x 1 * , d 1 * and ρ 1 * we make a move to The value of the objective function at x 1 * is 20.6912.

100% Boundary Points
Here the starting design is made up of design points obtained only from the boundaries only and which must satisfy the constraints.
Using 2 segments defined by We form the initial design measures  1 = ( 0.5 0 0.5 1.5 2 1.5 )  2 = ( 2 1 3 0 3 1.5 ) These measures together with the model form the design matrices X 1 and X 2 respectively as The associated information matrices are, respectively,  1 T  1 = ( 4.5 3.75 3.75 4.5 ) and  2 T  2 = ( 22 6.5 6.5 3.25 ) The average information matrix is Following the steps of the algorithm we obtain the optimal solution as f( 2 * ) = 20.1937where  2 * = ( 2.7313 1.6343 ) is the optimizer of the objective function.

50% Vertex Points, 50% Interior Points
Here the starting design is made up of 50% of design points obtained from the vertices and 50% of design points obtained from the interiors and which must satisfy the constraints.
Using 2 segments defined by We form the initial design measures )  2 = ( 2.5 1.5 2.5 0.5 3.5 0.5 ) These measures together with the model form the design matrices X 1 and X 2 respectively as ) X 2 = ( 2.5 1.5 2.5 0.5 3.5 0.5

)
The associated information matrices are, respectively, ) and  2 T  2 = ( 24.75 6.75 6.75 2.75 ) The average information matrix is  1 (  )= ( 10.9260 2.1242 2.1242 3.0940 ) Following the steps of the algorithm we obtain the optimal solution as f( Here the starting design is made up of 50% of design points obtained from the vertices and 50% of design points obtained from the boundaries and which must satisfy the constraints.
Using 2 segments defined by We form the initial design measures as )  2 = ( 2 1.5 3 0 3 0.5 ) These measures together with the model form the design matrices X 1 and X 2 , respectively as The associated information matrices are, respectively, ) and  2 T  2 = ( 17 4 4 2.5 ) The average information matrix is  1 (  ) = ( 9.5630 1.8878 1.8878 1.7430 ) Following the steps of the algorithm we obtain the optimal solution as f( Here the starting design is made up of 50% of design points obtained from the interiors and 50% of design points obtained from the boundaries and which must satisfy the constraints.
Using 2 segments defined by We obtain initial design measure from the segments as  1 = ( 0.5 0.5 1.5 0.5 1.5 2 )  2 = ( 2 2 2 0.5 3.5 0 ) These measures together with the model form the design matrices X 1 and X 2 respectively as X 1 = ( 0.5 0.5 1.5 0.5 1.5 2 The average information matrix is

)
Continuing the process, we obtain the starting point of search, the direction of search and the step-length of search,

Table 1 .
, the design measures, starting point of search, direction of search, step-length of search, determinant value of the average information matrix, optimal point reached, value of objective function, variance of optimizer as well as the number of iterative steps are summarized in Table1Summary statistics for illustration 1