Equivalence of Convex Problem Geometry and Computational Complexity in the Separation Oracle Model Robert M. Freund MIT Sloan School of Management, 50 Memorial Drive, Cambridge, MA 02142, USA email: [email protected]

Jorge R. Vera Dept. de Ingenier´ıa Industrial y de Sistemas, Facultad de Ingenier´ıa, Pontificia Universidad Cat´ olica de Chile, Campus San Joaqu´ın, Vicu˜ na Mackenna 4860 Santiago, CHILE email: [email protected] Consider the following supposedly-simple problem: compute x satisfying x ∈ S , where S is a convex set conveyed by a separation oracle, with no further information (e.g., no bounding ball containing or intersecting S, etc.). Our interest in this problem stems from fundamental issues involving the interplay of (i) the computational complexity of computing a point x ∈ S, (ii) the geometry of S, and (iii) the stability or conditioning of S under perturbation. Under suitable definitions of these terms, we show herein that problem instances with favorable geometry have favorable computational complexity, validating conventional wisdom. We also show a converse of this implication, by showing that there exist problem instances in certain families characterized by unfavorable geometry, that require more computational effort to solve. This in turn leads, under certain assumptions, to a form of equivalence among computational complexity, the geometry of S, and the conditioning of S. Our measures of the geometry of S, relative to a given (reference) point x ¯, are the aspect ratio A = R/r, as well as R and 1/r, where B(¯ x, R) ∩ S contains a ball of radius r. The aspect ratio arises in the analyses of many algorithms for convex problems, and its importance in convex algorithm analysis has been well-known for several decades. However, the terms R and 1/r in our complexity results are a bit counter-intuitive; nevertheless, we show that the computational complexity must involve these terms in addition to the aspect ratio even when the aspect ratio itself is small. This lower-bound complexity analysis relies on simple features of the separation oracle model of conveying S; if we instead assume that S is conveyed by a self-concordant barrier function, then it is an open challenge to prove such complexity lower-bound. Key words: Convex Optimization ; Ellipsoid Algorithm ; Computational Complexity MSC2000 Subject Classification: Primary: 90C25 , 90C60 ; Secondary: 52A41 , 52A40, 52A20 OR/MS subject classification: Primary: Programming: nonlinear, algorithms, theory ; Secondary: Mathematics: convexity

1. Introduction, Motivation, and Discussion Consider the following supposedly-simple problem: compute x satisfying x ∈ S , (1) where S ⊂ X is a convex set (bounded or not, open or not) conveyed by a separation oracle with no further information (e.g., no bounding ball containing or intersecting S, etc.), and X is a (finite) ndimensional vector space. Our interest in (1) stems from fundamental issues involving the interplay of three notions: (i) the computational complexity of computing a point x ∈ S, (ii) the geometry of S, and (iii) the stability or conditioning of S. In this paper we focus on the equivalence of computational complexity and a suitable measure of the geometry of S, which leads under certain assumptions to an equivalence of all three notions. There are two standard information models for convex sets, the separation oracle model and the (selfconcordant) barrier model. A separation oracle for S, see [12], is a subroutine that, given a point x ˆ as input, returns the statement “ˆ x ∈ S” if indeed this is the case, or if x ˆ ∈ / S, returns a hyperplane H with the property that x ˆ ∈ H − and S ⊂ H ++ . Here H ⊂ 0, next check if x ˆ := x ¯ + (w ˆ−x ¯)/θˆ is in S. If so, then (w, ˆ θ) T T we are done. Otherwise, the separation oracle for S outputs h 6= ˆ for all x ∈ S,  0 for which h x ≥ h x T T x ¯ ˆ which then implies that h (¯ x + (w − x ¯)/θ) ≥ h x ¯ + (w ˆ−x ¯)/θ for all (w, θ) ∈ W . Simplifying yields + T T T T ˆ ˆ ˆ in this H := {(w, θ) : θh w − θh x ¯ ≥ θ(h w ˆ−h x ¯)} as the requisite separating halfspace for (w, ˆ θ) case. Because W x¯ is a (translated) convex cone in 2 } for which S ⊂ H . Henceforth in this proof and other proofs 0 0 }.” If instead (x1 )1 > L0 +U , the oracle will return we simply denote this as “S¯ ⊂ {x ∈ L0 +U 2 2 L0 +U0 1 n 0 ¯ ¯ “x ∈ / S” together with “S ⊂ {x ∈ < : x1 < 2 }.” In the first case we will define L1 := L0 +U and 2 L0 +U0 U1 := U0 , whereas in the second case we define L1 := L0 and U1 := 2 . We will construct the output of the separation oracle in subsequent iterations in a manner that generalizes the above logic. After k oracle calls we will have two scalar values Lk and Uk satisfying Lk < Uk , and the algorithm will have ¯ together with separating halfspaces of generated x1 , . . . , xk for which the oracle has responded “xi ∈ / S” Li−1 +Ui−1 n ¯ the form “S ⊂ {x ∈ < : x1 > (or }.” 2 k+1 k ¯ together with the separating halfspace If instead (xk+1 )1 > Lk +U , the oracle will return “x ∈ / S” 2 k k “S¯ ⊂ {x ∈