Ross Anderson's Personal Website

MIT IAP Software Tools for Operations Research 2013: Integer Programming Callbacks

Wed, 13 Feb 2013 14:36:00 +0000

The Operations Research Center offered a new course this IAP, 15.S60 SSIM: Software Tools for Operations Research. The course consisted of the following modules (as summarized by Iain Dunning here):

introductory and advanced classes on R (Allison O’Hair, Andre Calmon, John Silberholz),
data visualization (Virot Chiraphadhanakul),
using Python for mathematical modelling (Iain Dunning),
convex optimization (Vishal Gupta),
distributed computing for optimization (John Silberholz),
customizing integer programming solvers with callbacks (me).

The tutorial I gave has a self contained wiki that is available to the public here. Briefly, the tutorial covered the following topics for MIP:

Good object oriented design for integer programming
Separation over exponentially many constraints with User Cuts (UserCutCallback) and Lazy Constraints (LazyConstraintCallback)
Giving CPLEX a starting solution (addMIPStart)
Generating integer solutions during branch and bound with a construction heuristic (HeuristicCallback)
Improving integer solutions found by CPLEX during branch and bound with an improvement heuristic (IncumbentCallback with HeuristicCallback)
Profiling and testing for IP solvers

The traveling salesmen problem is used as a running example. The implementation uses the Java interface to CPLEX.

From Separation to Optimization in CPLEX: Using User Cut Callbacks and Lazy Constraint Callbacks to Solve Large TSPs in Java

Sat, 16 Jun 2012 21:08:00 +0000

Update (February 2, 2013): some of the material here has been superceded by a more recent post based on the tutorial I gave for the 2013 MIT IAP. However, that tutorial does not discuss multi-threading or the Karger Stein algorithm, which are covered below.

Introduction

For many important problems in combinatorial optimization, known efficient algorithms (and approximation algorithms) rely on a separation oracle to dynamically generate violated constraints from an exponentially large pool of constraints that describe the problem. To prove that these algorithms run in polynomial time, the ellipsoid method and other theoretically efficient but impractical tools are often used. In practice however, dynamic constraint generation has been incorporated into simplex based integer programming algorithms successfully for many problems.

One such problem is the famous traveling salesman problem (TSP), one of the must well studied problems in combinatorial optimization. The problem is as follows: given a set of cities, or vertices $V$ a set of direct routes between cities, or edges $E$, and for each edge $e \in E$ a distance $d_e$, what is the shortest tour through all the cities that visits each city exactly once? As this problem is quite difficult, we often consider the special case where $d_e$ is assumed to be metric, the metric TSP.

The most powerful metric TSP solver is Concorde, which has been used to solve instances with as many as 15,000 cities exactly. Concorde uses linear programming and integer programming techniques, and can be configured to use multiple linear programming solvers. One such solver is CPLEX, one of the fastest and most reliable solvers for both linear and integer programming. However, concorde only uses CPLEX for linear programming and then runs its own custom algorithms to produce integer solutions.

With the release of CPLEX 12.3, a new API for dynamic constraint generation was created: Lazy Constraint Callbacks and User Cut Callbacks. In this article, we show how to use build a TSP solver entirely within CPLEX using these new dynamic constraint generation features. As general purpose integer programming solvers such as CPLEX continue to improve over time, a TSP solver that relies on a general purpose IP solver may one day surpass Concorde and other specialized tools. However, as Concorde uses heuristics which are very effective for TSP, these heuristics would need to be integrated with CPLEX for CPLEX to be competitive.

Integer Programming Formulations of the TSP

The TSP has several integer programming formulations. We will describe two, the “subtour elimination” formulation and the “cut set” formulation. In both formulations, for every edge $e \in E$, there is a binary decision variable $x_e$ that indicates where or not edge $e$ is included in the tour. For each node $v$, we let $\delta(v)$ be the set of edges incident to node $v$. For $S \subset V$, we let $E(S)$ be the set of edges with both endpoints in $S$. The subtour elimination formulation is given by

$\begin{align} &\min &\sum_{e \in E} d_e y_e&\\ &\text{subject to} &\sum_{e \in \delta(v)} y_e &= 2 &\forall v &\in V,\\ &&\sum_{e \in E(S)} y_e &\leq |S|-1 &\forall S &\subset V, \, S \neq V,\, S\neq 0,\\ &&y_e &\in \{0,1\} &\forall e &\in E.\\ \end{align}$

Let $\delta(S)$ for all $S \subset V$ be the set of edges with one endpoint in $S$ and the other in $V \setminus S$. In this notation, the cutset formulation is given by

$\begin{align} &\min &\sum_{e \in E} d_e y_e& \\ &\text{subject to} &\sum_{e \in \delta(v)} y_e &= 2 &\forall v &\in V,\\ && \sum_{e \in \delta(S)} y_e &\geq 2 &\forall S &\subset V, \, S \neq V,\, S\neq 0,\\ && y_e &\in \{0,1\} &\forall e &\in E.\\ \end{align}$

The two formulations are of equivalent strength, in that LP relaxations for both programs give the same polyhedron. The solution to the linear programming relaxation is known as the Held-Karp lower bound. In practice, the optimal solution tends to be within a few percentage points of this lower bound. Thus solving the LP relaxation and then doing a little branch and bound is generally a good technique for proving optimality of a solution. However, as the LP relaxations have exponentially many constraints, we need to do dynamic constraint generation.

Implementation in CPLEX

Here we show a few CPLEX based implementations of a TSP solver using dynamic constraint generation. CPLEX gives two options for generating constraints:

After every fractional solution is generated, we can have a UserCutCallback that allows us to search for constraints violated by the fractional solution.
After every integer solution is generated, we can have a LazyConstraintCallback that allows us to search for constraints violated by the integer solution.

In the case of TSP, the second search problem (identifying constraints violated by an integer solution) is much easier than the first. When all the edge variables are integer and the degree constraints are satisfied, the edge variables will just be set of loops. Thus we can determine in $O(|E|)$ time which subtour elimination or cutset constraints to add. However, when the edge variables are still fractional, we can only conveniently separate over the cutset constraints, and to do so we need to solve a minimum cut problem on the original graph but with edge weights as the value of the edge variable from the LP relaxation. The problem can be solved with high probability in $O(|V|^2 \log^3(|V|))$ with the Karger Stein Algorithm, significantly slower than the $O(|E|)$ time when all edge variables were integer. In practice, however, we find that using a combination of these two approaches is much faster than only separating integer solutions.

We take three approaches. First, we give a minimal solution that uses only LazyConstraintCallback. Then, we discuss some engineering and design issues with our first approach and show how to use the Guava libraries to make our code more reliable. Finally, we give a well engineered implementation using both UserCutCallbacks and LazyConstraintCallbacks. In each approach, we assume our input is given to us as an UndirectedGraph from the JUNG library and edge weights.

A minimal implementation using LazyConstraintCallback

First, we give the implementation, then we discuss some issues.

public class SloppyTsp {

  private static double tolerance = .00001;
  private UndirectedGraph graph;
  private IloCplex cplex;
  private IloIntVar[] edgeVars;
  private Map edgeIndex;
  private List edges;
  private Map vertexIndex;
  private List vertices;
  private LoopExtractor loopExtractor;

  public SloppyTsp(UndirectedGraph graph) throws IloException {
    super();
    this.graph = graph;
    this.cplex = new IloCplex();
    this.loopExtractor = LoopExtractorImpl.instance;
    this.edgeVars = cplex.boolVarArray(graph.getEdgeCount());
    this.edgeIndex = new HashMap();
    this.edges = new ArrayList(graph.getEdges());
    this.vertexIndex = new HashMap();
    this.vertices = new ArrayList(graph.getVertices());
    IloLinearIntExpr[] degreeConstraints =
        new IloLinearIntExpr[vertices.size()];
    int v = 0;
    for(V vertex: vertices){
      degreeConstraints[v] = cplex.linearIntExpr();
      vertexIndex.put(vertex, v++);
    }
    int e =0;
    for(E edge: edges){
      edgeIndex.put(edge, e++);
      IloIntVar edgeVar = edgeVars[edgeIndex.get(edge)];
      for(V vertex :graph.getEndpoints(edge)){
        degreeConstraints[vertexIndex.get(vertex)].addTerm(edgeVar, 1);
      }
    }
    for(int u = 0; u< vertices.size(); u++){
      cplex.addEq(degreeConstraints[u], 2);
    }
    cplex.use(new SubtourElimination());
  }

  public void solve() throws IloException {
    this.cplex.solve();
  }

  public List getTour() throws IloException{
    Set selected = new HashSet();
    double[] ipSolution = cplex.getValues(this.edgeVars);
    for(int e = 0; e< ipSolution.length; e++){
      if(Math.abs(ipSolution[e]-1) < tolerance){
        selected.add(edges.get(e));
      }
    }
    List tours = loopExtractor.subTours(graph, selected);
    if(tours.size() != 1){
      throw new RuntimeException();
    }
    return tours.get(0);
  }

  private class SubtourElimination extends IloCplex.LazyConstraintCallback{

    @Override
    protected void main() throws IloException {
      Set edgesUsed = new HashSet();
      double[] values = this.getValues(edgeVars);
      for(int e = 0; e < edges.size(); e++){
        if(Math.abs(values[e]-1)< tolerance){
          edgesUsed.add(edges.get(e));
        }
      }
      List tours = loopExtractor.subTours(graph, edgesUsed);
      if(tours.size() != 1){
        for(List subtour: tours){
          IloLinearIntExpr subtourVars = cplex.linearIntExpr();
          for(E edge: subtour){
            subtourVars.addTerm(edgeVars[edgeIndex.get(edge)], 1);
          }
          this.add(cplex.le(subtourVars, subtour.size()-1));
        }
      }
    }
  }
}
public interface LoopExtractor {

  public List subTours(UndirectedGraph graph, Set edgesSelected);

}

The implementation of the LoopExtractor is not important to the discussion, but is available in the zip file. We see that with not very much code, it is possible to do dynamic constraint generation. However, this implementation could be greatly improved from an engineering perspective. In particular:

We create a map between the edges of the graph and an index, and a second map (in the form of an array) from our index to the edge variables (and similarly with the vertices and constraints). Storing variables in a large array actually seems to be somewhat typical in the examples that come with CPLEX. However, we really just need a map between the edges of the graph and variables, the index is somewhat irrelevant. Further, the indices increase the chance of making a coding error, particularly when you have multidimensional arrays of variables.
We have no way of unit testing individual components of the IP.
We cannot reuse any of this code to solve related problems. For example, if we wanted to solve a multistage problem where in each time step, you solved a TSP, and there were some additional linking constraints to relate the problems between time steps, we would have to start from scratch. Fundamentally, this problem and our previous problem are quite similar: our code is not modular.

These problems are pervasive in the sample Java code that is distributed with CPLEX. In the next section, we address these issues with a slightly more complicated implementation.

An improved implementation of the LazyConstraintCallback solution

First, we get rid of the variable indices. To do this, one might be tempted to make a HashMap from the edges to the variables and a second HashMap from the variables to the edges, which would give us $O(1)$ lookup time in both directions. However, there is a better way: BiMap from the Guava library. Additionally, we will make a new class, EdgeVariables, to hold the BiMap. The purpose is two-fold. First, there are many other problems will modeled by integer and linear programs where you input a graph and want to create an edge for every variable, such as maximum weight matching. We can reuse our EdgeVariables class when solving these problems. Second is type safety. Suppose we want to write a function that takes in our edge variables and generates the degree constraints. If the function takes in a BiMap of edges and variables, if we had created another such map, we could pass in the wrong map. One might be concerned about the memory overhead and time wasted by creating all of these extra HashMaps, as they are not strictly necessary. However, at least in Java, the time and memory spent building these slightly heavier data structures (as compared to arrays) is inconsequential given that we are about to solve an integer program over these variables. The code to create the BiMap is below:

public class CplexUtil {
  public static  ImmutableBiMap makeBinaryVariables(IloCplex cplex,
      Iterable set) throws IloException{
    Builder ans = ImmutableBiMap.builder();
    for(T t: set){
      ans.put(t, cplex.boolVar());
    }
    return ans.build();
  }

  ...
}
public class EdgeVariables {

  private ImmutableBiMap edgeVars;

  public EdgeVariables(UndirectedGraph graph,
      IloCplex cplex) throws IloException{
    edgeVars = CplexUtil.makeBinaryVariables(cplex,
        graph.getEdges());
  }

  public ImmutableBiMap getEdgeVars() {
    return edgeVars;
  }
}

Notice that we actually create an ImmutableBiMap, a general good practice, as we don’t want to accidentally modify our edge variable mapping later. Next, we create the degree constraints. We use a similar technique.

public class CplexUtil {
  ...
  public static <T> ImmutableBiMap<T,IloLinearIntExpr> makeLinearIntExpr(
      IloCplex cplex, Iterable<T> set) throws IloException{
    Builder<T,IloLinearIntExpr> ans = ImmutableBiMap.builder();
    for(T t: set){
      ans.put(t, cplex.linearIntExpr());
    }
    return ans.build();
  }
  ...
}

public class DegreeConstraints<V,E> {

  private EdgeVariables<V,E> edgeVariables;
  private UndirectedGraph<V,E> graph;
  private IloCplex cplex;
  private ImmutableBiMap<V,IloLinearIntExpr> degree;
  private ImmutableBiMap<V,IloRange> degreeConstraint;
  private int degreeEquals;

  public DegreeConstraints(EdgeVariables<V, E> edgeVariables,
      UndirectedGraph<V, E> graph, IloCplex cplex,
      int degreeEquals) throws IloException {
    super();
    this.edgeVariables = edgeVariables;
    this.graph = graph;
    this.cplex = cplex;
    this.degreeEquals = degreeEquals;
    degree = CplexUtil.makeLinearIntExpr(cplex, graph.getVertices());
    for(E edge: graph.getEdges()){
      Pair<V> endpoints = graph.getEndpoints(edge);
      for(V vertex: endpoints){
        degree.get(vertex).addTerm(edgeVariables.getEdgeVars().get(edge), 1);
      }
    }
    Builder<V,IloRange> builder = ImmutableBiMap.builder();
    for(V vertex: degree.keySet()){
      builder.put(vertex, cplex.addEq(degree.get(vertex), degreeEquals));
    }
    degreeConstraint = builder.build();
  }

  public DegreeConstraints(UndirectedGraph<V, E> graph, IloCplex cplex,
      int degreeEquals) throws IloException  {
    this(new EdgeVariables<V,E>(graph,cplex),graph,cplex,degreeEquals);
  }

}

We also create a new class for the subtour elimination callback using the EdgeVariables class.

public class LazySubtourElimination<V,E> {

  private UndirectedGraph<V,E> graph;
  private IloCplex cplex;
  private EdgeVariables<V,E> edgeVariables;
  private boolean printOutput;
  private SubtourEliminationCallback callback;

  public LazySubtourElimination(UndirectedGraph<V, E> graph, IloCplex cplex,
                               EdgeVariables<V,E> edgeVariables,
                               boolean printOutput) throws IloException {
    super();
    this.graph = graph;
    this.cplex = cplex;
    this.edgeVariables = edgeVariables;
    this.callback = new SubtourEliminationCallback();
    cplex.use(callback);
  }

  public class SubtourEliminationCallback extends
      IloCplex.LazyConstraintCallback {
    @Override
    protected void main() throws IloException {
      Set<E> edgesUsed = new HashSet<E>();
      IloIntVar[] edgeVars = edgeVariables.getEdgeVars().values()
                                          .toArray(new IloIntVar[]{});
      double[] values = this.getValues(edgeVars);
      for(int i = 0; i < edgeVars.length; i++){
        if(CplexUtil.doubleToBoolean(values[i])){
          edgesUsed.add(edgeVariables.getEdgeVars()
                                     .inverse().get(edgeVars[i]));
        }
      }
      List<List<E>> loops = LoopExtractorImpl.instance
                                             .subTours(graph,edgesUsed);
      if(loops.size()>1){
        for(List<E> loop: loops){
          IloLinearIntExpr loopVars = cplex.linearIntExpr();
          for(E e: loop){
            loopVars.addTerm(edgeVariables.getEdgeVars().get(e), 1);
          }
          this.add(cplex.le(loopVars, loop.size()-1));
        }
        print("Added " + loops.size() + " lazy constraints");
      }
    }
  }

  private void print(String s){
    if(this.printOutput){
      System.out.println(s);
    }
  }
}

Note that we must resort to using arrays to extract the value of variables as LazyConstraintCallback.getValues() method is much faster than calling LazyConstraintCallback.getValue() inside a loop. This seems to be a weakness in either the CPLEX library or JNI. We create a min weight objective class similarly to how we created our constraint classes. Now all that remains is to put all our pieces together. In anticipation of having a second implementation based on the cutset formulation, we first create an abstract class for all TSP solutions based on the EdgeVariables class.

public abstract class AbstractTspFormulationEdgeBased<V,E> {

  protected UndirectedGraph<V,E> graph;
  protected Transformer<E,Double> edgeWeights;
  protected EdgeVariables<V,E> edgeVariables;
  protected IloCplex cplex;
  protected DegreeConstraints<V,E> degreeConstraints;
  protected MinEdgeWeightObjective<V,E> objective;
  protected List<E> optimalTour;
  protected double optimalCost;

  protected AbstractTspFormulationEdgeBased(UndirectedGraph<V,E> graph,
      Transformer<E,Double> edgeWeights, IloCplex cplex) throws IloException{
    this.graph = graph;
    this.edgeWeights = edgeWeights;
    this.cplex = cplex;
    this.edgeVariables = new EdgeVariables<V,E>(graph,cplex);
    this.degreeConstraints = new DegreeConstraints<V,E>(edgeVariables,graph,
                                                        cplex,2);
    this.objective = new MinEdgeWeightObjective<V,E>(graph,edgeWeights,
                                                     edgeVariables,cplex);
    cplex.setParam(IloCplex.IntParam.Threads, 4);
  }

  public void solve() throws IloException{
    this.cplex.solve();
    if(cplex.getStatus() == Status.Feasible ||
       cplex.getStatus() == Status.Optimal){
      this.optimalCost = cplex.getObjValue();
      this.optimalTour = extractTour();
    }
    else{
      this.optimalCost = 0;
      this.optimalTour = null;
    }
  }

  public void cleanUp(){
    cplex.end();
  }

  public List<E> getOptimalTour() {
    return optimalTour;
  }

  public double getOptimalCost() {
    return optimalCost;
  }

  protected List<E> extractTour() throws IloException{
    List<List<E>> tours = extractTours();
    if(tours.size() != 1){
      throw new RuntimeException();
    }
    return tours.get(0);
  }

  protected List<List<E>> extractTours() throws IloException{
    Set<E> edgesUsed = new HashSet<E>();
    for(E edge: edgeVariables.getEdgeVars().keySet() ){
      if(CplexUtil.doubleToBoolean(cplex.getValue(edgeVariables.getEdgeVars()
                                                               .get(edge)))){
        edgesUsed.add(edge);
      }
    }
    List<List<E>> tours = LoopExtractorImpl.instance
                                           .subTours(graph, edgesUsed);
    return tours;
  }
}

public class TspSubTourEliminationLazy<V, E> extends
    AbstractTspFormulationEdgeBased<V, E> {

  private LazySubtourElimination<V,E> lazySubtourElimination;

  public TspSubTourEliminationLazy(UndirectedGraph<V, E> graph,
                                   Transformer<E, Double> edgeWeights,
                                   boolean printOutput) throws IloException{
    this(graph,edgeWeights,printOutput,new IloCplex());
  }

  public TspSubTourEliminationLazy(UndirectedGraph<V, E> graph,
      Transformer<E, Double> edgeWeights, boolean printOutput, IloCplex cplex)
      throws IloException {
    super(graph, edgeWeights, cplex);
    this.lazySubtourElimination =
      new LazySubtourElimination<V,E>(graph, cplex,
                                      edgeVariables, printOutput);
  }
}

Given our new modular design, we could easily reuse our code to solve some kind of multistage optimization problem, and unit testing our code is now possible. For example:

@Test
public void testDegreeConstraints(){
  UndirectedGraph<Node,Node.Edge> graph = Node.makeGraph();
  Transformer<Node.Edge,Double> edgeWeights = Node.makeEdgeWeights();
  try {
    IloCplex cplex = new IloCplex();
    EdgeVariables<Node,Node.Edge> vars =
      new EdgeVariables<Node,Node.Edge>(graph,cplex);
    DegreeConstraints<Node,Node.Edge> degreeConstraints =
      new DegreeConstraints<Node,Node.Edge>(vars,graph,cplex,2);
    MinEdgeWeightObjective<Node, Node.Edge> objective =
      new MinEdgeWeightObjective<Node,Node.Edge>(graph,edgeWeights,
                                                 vars,cplex);
    cplex.solve();
    assertEquals(6,cplex.getObjValue(),tolerance);
    Set<Node.Edge> edgesUsed = new HashSet<Node.Edge>();
    for(Node.Edge edge: vars.getEdgeVars().keySet() ){
      if(CplexUtil.doubleToBoolean(
          cplex.getValue(vars.getEdgeVars().get(edge)))){
        edgesUsed.add(edge);
      }
    }
    List<List<Node.Edge>> tours =
      LoopExtractorImpl.instance.subTours(graph, edgesUsed);
    assertEquals(2,tours.size());
    assertEquals(3,tours.get(0).size());
    assertEquals(3,tours.get(1).size());
    Set<EnumSet<Node.Edge>> toursAsSet = new HashSet<EnumSet<Node.Edge>>();
    for(List<Node.Edge> tour: tours){
      toursAsSet.add(EnumSet.copyOf(tour));
    }
    Set<EnumSet<Node.Edge>> expected = new HashSet<EnumSet<Node.Edge>>();
    expected.add(EnumSet.of(Node.Edge.ab,Node.Edge.bc, Node.Edge.ac));
    expected.add(EnumSet.of(Node.Edge.de,Node.Edge.ef, Node.Edge.df));
    assertEquals(expected,toursAsSet);
    cplex.end();
  } catch (IloException e) {
    throw new RuntimeException(e);
  }
}

A fast solution with UserCutCallback and the Karger Stein algorithm for separation

Finally, we give a third implementation based on UserCutCallback. Recall that the UserCutCallback will allow us to look for and add violated constraints to fractional solutions. Doing so will allow us to solve the LP relaxation before branch and bound begins. Generally the LP relaxation is within a few percentage points of optimal, so there is little work to be done in the branch and bound stage. This greatly improves the performance of the algorithm.

For the TSP, violated constraints can be identified by solving the global min cut problem. We use the randomized Karger Stein algorithm, which will find the minimum cut with high probability. Our algorithm is still guaranteed to be correct, as even when Karger Stein fails by chance, we still have a fall back of our LazyConstraintCallback to catch any violated constraints. In fact, you can only use a UserCutCallback when the callback does not change the set of feasible integer points, as CPLEX does not guarantee that the UserCutCallback will be checked. In particular, if the first LP you solve is integral, the UserCutCallback will not be checked. Thus if you are using UserCutCallbacks to for dynamic constraint generation, you must also have a LazyCutCallback that will check the constraints on integer solutions. The key parts of the implementation are below:

public interface MinCutSolver<V,E> {

  /**
   * @param value the minimum value of a cut to keep
   * @return at least one cut less than value if such a cut exists, otherwise
   *         an empty list.
   */
  public Iterable<Cut<E>> findCutsLessThan(UndirectedGraph<V,E> graph,
      Transformer<E,Number> edgeWeights, double value);
}
public class UserCallbackCutSet<V,E> {

  private UndirectedGraph<V,E> graph;
  private IloCplex cplex;
  private EdgeVariables<V,E> edgeVariables;
  private MinCutSolver<V,E> minCutSolver;
  private double cutVal;
  private boolean haltUser;

  public UserCallbackCutSet(UndirectedGraph<V, E> graph, IloCplex cplex,
      EdgeVariables<V,E> edgeVariables,
      MinCutSolver<V,E> minCutSolver) throws IloException {
    this(graph,cplex,edgeVariables,minCutSolver,2);
  }

  public UserCallbackCutSet(UndirectedGraph<V, E> graph, IloCplex cplex,
      EdgeVariables<V,E> edgeVariables,MinCutSolver<V,E> minCutSolver,
      double cutVal) throws IloException {
    super();
    this.graph = graph;
    this.cplex = cplex;
    this.edgeVariables = edgeVariables;
    this.minCutSolver = minCutSolver;
    this.cutVal = cutVal;
    haltUser = false;
    cplex.use(new CutSetCallback());
  }

  private class CutSetCallback extends IloCplex.UserCutCallback {

    public CutSetCallback(){}

    @Override
    protected void main() throws IloException {
      if(!this.isAfterCutLoop()){
        return;
      }
      if(haltUser){
        return;
      }
      final Map<E,Double> edgeWeights =
        new HashMap<E,Double>(edgeVariables.getEdgeVars().size()*2);
      boolean isIntegral = true;
      IloIntVar[] edgeVars = edgeVariables.getEdgeVars().values()
                                          .toArray(new IloIntVar[]{});
      double[] edgeVarVals = this.getValues(edgeVars);
      for(int i = 0; i < edgeVars.length; i++){
        edgeWeights.put(edgeVariables.getEdgeVars().inverse()
                                                   .get(edgeVars[i]),
                        Double.valueOf(edgeVarVals[i]));
        isIntegral &= CplexUtil.isBinaryIntegral(edgeVarVals[i]);
      }
      if(isIntegral){
        System.err.println("Found integral in user callback");
        return;
      }
      Iterable<Cut<E>> cuts = minCutSolver.findCutsLessThan(graph,
        new Transformer<E,Number>(){
          @Override
          public Number transform(E arg0) {
            return edgeWeights.get(arg0);
          }
        }, 2);
      double bestCut = Double.MAX_VALUE;
      for(Cut<E> graphCutEdges: cuts){
        IloLinearIntExpr cut = cplex.linearIntExpr();
        double cutVal = graphCutEdges.getWeight();
        bestCut = Math.min(bestCut, cutVal);
        for(E edge: graphCutEdges.getEdges()){
          cut.addTerm(edgeVariables.getEdgeVars().get(edge), 1);
        }
        this.add(cplex.ge(cut, 2));
      }
      if(bestCut > cutVal){
        haltUser = true;
        System.out.println("halting user cuts");
      }
    }
  }
}

public class TspUserCutSetIp<V, E> extends
    AbstractTspFormulationEdgeBased<V, E> {

  private MinCutSolver<V,E> minCutSolver;
  private LazySubtourElimination<V,E> lazySubtourElimination;
  private UserCallbackCutSet<V,E> userCallbackCutSet;

  public TspUserCutSetIp(UndirectedGraph<V, E> graph,
      Transformer<E, Double> edgeWeights, MinCutSolver<V,E> minCutSolver,
      double cutVal, boolean printOutput) throws IloException {
    this(graph,edgeWeights,minCutSolver, cutVal,printOutput,new IloCplex());
  }

  public TspUserCutSetIp(UndirectedGraph<V, E> graph,
      Transformer<E, Double> edgeWeights,MinCutSolver<V,E> minCutSolver,
      double cutVal, boolean printOutput,IloCplex cplex) throws IloException {
    super(graph, edgeWeights, cplex);
    cplex.setParam(IloCplex.IntParam.Threads, 8);
    this.minCutSolver = minCutSolver;
    this.lazySubtourElimination =
      new LazySubtourElimination<V,E>(graph,cplex,edgeVariables,printOutput);
    this.userCallbackCutSet =
      new UserCallbackCutSet<V,E>(graph, cplex, edgeVariables,
                                  minCutSolver, cutVal);
  }
}

Note the somewhat mysterious UserCutCallback.isAfterCutLoop() in the UserCutCallback. If CPLEX successfully found another kind of cut to add to the problem, this section of code will prevent us from generating constraints and give CPLEX more chances to look for other cuts. In our case, it is a good idea to include this, as often CPLEX will be able to find cuts faster than we can, as solving the minimum global cut problem is difficult.

There are a few optimizations present in the code. First, we do not require that the global min cut problem for identifying violated constraints is solved to optimality, and we allow the global min cut solver to return multiple violated cuts. We then add all these constraints each callback. In fact, our Karger Stein implementation stops immediately when it finds cuts that are “good enough,” which greatly speeds up the algorithm. When we have found a solution that nearly satisfies all of the LP constraints, we “halt” the UserCutCallback and instead move to branch and bound and rely on the LazyCutCallback instead. We do this because when the LP relaxation is nearly satisfied, it takes much longer to find violated constraints (since we cannot terminate early). Also when the LP relaxation is nearly satisfied, we have a very good lower bound, which makes branch and bound very effective.

Conclusion

We saw that large constraint generation schemes for TSP and similar problems can be easily implemented using LazyConstraintCallback and a separation oracle. We briefly discussed basic principles of object oriented design applied to the design and testing of integer programs, using our TSP solver as an example. We integrated well established Java libraries of Guava, JUNG, and CPLEX Concert Technologies to accomplish our task with a small amount of code. Finally, we greatly improved our algorithm’s performance by using UserCutCallback and a Karger Stein separation oracle in conjunction with our LazyConstraintCallback.

At this point, our solver would probably benefit most from some good rounding schemes to generate integer solutions once we have solved the LP relaxation, and good local search heuristics to improve on integer solutions we have already found. CPLEX has additional callbacks for inserting integer solutions. Additionally, a more careful implementation of the Karger Stein algorithm (that ran faster and used less memory) would also likely be of value. It would be interesting to see if such a solver could compete with or surpass Concorde.

The complete source code from these examples can be downloaded from an Assembla subversion repository here. There is also a TSPLib parser and the TSPLib sample problems.

All Pairs Shortest Path in Parallel with Floyd Warshal in Java

Wed, 13 Jun 2012 11:54:00 +0000

The All Pairs Shortest Path (APSP) problem is to compute the shortest path between every pair of points in a directed weighted graph. The Floyd Warshall algorithm is a dynamic programming algorithm that solves the APSP problem in $O(V^3)$ time. The running time is impressive, as there are $O(V^2)$ pairs of nodes, so the average time spent per pair is $O(V)$, faster than solving a standard shortest path problem.

Provided below is a parallel implementation of the algorithm. There is also a wrapper for the JUNG library so this code can be used to solve APSP for a JUNG DirectedGraph, and JUnit tests, in this Assembla SVN repository. On a related note, JGraphT also provides an implementation of APSP, but it is not parallel.

Warning: do not forget to shut down your ExecutorService, we don’t do it!

Edit: There is an updated version in the Assembla repository that supports both single threaded and multi threaded modes. Also there is a Demo class to test the performance on large random graphs. On a graph of 3000 nodes, the single threaded mode took 130 seconds, while the multithreaded mode (on an 8 core machine) took 37 seconds. The single threaded mode only seems to be faster for graphs with about 100 nodes or fewer, and on these graphs, the running time is about 15 milliseconds.

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Future;

public class ParallelFloydWarshall {
  
  private ExecutorService exec;
  private int numThreads;
  private double[] current;
  private double[] next;
  
  private int[] maxIndex;
  private int numNodes;
  private boolean solved;
  
  private int getIndex(int i, int j){
    return i*numNodes+j;
  }
  
  private int getI(int index){
    return index / numNodes;
  }
  
  private int getJ(int index){
    return index % numNodes;
  }
  
  /**
   * @param numNodes the number of nodes in the graph
   * @param distances the matrix of distances between nodes, indexed from 0 to
   *                  numNodes-1.  distances[i][j] cost of a directed edge from
   *                  i to j.  Must be Double.POSITIVE_INFINITY if the edge is
   *                  not present.  distance[i][i] is a self arc (allowed).
   */
  public ParallelFloydWarshall(int numNodes, double[][] distances,
                               ExecutorService exec, int numThreads){
    this.exec = exec;
    this.numThreads = numThreads;
    this.numNodes = numNodes;
    this.current = new double[numNodes*numNodes];
    this.next = new double[numNodes*numNodes];
    this.maxIndex = new int[numNodes*numNodes];
    Arrays.fill(maxIndex, -1);
    for(int i = 0; i < numNodes; i++){
      for(int j = 0; j < numNodes; j++){
        current[getIndex(i,j)] = distances[i][j];
      }
    }
    this.solved = false;
  }
  
  public void solve(){
    if(solved){
      throw new RuntimeException("Already solved");
    }
    for(int k = 0; k < numNodes; k++){
      List<Callable<Boolean>> tasks = new ArrayList<Callable<Boolean>>();
      if(current.length < numThreads){
        for(int i = 0; i < current.length; i++){
          tasks.add(new FloydJob(i,i+1,k));
        }
      }
      else{
        for(int t = 0; t < numThreads; t++){
          int lo = t*current.length/numThreads;
          int hi = (t+1)*current.length/numThreads;
          tasks.add(new FloydJob(lo,hi,k));
        }
      }
      try {
        List<Future<Boolean>> results = this.exec.invokeAll(tasks);
        for(Future<Boolean> result : results){
          if(!result.get().booleanValue()){
            throw new RuntimeException();
          }
        }
      } catch (InterruptedException e) {
        throw new RuntimeException(e);
      } catch (ExecutionException e) {
        throw new RuntimeException(e);
      }
      double[] temp = current;
      current = next;
      next = temp;      
    }
    next = null;
    solved = true;
  }
  
  /**
   * 
   * @param i must lie in in [0,numNodes)
   * @param j must lie in in [0,numNodes)
   * @return the length of the shortest directed path from node i to node j.
   *         If i == j, gives the shortest directed cycle starting at node i
   *          (note that the graph may contain nodes with self loops).  Returns
   *         Double.POSITIVE_INFINITY if there is no path from i to j.
   */
  public double shorestPathLength(int i, int j){
    if(!solved){
      throw new RuntimeException("Must solve first");
    }
    return this.current[getIndex(i,j)];
  }
  /**
   * Example: If the path from node 2 to node 5 is an edge from 2 to 3 and then
   * an edge from 3 to 5, the return value will be
   * Arrays.asList(Integer.valueOf(2),Integer.valueOf(3),Integer.valueOf(5));
   * 
   * @param i the start of the directed path
   * @param j the end of the directed path
   * @return The shortest path starting at node i and ending at node j, or null
   *         if no such path exists.
   */
  public List<Integer> shortestPath(int i, int j){    
    if(current[getIndex(i,j)] == Double.POSITIVE_INFINITY){
      return null;
    }
    else{
      List<Integer> ans = new ArrayList<Integer>();      
      ans.add(Integer.valueOf(i));
      shortestPathHelper(i,j,ans);
      return ans;
    }
  }
  
  public void shortestPathHelper(int i, int j, List<Integer> partialPath){
    int index = getIndex(i,j);
    if(this.maxIndex[index] < 0){
      partialPath.add(Integer.valueOf(j));
    }
    else{
      shortestPathHelper(i,this.maxIndex[index],partialPath);
      shortestPathHelper(this.maxIndex[index],j,partialPath);
    }
  }
  
  private class FloydJob implements Callable<Boolean>{
    
    private final int lo;
    private final int hi;
    private final int k;
    
    public FloydJob(int lo, int hi, int k){
      this.lo = lo;
      this.hi = hi;
      this.k = k;
    }

    @Override
    public Boolean call() throws Exception {
      for(int index = lo; index < hi; index++){
        int i = getI(index);
        int j = getJ(index);
        double alternatePathValue = current[getIndex(i,k)]
                                      + current[getIndex(k,j)];
         
        if(alternatePathValue < current[index]){
          next[index] = alternatePathValue;
          maxIndex[index] = k;
        }
        else{
          next[index] = current[index];
        }
      }
      return true;
    }
  }
}

Sorting in Parallel in Java with Executors

Sat, 24 Dec 2011 03:05:00 +0000

Sorting an array in Java (version 6 and below) is typically accomplished using Arrays.sort(), which runs on a single thread. In an application where sorting large arrays is a bottleneck, it could be desirable to parallelize this computation. In this post, we combine the existing Java infrastructure of ExecutorService and Arrays.sort() to implement a new sorting algorithm. This algorithm gets a nice speed up from running in parallel, and at the same time gets the benefits of all optimizations in the Arrays.sort() implementation, as the bulk of the work is done inside calls to this function.

We will refer to this algorithm as HeavySort. We give an informal but intuitive description first. We are given a list $A$ of length $n$ to sort using $m$ threads and a single threaded $\mathop{\text{sort}}()$ implementation. We split $A$ into $m$ equal sized pieces, denoted $B_i$, and sort each piece with $\mathop{\text{sort}}(B_i)$. We now want to merge these pieces $B_i$ back together, but in parallel. We cut $B_1$ into $m$ further equally spaced subsections. Then for each subsection of $B_1$, letting $h$ be the highest element and $\ell$ be the lowest element (these are at the ends, we can find them in constant time), we can in parallel calculate how many elements from $A$ should be merged with this section, by going through the other pieces $B_2,\ldots, B_m$ and doing binary searches for $h$ and $\ell$. Finally, once the number of elements that will be merged with each subsection of $B_1$ is known, we can compute their offset in the final array, then run the standard merging procedure in parallel to write the fully sorted results on top of $A$.

We summarize the above with a more formal description below.

Input: An list $A$ of length $n$ to be sorted with $m$ threads, and a non-parallelizable sorting algorithm $\mathop{\text{sort}}()$

Algorithm:

Divide $A$ into $m$ pieces, copy them into new lists $B_i$ for $i = 1,\ldots,m$, and sort them with $\mathop{\text{sort}}()$.
Build a list $C$ with first the smallest element of $A$, then $m-1$ points evenly spaced from the interior of $B_1$, then the largest element of $A$.
Compute $D[i]$, the number of elements of $A$ that are between $C[i]$ and $ C[i+1]$ (use binary searches).
For each $i = 1,\ldots,m$, merge the values of the sorted lists $B_1,\ldots,B_m$ between $C[i]$ and $C[i+1]$. Copy these values back onto $A$ starting at $ D[1] + \cdots + D[i-1] $

Here is a brief analysis of the the running time. Step 1 takes $ O(n/m \log(n/m))$ for each of $ m$ sorts which can be accomplished in parallel. Step 2 can be accomplished in $ O(m)$ time as we can find the largest and smallest elements of $A$ by checking the end points of each $B_i$. For step 3, we can compute each $D[i]$ in parallel. For $ j = 1,\ldots,m$, we do two binary searches of $B_j$ for $C[i]$ and $C[i+1]$, taking $ O(m \log(n/m))$ in each thread. For step 4, as should be about $ n/m$ elements of $A$ with values between $c[i]$ and $c[i+1]$, and for each element we need to spend $ O(m)$ time to determine which of the $B_j$ is next, we can complete step 4 in $ O(n)$ using $ m$ threads. Thus the total running time is $ O(n/m\log(n/m) + n)$. Note that for all practical purposes ($m \in O(\log n) $) this is equal to $ O(n/m\log(n/m) ) $, i.e. the running time is dominated by Step 1 and Arrays.sort() as desired. Note that in the final step, performance could be improved by using a binary heap to determine which $B_j$, but $ m$ would have to be quite large for this to be practical.

Implicit in this analysis is the assumption that the values in $B_1$ are representative of $A$. While this assumption would usually be true if $A$ began in uniformly random order, it would be violated if $A$ began sorted or nearly sorted. When the assumption is violated, the entire merging would be done by a single thread, making step 4 take $ O(mn)$.

The algorithm is called HeavySort as it uses twice the memory required to store $A$. The extra memory allocation is only necessary as we try and merge in parallel, as is step 3. Since for small $ m$, the sorting in step 1 will be dominant, it would be interesting to see if the cost of making the memory allocation and the binary searches in step 3 are actually saved by parallelizing the merging.

My objective while designing this algorithm was that it would be easy to implement the sorting with Java’s built in Arrays.sort(), and the parallelization with Java’s ExecutorService. The algorithm is implemented in a single class that can be dropped into any Java project, version 1.5 or greater. Below is the source code, JUnit tests, and an example. In practice, I had a 3-4x speedup over Arrays.sort() using 6 threads. The source can be downloaded here.

package parallelSorting;

import java.util.Arrays;
import java.util.Collections;
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

import parallelSorting.HeavySort.ArrayFactory;

public class HeavySortMain {
  
  public static void main(String [] args){
    int  problemSize = 200000000;
    int numThreads = 6;
    Random random = new Random();
    Integer[] sortArray = new Integer[problemSize];
    for (int i=0; i< problemSize ;i++){
      sortArray[i] = random.nextInt(Integer.MAX_VALUE );
    }
    long startTime2 = System.currentTimeMillis();
    Arrays.sort(sortArray);
    System.out.println("Single Threaded Sort: time taken " + 
        (System.currentTimeMillis() - startTime2));
    Collections.shuffle(Arrays.asList(sortArray));
    final ExecutorService executor = Executors.newFixedThreadPool(numThreads);
    long startTime = System.currentTimeMillis();
    ArrayFactory<Integer> factory = new ArrayFactory<Integer>(){

      @Override
      public Integer[] buildArray(int length) {
        return new Integer[length];
      }

    };
    HeavySort.sort(sortArray,executor,numThreads,factory);
    System.out.println("Multi-Threaded sort: time taken " + 
        (System.currentTimeMillis() - startTime));
    

    for (int i=0; i<sortArray.length-1; i++){
      if(sortArray[i] > sortArray[i+1]){
        System.err.println("Error: element at " + i 
            + " : " + sortArray[i]  );
        System.err.println("Error: element at " + 
            (i+1) + " : " + sortArray[i+1]  );
      }

    }
    executor.shutdown();
  }

}

package parallelSorting;


import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Future;

public class HeavySort {

  private static boolean noisy = true;
  
  public static <T extends Comparable<T>> void sort(T[] data, 
      ExecutorService service, int numThreads, 
      ArrayFactory<T> arrayFactory){
    if(data.length <= 1){
      return;
    }
    List<Callable<Boolean>> tasks = new ArrayList<Callable<Boolean>>();
    int[] startingPoints = new int[numThreads];
    for(int i = 0; i < numThreads;i++){
      int lo = data.length*i/numThreads;
      startingPoints[i] = lo;
      int hi = data.length*(i+1)/numThreads;
      tasks.add(new SortSubsequence<T>(data,lo,hi));
    }
    long sortingTime = System.currentTimeMillis();
    try {
      List<Future<Boolean>> results = service.invokeAll(tasks);
      for(Future<Boolean> result: results){
        if(!result.get().booleanValue()){
          throw new RuntimeException();
        }
      }
    } catch (InterruptedException e) {
      throw new RuntimeException(e);
    } catch (ExecutionException e) {
      throw new RuntimeException(e);
    }
    if(noisy){
      System.out.println("Sorting Time " + 
          (System.currentTimeMillis() - sortingTime));
    }
    List<T> dividers = new ArrayList<T>();
    int hi = data.length/numThreads;
    dividers.add(null);
    for(int i = 1; i < numThreads; i++){
      dividers.add(data[(hi*i)/numThreads]);
    }
    dividers.add(null);
    List<Callable<T[]>> merges = new ArrayList<Callable<T[]>>();
    for(int i = 0 ; i < numThreads; i++){
      merges.add(new MergeSubsequences<T>(dividers.get(i),
          dividers.get(i+1),data,startingPoints,arrayFactory));
    }
    List<T[]> resultsCollected = new ArrayList<T[]>();
    long mergingTime = System.currentTimeMillis();
    try {
      List<Future<T[]>> results = service.invokeAll(merges);
      for(Future<T[]> result: results){
        resultsCollected.add(result.get());
      }
    } catch (InterruptedException e) {
      throw new RuntimeException(e);
    } catch (ExecutionException e) {
      throw new RuntimeException(e);
    }
    if(noisy){
      System.out.println("Merging Time " + 
          (System.currentTimeMillis() - mergingTime));
    }
    List<Callable<Boolean>> pastes = new ArrayList<Callable<Boolean>>();
    int startingPoint = 0;
    for(int i = 0 ; i < numThreads; i++){
      pastes.add(new Paste<T>(startingPoint, 
          data,resultsCollected.get(i)));
      startingPoint+= resultsCollected.get(i).length;
    }
    long pastingTime = System.currentTimeMillis();
    try {
      List<Future<Boolean>> pastesResults = service.invokeAll(pastes);
      for(Future<Boolean> result: pastesResults){
        if(!result.get().booleanValue()){
          throw new RuntimeException();
        }
      }
    } catch (InterruptedException e) {
      throw new RuntimeException(e);
    } catch (ExecutionException e) {
      throw new RuntimeException(e);
    }
    if(noisy){
      System.out.println("Pasting Time " + 
          (System.currentTimeMillis() - pastingTime));
    }
  }

  public static interface ArrayFactory<T extends Comparable<T>>{
    public T[] buildArray(int length);
  }

  private static class Paste<T extends Comparable<T>> implements
      Callable<Boolean>{

    private int lo;
    private T[] data;
    private T[] source;

    public Paste(int lo, T[] data, T[] source) {
      super();
      this.lo = lo;
      this.data = data;
      this.source = source;
    }

    @Override
    public Boolean call() throws Exception {
      System.arraycopy(source, 0, data, lo, source.length);
      return Boolean.valueOf(true);
    }
  }

  private static class MergeSubsequences<T extends Comparable<T>> implements
      Callable<T[]>{

    private T lo;
    private T hi;
    private T[] data;
    private int[] startingPoints;
    private int[] endPoints;
    private ArrayFactory<T> arrayFactory;

    public MergeSubsequences(T lo, T hi, T[] data, 
        int[] startingPoints, ArrayFactory<T> arrayFactory) {
      super();
      this.arrayFactory = arrayFactory;
      this.lo = lo;
      this.hi = hi;
      this.data = data;
      this.startingPoints = startingPoints;
      this.endPoints = new int[startingPoints.length];
      for(int i = 0; i < startingPoints.length-1; i++){
        endPoints[i] = startingPoints[i+1];
      }
      endPoints[endPoints.length-1] = data.length;
    }

    @Override
    public T[] call() throws Exception {

      int[] currentLocationBySection = Arrays.copyOf(
          startingPoints, startingPoints.length);
      int[] upperBoundsBySection = Arrays.copyOf(
          endPoints, endPoints.length);

      if(lo != null){
        for(int i = 0; i < currentLocationBySection.length; i++){
          currentLocationBySection[i] = Arrays.binarySearch(
              data, startingPoints[i], endPoints[i], lo);
          if(currentLocationBySection[i] < 0){
            currentLocationBySection[i] = 
              -currentLocationBySection[i] - 1;
          }
        }
      }
      if(hi != null){
        for(int i = 0; i < upperBoundsBySection.length; i++){
          upperBoundsBySection[i] = Arrays.binarySearch(
              data, startingPoints[i], endPoints[i], hi);
          if(upperBoundsBySection[i] < 0){
            upperBoundsBySection[i] = 
              -upperBoundsBySection[i] - 1;
          }
        }
      }
      boolean[] sectionsInBounds = 
        new boolean[currentLocationBySection.length];
      Arrays.fill(sectionsInBounds, true);
      int numSectionsInBounds = sectionsInBounds.length;
      int totalItems = 0;
      for(int i = 0; i < sectionsInBounds.length; i++){
        if(currentLocationBySection[i] >= upperBoundsBySection[i]){
          sectionsInBounds[i] = false;
          numSectionsInBounds--;
        }
        else{
          totalItems += upperBoundsBySection[i] -
          currentLocationBySection[i];
        }    
      }
      T[] ans = arrayFactory.buildArray(totalItems);
      int ansInd = 0;
      while(numSectionsInBounds > 0){
        int bestSection = -1;
        T best = null;
        for(int i = 0; i < sectionsInBounds.length; i++){
          if(sectionsInBounds[i]){
            if(best == null || 
                data[currentLocationBySection[i]].compareTo(best)
                < 0){
              bestSection = i;
              best = data[currentLocationBySection[i]];
            }
          }        
        }        
        ans[ansInd] = best;
        ansInd++;
        currentLocationBySection[bestSection]++;
        if(currentLocationBySection[bestSection] 
                                    >= upperBoundsBySection[bestSection]){
          sectionsInBounds[bestSection] = false;
          numSectionsInBounds--;
        }
      }
      return ans;
    }
  }

  private static class SortSubsequence<T extends Comparable<T>> implements
      Callable<Boolean>{

    private T[] data;
    private int lo;
    private int hi;

    public SortSubsequence(T[] data, int lo, int hi) {
      super();
      this.data = data;
      this.lo = lo;
      this.hi = hi;
    }

    @Override
    public Boolean call() throws Exception {
      Arrays.sort(data,lo,hi);
      return Boolean.valueOf(true);
    }
  }
}

package parallelSorting;

import static org.junit.Assert.*;

import java.util.Arrays;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

import org.junit.Test;

import parallelSorting.HeavySort.ArrayFactory;

public class HeavySortTest {
  
  private static ArrayFactory<Integer> integerArrayFactory = 
    new ArrayFactory<Integer>(){
      @Override
      public Integer[] buildArray(int length) {
        return new Integer[length];
      }      
    };
    
    private static Integer[] zeroArray(){
      return new Integer[0];
    }
    
    private static Integer[] oneArray(){
      return new Integer[]{Integer.valueOf(-12)};
    }
    
    private static Integer[] twoArray(){
      return new Integer[]{Integer.valueOf(300), Integer.valueOf(100)};
    }
    
    private static Integer[] threeArray(){
      return new Integer[]{Integer.valueOf(-10), 
          Integer.valueOf(-5), Integer.valueOf(-1)};
    }    
    
    private static Integer[] nineArray(){
      return new Integer[]{14, 4 ,100,140,-4,8,30,4,-20 };
    }    

  @Test
  public void test() {
    for(int i : new int[]{1,3,4,5,8,10}){
      ExecutorService exec = Executors.newFixedThreadPool(i);
      {
        Integer[] zero = zeroArray();
        HeavySort.sort(zero, exec, i, integerArrayFactory);
        Integer[] ans = zeroArray();
        Arrays.sort(ans);
        assertArrayEquals(ans,zero);
      }
      {
        Integer[] one = oneArray();
        HeavySort.sort(one, exec, i, integerArrayFactory);
        Integer[] ans = oneArray();
        Arrays.sort(ans);
        assertArrayEquals(ans,one);
      }
      {
        Integer[] two = twoArray();
        HeavySort.sort(two, exec, i, integerArrayFactory);
        Integer[] ans = twoArray();
        Arrays.sort(ans);
        assertArrayEquals(ans,two);
      }
      {
        Integer[] three = threeArray();
        HeavySort.sort(three, exec, i, integerArrayFactory);
        Integer[] ans = threeArray();
        Arrays.sort(ans);
        assertArrayEquals(ans,three);
      }
      {
        Integer[] nine = nineArray();
        HeavySort.sort(nine, exec, i, integerArrayFactory);
        Integer[] ans = nineArray();
        Arrays.sort(ans);
        assertArrayEquals(ans,nine);
      }
    }
  }
}

Labeling Commutative Diagrams with Hyperlinks in LaTeX

Mon, 23 May 2011 15:26:00 +0000

In the example below, a commutative diagram was drawn using Tikz. The arrows in the diagram have labels that link to the corresponding theorem below.

You can download this example here. The LaTeX code used to generate this figure is below.

\documentclass{article}
\pagestyle{empty}
\usepackage{tikz,amsmath}
\usetikzlibrary{arrows,shapes,matrix}
\newtheorem{thm}{Theorem}
\usepackage{hyperref}
\hypersetup{
  colorlinks=true,
}
\begin{document}
\begin{figure}
  \centering
  \begin{tikzpicture}
    \matrix(m)[matrix of math nodes, row sep=10em, column sep=10em]
    {Q^n(t)&Q^n(\infty)\\
      q(t)&q(\infty)\\};
    \path[->]
    (m-1-1) edge node[above]{
      \hyperref[thm:stability]{Theorem \ref{thm:stability}}}
    node[below] {$t \to \infty$} (m-1-2)
    (m-1-1) edge node[left]{
      \hyperref[thm:convergence]{Theorem \ref{thm:convergence}}}
    node[right] {$\displaystyle \lim_{n \to \infty} \frac{Q^n(t)}{n}$}
    (m-2-1)
    (m-2-1) edge node[above]{
      \hyperref[thm:fixedPoint]{Theorem \ref{thm:fixedPoint}}}
    node[below] {$t \to \infty$} (m-2-2)
    (m-1-2) edge node[left]{
      \hyperref[thm:interchange]{Theorem \ref{thm:interchange}}}
    node[right] {
      $\displaystyle \lim_{n \to \infty} \frac{Q^n(\infty)}{n}$}
    (m-2-2)
    ;    
  \end{tikzpicture}
\end{figure}
\newpage
\begin{thm}
  \label{thm:stability}
  When $\rho < 1$, for all $n$, $Q^n(t) \Rightarrow Q^n(\infty)$ as
  $t \to \infty$.
\end{thm}
\newpage
\begin{thm}
  \label{thm:convergence}
  For any $q(0)$, when $Q^n(0)/n \to q(0)$ almost surely as $n \to
  \infty$, then $Q^n(t)/n \to q(t)$ almost surely and uniformly on
  compact sets $[0,T]$.
\end{thm}
\newpage
\begin{thm}
  \label{thm:fixedPoint}
  As $t \to \infty$, $q(t) \to q(\infty)$ for all $q(0)$.
\end{thm}
\newpage
\begin{thm}
  \label{thm:interchange}
  As $n \to \infty$, $Q^n(\infty)/n \to q(\infty)$ in probability.
\end{thm}
\end{document}