http://mhyee.com/Ming-Ho's Blog2016-11-01T04:00:00ZMing-Ho Yeehttp://mhyee.comtag:mhyee.com,2016-11-01:/blog/pl_blog_response.htmlMy Thoughts on "Bending the Dynamic vs Static Language Tradeoff"2016-11-01T04:00:00Z2016-11-01T04:00:00Z<p>Last week, Jamie Wong wrote about the <a href="http://jamie-wong.com/bending-the-pl-curve/">tradeoffs between dynamically and
statically typed languages</a>. As a topic I’m very interested in, there
was a lot I wanted to say.</p>
<p>If you have comments or if you find errors, please let me know!</p>
<hr>
<p>I really enjoyed reading Jamie’s post, and felt that he did a great job
comparing the two camps, and then exploring some recent developments. The post
reminded me of a <a href="https://twitter.com/jlfwong/status/580390984271421440">Twitter conversation</a> I had with Jamie a while back.
I mentioned <a href="http://homes.soic.indiana.edu/jsiek/what-is-gradual-typing/">gradual typing</a>, which attempts to bridge this
static/dynamic tradeoff. Type annotations are optional, so you can write code
that mixes typed and untyped expressions/functions/modules. Gradual typing is
still an active area of research, but it’s promising to see languages like Hack
and TypeScript.</p>
<p>Sometimes, it’s useful to have a different categorization of languages. We could
also think of a static/dynamic spectrum, rather than a strict divide. Sometimes
people use strong/weak typing. (See Eric Lippert’s discussion on <a href="https://ericlippert.com/2012/10/15/is-c-a-strongly-typed-or-a-weakly-typed-language/">whether C# is
strongly typed or weakly typed</a>.) And in one course I took,
a professor suggested a static/dynamic/“operation” triangle, where assembly (and
to an extent, C) has “operation types” rather than static types or dynamic
types. Assembly sort of has types, but they’re not really static or dynamic
types.</p>
<p>We could also take a step back and consider dynamic <em>languages</em> rather than
dynamically <em>typed</em> languages. There’s a lot of overlap between the two, and
I don’t have a good example of a dynamic language that is not dynamically typed.
But a dynamic language allows you to modify objects at run time. Of course, this
gives you more expressive power, faster iteration speed, better debugging
support, etc., but then it can be harder to reason about your program (both for
the programmer and other tools, as Jamie has <a href="http://jamie-wong.com/2013/07/12/grep-test/">written about</a>). And
dynamic languages are also inherently slow. Every time you look up an object
member, you basically have to look it up in a map.</p>
<p>Moving onto more specific comments…</p>
<blockquote>
<p><a href="http://jamie-wong.com/bending-the-pl-curve/#iteration-speed">Iteration Speed</a></p>
<p>Dynamically typed languages do great here.</p>
</blockquote>
<p>I would also add “prototyping.” Using the <code>render</code> method as an example, let’s
say I want to refactor its signature, but I want to do a quick prototype first.
In a dynamically typed language, I can change only the parts I care about and
not worry about type errors in other parts of the program. In a statically typed
language, I have to change everything before I can even compile my code and see
if my prototype is on the right track.</p>
<blockquote>
<p><a href="http://jamie-wong.com/bending-the-pl-curve/#iteration-speed-1">Iteration Speed</a></p>
<p>This is arguably the biggest downside of statically typed languages. Type
checking, as it turns out, is frequently slow.</p>
</blockquote>
<p>I’d say that this is more of a “AOT compilation” vs “interpreted or JITted”
issue. (By the way, <a href="https://root.cern.ch/cling">Cling</a> is an interesting project, as
a C++ interpreter.) Yes, type checking can be slow, but there are other reasons
why compilation might be slow.</p>
<p>When I was an intern at Microsoft, someone did an informal experiment and found
that a third of the C++ compiler’s time was spent on I/O, because <code>#include</code> is
a dumb copy-and-paste. I’ve also found that heavy use of templates (because they
need to be expanded) is slow, and when I was building Chromium, linking was the
bottleneck, not code generation.</p>
<p>The Scala compiler is also known to be very slow. There’s something like twenty
phases, and each phase basically traverses over the entire syntax tree. The
new/next/prototype Scala compiler, Dotty, tries to fuse phases together into
a single traversal, which significantly improves performance. (I’m actually not
that familiar with the details.)</p>
<p>Going back to slow type checking, <a href="https://gist.github.com/mhyee/11129840">OCaml’s type inference has worst-case
exponential complexity</a>, and if your type system is Turing-complete
(e.g. <a href="http://matt.might.net/articles/c++-template-meta-programming-with-lambda-calculus/">C++</a>, <a href="https://arxiv.org/abs/1605.05274">Java</a>, <a href="https://gist.github.com/mhyee/38a895277f246f6c79332d6c7ca32f82">Scala</a>,
<a href="http://www.lochan.org/keith/publications/undec.html">Haskell</a>), then your type checker might not terminate (unless
your stack overflows first). But these are all incredibly unusual circumstances.</p>
<blockquote>
<p><a href="http://jamie-wong.com/bending-the-pl-curve/#correctness-checking-1">Correctness Checking</a></p>
<p>In C++, the compiler will quite happily let you do this:</p>
<p><code>User* a = nullptr;
a->setName("Gretrude");</code></p>
<p>Haskell and Scala do their best to dodge this problem by not letting you have
<code>null</code>, instead representing optional fields explicitly with an
<code>Maybe User</code>/<code>Option[User]</code>.</p>
<p>[…]</p>
<p><a href="http://jamie-wong.com/bending-the-pl-curve/#debugging-support-1">Debugging Support</a></p>
<p>A particularly nasty class of this where you don’t get any interactive console
at all to debug is complex compile errors [e.g. template errors].</p>
</blockquote>
<p>C++17 now has <a href="http://en.cppreference.com/w/cpp/utility/optional">std::optional</a>. There is also a Technical
Specification (that just missed getting into C++17) called Concepts Lite, which
allows constraints on templates. The goal is to make template error messages
easier to understand. <a href="https://isocpp.org/blog/2013/02/concepts-lite-constraining-templates-with-predicates-andrew-sutton-bjarne-s">Here’s a short example (though it links to an old draft of
the concepts proposal)</a>.</p>
<blockquote>
<p><a href="http://jamie-wong.com/bending-the-pl-curve/#stuck">Stuck</a></p>
<p>You see a similar middle ground emerging in the Object Oriented vs. Functional
holy war with languages like Scala and Swift taking an OO syntax, functional
thinking approach, and JavaScript being kind of accidentally multi-paradigm.</p>
</blockquote>
<p>Just a nitpick, but Scala was always designed to be object-oriented and
functional, rather than functional with OO syntax. Other languages have also
been designed as multi-paradigm (e.g. Ruby, Python), and some languages are
borrowing features/ideas from other paradigms (e.g. C++ and Java adding
lambdas).</p>
<blockquote>
<p><a href="http://jamie-wong.com/bending-the-pl-curve/#type-inference">Type Inference</a></p>
<p>It’s also now made its way into C++ via the C++11 <code>auto</code> keyword, and is
a feature of most modern statically typed languages like Scala, Swift, Rust,
and Go.</p>
</blockquote>
<p>Type inference is very much from the “static types” camp, but was more
associated with functional programming languages. Some people will also make the
distinction between type inference à la Hindley-Milner type inference, as
opposed to <code>auto</code> which is “take the right-hand side expression’s type and make
it the type of the left-hand side variable.” Type inference in Haskell is the
former, while type inference in Go is the latter.</p>
<p>Actually, that’s a little unfair to <code>auto</code>, since it’s <a href="http://thbecker.net/articles/auto_and_decltype/section_01.html">similar to (but not
exactly the same as) function template argument deduction</a>. In C++14,
you can even write:</p>
<pre><code class="language-c++"><span class="k">auto</span> <span class="n">plus1</span> <span class="o">=</span> <span class="p">[](</span><span class="k">auto</span> <span class="n">x</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="n">x</span> <span class="o">+</span> <span class="mi">1</span><span class="p">;</span> <span class="p">};</span></code></pre>
<p>This declares a generic lambda function and binds it to <code>plus1</code>. However, this
is just shorthand for using templates. If you never call <code>plus1</code>, the template
will never be instantiated, so you could argue that <code>auto</code> isn’t doing any type
inference at that point.</p>
<p>Compared to Haskell, if you write:</p>
<pre><code class="language-haskell"><span class="kr">let</span> <span class="n">plus1</span> <span class="ow">=</span> <span class="nf">\</span><span class="n">x</span> <span class="ow">-></span> <span class="n">x</span> <span class="o">+</span> <span class="mi">1</span></code></pre>
<p>The type inference engine will look at the function body and infer that <code>x</code> is
an instance of the <code>Num</code> type class.</p>
<p>Anyway, going back to writing out types, when I write Scala or Haskell,
I usually write out all my types for function declarations, because I consider
it a form of documentation. But I’ll generally leave them out for local
variables, unless it’s something really complicated or the compiler infers the
wrong type for me.</p>
<blockquote>
<p><a href="http://jamie-wong.com/bending-the-pl-curve/#decoupling-type-checking-from-code-generation">Decoupling Type Checking from Code Generation</a></p>
<p>If you define your language very carefully, you can make the compiler output
not dependent on the types (i.e. ignore the type information completely), and
then run type checking completely separately.</p>
</blockquote>
<p>This is where I disagree, but I’m probably coming at it from a different angle.
If you’re compiling TypeScript to JavaScript, then I guess it makes sense, but
I’m thinking about implementing a JavaScript JIT. For example, let’s say you’re
evaluating <code>x + x.</code> You don’t know what <code>x</code> is, so you have to check its type
and then dispatch to the correct <code>+</code> method. An optimizing JIT might eventually
figure out that <code>x + x</code> is always integer arithmetic, so it’ll generate machine
code for that. However, the generated code needs to include a guard, because
there’s a possibility that in some future call, <code>x</code> is not an integer, and so
the JIT will need to de-optimize. But if the language had types and <code>x</code> was an
integer, then the JIT can eliminate the guard. I believe
<a href="http://plg.uwaterloo.ca/~dynjs/strongscript/">StrongScript</a> does this (and more).</p>
<p>JITs for static languages (like Java) will do something similar for method
dispatches. If <code>x.foo()</code> is a polymorphic call, but always dispatches to
a single implementation, the JIT will compile it as a static dispatch. Of
course, it still needs to insert a guard, in case <code>x.foo()</code> isn’t actually
monomorphic.</p>
<blockquote>
<p><a href="http://jamie-wong.com/bending-the-pl-curve/#better-compiler-error-messages">Better Compiler Error Messages</a></p>
<p>There have been numerous attempts to make debugging compilation errors
a non-issue by having sensible human-readable error messages, notably in Elm.</p>
</blockquote>
<p>Dotty is working on significantly better error messages for Scala. They
published a <a href="http://scala-lang.org/blog/2016/10/14/dotty-errors.html">blog post</a> just a few weeks ago. And part of having
good error messages is designing your compiler and infrastructure to track and
provide that information.</p>
<p>And that’s everything I wanted to say. There was just one small point
I disagreed with, but overall, I enjoyed the article. I’ve also felt the
frustration when switching between dynamically typed and statically typed
languages. But I’m also optimistic about the future. (It also means job
opportunities for when I graduate!)</p>
<p><em>I would like to thank Jamie Wong for his feedback and discussion, which has
improved this post.</em></p>
Last week, Jamie Wong wrote about the tradeoffs between dynamically and statically typed languages. As a topic I'm very interested in, there was a lot I wanted to say.tag:mhyee.com,2013-12-16:/blog/fydp6.htmlFourth-Year Design Project, Part 6: Fixing the Partitioned Guided Improvement Algorithm2013-12-16T05:00:00Z2013-12-16T05:00:00Z<p>In the <a href="/blog/fydp5.html">previous post</a>, we started our discussion on the <em>partitioned
guided improvement algorithm</em> (PGIA). We covered some background information, an
interesting idea our group developed, and how it fails for problems with more
than two objectives.</p>
<p>In this post, we’ll conclude our discussion on PGIA. We’ll extend the algorithm
for any number of objectives, cover an example problem with three objectives,
and finally, discuss preliminary results and future work.</p>
<p>At this point, I will assume familiarity with <a href="/blog/fydp2.html">multi-objective
optimization</a>, the <a href="/blog/fydp3.html">guided improvement algorithm</a>, and the
<a href="/blog/fydp5.html">previous post</a> on PGIA. This post is likely the most difficult one in my
blog series, so please feel free to ask questions in the comments.</p>
<h2>Fixing the algorithm</h2>
<p>When we concluded the previous post, we investigated the reasons why PGIA only
worked with two objectives: dominating a locally optimal solution requires
finding a solution in the “empty” region.</p>
<p>I’ve copied a diagram from my previous post, where we’ve found a Pareto point
and split the search space into four regions. The solution in Region B can only
be dominated by solutions in Region B or Region D; specifically, the patterned
area in the diagram. If the solution we found is locally optimal, then there is
no better solution in Region B. Additionally, there is no better solution in
Region D because nothing dominates the originally discovered Pareto point.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig5-4.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig5-4.svg" width="400" height="260" style="border:none">
</a>
</div>
<p>The problem with three or more objectives is that we cannot <em>easily</em> make this
guarantee.</p>
<p>Let’s look at the general problem with <code>N</code> metrics that we want to maximize,
<tt>(m<sub>1</sub>, m<sub>2</sub>, …, m<sub>N</sub>)</tt>. Suppose the first
Pareto point we find is <tt><strong>P</strong> = (p<sub>1</sub>, p<sub>2</sub>,
…, p<sub>N</sub>)</tt>. When we split the space, we get <tt>2<sup>N</sup></tt>
regions. Each region is described by constraints of the form
<tt>{(m<sub>1</sub>, m<sub>2</sub>, …, m<sub>N</sub>) such that m<sub>1</sub>
■ p<sub>1</sub>, m<sub>2</sub> ■ p<sub>2</sub>, …, m<sub>N</sub>
■ p<sub>N</sub>}</tt>, where ■ is ≥ for “better than or equal” or
< for “worse than”. For shorthand, I’ll simply write
<tt>(■p<sub>1</sub>, ■p<sub>2</sub>, …,
■p<sub>N</sub>)</tt>. Keep in mind that <tt>m<sub>i</sub></tt> represents
the <tt>i</tt>th <em>metric</em> and will vary, while <tt>p<sub>i</sub></tt> represents
the <tt>i</tt>th <em>metric value</em> and is fixed.</p>
<p>The (dominated) region described by the constraints <tt>(<p<sub>1</sub>,
<p<sub>2</sub>, …, <p<sub>N</sub>)</tt> is excluded since all solutions
within that region are dominated by <tt><strong>P</strong></tt>. The (empty)
region described by the constraints <tt>(≥p<sub>1</sub>,
≥p<sub>2</sub>, …, ≥p<sub>N</sub>)</tt> is excluded since there
are no solutions that dominate <tt><strong>P</strong></tt>, by definition of
a Pareto point.</p>
<p>Let’s look at the regions directly “adjacent” to the empty region, that is,
regions with exactly one “worse than” constraint. The following argument will be
the same for all <code>N</code> of these regions, so we’ll simply consider the region with
constraints <tt>(<p<sub>1</sub>, ≥p<sub>2</sub>, …,
≥p<sub>N</sub>)</tt>, where the remaining constraints are
<tt>≥p<sub>i</sub></tt>.</p>
<p>For any given solution within this region, a dominating solution exists if we
improve at least one of the metrics (and satisfy the problem constraints)<sup><a href="#n1" id="t1">1</a></sup>. If we increase only the objectives
<tt>m<sub>2</sub>, m<sub>3</sub>, … m<sub>N</sub></tt>, we will remain in this
region. In contrast, if we increase <tt>m<sub>1</sub></tt>, we will eventually
reach a different region, the one described by constraints
<tt>(≥p<sub>1</sub>, ≥p<sub>2</sub>, …,
≥p<sub>N</sub>)</tt>.</p>
<p>In other words, any locally optimal solution in <tt>(<p<sub>1</sub>,
≥p<sub>2</sub>, …, ≥p<sub>N</sub>)</tt> might be dominated by
a solution in <tt>(≥p<sub>1</sub>, ≥p<sub>2</sub>, …,
≥p<sub>N</sub>)</tt>., but since that region is empty, all locally optimal
solutions in <tt>(<p<sub>1</sub>, ≥p<sub>2</sub>, …,
≥p<sub>N</sub>)</tt> are globally optimal.</p>
<p>The same argument applies to the other regions with exactly one “worse than”
constraint. Since all these regions are independent<sup><a href="#n2" id="t2">2</a></sup> and do not overlap each other, we can search them in
parallel.</p>
<p>Suppose we have gone and found all locally optimal solutions in the <code>N</code> regions
with exactly one “worse than” constraint. We know that these solutions are
globally optimal.</p>
<p>Now let’s look at the regions with exactly two “worse than” constraints. Without
loss of generality, we’ll consider <tt>(<p<sub>1</sub>, <p<sub>2</sub>,
…, ≥p<sub>N</sub>)</tt>, where the remaining constraints are
<tt>≥p<sub>i</sub></tt>. Using a similar argument from before, if we want
to find a better solution in a different region, we have to increase
<tt>m<sub>1</sub></tt>, <tt>m<sub>2</sub></tt>, both. This pushes us into one of
the following regions <tt>(<p<sub>1</sub>, ≥p<sub>2</sub>, …,
≥p<sub>N</sub>)</tt>, <tt>(≥p<sub>1</sub>, <p<sub>2</sub>, …,
≥p<sub>N</sub>)</tt>, or <tt>(≥p<sub>1</sub>, ≥p<sub>2</sub>, …,
≥p<sub>N</sub>)</tt>.</p>
<p>We already know the last one is empty. However, at this point, we’ve already
found the locally optimal solutions in <tt>(<p<sub>1</sub>,
≥p<sub>2</sub>, …, ≥p<sub>N</sub>)</tt> and
<tt>(≥p<sub>1</sub>, <p<sub>2</sub>, …, ≥p<sub>N</sub>)</tt>.
We can use those solutions to create exclusion constraints for our search in
<tt>(<p<sub>1</sub>, <p<sub>2</sub>, …, ≥p<sub>N</sub>)</tt>, thus
avoiding locally optimal solutions that are dominated by external solutions. In
other words, we can guarantee that locally optimal solutions in
<tt>(<p<sub>1</sub>, <p<sub>2</sub>, …, ≥p<sub>N</sub>)</tt>
are globally optimal.</p>
<p>We can apply the same argument to all regions with exactly two “worse than”
constraints and search them in parallel. Once these regions are done, we can
search all regions with exactly three “worse than” constraints, then four, and
so on until we finish with <code>N - 1</code> “worse than” constraints<sup><a href="#n3" id="t3">3</a></sup>. At every step of this process, we can guarantee locally
optimal solutions are globally optimal by using exclusion constraints based on
all previously found optimal solutions.</p>
<p>Our algorithm now has multiple steps with dependencies, which will require
sequential processing. However, we have still broken the problem into smaller
pieces, and we can still perform some of the searching in parallel. We just
cannot search all <tt>2<sup>N</sup></tt> regions in parallel.</p>
<h2>An optimization and some more notation</h2>
<p>Consider the regions <tt>(≥p<sub>1</sub>, ≥p<sub>2</sub>, …,
<p<sub>N</sub>)</tt> and <tt>(<p<sub>1</sub>, <p<sub>2</sub>, …,
≥p<sub>N</sub>)</tt>, where the former has exactly one “worse than”
constraint and the latter has exactly two “worse than” constraints. From our
previous description, we have to finish searching in the first region before we
can start searching in the second region, because it has fewer “worse than”
constraints.</p>
<p>However, these two regions are independent. Region <tt>(≥p<sub>1</sub>,
≥p<sub>2</sub>, …, <p<sub>N</sub>)</tt> depends only on
<tt>(≥p<sub>1</sub>, ≥p<sub>2</sub>, …,
≥p<sub>N</sub>)</tt>, while region <tt>(<p<sub>1</sub>,
<p<sub>2</sub>, …, ≥p<sub>N</sub>)</tt> depends only on
<tt>(≥p<sub>1</sub>, <p<sub>2</sub>, …, ≥p<sub>N</sub>)</tt>,
<tt>(<p<sub>1</sub>, ≥p<sub>2</sub>, …, ≥p<sub>N</sub>)</tt>,
and <tt>(≥p<sub>1</sub>, ≥p<sub>2</sub>, …,
≥p<sub>N</sub>)</tt>. In other words, we can search these two regions in
parallel. No solution in one can dominate a solution in the other.</p>
<p>We can create a dependency graph that illustrates which regions we can search in
parallel. However, our current notation is a little difficult to work with, so
we’ll use binary strings, similar to how our implementation uses bit sets. The
binary string <tt>b<sub>1</sub>b<sub>2</sub>…b<sub>N</sub></tt>, represents
a region <tt>(■p<sub>1</sub>, ■p<sub>2</sub>, …,
■p<sub>N</sub>)</tt> where <tt>b<sub>i</sub></tt> is <code>1</code> for “better than
or equal” constraints and <code>0</code> for “worse than” constraints.</p>
<p>Our dependency graph will be a directed acyclic graph where each vertex
represents a region, and an edge directed from <code>R</code> to <tt>R′</tt> means
<code>R</code> is a dependency of <tt>R′</tt>. In other words, we need to search in
<code>R</code> before we can search in <tt>R′</tt>, because a solution in <code>R</code> might
dominate a locally optimal solution in <tt>R′</tt>.</p>
<p>For a problem with <code>N</code> objectives, our graph will have <tt>2<sup>N</sup></tt>
vertices, where each vertex is labelled with a binary string of length <code>N</code>.
A directed edge from <code>R</code> to <tt>R′</tt> exists if and only if
<tt>R′</tt> is the same string as <code>R</code>, but with <em>exactly a single <code>1</code> changed
to a <code>0</code></em>.</p>
<p>Let’s examine why this works. Suppose the <code>1</code> we changed to a <code>0</code> corresponds to
the digit <tt>b<sub>i</sub></tt>, which represents metric
<tt>m<sub>i</sub>,</tt>. The edge from <code>R</code> to <tt>R′</tt> means <code>R</code> is
a dependency of <tt>R′</tt>, or that a solution in <code>R</code> might dominate
a locally optimal solution in <tt>R′</tt>. Assuming we can satisfy the
problem constraints, we can find this better solution in <code>R</code> by taking the
solution in <tt>R′</tt> and setting <tt>m<sub>i</sub></tt> to be greater
than or equal to <tt>p<sub>i</sub></tt>.</p>
<p>It is also possible to find a better solution by increasing multiple metric
values. This would correspond to changing multiple <code>0</code>s to <code>1</code>s. However, the
dependencies and the “dominates” relationship are transitive; if solution A is
dominated by solution B and solution B is dominated by solution C, then solution
A is dominated by solution C.</p>
<p>Remember, the key idea for this algorithm is that we search the dependencies in
order, and then use the known Pareto points to construct exclusion constraints.
Therefore, when we search within a region, we exclude any locally optimal points
that we know are dominated. The remaining locally optimal points are globally
optimal, because we did not find anything that could dominate them.</p>
<h2>An example problem with three objectives</h2>
<p>To visualize how this algorithm works, we’ll walk through an example problem
with three objectives. This example problem is based on the one from the
<a href="/blog/fydp5.html">previous post</a>, but slightly modified for demonstration purposes.</p>
<p>First, we’ll consider all existing solutions, even non-optimal ones. We will
assume that the first Pareto point, <tt><strong>P</strong> = (10, 11, 9)</tt>,
has already been found, and that we have already split the space accordingly.
This time, the regions are labelled with binary strings instead of letters.</p>
<table style="border-collapse:separate;border-spacing:10px;margin:0px auto;border:1px">
<thead>
<tr>
<th>Region</th>
<th>Constraint</th>
<th>Metric points of solutions</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<tt>000</tt><sup><a href="#n4" id="t4">4</a></sup>
</td>
<td><tt>(<10, <11, <9)</tt></td>
<td>
<tt>(5, 6, 4)</tt>
</td>
</tr>
<tr>
<td><tt>001</tt></td>
<td><tt>(<10, <11, ≥9)</tt></td>
<td><tt>(9, 8, 12)</tt></td>
</tr>
<tr>
<td><tt>010</tt></td>
<td><tt>(<10, ≥11, <9)</tt></td>
<td><tt>(7, 13, 8)</tt></td>
</tr>
<tr>
<td><tt>011</tt></td>
<td><tt>(<10, ≥11, ≥9)</tt></td>
<td>
<tt>(5, 12, 10)</tt>, <tt>(6, 12, 12)</tt>
</td>
</tr>
<tr>
<td><tt>100</tt></td>
<td><tt>(≥10, <11, <9)</tt></td>
<td>
<tt>(11, 10, 8)</tt>, <tt>(14, 10, 8)</tt>
</td>
</tr>
<tr>
<td><tt>101</tt></td>
<td><tt>(≥10, <11, ≥9)</tt></td>
<td><tt>(11, 9, 10)</tt></td>
</tr>
<tr>
<td><tt>110</tt></td>
<td><tt>(≥10, ≥11, <9)</tt></td>
<td><tt>(11, 14, 8)</tt></td>
</tr>
<tr>
<td><tt>111</tt></td>
<td><tt>(≥10, ≥11, ≥9)</tt></td>
<td>None; no solutions dominate <tt><strong>P</strong></tt>
</td>
</tr>
</tbody>
</table>
<br>
<p>The dependency graph for this problem is shown below. For completeness, I have
additionally included the vertices for <code>111</code> and <code>000</code>, even though we skip them
in the algorithm.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig6-1.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig6-1.svg" width="400" height="260" style="border:none">
</a>
</div>
<br>
<p>Since <code>111</code> is empty, we start by searching <code>110</code>, <code>101</code>, and <code>011</code> in parallel.
In <code>110</code> and <code>101</code>, the only solutions are <code>(11, 14, 8)</code> and <code>(11, 9, 10)</code>,
respectively. Therefore, these solutions are locally optimal. In <code>011</code>, we first
find that <code>(6, 12, 12)</code> is the locally optimal solution, dominating <code>(5, 12,
10)</code>, since it has improved at least one metric value without worsening the
others. Specifically, the first and third metric values are better, while the
second value is no worse.</p>
<p>The locally optimal solutions we discovered are also globally optimal, because
any better solution from a different region must exist in <code>111</code>, which is empty.</p>
<p>If we finished <code>110</code> and <code>101</code> first, the dependency graph indicates we can
proceed to <code>100</code>, even though we could still be searching in <code>011</code>. This is
because no solution in <code>100</code> can dominate a solution in <code>011</code>, and vice versa.
We could not make a metric better without making some other metric worse.</p>
<p>Suppose we have finished searching <code>110</code>, <code>101</code>, and <code>011</code>, and we have the
corresponding exclusion constraints to search in <code>001</code>, <code>010</code>, and <code>100</code>.</p>
<p>In <code>001</code>, we find the solution <code>(9, 8, 12)</code>, which is locally optimal and not
dominated by the known solutions. Therefore, it is globally optimal.</p>
<p>In <code>010</code>, the exclusion constraints prevent us from returning <code>(7, 13, 8)</code> as
a solution, because it is dominated by <code>(11, 14, 8)</code>. Thus, there are no locally
optimal solutions in this region.</p>
<p>Finally, in <code>100</code>, the exclusion constraints prevent us from yielding <code>(11, 10, 8)</code>.
However, nothing excludes <code>(14, 10, 8)</code>, so it is both locally and globally
optimal.</p>
<p>In the diagram below, I have plotted the three-dimensional graph, without
solutions. Note the two regions shaded grey; the darker region is excluded
because all solutions are dominated by <tt><strong>P</strong></tt>, and the
lighter region is excluded because there are no solutions that dominate
<tt><strong>P</strong></tt>.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig6-2.png" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig6-2.png" width="400" height="260" style="border:none">
</a>
</div>
<br>
<p>I plotted this graph with MATLAB (specifically, MuPAD). The code is provided
below, and I have also included the code for plotting the solutions.</p>
<pre><code class="language-matlab"><span class="n">plot</span><span class="p">(</span>
<span class="n">plot</span><span class="p">::</span><span class="n">Box</span><span class="p">(</span>10<span class="p">..</span>15<span class="p">,</span> 11<span class="p">..</span>16<span class="p">,</span> 9<span class="p">..</span>14<span class="p">,</span> <span class="n">FillColor</span><span class="p">=</span><span class="n">RGB</span><span class="p">::</span><span class="n">Grey80</span><span class="p">.[</span>0<span class="p">.</span>8<span class="p">]),</span>
<span class="n">plot</span><span class="p">::</span><span class="n">Box</span><span class="p">(</span>10<span class="p">..</span>15<span class="p">,</span> 11<span class="p">..</span>16<span class="p">,</span> 4<span class="p">..</span>9<span class="p">,</span> <span class="n">Filled</span> <span class="p">=</span> <span class="n">FALSE</span><span class="p">),</span>
<span class="n">plot</span><span class="p">::</span><span class="n">Box</span><span class="p">(</span>10<span class="p">..</span>15<span class="p">,</span> 6<span class="p">..</span>11<span class="p">,</span> 9<span class="p">..</span>14<span class="p">,</span> <span class="n">Filled</span> <span class="p">=</span> <span class="n">FALSE</span><span class="p">),</span>
<span class="n">plot</span><span class="p">::</span><span class="n">Box</span><span class="p">(</span>10<span class="p">..</span>15<span class="p">,</span> 6<span class="p">..</span>11<span class="p">,</span> 4<span class="p">..</span>9<span class="p">,</span> <span class="n">Filled</span> <span class="p">=</span> <span class="n">FALSE</span><span class="p">),</span>
<span class="n">plot</span><span class="p">::</span><span class="n">Box</span><span class="p">(</span>5<span class="p">..</span>10<span class="p">,</span> 11<span class="p">..</span>16<span class="p">,</span> 9<span class="p">..</span>14<span class="p">,</span> <span class="n">Filled</span> <span class="p">=</span> <span class="n">FALSE</span><span class="p">),</span>
<span class="n">plot</span><span class="p">::</span><span class="n">Box</span><span class="p">(</span>5<span class="p">..</span>10<span class="p">,</span> 11<span class="p">..</span>16<span class="p">,</span> 4<span class="p">..</span>9<span class="p">,</span> <span class="n">Filled</span> <span class="p">=</span> <span class="n">FALSE</span><span class="p">),</span>
<span class="n">plot</span><span class="p">::</span><span class="n">Box</span><span class="p">(</span>5<span class="p">..</span>10<span class="p">,</span> 6<span class="p">..</span>11<span class="p">,</span> 9<span class="p">..</span>14<span class="p">,</span> <span class="n">Filled</span> <span class="p">=</span> <span class="n">FALSE</span><span class="p">),</span>
<span class="n">plot</span><span class="p">::</span><span class="n">Box</span><span class="p">(</span>5<span class="p">..</span>10<span class="p">,</span> 6<span class="p">..</span>11<span class="p">,</span> 4<span class="p">..</span>9<span class="p">,</span> <span class="n">FillColor</span><span class="p">=</span><span class="n">RGB</span><span class="p">::</span><span class="n">Grey50</span><span class="p">.[</span>0<span class="p">.</span>8<span class="p">]),</span>
<span class="o">/*</span> <span class="n">These</span> <span class="n">are</span> <span class="n">the</span> <span class="p">(</span><span class="n">globally</span><span class="p">)</span> <span class="n">optimal</span> <span class="n">solutions</span> <span class="o">*/</span>
<span class="n">plot</span><span class="p">::</span><span class="n">PointList3d</span><span class="p">(</span>
<span class="p">[[</span>9<span class="p">,</span>8<span class="p">,</span>12<span class="p">],</span> <span class="p">[</span>6<span class="p">,</span>12<span class="p">,</span>12<span class="p">],</span> <span class="p">[</span>14<span class="p">,</span>10<span class="p">,</span>8<span class="p">],</span> <span class="p">[</span>11<span class="p">,</span>9<span class="p">,</span>10<span class="p">],</span> <span class="p">[</span>11<span class="p">,</span>14<span class="p">,</span>8<span class="p">]],</span>
<span class="n">PointStyle</span><span class="p">=</span><span class="n">FilledSquares</span><span class="p">),</span>
<span class="o">/*</span> <span class="n">These</span> <span class="n">solutions</span> <span class="n">are</span> <span class="n">not</span> <span class="n">globally</span> <span class="n">optimal</span> <span class="o">*/</span>
<span class="n">plot</span><span class="p">::</span><span class="n">PointList3d</span><span class="p">(</span>
<span class="p">[[</span>5<span class="p">,</span>6<span class="p">,</span>4<span class="p">],</span> <span class="p">[</span>7<span class="p">,</span>13<span class="p">,</span>8<span class="p">],</span> <span class="p">[</span>5<span class="p">,</span>12<span class="p">,</span>10<span class="p">],</span> <span class="p">[</span>11<span class="p">,</span>10<span class="p">,</span>8<span class="p">]],</span>
<span class="n">PointStyle</span><span class="p">=</span><span class="n">FilledCircles</span><span class="p">)</span>
<span class="p">)</span></code></pre>
<h2>Preliminary results</h2>
<p>Our group has implemented both the algorithm and the optimization described
above. I will not be discussing the implementation this time<sup><a href="#n5" id="t5">5</a></sup>, but you can find the code on <a href="https://github.com/TeamAmalgam/kodkod/pull/40">GitHub</a>.</p>
<p>With our implementation, we have been able to run our preliminary tests. Below,
we have a graph and a table comparing incremental solving (IGIA), checkpointed
solving (CGIA), the overlapping guided improvement algorithm (OGIA), and PGIA.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig6-3.png" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig6-3.png" width="647" height="415" style="border:none">
</a>
</div>
<br>
<p>In the graph, lower numbers are better, indicating that less time was spent
solving the problem. Also, note that two of the queens problems are missing
a bar for IGIA; for one of them, we never attempted the test, and for the other,
the bar distorts the entire graph because it is so large. The original GIA
results are not included, since we are now comparing our algorithms against
IGIA. Finally, recall that these are informal test results, run on the
undergraduate computer science servers.</p>
<table style="border-collapse:separate;border-spacing:10px;margin:0px auto;border:1px">
<thead>
<tr>
<th><!-- empty --></th>
<th>Incremental Guided Improvement Algorithm</th> <!-- 1da4839 -->
<th>Checkpointed Guided Improvement Algorithm</th> <!-- 13ab50f -->
<th>Overlapping Guided Improvement Algorithm</th> <!-- 8eaa3d6 -->
<th>Partitioned Guided Improvement Algorithm</th> <!-- 8f8626f -->
</tr>
</thead>
<tbody>
<tr>
<td>9 queens, 5 metrics</td>
<td>2 hours, 0 min</td>
<td>51 min</td>
<td>50 min</td>
<td>42 min</td>
</tr>
<tr>
<td>9 queens, 6 metrics</td>
<td>4 days, 16 hours, 43 min</td>
<td>1 hour, 39 min</td>
<td>1 hour, 53 min</td>
<td>1 hour, 18 min</td>
</tr>
<tr>
<td>9 queens, 7 metrics</td>
<td>Never attempted<sup><a href="#n6" id="t6">6</a></sup>
</td>
<td>3 hours, 37 min</td>
<td>4 hours, 4 min</td>
<td>3 hours, 4 min</td>
</tr>
<tr>
<td>Search and rescue, 5 metrics</td>
<td>3 hours, 0 min</td>
<td>1 hour, 26 min</td>
<td>2 hours, 38 min</td>
<td>1 hour, 44 min</td>
</tr>
<tr>
<td>Search and rescue, 6 metrics</td>
<td>2 hours, 8 min</td>
<td>59 min</td>
<td>1 hour, 11 min</td>
<td>2 hours, 5 min</td>
</tr>
<tr>
<td>Search and rescue, 7 metrics</td>
<td>5 hours, 1 min</td>
<td>2 hours, 16 min</td>
<td>2 hours, 46 min</td>
<td>3 hours, 10 min</td>
</tr>
</tbody>
</table>
<br>
<p>The results are similar to OGIA, though in some cases PGIA can be better or
worse. Again, we also see that CGIA, a single-threaded approach, beats PGIA,
a multi-threaded approach, in some of the tests.</p>
<p>However, when we tried some of our extremely large tests, CGIA and OGIA fell
behind<sup><a href="#n7" id="t7">7</a></sup>.</p>
<table style="border-collapse:separate;border-spacing:10px;margin:0px auto;border:1px">
<thead>
<tr>
<th><!-- empty --></th>
<th>Checkpointed Guided Improvement Algorithm</th> <!-- 13ab50f -->
<th>Overlapping Guided Improvement Algorithm</th> <!-- 8eaa3d6 -->
<th>Partitioned Guided Improvement Algorithm</th> <!-- 8f8626f -->
</tr>
</thead>
<tbody>
<tr>
<td>9 rooks, 5 metrics</td>
<td>21 days, 9 hours, 36 min</td>
<td>5 days, 8 hours, 16 min</td>
<td>4 days, 15 hours, 28 min</td>
</tr>
<tr>
<td>9 rooks, 6 metrics</td>
<td>Never attempted<sup><a href="#n8" id="t8">8</a></sup>
</td>
<td>16 days, 11 hours, 56 min</td>
<td>5 days, 17 hours, 19 min</td>
</tr>
<tr>
<td>9 rooks, 7 metrics</td>
<td>Never attempted</td>
<td>19 days, 39 min</td>
<td>5 days, 15 hours, 52 min</td>
</tr>
</tbody>
</table>
<br>
<p>We suspect this extremely poor performance is because the rooks problems have
fewer constraints than the queens problems. Thus, there are far more solutions
to find, and PGIA can better handle this situation. However, it’s important to
note that these problems are extremely large and contrived, which we likely
won’t see with real world problems.</p>
<h2>Future work</h2>
<p>At the time of writing, we are interested in three further improvements to PGIA.</p>
<p>The first, similar to OGIA, is to build PGIA on top of CGIA. Again, we would
need to look at reducing memory usage first.</p>
<p>Next, we could recursively split the regions. To keep things simple, our first
implementation only splits the space once, and uses regular GIA within each
region. We’re interested in what happens if we continue splitting the regions
and making them smaller. However, we probably want to limit the number of
recursive calls, otherwise there will be too much overhead.</p>
<p>Finally, we’re interested in whether we can find a “good” Pareto point to split
the space. Currently, we take whatever Pareto point we find first. However, this
may result in regions with different sizes. It may be more effective for us to
choose a Pareto point that gives us regions with roughly the same size, or
better, a Pareto point that evenly distributes the solutions in each of the
regions.</p>
<h2>Conclusion</h2>
<p>In this post, we concluded our discussion of the <em>partitioned guided improvement
algorithm</em>. We had previously discussed the background and inspiration for PGIA,
and in this post, we completed the algorithm and walked through an example
execution. The algorithm is far more complicated than OGIA, but it guarantees no
duplicate solutions.</p>
<p>At this point, the blog series has covered all the work our group has
accomplished so far. I have no further blog posts planned for this series, as
they will depend on what our group works on, from now until March 2014. However,
potential topics include the results of our further investigations, or results
from rigorous performance evaluations.</p>
<p>Thank you for reading this blog series. I hope the posts have been informative
and interesting, and that it was worth your time. If you still want to learn
more, you can look at our <a href="/fydp.html">documents and repositories</a>, or
<a href="/about.html">contact</a> me.</p>
<p><em>I would like to thank Talha Khalid, Chris Kleynhans, Zameer Manji, and Arjun
Sondhi for proofreading this post.</em></p>
<h2>Notes</h2>
<ol>
<li><p><a style="text-decoration: none;" id="n1" href="#t1">^</a> Remember, such
a solution only exists if it satisfies the problem constraints. However,
the <em>possibility of existence</em> is good enough here, because we want to
guarantee that <em>no solution exists</em>.</p></li>
<li><p><a style="text-decoration: none;" id="n2" href="#t2">^</a> That is,
a solution in one region cannot dominate a solution in another region. We
demonstrated this by identifying the regions a dominating solution could
exist in.</p></li>
<li><p><a style="text-decoration: none;" id="n3" href="#t3">^</a> We do not
concern ourselves with the region with <code>N</code> “worse than” constraints,
because we already know all of its solutions are dominated by
<tt><strong>P</strong></tt>.</p></li>
<li><p><a style="text-decoration: none;" id="n4" href="#t4">^</a> This row can be
read as “Region <code>000</code>, with constraint <tt>(<10, <11, <9)</tt>,
contains the solution at <code>(5, 6, 4)</code>.”</p></li>
<li><p><a style="text-decoration: none;" id="n5" href="#t5">^</a> I tried
discussing the implementation details in an early draft, but a high-level
description was not very enlightening. I would have to start describing
what sort of concurrency constructs we used in Java, but the discussion
then becomes tedious.</p></li>
<li><p><a style="text-decoration: none;" id="n6" href="#t6">^</a> IGIA had very
little improvement over GIA for the 9 queens problems, so we never
attempted this case, which took over fifty days with GIA.</p></li>
<li><p><a style="text-decoration: none;" id="n7" href="#t7">^</a> Remember, this
is the same CGIA that took just under four hours for 9 queens with
7 metrics, which took over 50 days on the original GIA.</p></li>
<li><p><a style="text-decoration: none;" id="n8" href="#t8">^</a> We did not
attempt these problems with CGIA, because we suspected the tests would take
too long.</p></li>
</ol>
In this post, we'll conclude our discussion on PGIA. We'll extend the algorithm for any number of objectives, cover an example problem with three objectives, and finally, discuss preliminary results and future work.tag:mhyee.com,2013-12-02:/blog/fydp5.htmlFourth-Year Design Project, Part 5: Attempting the Partitioned Guided Improvement Algorithm2013-12-02T05:00:00Z2013-12-02T05:00:00Z<p>In the <a href="/blog/fydp4.html">previous post</a>, I covered the <em>overlapping guided improvement
algorithm</em> (OGIA), which is the first of our group’s two multi-threaded approaches. We
saw some dramatic improvements, but found that there is still a lot of duplicate
work.</p>
<p>The subject of our next few posts is the <em>partitioned guided improvement
algorithm</em>, our other multi-threaded approach. In contrast to OGIA, we trade
simplicity for the guarantee that no duplicate solutions are found. In this post
specifically, we will cover a little background, the inspiration behind this
algorithm, and finally, a mistake we made.</p>
<p>Again, before continuing, you may want to refresh your memory on <a href="/blog/fydp2.html">multi-objective
optimization</a> and the <a href="/blog/fydp3.html">guided improvement algorithm</a> (GIA).</p>
<h2>Background</h2>
<p>There were other groups before ours that worked with Professor Rayside to
improve the guided improvement algorithm. In fact, all the other improvement
ideas I have covered so far (<a href="/blog/fydp3.html">incremental and checkpointed solving</a>, and
<a href="/blog/fydp4.html">OGIA</a>) originated from these groups or other collaborators. In contrast,
the partitioned guided improvement algorithm (PGIA) was an idea we developed on
our own, though the concept of “dividing the search space” is not an original
idea.</p>
<p>Recall how we can plot solutions on a graph. This graph represents a <em>search
space</em> of solutions, with a dimension (or axis on the graph) for each objective
in the problem<sup><a href="#n1" id="t1">1</a></sup>. The basic concept of PGIA,
then, is to divide the search space into regions that can be searched in
parallel<sup><a href="#n2" id="t2">2</a></sup>.</p>
<p>The interesting question is <em>how</em> we can divide the search space. Ideally, we
want to divide the search space so that a <em>locally optimal</em> solution in a region
is <em>globally optimal</em> for the whole problem. If we have this property, we can
treat each region as a separate problem, search the regions in parallel, and
then combine the solutions. We would not have to worry about duplicate
solutions, or solutions from one region dominating solutions from another
region.</p>
<p>I’ve illustrated some of the previous attempts to divide the search space. In
these examples, we’re dealing with two metrics (<code>m1</code> and <code>m2</code>), so the search space
is two-dimensional.</p>
<p>Unfortunately, none of the attempts to divide the search space work, as locally
optimal solutions may not be globally optimal. To demonstrate this, each diagram
contains two solutions. One, marked with a dot, is locally optimal, but not
globally optimal. The other, marked with an X, is both locally and globally
optimal. The region it dominates is shaded, so we can see how it dominates the
other solution.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig5-1.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig5-1.svg" width="600" height="390" style="border:none">
</a>
</div>
<p>A locally optimal solution that is not globally optimal is problematic for
a few reasons. First, we wasted some work, finding a solution that we must
discard. Also, the regions are no longer independent. Finally, we no longer have
the property that the algorithm yields solutions as they are found. We have
additional work to determine if a solution is actually globally optimal, which
can only be done after all regions have been searched.</p>
<p>Our group picked up the work around this point. We were wondering how expensive
the additional work would be, and if it even mattered in the long run. We were
also wondering if we could minimize wasted work by finding “safe” regions, where
locally optimal solutions were always globally optimal. We likely would have
continued in this direction, if not for a key observation we made.</p>
<h2>The key observation</h2>
<p>The inspiration came when we drew the graph slightly differently. Normally, when
a Pareto point is found, we would shade the region dominated by that Pareto
point and exclude it from the search. However, we can shade and exclude another
region —– the “empty” area that dominates the Pareto point. There can be no
solutions in that region, since we failed to find one while climbing up to the
Pareto front.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig5-2.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig5-2.svg" width="400" height="260" style="border:none">
</a>
</div>
<p>By drawing the graph this way, we can think of the first Pareto point as
splitting the search space into four regions, with two regions immediately
excluded. This leaves two regions we have to search in. Furthermore, in both
regions, a locally optimal solution is also globally optimal. In other words, we
now have two independent regions that we can search in parallel.</p>
<p>We could also recursively apply this algorithm to each of the regions, as shown
below.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig5-3.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig5-3.svg" width="400" height="260" style="border:none">
</a>
</div>
<p>However, this raises a few issues. If we keep recursively splitting, the number
of regions increases exponentially. We may require a lot of overhead in our
implementation to manage these regions. We may produce many small regions and
spend a lot of effort, only to find no solutions<sup><a href="#n3" id="t3">3</a></sup>. Even if we split the search space only once, the number of
regions grows exponentially as we increase the number of objectives; for
<code>N</code> objectives, we will get <tt>2<sup>N</sup> – 2</tt> regions.</p>
<p>We started thinking about how we could handle these issues, but abandoned them
when we discovered a serious problem<sup><a href="#n4" id="t4">4</a></sup>.</p>
<h2>Discovering a flaw in the algorithm</h2>
<p>Some of our test cases were failing. We examined the simplest failing test,
a problem with three objectives. The regions were being divided properly and
the solutions found were all locally optimal. However, compared to GIA, there
was an extra solution that was not globally optimal. Our claim that locally
optimal solutions are globally optimal was incorrect.</p>
<p>To see how this might be possible, let’s look at the test case. With three
objectives, the number of regions to search in is <tt>2<sup>3</sup>
– 2 = 6</tt>. Our constraints will be given in terms of the first Pareto point
the algorithm found, which is <tt><strong>P</strong> = (10, 11, 9)</tt>. The
constraints will be of the form <tt>{(m<sub>1</sub>, m<sub>2</sub>,
m<sub>3</sub>) such that m<sub>1</sub> ■ 10, m<sub>2</sub> ■ 11,
m<sub>3</sub> ■ 9}</tt>, where ■ is ≥ for “better than or
equal” or < for “worse than”. For shorthand, I’ll simply write
<tt>(■10, ■11, ■9)</tt>.</p>
<p>In the table below, I’ve listed the constraints for each of the regions, as well
as the metric points of the locally optimal metric solutions in each region. In
this particular problem, we found one locally optimal solution in each region.</p>
<table style="border-collapse:separate;border-spacing:10px;margin:0px auto;border:1px">
<thead>
<tr>
<th>Region</th>
<th>Constraint</th>
<th>Metric points of locally optimal solutions</th>
</tr>
</thead>
<tbody>
<tr>
<td>A</td>
<td><tt>(<10, <11, <9)</tt></td>
<td>Excluded; all solutions dominated by <tt><strong>P</strong></tt>
</td>
</tr>
<tr>
<td>B</td>
<td><tt>(<10, <11, ≥9)</tt></td>
<td><tt>(9, 8, 12)</tt></td>
</tr>
<tr>
<td>C</td>
<td><tt>(<10, ≥11, <9)</tt></td>
<td><tt>(7, 13, 8)</tt></td>
</tr>
<tr>
<td>D</td>
<td><tt>(<10, ≥11, ≥9)</tt></td>
<td><tt>(6, 12, 12)</tt></td>
</tr>
<tr>
<td>E</td>
<td><tt>(≥10, <11, <9)</tt></td>
<td><tt>(14, 10, 8)</tt></td>
</tr>
<tr>
<td>F</td>
<td><tt>(≥10, <11, ≥9)</tt></td>
<td><tt>(11, 9, 10)</tt></td>
</tr>
<tr>
<td>G</td>
<td><tt>(≥10, ≥11, <9)</tt></td>
<td><tt>(11, 14, 8)</tt></td>
</tr>
<tr>
<td>H</td>
<td><tt>(≥10, ≥11, ≥9)</tt></td>
<td>None; no solutions dominate <tt><strong>P</strong></tt>
</td>
</tr>
</tbody>
</table>
<br>
<p>As expected, the metric points obey the constraints. They are also locally
optimal, though I have not shown the (locally) dominated metric points. However,
note that the metric point <code>(7, 13, 8)</code> is dominated by <code>(11, 14, 8)</code>, which is
from a different region. In fact, for any given point in Region C, it is
possible a better one exists in Region G —– we simply take the same metric
values, but set the first one to be greater than or equal to <code>10</code>. Of course, that
point is only a valid solution if it meets the problem constraints, so it may
not actually exist. The <em>possibility</em> of its existence is the problem, since we
cannot guarantee the point in Region C is globally optimal.</p>
<p>Unfortunately, we cannot simply ignore Region C, just because we might be able
to find a better solution in Region G. For example, suppose the locally optimal
metric point in Region G is <code>(11, 14, 8)</code>, and the locally optimal metric point
in Region C is <code>(7, 15, 8)</code>. Neither solution dominates the other.</p>
<p>This is our dilemma: <em>some</em> locally optimal solutions are globally optimal, but
not <em>all</em> locally optimal solutions are globally optimal.</p>
<p>Before we conclude this post, it’s interesting —– and constructive —– to
examine why the algorithm worked with two objectives. Consider the diagram
below, where we have a locally optimal solution in Region B. I’ve marked the
area where we have to search to find a better solution, and it overlaps regions
B and D. Thus, the only way to find a better solution in a different region is
to find one in Region D. However, this is impossible, because Region D is empty.
Therefore, any locally optimal solution in Region B is globally optimal. (A
similar argument applies to Region C.)</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig5-4.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig5-4.svg" width="400" height="260" style="border:none">
</a>
</div>
<h2>Conclusion</h2>
<p>In this post, I introduced the <em>partitioned guided improvement algorithm</em>, an
attempt to divide the search space into regions that can be searched in
parallel. We discussed some of the previous attempts, as well as the inspiration
behind PGIA. Unfortunately, as presented, PGIA only works for bi-objective
problems.</p>
<p>In the <a href="/blog/fydp6.html">next part</a>, we’ll continue our discussion of the partitioned guided
improvement algorithm. We’ll see how the algorithm can be fixed so it works with
any number of objectives.</p>
<p><em>I would like to thank Talha Khalid and Chris Kleynhans for proofreading this
post.</em></p>
<h2>Notes</h2>
<ol>
<li><p><a style="text-decoration: none;" id="n1" href="#t1">^</a> Visualizing the
search space for a bi-objective problem is straightforward, since the
search space is a plane. A problem with three objectives is harder to
visualize, but still feasible. Unsurprisingly, trying to visualize four or
more dimensions is extremely difficult, if not impossible.</p></li>
<li><p><a style="text-decoration: none;" id="n2" href="#t2">^</a> In one of the
notes left behind was the warning “Beware: ideas that seem to intuitively
work in two dimensions do not always generalize to three or more
dimensions.”</p></li>
<li><p><a style="text-decoration: none;" id="n3" href="#t3">^</a> In early
prototypes, we observed that recursively splitting the search space helped
with performance, but after a while, performance degraded. We suspect that
the exponential number of regions, along with empty regions requiring
effort, is the cause.</p></li>
<li><p><a style="text-decoration: none;" id="n4" href="#t4">^</a> While working on
this post, I managed to find some very old code from previous groups. It
appears they had the same idea we had of splitting the search space.
However, at the top of the file was the comment “Has correctness problems,”
so I suspect we both encountered the same problem.</p></li>
</ol>
The subject of our next few posts is the partitioned guided improvement algorithm ... In this post specifically, we will cover a little background, the inspiration behind this algorithm, and finally, a mistake we made.tag:mhyee.com,2013-11-14:/blog/fydp4.htmlFourth-Year Design Project, Part 4: The Overlapping Guided Improvement Algorithm2013-11-14T05:00:00Z2013-11-14T05:00:00Z<p>In the <a href="/blog/fydp3.html">previous post</a>, I finished discussing all the relevant background
material for our group’s design project. I also described two approaches we were
exploring: incremental and checkpointed solving.</p>
<p>However, these approaches are single-threaded and do not take advantage of
multi-core processors. Therefore, our group is also interested in
multi-threaded<sup><a href="#n1" id="t1">1</a></sup> approaches. In this post,
I will be discussing the <em>overlapping guided improvement algorithm</em>, which is
the first of our two ideas. I also want to describe what problems we ran into
and fixed, and what new ideas we have.</p>
<p>If you need to refresh your memory on <a href="/blog/fydp2.html">terminology</a> and the <a href="/blog/fydp3.html">guided
improvement algorithm</a>, please take some time to review the two previous
posts. Again, I am also happy to answer any questions in the comments.</p>
<h2>The overlapping guided improvement algorithm</h2>
<p>The idea behind the overlapping guided improvement algorithm (OGIA) is actually
quite straightforward: we run multiple instances of GIA in parallel. The main
program will start up multiple threads, and each thread will follow the old
algorithm of finding a starting point, climbing up to the Pareto front, and then
repeating.</p>
<p>In the best case, all the threads do useful work, finding unique solutions, and
we speed up the algorithm immensely. In the worst case, only one thread does
useful work and the other threads find only duplicate solutions. This wasted
work is acceptable for our purposes, since we want to reduce the total time
between the start and finish of the program. Since the wasted work is done in
parallel, OGIA is no worse<sup><a href="#n2" id="t2">2</a></sup> than GIA.</p>
<h2>Implementation details</h2>
<p>As described above, the idea behind OGIA is rather straightforward. However,
there are many implementation details to consider.</p>
<p>For example, we need a way to identify duplicate solutions so that we never
yield the same solution twice. We also want to be smart about our <em>magnifier
task</em><sup><a href="#n3" id="t3">3</a></sup> —– if we can easily identify
duplicate solutions, then there is no point having multiple threads run the
magnifier task on the same Pareto point. To do this, we have a <em>solution
deduplicator</em><sup><a href="#n4" id="t4">4</a></sup> that keeps track of all the
Pareto points found, and also maintains the <em>global exclusion
constraints</em><sup><a href="#n5" id="t5">5</a></sup>.</p>
<p>When a thread finds a Pareto point, it reports it to the solution deduplicator.
If the Pareto point is a new solution, the thread adds a magnifier task for that
Pareto point to the task queue<sup><a href="#n6" id="t6">6</a></sup>. Next,
whether we have a duplicate or not, the thread asks the solution deduplicator
for the updated global exclusion constraints to find a new starting solution.</p>
<p>Thus, the solution deduplicator is responsible for keeping the set of unique
Pareto points found, and informing threads if the solution they found was
a duplicate or not. Since a magnifier task is queued only once per Pareto point,
we can be sure that the magnifier tasks will not waste any work.</p>
<p>Below, I’ve listed some pseudocode for the solution deduplicator. Our
implementation is thread-safe, but I have omitted those details from the
pseudocode.</p>
<pre><code class="language-ruby"><span class="no">SolutionDeduplicator</span><span class="p">:</span>
<span class="n">globalExclusionConstraints</span> <span class="o">=</span> <span class="n">empty</span>
<span class="n">solutionHashTable</span> <span class="o">=</span> <span class="n">empty</span>
<span class="c1"># Called when a thread has found a new solution.</span>
<span class="no">PushNewSolution</span><span class="p">(</span><span class="n">solution</span><span class="p">)</span>
<span class="k">if</span> <span class="n">solutionHashTable</span><span class="o">.</span><span class="n">contains</span><span class="p">(</span><span class="n">solution</span><span class="p">)</span>
<span class="c1"># We have a duplicate solution</span>
<span class="k">return</span> <span class="kp">false</span>
<span class="k">else</span>
<span class="c1"># Not a duplicate, so add the solution.</span>
<span class="n">solutionHashTable</span><span class="o">.</span><span class="n">add</span><span class="p">(</span><span class="n">solution</span><span class="p">)</span>
<span class="c1"># Update global exclusion constraints.</span>
<span class="n">globalExclusionConstraints</span> <span class="o">=</span> <span class="n">globalExclusionConstraints</span> <span class="no">AND</span>
<span class="ow">not</span> <span class="n">dominated</span> <span class="n">by</span> <span class="n">solution</span>
<span class="k">return</span> <span class="kp">true</span>
<span class="k">end</span>
<span class="k">end</span></code></pre>
<p>Here is how we have modified the double-nested loop from GIA, and packaged it as
a solution finder task for the OGIA threads.</p>
<pre><code class="language-ruby"><span class="no">SolutionFinder</span><span class="p">(</span><span class="n">solution</span><span class="p">):</span>
<span class="k">while</span> <span class="n">solution</span> <span class="n">exists</span>
<span class="c1"># Climb up to the Pareto front.</span>
<span class="k">while</span> <span class="n">solution</span> <span class="n">exists</span>
<span class="n">prevSolution</span> <span class="o">=</span> <span class="n">solution</span>
<span class="c1"># Find a better solution.</span>
<span class="n">solution</span> <span class="o">=</span> <span class="no">Solve</span><span class="p">(</span><span class="n">problemConstraints</span> <span class="no">AND</span> <span class="n">dominates</span> <span class="n">prevSolution</span><span class="p">)</span>
<span class="k">end</span>
<span class="c1"># Nothing dominates prevSolution; it's a Pareto point.</span>
<span class="c1"># Report to the solution deduplicator.</span>
<span class="n">isUniqueSolution</span> <span class="o">=</span> <span class="no">PushNewSolution</span><span class="p">(</span><span class="n">prevSolution</span><span class="p">)</span>
<span class="c1"># Queue magnifier task if solution was unique.</span>
<span class="k">if</span> <span class="n">isUniqueSolution</span>
<span class="n">tasks</span><span class="o">.</span><span class="n">queue</span><span class="p">(</span><span class="no">Magnifier</span><span class="p">(</span><span class="n">solution</span><span class="p">))</span>
<span class="k">end</span>
<span class="c1"># Throw the dart. Find a new solution.</span>
<span class="n">solution</span> <span class="o">=</span> <span class="no">Solve</span><span class="p">(</span><span class="n">problemConstraints</span> <span class="no">AND</span> <span class="n">globalExclusionConstraints</span><span class="p">)</span>
<span class="k">end</span></code></pre>
<p>This was the idea for our first implementation. However, we discovered a serious
problem: all the threads would start at the same starting solution, and then
find the same better solutions. Apart from the magnifier tasks, only one thread
was doing useful work. There was almost no improvement over the single-threaded
GIA.</p>
<p>While the idea to fix this problem was straightforward —– have each thread find
a different starting solution —– the actual implementation was less so.
Originally, we had hoped that the non-determinism in our SAT solver would find
different starting solutions, but this was not the case. We were also
unsuccessful in forcing this behaviour with different random seeds. In the end,
our only option was to find starting solutions in sequence, allowing us to use
exclusion constraints from the previous solutions. Unfortunately, this means we
have some unavoidable sequential work at the beginning of the algorithm.</p>
<p>Here is the initialization code from the main thread, which is responsible for
finding the starting solutions and starting the other worker threads.</p>
<pre><code class="language-ruby"><span class="no">Main</span><span class="p">:</span>
<span class="n">tasks</span> <span class="o">=</span> <span class="n">empty</span>
<span class="n">intialPointConstraint</span> <span class="o">=</span> <span class="n">problemConstraints</span>
<span class="k">for</span> <span class="mi">1</span><span class="o">.</span><span class="n">.numberOfThreadsToUse</span>
<span class="c1"># Get a unique starting point.</span>
<span class="n">solution</span> <span class="o">=</span> <span class="no">Solve</span><span class="p">(</span><span class="n">initialPointConstraint</span><span class="p">)</span>
<span class="k">if</span> <span class="n">solution</span> <span class="n">exists</span>
<span class="n">initialPointConstraint</span> <span class="o">=</span> <span class="n">initialPointConstraint</span> <span class="no">AND</span>
<span class="ow">not</span> <span class="n">dominated</span> <span class="n">by</span> <span class="n">solution</span>
<span class="c1"># Queue a solution finder task.</span>
<span class="n">tasks</span><span class="o">.</span><span class="n">queue</span><span class="p">(</span><span class="no">SolutionFinder</span><span class="p">(</span><span class="n">solution</span><span class="p">))</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="c1"># Wait until all threads have finished.</span>
<span class="n">tasks</span><span class="o">.</span><span class="n">wait</span></code></pre>
<p>For the full implementation details, you can find our code on <a href="https://github.com/TeamAmalgam/kodkod/pull/36">GitHub</a>.</p>
<h2>Preliminary results</h2>
<p>Here is the table from last time, with OGIA results added, and GIA results
removed, since our new work is built on top of IGIA. Recall that IGIA uses
incremental solving, while CGIA uses checkpointed solving. Again, these are
informal test results, run on the undergraduate computer science servers.</p>
<table style="border-collapse:separate;border-spacing:10px;margin:0px auto;border:1px">
<thead>
<tr>
<th><!-- empty --></th>
<th>IGIA</th> <!-- 1da4839 -->
<th>CGIA</th> <!-- 13ab50f -->
<th>OGIA</th> <!-- 8eaa3d6 -->
</tr>
</thead>
<tbody>
<tr>
<td>9 queens, 5 metrics</td>
<td>2 hours, 0 min</td>
<td>51 min</td>
<td>50 min</td>
</tr>
<tr>
<td>9 queens, 6 metrics</td>
<td>4 days, 16 hours, 43 min</td>
<td>1 hour, 39 min</td>
<td>1 hour, 53 min</td>
</tr>
<tr>
<td>9 queens, 7 metrics</td>
<td>Never attempted<sup><a href="#n7" id="t7">7</a></sup>
</td>
<td>3 hours, 37 min</td>
<td>4 hours, 4 min</td>
</tr>
<tr>
<td>Search and rescue, 5 metrics</td>
<td>3 hours, 0 min</td>
<td>1 hour, 26 min</td>
<td>2 hours, 38 min</td>
</tr>
<tr>
<td>Search and rescue, 6 metrics</td>
<td>2 hours, 8 min</td>
<td>59 min</td>
<td>1 hour, 11 min</td>
</tr>
<tr>
<td>Search and rescue, 7 metrics</td>
<td>5 hours, 1 min</td>
<td>2 hours, 16 min</td>
<td>2 hours, 46 min</td>
</tr>
</tbody>
</table>
<br>
<p>Once again, we see a dramatic improvement over IGIA. Interestingly, CGIA,
a single-threaded approach, performs better than OGIA, a multi-threaded
approach. However, in the same way we built OGIA on top of IGIA, we can build
OGIA on top of CGIA.</p>
<h2>Future work</h2>
<p>At the time of writing, we have four open issues to explore.</p>
<p>As noted above, we can build OGIA on top of CGIA. This would seem to be a very
easy way of getting significant improvements, but the situation is actually more
complicated. Both CGIA and OGIA greatly increase the algorithm’s memory usage.
We would probably have to reduce memory usage, before we could combine and use
both of them effectively.</p>
<p>The next issue is how we queue up magnifier tasks. Due to our implementation,
the magnifier tasks do not run until all the (unique) Pareto points have been
found<sup><a href="#n8" id="t8">8</a></sup>. This is not a serious problem, but
a nice property of GIA is that it yields solutions as they are found. With the
magnifier tasks deferred, we lose this property, and do not yield solutions
until the first magnifier task runs. We should be able to fix this easily, by
prioritizing magnifier tasks over the solution finder tasks.</p>
<p>Another minor issue is that the algorithm waits until every thread has finished,
meaning we are as fast as the slowest thread. However, we can do better than
this. If a solution finder task fails to find a new starting solution, then this
means all Pareto points have been found. The other solution finder tasks are
either trying to find a new starting solution (in which case, they will fail
like the first task), or they are climbing up to a Pareto point (which will be
a duplicate, since no new Pareto points exist). Therefore, we can cancel the
other solution finder tasks.</p>
<p>The final issue is much more open-ended. Our group has noticed that, even with
our dramatic improvements, there is still a lot of wasted work. Reducing this
wasted work could improve OGIA even more.</p>
<p>One idea we want to explore is how we handle the global exclusion constraints.
Currently, these constraints are based on the set of (known) Pareto points. We
suspect that since it can take a long time before Pareto points are found, the
global exclusion constraints are not updated frequently enough. This leads to
other tasks using “stale” information and performing duplicate work. If the
global exclusion constraints were based on non-optimal solutions, they would be
updated more frequently, and other threads would have “fresher” information.</p>
<h2>Conclusion</h2>
<p>In this post, I described the <em>overlapping guided improvement algorithm</em>,
highlighted some of its implementation details, and discussed some of the open
issues our group will be addressing in the future.</p>
<p>In the <a href="/blog/fydp5.html">next part</a>, I will be discussing our other multi-threaded
approach, the <em>partitioned guided improvement algorithm</em>. It lets us eliminate
all the duplicate work we had with OGIA, but the price is a far more complicated
algorithm.</p>
<p><em>I would like to thank Chris Kleynhans, Zameer Manji, and Arjun Sondhi for
proofreading this post.</em></p>
<h2>Notes</h2>
<ol>
<li><p><a style="text-decoration: none;" id="n1" href="#t1">^</a> Loosely put,
a thread is a portion of a program that can be executed concurrently with
other portions of the program. On a multi-core processor, each core can
execute a separate thread simultaneously. In contrast, a single-core
processor must switch between the different threads. With a multi-threaded
approach, we can run portions of the program in parallel.</p></li>
<li><p><a style="text-decoration: none;" id="n2" href="#t2">^</a> There will be
some overhead when we check for duplicates. However, this is a very cheap
operation.</p></li>
<li><p><a style="text-decoration: none;" id="n3" href="#t3">^</a> Recall that
multiple solutions may exist at the same Pareto point. The magnifier task
examines a Pareto point and yields all the solutions at that point.</p></li>
<li><p><a style="text-decoration: none;" id="n4" href="#t4">^</a> The solution
deduplicator is backed by a hash table, which means adding solutions and
checking for duplicates is extremely cheap.</p></li>
<li><p><a style="text-decoration: none;" id="n5" href="#t5">^</a> Recall that in
GIA, when a Pareto point is found, the exclusion constraints are updated so
that new starting solutions are not dominated by the existing Pareto
points. We do the same thing here, except that the exclusion constraints
are based on all Pareto points found, not just the ones found by the
current thread.</p></li>
<li><p><a style="text-decoration: none;" id="n6" href="#t6">^</a> When a thread is
finished its task, it will check the queue for a new task. As a result,
magnifier tasks are deferred until all Pareto points have been found.</p></li>
<li><p><a style="text-decoration: none;" id="n7" href="#t7">^</a> IGIA had very
little improvement over GIA for the 9-queens problems, so we never attempted
this case.</p></li>
<li><p><a style="text-decoration: none;" id="n8" href="#t8">^</a> Recall the
argument for GIA: the starting solution must not be dominated by any
existing Pareto point. If we cannot find such a solution, then all
solutions are either Pareto points or dominated by existing Pareto points.</p></li>
</ol>
In the previous post, I finished discussing all the relevant background material for our group's design project. In this post, I will be discussing the overlapping guided improvement algorithm, which is the first of our two [multi-threaded] ideas.tag:mhyee.com,2013-11-09:/blog/fydp3.htmlFourth-Year Design Project, Part 3: The Guided Improvement Algorithm2013-11-09T05:00:00Z2013-11-09T05:00:00Z<p>In the <a href="/blog/fydp1.html">first</a> <a href="/blog/fydp2.html">two</a> posts of this blog series, we discussed
multi-objective optimization: what it is and why it’s important. I concluded the
previous post by posing a question: how do we actually solve multi-objective
optimization problems?</p>
<p>My next few posts will be answering that question. More specifically, this post
will be about the <em>guided improvement algorithm</em>, while the other posts will be
about <em>improvements</em>.</p>
<p>There will be a lot of terminology in this post, so if you need a reminder,
please take some time to review my <a href="/blog/fydp2.html">previous post</a>. Also, the material in
this post will be more difficult than my previous ones —– if anything is
unclear, I am more than happy to clarify in the comments.</p>
<h2>The guided improvement algorithm</h2>
<p>The guided improvement algorithm (GIA) was first <a href="http://dspace.mit.edu/handle/1721.1/46322">introduced</a> by
Rayside<sup><a href="#n1" id="t1">1</a></sup>, Estler, and Jackson. The main
idea is to formulate the optimization problem as a set of constraints, and then
use a SAT solver<sup><a href="#n2" id="t2">2</a></sup> to find solutions that
satisfy those constraints. Furthermore, GIA also augments the constraints so we
can find <em>better</em> solutions.</p>
<p>We start by passing the problem constraints to the SAT solver and asking for
some solution. Informally, we call this step “throwing a dart,” since we do not
care how good the first solution is.</p>
<p>Next, GIA asks the SAT solver for a solution that dominates the previous one.
Specifically, GIA augments the constraints to specify that <em>all</em> the metrics of
the new solution are <em>at least as good</em> as the current solution’s metric values,
and that <em>at least one</em> of the metrics is <em>strictly better</em>.</p>
<p>The algorithm repeats this process, “climbing” up to the Pareto front, until it
cannot find a better solution. Thus, the last solution found is a Pareto point,
by definition. Note that, due to continually finding a better solution, the
Pareto point found will dominate the starting solution.</p>
<p>Recall that a Pareto point may contain multiple solutions. To handle this case,
the algorithm performs a task we call the “magnifier.” It searches for other
solutions, asking the SAT solver for solutions that have the same metric values
as the Pareto point just found.</p>
<p>The algorithm is not finished yet, as there may be more Pareto points. We may be
able to find a solution with a better metric value, at the cost of worsening
a different metric value. Therefore, we need to start a new climb up to the
Pareto front. However, we also want to guarantee that we find a new Pareto
point. To do this, GIA adds extra constraints to ensure that the starting point
is not dominated by any of the known Pareto points. Since the climb finishes at
a Pareto point that dominates the starting solution, and our starting point is
not dominated by a known Pareto point, we will end up at a new Pareto point.</p>
<p>The algorithm repeats this process: throwing a dart, climbing up to the Pareto
front, and running the magnifier on the Pareto point. If GIA throws a dart but
finds no solution, then this means there is no new solution that is not
dominated by an existing Pareto point. In other words, all solutions are either
Pareto points, or dominated by existing Pareto points. Thus, there is no new
Pareto point to be discovered.</p>
<p>We have an implementation of the algorithm on <a href="https://github.com/TeamAmalgam/kodkod/blob/master/src/kodkod/multiobjective/algorithms/GuidedImprovementAlgorithm.java">GitHub</a>. However, there are
a lot of implementation details, so the following pseudocode may be easier to
understand:</p>
<pre><code class="language-ruby"><span class="n">exclusionConstraints</span> <span class="o">=</span> <span class="n">empty</span>
<span class="c1"># Throw the dart. Get the first solution.</span>
<span class="n">solution</span> <span class="o">=</span> <span class="no">Solve</span><span class="p">(</span><span class="n">problemConstraints</span><span class="p">)</span>
<span class="k">while</span> <span class="n">solution</span> <span class="n">exists</span>
<span class="c1"># Climb up to the Pareto front.</span>
<span class="k">while</span> <span class="n">solution</span> <span class="n">exists</span>
<span class="n">prevSolution</span> <span class="o">=</span> <span class="n">solution</span>
<span class="c1"># Find a better solution.</span>
<span class="n">solution</span> <span class="o">=</span> <span class="no">Solve</span><span class="p">(</span><span class="n">problemConstraints</span> <span class="no">AND</span> <span class="n">dominates</span> <span class="n">prevSolution</span><span class="p">)</span>
<span class="k">end</span>
<span class="c1"># Nothing dominates prevSolution, so it's a Pareto point.</span>
<span class="c1"># Magnifier; find all solutions at the Pareto point.</span>
<span class="k">while</span> <span class="n">s</span> <span class="o">=</span> <span class="no">Solve</span><span class="p">(</span><span class="n">problemConstraints</span> <span class="no">AND</span> <span class="n">equals</span> <span class="n">prevSolution</span><span class="p">)</span>
<span class="k">yield</span> <span class="n">s</span>
<span class="k">end</span>
<span class="c1"># Augment the constraints. Exclude solutions dominated by known Pareto points.</span>
<span class="n">exclusionConstraints</span> <span class="o">=</span> <span class="n">exclusionConstraints</span> <span class="no">AND</span> <span class="ow">not</span> <span class="n">dominated</span> <span class="n">by</span> <span class="n">prevSolution</span>
<span class="c1"># Throw the dart. Find a new solution.</span>
<span class="n">solution</span> <span class="o">=</span> <span class="no">Solve</span><span class="p">(</span><span class="n">problemConstraints</span> <span class="no">AND</span> <span class="n">exclusionConstraints</span><span class="p">)</span>
<span class="k">end</span></code></pre>
<h2>An example execution of the guided improvement algorithm</h2>
<p>To better illustrate how GIA works, let’s walk through an execution. We’ll solve
a problem with two metrics, <code>m1</code> and <code>m2</code>. We’ll plot the metric points on
a graph like last time. I’ll use a dot to represent a solution that exists but
is unknown to the algorithm, and an X to represent a solution that was found by
the algorithm.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig3-1.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig3-1.svg" width="400" height="260">
</a>
</div>
<p>The algorithm throws a dart and finds its first solution<sup><a href="#n3" id="t3">3</a></sup>, which I’ve marked with an X. Next, it begins the climb up
to the Pareto front, searching for a better solution. The algorithm wants
a solution that dominates the previous one, so it ignores the shaded area<sup><a href="#n4" id="t4">4</a></sup>.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig3-2.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig3-2.svg" width="400" height="260">
</a>
</div>
<p>The algorithm continues its climb up to the Pareto front, searching for a better
solution each time.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig3-3.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig3-3.svg" width="400" height="260">
</a>
</div>
<p>We continue this process until no better solution can be found. Thus, the
previous solution is a Pareto point. The algorithm must now run the magnifier on
the Pareto point. To keep the example and diagrams simple, we’ll say there are
no other solutions at the Pareto point.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig3-4.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig3-4.svg" width="400" height="260">
</a>
</div>
<p>Now GIA needs to start its climb again. To ensure that the climb reaches a new
Pareto point, we need a starting solution that is not dominated by any of the
known Pareto points<sup><a href="#n5" id="t5">5</a></sup>. In the diagram below,
the new solution must exist outside this shaded area.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig3-5.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig3-5.svg" width="400" height="260">
</a>
</div>
<p>The algorithm continues this process of throwing a dart, climbing up to the
Pareto front, and running magnifier on the Pareto point. The second dart throw
is shown below.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig3-6.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig3-6.svg" width="400" height="260">
</a>
</div>
<p>The algorithm has found a better solution. It tries to find an even better one,
but cannot, so the algorithm has found another Pareto point.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig3-7.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig3-7.svg" width="400" height="260">
</a>
</div>
<p>The algorithm must throw a dart and start the climb again. The new solution must
not be dominated by the known Pareto points.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig3-8.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig3-8.svg" width="400" height="260">
</a>
</div>
<p>Here, the dart throw just happened to land on another Pareto point.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig3-9.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig3-9.svg" width="400" height="260">
</a>
</div>
<p>GIA throws a dart but cannot find a new solution. All solutions are either
optimal or dominated by the optimal solutions. Therefore, the algorithm can now
terminate.</p>
<div style="text-align:center">
<a href="http://files.mhyee.com/fydp/images/fig3-10.svg" style="border:none">
<img src="http://files.mhyee.com/fydp/images/fig3-10.svg" width="400" height="260">
</a>
</div>
<h2>The problem with the guided improvement algorithm</h2>
<p>The algorithm is very slow. Every call to the SAT solver is an extremely
expensive operation<sup><a href="#n6" id="t6">6</a></sup>. Furthermore, as a SAT
solver only understands Boolean algebra, we need to translate everything,
including arithmetic (which is used when adding up metric values)<sup><a href="#n7" id="t7">7</a></sup>.</p>
<p>The solving time for small problems is typically manageable. However, for larger
problems, GIA will not complete in any reasonable time. Worse, the difference
between “small” and “large” can be sudden and surprising. For example, one of
the our largest benchmarks is multi-objective 9-queens. It takes about three
hours to solve with five metrics. Add another metric and it takes almost five
days. With seven metrics, it takes over fifty days.</p>
<p>The algorithm’s performance is not limited to contrived problems such as
multi-objective 9-queens. We also see it with our search and rescue benchmark,
a real-world problem I mentioned in my <a href="/blog/fydp1.html">first post</a>. It takes over ten
hours to solve the seven metric version.</p>
<p>This is the reason for our group’s fourth-year design project. The guided
improvement algorithm is slow and scales poorly for large problems. Our project
is to design, implement, and test improvements to the algorithm.</p>
<h2>Incremental and checkpointed solving</h2>
<p>Before I conclude this post, I’d like to describe two improvements our group is
exploring: incremental and checkpointed solving.</p>
<p>We’re also working on other approaches, but incremental and checkpointed solving
apply to the way GIA interacts with the SAT solver, rather than the algorithm
itself<sup><a href="#n8" id="t8">8</a></sup>. The advantage here is that our
other approaches can build on top of incremental and checkpointed solving.</p>
<p>The issue we’re trying to address is that every time the algorithm asks the SAT
solver for a new solution, we waste a lot of work. As the SAT solver is
searching for a solution, it builds up state and slowly “learns.” However, that
is all discarded when we ask the SAT solver for a new solution, and it has to
start from scratch.</p>
<p>Some SAT solvers are capable of <em>incremental solving</em>. After solving a formula,
we add new constraints, and the SAT solver can reuse everything it “learned”
while solving the previous formula. This scenario perfectly fits how GIA
continually finds better solutions. We can keep the state up until we find
a Pareto point.</p>
<p>We <a href="https://github.com/TeamAmalgam/kodkod/pull/16">modified our implementation</a> to take advantage of incremental solvers.
When we ran our informal benchmarks, we saw tests run about twice as fast.</p>
<p>However, there is still room for improvement. Incremental solving allows us to
add new constraints without losing state, but the state is cleared when we
remove constraints. Unfortunately, this happens once the algorithm has found
a Pareto point —– it needs to step down from the Pareto front and find a new
starting solution.</p>
<p><em>Checkpointed solving</em> allows us to “roll back” to a previous state. Now, we can
work our way up to the Pareto front without losing state, and also roll back and
find a new starting solution without losing state. Thus, we no longer have to
discard any state.</p>
<p>It turns out very few SAT solvers support this functionality, so we had to
modify an existing SAT solver. It was a lot of work, but we <a href="https://github.com/TeamAmalgam/kodkod/pull/39">eventually
succeeded</a>. With checkpointed solving, we are about twice as fast as
incremental solving.</p>
<p>I’ve listed how long it takes to solve the 9-queens and search and rescue
problems with our algorithms: (base) GIA, incremental GIA (IGIA), and
checkpointed GIA (CGIA). We ran these informal tests on the shared undergraduate
computer science servers<sup><a href="#n9" id="t9">9</a></sup>.</p>
<table style="border-collapse:separate;border-spacing:10px;margin:0px auto;border:1px">
<thead>
<tr>
<th><!-- empty --></th>
<th>GIA</th> <!-- e2ae93c -->
<th>IGIA</th> <!-- 1da4839 -->
<th>CGIA</th> <!-- 13ab50f -->
</tr>
</thead>
<tbody>
<tr>
<td>9 queens, 5 metrics</td>
<td>2 hours, 53 min</td>
<td>2 hours, 0 min</td>
<td>51 min</td>
</tr>
<tr>
<td>9 queens, 6 metrics</td>
<td>4 days, 19 hours, 55 min</td>
<td>4 days, 16 hours, 43 min</td>
<td>1 hour, 39 min</td>
</tr>
<tr>
<td>9 queens, 7 metrics</td>
<td>Over 50 days<sup><a href="#n10" id="t10">10</a></sup>
</td>
<td>Never attempted<sup><a href="#n11" id="t11">11</a></sup>
</td>
<td>3 hours, 37 min</td>
</tr>
<tr>
<td>Search and rescue, 5 metrics</td>
<td>6 hours, 55 min</td>
<td>3 hours, 0 min</td>
<td>1 hour, 26 min</td>
</tr>
<tr>
<td>Search and rescue, 6 metrics</td>
<td>4 hours, 55 min</td>
<td>2 hours, 8 min</td>
<td>59 min</td>
</tr>
<tr>
<td>Search and rescue, 7 metrics</td>
<td>10 hours, 38 min</td>
<td>5 hours, 1 min</td>
<td>2 hours, 16 min</td>
</tr>
</tbody>
</table>
<br>
<p>As you can see, CGIA is very impressive. However, we still have some ideas to
refine it —– in particular, our implementation is not very memory efficient.</p>
<h2>Conclusion</h2>
<p>In this post, I described and showed how the guided improvement algorithm can
solve a multi-objective optimization problem. However, GIA can be incredibly
slow, which makes it less useful for large problems. I also briefly described
two promising approaches to improving the GIA: incremental solving and
checkpointed solving.</p>
<p>So far, these two approaches have been single-threaded, and cannot take
advantage of multi-core processors. We are also interested in parallelizing GIA.
In the <a href="/blog/fydp4.html">next part</a>, I’ll describe the <em>overlapping guided improvement
algorithm</em>, which is one of our approaches for a parallel GIA.</p>
<p><em>I would like to thank Chris Kleynhans, Zameer Manji, and Arjun Sondhi for
proofreading this post.</em></p>
<h2>Notes</h2>
<ol>
<li><p><a style="text-decoration: none;" id="n1" href="#t1">^</a> Professor
Rayside is the advisor for our fourth-year design project.</p></li>
<li><p><a style="text-decoration: none;" id="n2" href="#t2">^</a> In the Boolean
satisfiability problem (SAT), formulas are expressed in Boolean algebra,
where each variable is either 1 (true) or 0 (false). The operators are AND,
OR, and NOT. The problem is to determine whether there exists an assignment
of values such that the formula evaluates to “true.” For our purposes, we
will treat the SAT solver as a “black box”: we give it constraints, and it
either gives us a solution that satisfies the constraints or it declares
that the constraints cannot be satisfied.</p></li>
<li><p><a style="text-decoration: none;" id="n3" href="#t3">^</a> Any solution
could be the first solution. It could be a bad one, a solution that already
dominates another solution (as in this example), or even an optimal
solution.</p></li>
<li><p><a style="text-decoration: none;" id="n4" href="#t4">^</a> Solutions in the
shaded area fall in two categories: 1) they are dominated by the previous
solution; or 2) they do not dominate and are not dominated by the previous
solution. In other words, solutions here do not dominate the previous
solution. Solutions in the unshaded area, however, will dominate the
previous solution.</p></li>
<li><p><a style="text-decoration: none;" id="n5" href="#t5">^</a> If we start in
the shaded area, our search for a better solution could lead us to a new
Pareto point or an existing one. (Note that the discovered Pareto point is
right at the corner of the shaded area.) However, if we start outside the
shaded area, searching for a better solution will never take us to an
existing Pareto point.</p></li>
<li><p><a style="text-decoration: none;" id="n6" href="#t6">^</a> Readers with
a computer science background may recognize SAT as an NP-complete problem.
A gross simplification is to say that there is currently no known efficient
algorithm for SAT. Whether any efficient algorithm exists (for SAT or
similar problems) is the subject of the <a href="http://www.claymath.org/millennium/P_vs_NP/">P vs NP problem</a>, arguably
the most important unsolved problem in computer science.</p></li>
<li><p><a style="text-decoration: none;" id="n7" href="#t7">^</a> This is
a technique called “bit-blasting,” where adder and multiplier circuits are
created and then converted to SAT. As a consequence, the input formulas to
the SAT solver are extremely large.</p></li>
<li><p><a style="text-decoration: none;" id="n8" href="#t8">^</a> Of course,
another approach is to replace our current SAT solver with a better one.</p></li>
<li><p><a style="text-decoration: none;" id="n9" href="#t9">^</a> There are three
servers, each with 48 cores and 128 GB of memory. Each processor is
a twelve-core AMD Opteron 6100.</p></li>
<li><p><a style="text-decoration: none;" id="n10" href="#t10">^</a> After running
the test for 50 days, the server was rebooted and our test was lost. We
decided not to try again, until after we had made significant improvements
to the algorithm.</p></li>
<li><p><a style="text-decoration: none;" id="n11" href="#t11">^</a> Seeing as IGIA
had very little improvement for the other 9-queens problems, we decided it
was not worth testing it with 9-queens, seven metrics.</p></li>
</ol>
I concluded my previous post by posing a question: how do we actually solve multi-objective optimization problems? This post will answer that question by discussing the guided improvement algorithm.