Thursday, January 1, 2015

HPC@home

I spent a fair amount of my vacation time with comparing several OpenCL-enabled boards. I looked for a "cheap" board in terms of price/performance and operational costs (power consumption) for massively parallel workload. I found that the best performing boards available (for almost anyone) are AMD Radeon R9 295X2 and NVIDIA GeForce GTX TITAN Z.The following table summarizes my findings.


Radeon R9 295X2GeForce GTX TITAN Z
Price (USD)11002200
Single precision performance (TFLOPS)11.4668.122
Double precision performance (TFLOPS)1.4332.66
Thermal Design Power (W)500375
OpenCL version1.21.1

There are of course many more criteria, but these are the most important ones that affect my decision.

So the absolute winner is Radeon R9 295X2. Yes, it is a little bit power hungry, but its 22.9 GFLOPS/W is slightly better than the 21.7 GFLOPS/W of GeForce GTX TITAN Z. If you cannot afford to build a nuclear reactor in your room, the Radeon R9 290X is also a good choice with 18.78 GFLOPS/W.
Just a side note, but I really don't know what to think about GeForce GTX TITAN Z. Its FP64 performance is remarkable, but the board is a bit overpriced. It is also an open question, how the board performs using OpenCL, since NVIDIA's support for CUDA is far better than for OpenCL.

References:

Thursday, April 4, 2013

Code+tests == specification?

Have you worked together with Agile enthusiasts (or fanatics) in the past? If the answer is yes, you might know what I want to talk about.
One of the core concepts of the agile movement is "working code over comprehensive documentation". I strongly agree with it. Many of us know of projects failed or missed deadlines because of the big up front specification and design. It usually does not work if requirements can change during the design or implementation phase. Unfortunately many agile enthusiasts simply state that time spent preparing the specification is useless, as the code and test suite themselves replace specification. I think they try to put the cart behind the horse.
First, without enough time spent on up front specification and design, the project usually ends up with a sub-optimal architecture, ad hoc solutions and endless refactoring. The more large scale the project is, the greater the chance of failure or at least missing deadlines.
Second, you need a specification to make sure you build the right product. Yes, you can show the result of each iteration to the customer, but his/her opinion it not a proof. He/she might not be aware of other things than the UI and a subset of the functional requirements. Without a written specification you cannot prove anything. Sure, you can test the implementation, but unit tests can only prove the presence of bugs, they cannot prove their absence. Even Test-Driven Development won't help you, since tests may be incomplete or may contain bugs and the code reflects this. The only way to prove correctness is to test each and every feature based on the written specification. This can be automated by creating an acceptance test suite, but first you need a specification to write acceptance tests.
Years before, I was attended to a conference organized by a local computer science society. One of the presenters talked about a case study of a military-grade project done by using Extreme Programming. The presenter was proud of the fact that they only used post-it notes created on iteration planning meetings. I was curious if they had to verify the resulting system and how could they do it without a specification, only based on these post-it notes? Unfortunately the session ran out of time, and the presenter left the room, leave me no chance to talk him.
To recap, in most of the cases, skipping the specification step is not something that can be done without negative consequences. We are agile enough if we spend some time for creating a specification and a conceptual architecture up front. This is an overhead, of course, but it'll pays back in the long run.

Monday, March 18, 2013

Google Summer of Code

New year, new Google Summer of Code. If I would be a student again, I'd seriously consider to apply to one of the projects proposed by the Java Pathfinder team. I wish their proposals will be accepted by Google.

Wednesday, February 27, 2013

Nulls Are Mostly Harmful


This is an old thing, but it should be emphasized again and again. Nulls are mostly harmful, because
  • they force programmers to pollute the code with null checks
  • uninitialized references may easily cause runtime errors
  • null semantics is ambiguous, null may represent
    • an unintentionally uninitialized reference
    • a legal value (an intentionally uninitialized reference)
    • a lack of result of an operation
    • a result of a failed operation
    • a result of a human (usually programming) error
    • etc.
So avoid them by using
  • defensive programming practices
    • never return null, only in the rare case when the operation has clear semantics, and
    • explicitly document this decision in a formal way (e.g. using the javax.annotation.Nullable annotation)
    • use the Null Object design pattern if you should represent a legal null value
    • annotate the code with javax.validation.constraints.NotNull to be able to verify runtime reference nullness
  • validation using static and runtime analysis
    • static analysis is fast and well-integrated with most of the popular IDEs (see The Checker Framework and FindBugs), but it only finds errors that can be detected in compile-time
    • runtime analysis is more accurate, but slower and requires a testing phase in the development process (see Java Pathfinder and Hibernate Validator).

Monday, January 14, 2013

Java PathFinder Maven repository how-to

I set up a Maven repository for Java PathFinder projects in the past few days. I spent long hours to figure out the solution for some problems, so I decided to share my experiences.

Install a Maven repository manager

The first step is to install a Maven repository manager. There are many good open-source candidates like Artifactory or Apache Archiva, but my favourite is Sonatype Nexus. There is a standalone version which runs in a Jetty container, but it is also distributed as a war file that can be deployed in almost any JavaEE container. Its install process is very straightforward, consisting of a few easy steps.
After you have a working repository manager, you should create a Maven repository for JPF. It can be done using the web-based admin interface of Nexus. (Do not forget to change the default password of admin and deployment users.)
At the left panel you find the "Repositories" link which opens the "Repositories" tab. Click the "Add" button on the toolbar and choose the "Hosted Repository" from the list, then fill the "Repository ID" and "Repository Name" fields. After you save the configuration, your repository can be accessed in {URL of your Nexus deployment}/content/repositories/.

Generate and upload the artifacts

At first, clone the Mercurial repositories of JPF that you want to hosts in your Maven repository. Then you should run Ant build for each project to create the jar files that you upload to the Maven repository.
After this step, you are ready to upload the artifacts. It can be done using the admin UI of Nexus or from command line by executing mvn deploy for each jar file:
mvn deploy:deploy-file -Durl={URL of your Nexus deployment}/content/repositories/{repository ID} -DrepositoryId={repository ID} -Dpackaging=jar -Dfile={jar file} -DgroupId=gov.nasa.jpf -DartifactId={name of the jar without the extension} -Dversion={JPF module version number}
Before issuing this command, make sure that your Maven settings contain a repository entry for the repository ID and the corresponding username/password entries are set.


Automate

The JPF projects are constantly changing, so you might want to update your Maven repository with the artifacts containing these changes. The next step is to create a script to poll the Mercurial repositories, compile the sources and upload the jar files to the Maven repository. I created a small Bash shell script for this task:
#!/bin/bash
 

nexus_repo='{URL of your Nexus deployment}/content/repositories/{repository ID}'
jpf_home='{path to your local Mercurial repositories}'
repo_id='{repository ID}'
group_id='gov.nasa.jpf'
module_version='{JPF module version number}'

for repo in $jpf_home/jpf-*; do

    cd $repo
    changesets=$(hg incoming)
    if [ -v "$changesets" ] || [ ! -d build ]; then

        hg pull -u && ant || continue
        for jar in build/jpf*.jar; do
            if [ -f $jar ]; then
                artifact_id=${jar:6:-4}

                mvn deploy:deploy-file -Durl=$nexus_repo -DrepositoryId=$repo_id -Dpackaging=jar -Dfile=$jar -DgroupId=$group_id -DartifactId=$artifact_id -Dversion=$module_version -Ddescription="$changesets"
            fi

        done
    fi
done
This script updates the local Mercurial repositories, executes the Ant build task for each JPF project and uploads the resulting jars to the Maven repository. In case you want to run the script periodically, you may create a cron job running couple of times a day or a week.

..and done.

Tuesday, October 30, 2012

Butterfly-effect in unit tests

One of our projects suffered a significant delay in development. Programmers complain that they cannot make progress as planned, because of the complexity of the legacy code base.
A quick look revealed that all the unit tests show a typical testing antipattern: they are optimized for 100% code coverage. In order to reach this, the SUT is modified to expose the internals of the code, not only the public interface of it. All methods' visibility were changed to public/protected, and tests covered not only the method calls and their results, but the very details of the implementation as well. This led to a very fragile environment, where even a tiny little change in the implementation could potentially break most of the unit tests. Moreover, the test were structured very poorly, effectively rendering the readability to the bare minimum.
After we finished the code review, it was clear that we need to refactor the whole test suite, not only some tests of it. First we should change the SUT, and revert all the visibility related changes. Then we dropped all the tests that verified the internals of the SUT. The remaining tests were refactored to increase the readability. We also needed to make changes to these tests to verify the right thing. Finally, we wrote some new tests against the public API of the SUT to reach the desired coverage level.
To sum up:
  1. Unit tests are part of the documentation. They should be readable and clearly structured (given, when, then).
  2. Unit tests are only useful if they increase the productivity of the developers by helping them catching bugs earlier. Failing to comply this requirement means that they do not add value to the development process.

Saturday, March 19, 2011

Unit testing reloaded: Java Path Finder

Unit tests are good. Sometimes we hate them because of the work overhead they require, but almost everyone agree on their usefulness. So the question is: what could we do to eliminate (or at least minimize) this overhead? The answer is quite simple: we should minimize the number of separate unit tests. It is not impossible, since most of the test cases are based on simple assertions, so they can be easily integrated with the source code. In order to do this we need a simple but powerful notation to express the constraints, since assertions scattered across the source statements obscure the code. Annotations are very helpful in this case. We can create annotations for classes and field variables as well as input parameters and return values of operations (we are talking about the design by contract idiom).
Defining annotations is easy, but it is very time consuming to write a processor for the processing of the statements written in these annotations. Fortunately, we do not have to implement anything, we can use the Java Path Finder. Its jpf-aprop module fully supports the design by contract idiom.
JPF is a real multi-purpose tool, and it has a moderate learning curve. Unfortunately, the online documentation is a bit outdated, but there are many publications and examples that can be downloaded from the site, so an average Java programmer can use the tool without difficulties.