Thursday, 30 May 2013

ISC'13 Preview - Supercomputing in Leipzig

ISC'13 - the summer's big international conference for the world of supercomputing - is only a few weeks away. As usual, I will be attending, along with several of my NAG colleagues.

This year, ISC will be in Leipzig - a new location. That means as well as the #HPC [do I need hashtags in a blog post???] fun of the conference itself, we will all have to learn the foibles of a new host city. Which bar will replace the cornerstone that was the Radisson Blu in Hamburg? How quickly can we learn the layout of the conference center itself?

More to the point, what will be the big #HPC [there I go again] news stories and themes to watch out for in Leipzig? I have already been interviewed by InsideHPC as an early ISC'13 preview, and have written a blog for the official ISC blog (comparing #HPC and #F1) - but here are some further thoughts.

Take the MIC

Well, as far as big #HPC [getting silly now] news goes, I'm glad I didn't try to write this blog last week! I think the stand out news story of ISC'13 is pretty obvious (barring another dramatic reveal) - the excellent expose by HPC Wire of China's MIC (sorry Xeon Phi) based 50+ petaflops supercomputer. Yes, at the ISC'13 issue of the Top500 list, China's Tianhe-2 supercomputer will smash aside the USA's Titan supercomputer at ORNL to take the crown as the world's fastest supercomputer - by a huge margin.

Indeed, in a painful turn of events, Titan may be dumped out of the Top500 leadership position before it has even passed it's acceptance tests! I'll reserve further comment on the Tianhe-2 story for a upcoming blog post at my own HPC blog (

I believe Tianhe-2 will not be the only Xeon Phi news story at ISC'13 - there will probably be a few other Xeon Phi based supercomputers to be announced or updated, along with several product announcements etc (including some from NAG).

Wednesday, 15 May 2013

Calling the NAG C Library from kdb+

Kx Systems was founded in 1993 to address a simple problem: the inability of traditional relational database technology to keep up with escalating volumes of data. This problem is even more acute today, where data records, especially in the financial industry, number in the trillions. To provide users to fast data, the founders of Kx have developed kdb+ and vector-processing programming language 'Q'.

Kdb+ has the ability to quickly process data and for more complex analysis it has access to other programming languages. To call the NAG C Library from kdb+, a wrapper must be placed around the NAG routine, taking care when passing data types. The wrapped functions must then be compiled into an external library. Below is an example for calling a NAG regression routine:

#include <nag.h>
#include <nagg02.h>
#include <string.h>
#include <nag_stdlib.h>
#define KXVER
3          // Using version 3, this must come before #include "k.h"
#include"k.h"              // kx header file containing K structures

K regression(K x, K y)          //The two arrays we will input from kdb+
       double *wt=0;
       NagError fail;            
       K kout;                         //Array that we will return to kdb+
       kout=ktn(KF,7);            //Allocate kout - it will have 7 elements. KF indicates
                                          // it is an array of floating point numbers
       F*out=kF(kout);            //Set pointer to kout - comes after kout is allocated

       F*(x_input)=kF(x);         //Need to set pointers to input arrays into NAG
       nag_simple_linear_regression( Nag_AboutMean, (Integer) x->n, x_input, y_input, wt, &out[0], &out[1], &out[2], &out[3], &out[4], &out[5], &out[6], &fail );

       if(fail.code != NE_NOERROR)
              return r0(kout),krr(ss(fail.message));    //Free memory and return any
                                                                         //NAG fail to Q
       return kout;

In production level code, we should really check array bounds and data types. If you would like, you can add the following checks:

     if(x->n != y->n) return krr("data not same length");
     if(x->t != -KF || y->t != -KF) return krr("input data wrong type");

We must then compile the above code into a sharable library. On Linux the command is:
gcc -shared -fPIC nag_regression.c -I/[NAG_Directory]/include  [NAG_Directory]/lib/ -o

Before we start the Q language, make sure the library can be located:

Finally, we load the library in Q and test the NAG example program:
q)nag_reg:`nag_regression 2:(`regression;2)
q)nag_reg[1.0 0.0 4.0 7.5 2.5 0.0 10.0 5.0;20.0 15.5 28.3 45.0 24.5 10.0 99.0 31.2]
7.598166   7.090489   6.685763   1.322359   .8273433   965.2454   6

Outputs from the regression are the regression coefficients, the standard errors, the R-squared, sum of squares, and degrees of freedom (which all agree with the NAG example).

Interested in other NAG routines in kdb+? Email and let us know! Many thanks to the support team at Kx Systems.