Posts

Showing posts from 2012

Bitwise Reproducibility with the NAG Libraries

I've written in this blog before about the problems of Wandering Precision - where the results computed by a program are not consistent, even when running exactly the same program on the same machine with the same data several times in a row.

At SC12 in Salt Lake City a couple of weeks ago I took part in a "Birds of a Feather" session organised by Intel, where problems like these, associated with bitwise reproducibility of results, were discussed. On the panel, apart from myself, were representatives of Intel's Math Kernel Library, The MathWorks (producers of MATLAB), German engineering company GNS, and the University Of California, Berkeley.

We all gave brief presentations discussing the impact of non-reproducible results on our work or on our users, and ways around the problem. It turned out that most of
our presentations were remarkably similar, involving tales of disgruntled users who were not happy about accepting such varying results. This is true even when the…

Now the games are over – what’s our legacy? How can we inspire a generation?

Image
I really didn’t expect to love the Olympics as much as I did. I’m not really into sport and have never taken much interest before, but this time I was absolutely enthralled. London, being the host city, obviously had a bearing on my enthusiasm and enjoyment, but what now the games are over? A lot of talk centred around the Olympics 2012 ethos of ‘inspiring a generation’ and this got me thinking about NAG’s legacy and how we too can inspire.

NAG has had a busy year as well; we've managed to produce significant new releases of our four main numerical software products, the C and Fortran Libraries, the Library for SMP & Multicore and the Toolbox for MATLAB®. If you study the history of the Library you start to understand how truly amazing it is. So is the Library NAG’s legacy? Well, funnily enough, our legacy is right there in each of the new releases. That’s why the NAG Library is amazing. It’s an ever changing, ever improving set of codes that have been developed by people th…

The making of “1000x” – unbalanced supercomputing

I had a bit of a rant in my article published at HPCwire this week - “Chasing1000x: The future of supercomputing is unbalanced”.

The gist of my rant is that the supercomputing community pays great attention to the next factor of 1000x in performance – and I firmly agree that next 1000x is highly valuable to the HPC community and the wider economy. But, we should give equal attention to 1000x in other areas, notably ease-of-use and growth of the user-base. And, critically, give equal peer recognition to those promoting that growth and pursuing ease-of-use and ease-of-access, not reserve all our “visionary” accolades for those figuring out the details of the first exascale computers.

However, I planted an appropriate pun in the title of the article itself. The obvious meaning, in the context of the article, is that the future of supercomputing is unbalanced with respect to the focus on performance versus community growth etc. However, the double meaning should be readily recognizable to…

The NAG SMP and Multicore Library on the Cloud

Image
In my last post we looked at the NAG serial library on Amazon's EC2. We noticed that the CPU utilization was not anywhere close to the capacity I had on the Cloud. Below, I have loaded the NAG SMP and Multicore Library on Amazon's EC2 in hopes of utilizing all the virtual cores.
Routines Tested The EC2 instance used was a High-CPU, c1.xlarge instance (7GB of memory and 8 Virtual Cores). The SMP Library contains 204 tuned and a total of 337 enhanced routines for use on Multicore machines from which I tested f11me(Sparse Matrix Factorization) and g02bn(Kendall/Spearman rank coefficient). While I had a maximum of 8 cores available, I decided to increase the number of threads beyond this, just to see the result. Below you will see how the time taken scales as the number of threads increases (click to enlarge):


Both routines scale well as you increase the number of threads, but f11me takes longer with 12 threads as opposed to 8! I suspect the slowdown is a result of dependencies bet…

NAG on the Cloud: Part 2

Image
In case you missed Part 1 of NAG on the Cloud, I discussed calling the Library on Windows Azure using the C# library and Microsoft's Cloud Numerics. We now turn to a different cloud provider. Amazon's Elastic Cloud Computing (EC2) allows the user to buy or rent instances of virtual computers and pay for only what you use. Below, I'll discuss how to put the NAG Library on EC2 and give some initial performance tests.

Creating an Instance There are a plethora of videos on creating Amazon EC2 instances. If you are going to install and run the NAG Library, make sure you start a Virtual Private Cloud when creating the instance. Private Clouds can be created at no additional cost on EC2 and remain compatible with NAG's Kusari License management system. I decided to create a couple tests:

My EC2 Console Instances
The above instances vary in size, memory, and type of OS running. When the instance is created, you can gain access to it via ssh. To load the library onto EC2, jus…

What's new at Mark 23? Linear quantile regression

Image
This is the first in an occasional series of posts highlighting new functionality in the latest release (Mark 23) of the NAG Library.  Here, we describe linear quantile regression, which has just been added to the collection of regression techniques that is already available in the Library.

Regression techniques are concerned with modelling and analyzing the relationship between a dependent (or response) variable and one or more independent (or explanatory) variables.  More specifically, they enable the user to understand how the typical value of the response variable changes when one or more of the explanatory variables are varied.  A common example of regression analysis is linear least-squares regression, which is concerned with modelling the behaviour of the conditional meanof the response variable and which, as its name implies, employs the well-known method of least-squares.  By contrast, linear quantile regression models one of the conditional quantiles of the response variable …

A cure for the common code: NAG at The Trading Show Chicago

Image
A few weeks ago, my colleagues and I attended The Trading Show Chicago, which looks at the world of quantitative financial analysis, trading strategy and technology.  The conference was preceded by two technology workshops; I attended one of these, which was on the use of independent component analysis (ICA) and so-called second order statistics for hedge fund data.  After the session, I had an interesting conversation with the presenters (Prof Andrew Kumiega from IIT and Dr Greg Sterijevski from UBS O’Connor) afterwards about ICA, NAG’s functionality – with which they were already familiar, and complementary about – and the way in which it could be used in ICA.


Following a plenary session on the opening day of the conference, the proceedings were split into four streams: Trading Congress, HFT World, Quant Invest and Exchange Technology.  There was a very nice talk by Professor Emanuel Derman on the use of metaphor, models and theory by quants; I had a brief chat with Derman afterward…

The Matrix Square Root, Blocking and Parallelism

Image
NAG recently embarked on a ‘Knowledge Transfer Partnership’ with the University of Manchester to introduce matrix function capabilities into the NAG Library. As part of this collaboration, Nick Higham (University of Manchester), Rui Ralha (University of Minho, Portugal) and I have been investigating how blocking can be used to speed up the computation of matrix square roots.
There is plenty of interesting mathematical theory concerning matrix square roots, but for now we’ll just use the definition that a matrix X is a square root of A if X2=A. Matrix roots have applications in finance and population modelling, where transition matrices are used to describe the evolution of a system from over a certain time interval, t. The square root of a transition matrix can be used to describe the evolution for the interval t/2. The matrix square root also forms a key part of the algorithms used to compute other matrix functions.
To find a square root of a matrix, we start by computing a Schur decom…

Using NAG and LabVIEW in a 64 bit environment

Image
This is the latest in a series of blog posts about enhancing LabVIEW applications by using NAG methods and routines; previously, we've described in detail how to invoke methods from the NAG Library for .NET in LabVIEW, and how to call routines from the NAG Fortran and C libraries from within that programming environment.  In addition, we supplemented those descriptions with an archive of examples which is available from the NAG LabVIEW page.

The examples we looked at previously were all in the 32 bit environment, but some users have asked whether all this works in the 64 bit world, being keen to take advantage of the larger address space of that architecture.  Indeed it does, as we shall show here.

So just how expensive is Marshaling?

One complication in using the NAG C Library from .NET is callback functions which, in C, have arrays in their parameter lists. Take for example the optimization algorithm e04nfc(nag_opt_qp) for quadratic programming problems. This NAG function requires a callback function qphess of the following C prototype: void qphess(Integer n, Integer jthcol, constdouble h[], Integer tdh, constdouble x[], double hx[], Nag_Comm *comm);
In C# the corresponding delegate is

publicdelegatevoidNAG_E04NFC_QPHESS (int n, int jthcol, IntPtr h_ptr, int tdh, IntPtr x_ptr, [In, Out] IntPtr hx, refCommStruct comm);

If you follow the C example program for nag_opt_qp as well as the style of the NAG C# examples you will write something like this for qphess in C#:
staticvoid qphess0(int n, int jthcol, IntPtr h_ptr, int tdh, IntPtr x_ptr, [In, Out] IntPtr hx_ptr, refCommStruct comm)         { double[] xloc = newdouble[n]; double[] hloc = newdouble[n * n]; double[] hxloc = newdouble[n];
Marshal.Copy(h_ptr, hloc, 0, n * n); Marsh…

ISC'12 Hamburg Preview

ISC'12 - the summer's big international conference for the world of supercomputing - is next month in Hamburg.

I will be attending, along with several of my NAG colleagues. Will you be attending? What will you be looking to learn? I will be listening out for these five key topics.

GPU vs MIC vs Other

As at ISC'11 last year (and SC11), I think there will be a strong fight for attention in the key area of manycore/GPU devices - and a matching search for evidence of real progress. So far the loudest voice has been NVidia and CUDA, especially following NVidia's successful GTC event recently. However, interest in Intel's MIC (Knights Corner) is strong and growing - MIC has often been a big discussion topic in workshops, conferences and meetings over the last year. As the MIC product launch gets closer, people will be making obvious comparisons with NVidia's Kepler announced at the GTC.

What about others - will anyone else develop a strong voice in this manycore world…

An Affair with the Java Native Access (JNA)

I was recently speaking to a colleague about my first couple projects here at NAG. The first project was learning to call the Library from Python using c-types (thanks to Mike Croucher’s blog which helped immensely). Next, was a project using the Java Native Interface (JNI), which I had difficulty using. After hearing the above two pieces of information, my colleague recommended I look into Java Native Access (JNA) as it was very similar to c-types in Python. Thus began a brief love affair! I say ‘love affair’ because my experience the JNA was a bit of a roller coaster of highs and lows. In the beginning, the JNA and I got along great. As time went on, I was left sitting at the computer screen wondering what to do next, hoping for the JNA to fix things.Background of JNINAG already has a thorough technical report on our website for calling the NAG Library using the Java Native Interface. This includes creating header files, compiling java files, compiling the interface library, and run…

Optimization, statistics, big data and business analytics

Image
Last month, we attended the INFORMS 2012 conference in order to learn more about current activities in the field of business analytics, and to present the results of some of the work we've done in this area.  The meeting kicked off with a series of interesting technology workshops run by commercial companies as a means of promulgating their software systems; the main insight I got from them was the importance that the community places on high-quality optimization solvers in areas like prescriptive analytics, in which quantitative methods are employed to help make better decisions in business.

The NAG Library contains a variety of optimization routines (for both local and global minimization) - along with, of course, a wide range of solvers for other types of problems in analytics (such as statistical analysis, correlation and regression modelling and time series analysis) and in a variety of other numerical areas.  At the conference, we presented the results of some consultancy wo…

NAG on the Cloud

Image
I have been at NAG for 3 months now and one of my first tasks here was the topic of cloud computing. Customers have been inquiring as to whether they can utilize the NAG library on the hundreds of cores available on Cloud services like Microsoft's Azure and Amazons EC2. Below you will find a preliminary report of calling the NAG Library for .NET on Windows Azure.

I began with Microsoft's Cloud Numerics; a .NET analytical library that can easily be scaled out to Windows Azure for large computations. Cloud Numerics provides a library of about 400 Mathematical and Statistical functions that the user can call (in this case, from C#). Since NAG supplies the library in a .NET framework, I decided this was a good way to start.

Getting an account and all the correct software downloaded can be a challenge. I actually found this example quite useful for installation, setup, and deployment of Cloud Numerics on Azure.



To start calling NAG functions from the MSCloudNumerics example program…

How To: Call Brent's Root-Finding Algorithm From C#

As a Senior Technical Consultant for NAG, I answer many customer questions covering many topics. I thought I’d write up one such question I recently received from a NAG C Library user, as the answer may be useful to others.

Q: In looking through the C# associated info, I found many examples of InteropService calls from C# to the C Library (CLW3209DA_nag.dll). Have any examples been posted for the "c05" functions, e.g. nag_zero_cont_func_brent_bsrch(c05agc)?

A: I'm glad you asked! By the time you reach the end of this post, there will be one.J
In working with the NAG C Library from C#, there are three main factors to which we must attend. The first is how to represent the NAG C Library structure types in C#, and for C# this has largely been taken care of for you in NAGCFunctionsAPI.cs.

How to worry about everything (and nothing)

Worry - To feel uneasy or concerned about something. In my last post I wrote about managing your e-mail inbox (in the narrow sense) and, more broadly, managing your work and commitments. I ended that last post with this quote: "If you worry about everything, then you don't have to worry about anything." At least one reader suggested that I owed them an explanation and so here it is.

You'll recall that I challenged you (and me) to get our e-mail inboxes empty at least once per week in a systematic way, as follows:
Go through you inbox one message at a time and ask yourself "Is there an action required?"

If the answer is "No", then either delete as trash, file it elsewhere for later reference or file it in your "great things I'd like to do someday but don't know when" folder. Be ruthless.

Adding functionality to Excel using the NAG Library for .NET

Image
Much of our work at NAG is devoted to creating new implementations of our numerical libraries and attempting to make their algorithms available from as many languages and packages as possible, so that our users have access to them from whichever environment they're working in. Thus, users of packages such as MATLAB® (and similar packages such Octave), LabVIEW and Maple, and programmers working in languages like Java, Python and Visual Basic (along with, of course, more traditional languages such as C and Fortran) have all been making use of NAG algorithms to enhance their applications and solve numerical problems for a long time.

Microsoft Excel® users can easily access NAG routines from both the NAG Fortran Library and the NAG C Library, because they are distributed as Dynamic Link Libraries (DLLs). For example, my colleague Marcin Krzysztofik has recently described how to solve a nonlinear least-squares problem in Excel using the nag_opt_nlin_lsq (e04unc) routine from the NAG …

Upgrades - hotels, airlines and HPC

I was pointed to this short but interesting blog today: "What's your upgrade?" by @therichbrooks, which makes the point that customers like it when businesses over-deliver on expectations. It is easy to understand what over-deliver might mean for hotels, airlines, rental cars, etc. - upgrades! - but it is equally important for other businesses to consider.

In the contexts of High Performance Computing (HPC) and of software, upgrades are a part of the routine. This covers both upgrades to newer or more powerful hardware (e.g. see the recent upgrades to the Cray supercomputers at HECToR - the UK's national supercomputing service - run by partners including NAG for CSE Support); and software upgrades for new features etc. However, these are all expected and planned upgrades - whilst they do deliver more to the customers, they are not a "over-delivery". And of course, for the service teams, upgrades mean hard work installing, testing, benchmarking and documenti…

Self Improvement - An Algorithm for getting to "empty"

Whether we like it or not, at NAG and many other organizations, we live in an "e-mail" culture meaning that e-mails are how we communicate, receive and retain information. For many of us, e-mails also document both what we have done and what we still have to do. If you are like most in this culture, your e-mail inbox is the hub of your work life. I'm going to suggest an inbox "experiment" for you but first, a little fun.

One of my favorite ways to get to know someone is to ask how they use their inbox. It's almost a litmus test for personalities. So, what does yours say about you? Is yours:

The Black Hole: E-mail gets sucked in but never leaves, a filing cabinet with one gigantic drawer and two folders labelled "In" and "Sent". Periodically, either due to an inspired desire to get organized or "intervention" from a systems administrator, the inbox gets purged and the cycle begins anew.

The Formula One Pit: E-mail comes racing i…

How to solve a NLLS problem using SQP method in Excel?

NLLS stands for nonlinear least-squares and SQP is sequential quadratic programming. So essentially this is an optimization problem, and everyone knows that NAG Library's chapter e04 is the best place to look for optimization solvers. The appropriate NAG routine in our C Library is nag_opt_nlin_lsq (e04unc).
A few weeks ago one of our users contacted NAG and asked for an example program of using e04unc in Excel. NAG and Excel page has quite a few examples and guidelines about using NAG Library in Excel, but we didn't have this particular one.
I wrote this example and now it is available for download on the Excel page. I encourage readers of this blog to download it and play with it on your own. It wasn't difficult to create it, but there was one issue that caused me a nasty headache. Some routines that have callback functions (just as e04unc does) where a vector or matrix is passed to/from a subroutine require usage of Windows API subroutine RtlMoveMemory.

Girls, Geeks, Twitter and Me.

Image
When I get to work one of the first things that I do each morning is check out what’s happening on my Twitter timeline. One Thursday, a couple of weeks ago, one particular tweet caught my eye. It lead me to a great blog 'Girls can love computing; someone just needs to show them how' about the Manchester Girl Geeks. They are a group who are trying to encourage more girls and women to be interested in maths, science and technology. Being a girl myself, (OK, a woman really), and working for a mathematical software company, the article sparked a real interest.





When I was at school, maths wasn’t my best subject, well actually and I’m going to be completely honest with you, it was my worst subject. My fear of all things mathematical started after being made to stand in front of the class reciting times tables. So it’s somewhat ironic that I found myself working at a numerical software company a few years ago, albeit in the marketing department. Had the 'Girl Geeks' been arou…

Cloud computing or HPC? Finding trends.

Image
Enable innovation and efficiency in product design and manufacture by using more powerful simulations. Apply more complex models to better understand and predict the behaviour of the world around us. Process datasets faster and with more advance analyses to extract more reliable and previously hidden insights and opportunities.

All ambitions that will probably resonate with those seeking scientific advances, commercial innovation, industrial growth and more cost-effective research. Underpinning all of the above is the use of more powerful computing methods and technologies. Faster and more capable computers - but equally important - more advanced and better performing algorithms and software implementations.

It's a pretty convincing story for those who take the time to listen - whether business leaders, governments, or research funders. Even in these challenging economic times, it has led to investments from industry and governments for this reason - the potential return is well d…

Self-Improvement

Winston Churchill once said "The pessimist sees difficulty in every opportunity. The optimist sees the opportunity in every difficulty." (for you confirmed pedants, it may have been L.P. Jacks)

My custom is to use the time away from work at the end of the year to think about what I want to do differently in the year ahead. Among the topics that came up was e-mail, the bane of my life and perhaps yours as well. I get hundreds every day (and that doesn't include the SPAM).

Being a lifelong optimist I've decided to make e-mail less of a pain in my life, both work and personal. Being an occasional realist, I recognize that I have a limited number of options and they must focus on what I can do.

So, here's my plan for 2012 (with acknowledgement to Scott Belsky and Stever Robbins who supplied some of the ideas and got me thinking). See Disrupt Your Inbox and What you should never say in an e-mail Experiment with three-sentence emails when I need an answer from someone.…

Question one: Where's my phone?

Image
The end of the year at NAG is always celebrated with a rather splendid Christmas lunch, which is generously paid for by the company as an acknowledgement of the hard work its employees have put in over the previous twelve months. Accordingly, it provides an occasion for some much-needed relaxation and refreshment before the Christmas break. It's also the time for the NAG Christmas Quiz which, whilst not necessarily contributing to the participants' relaxation, usually provides some entertainment or diversion for those who care about such arcane matters as the number of hearts an octopus has or Paul McCartney's middle name. One of those tortured souls is the present author, who cleverly realized some time ago that the only way to be sure of knowing all the answers to the questions was to set them.

Setting quiz questions in the connected age - particularly for a technical-savvy band such as the employees of NAG - can present a few challenges, however. For example, anyone…