## Wednesday, 28 November 2012

### Bitwise Reproducibility with the NAG Libraries

At SC12 in Salt Lake City a couple of weeks ago I took part in a "Birds of a Feather" session organised by Intel, where problems like these, associated with bitwise reproducibility of results, were discussed. On the panel, apart from myself, were representatives of Intel's Math Kernel Library, The MathWorks (producers of MATLAB), German engineering company GNS, and the University Of California, Berkeley.

We all gave brief presentations discussing the impact of non-reproducible results on our work or on our users, and ways around the problem. It turned out that most of

our presentations were remarkably similar, involving tales of disgruntled users who were not happy about accepting such varying results. This is true even when the same

users completely understand the fact that numerical software is subject to rounding errors, and that the varying results they see are not incorrect - they may be equally good approximations to an ideal mathematical result. But they just don't like inconsistency - in fact, often they would prefer to get an answer that is accurate to fewer digits of precision so long as it is consistent, rather than inconsistent but slightly more accurate results.

As it happens, we do have some control over these inconsistent results, which are largely due to optimizations introduced by clever compilers that are designed to take advantage of fast SIMD instructions like SSE and AVX that are available on modern hardware. By using appropriate compiler flags on the Intel Fortran and C compilers, for example, we can avoid these optimizations, at the cost of making the code run up to 12 or 15% slower (according to Intel).

## Wednesday, 24 October 2012

### Now the games are over – what’s our legacy? How can we inspire a generation?

London 2012 Olympic Park |

NAG has had a busy year as well; we've managed to produce significant new releases of our four main numerical software products, the C and Fortran Libraries, the Library for SMP & Multicore and the Toolbox for MATLAB®. If you study the history of the Library you start to understand how truly amazing it is. So is the Library NAG’s legacy? Well, funnily enough, our legacy is right there in each of the new releases. That’s why the NAG Library is amazing. It’s an ever changing, ever improving set of codes that have been developed by people that work here or contributed by well-known and maybe not so quite well-known numerical analysts, statisticians and computer scientists from all around the world.

Back in the day – 1970 to be precise – Brian Ford, NAG’s Founding Director, was inspired with a vision of a collective inter-university numerical library and set about creating the first NAG Library. Mentored by the legendary Jim Wilkinson, Brian established NAG as an organisation that was, and still is not-for-profit. Brian said of Jim,

Brian's key ideals were voluntary collaboration and quality in every phase of the activity. Over the years mathematical and statistical routines have been contributed by some highly regarded people:“he gave us his invaluable numerical linear algebra software, and his contacts".

## Friday, 12 October 2012

### The making of “1000x” – unbalanced supercomputing

*Chasing1000x: The future of supercomputing is unbalanced*”.

The gist of my rant is that the supercomputing community pays great attention to the next factor of 1000x in performance –

*and I firmly agree that next 1000x is highly valuable to the HPC community and the wider economy*. But, we should give equal attention to 1000x in other areas, notably ease-of-use and growth of the user-base. And, critically, give

**equal peer recognition to those promoting that growth and pursuing ease-of-use and ease-of-access**, not reserve all our “visionary” accolades for those figuring out the details of the first exascale computers.

However, I planted an appropriate

**pun in the title of the article itself**. The obvious meaning, in the context of the article, is that the future of supercomputing is unbalanced with respect to the focus on performance versus community growth etc. However, the double meaning should be readily recognizable to anyone active watching or developing the HPC technology roadmaps. The future of supercomputers is unbalanced – i.e., the broad summary of the many technology roadmaps out there is that future supercomputers (at all scales) will be less balanced in some key performance attributes than current technology.

## Friday, 28 September 2012

### The NAG SMP and Multicore Library on the Cloud

**Routines Tested**

Both routines scale well as you increase the number of threads, but f11me takes longer with 12 threads as opposed to 8! I suspect the slowdown is a result of dependencies between threads. In order for some to start, they may have to wait on other parts of the matrix factorization to finish. G02bn on the other hand doesn't require communication between threads so each one can run independently. This routine slightly benefited from running on 12 vs. 8 threads.

## Wednesday, 29 August 2012

### NAG on the Cloud: Part 2

**Creating an Instance**

**My EC2 Console Instances**

The above instances vary in size, memory, and type of OS running. When the instance is created, you can gain access to it via ssh. To load the library onto EC2, just download the .tgz file of your choice from the NAG website and secure copy it up there:

$scp -i VPCkey.pem fll6a23dfl.tgz ubuntu@50.112.131.89:/home/ubuntu

Here VPCkey.pem is a key file that only allows my computer to access this Cloud instance. Once the Library is copied, we can unzip it and the installation instructions remain the same as if the computer were right in front of you!

## Tuesday, 14 August 2012

### What's new at Mark 23? Linear quantile regression

*linear quantile regression*, which has just been added to the collection of regression techniques that is already available in the Library.

Regression techniques are concerned with modelling and analyzing the relationship between a dependent (or

*response*) variable and one or more independent (or

*explanatory*) variables. More specifically, they enable the user to understand how the typical value of the response variable changes when one or more of the explanatory variables are varied. A common example of regression analysis is

*linear least-squares regression*, which is concerned with modelling the behaviour of the

*conditional mean*of the response variable and which, as its name implies, employs the well-known method of least-squares. By contrast, linear quantile regression models one of the conditional quantiles of the response variable (see this note for more information on its formal definition). The solution of this problem is obtained using linear programming techniques; least-squares methods cannot be used here.

## Tuesday, 24 July 2012

### A cure for the common code: NAG at The Trading Show Chicago

John Morrissey, Paul Hipes and Brian Spector offering the benefit of their numerical expertise in the NAG Clinic at The Trading Show Chicago |

Following a plenary session on the opening day of the conference, the proceedings were split into four streams: Trading Congress, HFT World, Quant Invest and Exchange Technology. There was a very nice talk by Professor Emanuel Derman on the use of metaphor, models and theory by quants; I had a brief chat with Derman afterwards, during which he referred to NAG in positive terms. After the conference, my colleague kindly lent me Derman’s book, "

*My Life As A Quant*" which provides some valuable insights on his evolving views about the applicability of models to quantitative analysis, as well as interesting background on his personal history (he started his academic career as a particle physicist, switching to finance when he joined Goldman Sachs in 1985 to become one of the first so-called POWS – i.e. Physicists On Wall Street).

## Wednesday, 11 July 2012

### The Matrix Square Root, Blocking and Parallelism

*X*is a square root of

*A*if

*X*Matrix roots have applications in finance and population modelling, where transition matrices are used to describe the evolution of a system from over a certain time interval,

^{2}=A.*t*. The square root of a transition matrix can be used to describe the evolution for the interval

*t/2*. The matrix square root also forms a key part of the algorithms used to compute other matrix functions.

*U*of the resulting upper triangular matrix

*T*can then be found via a simple recurrence over the elements

*U*and

_{ij }*T*:

_{ij}## Monday, 18 June 2012

### Using NAG and LabVIEW in a 64 bit environment

The examples we looked at previously were all in the 32 bit environment, but some users have asked whether all this works in the 64 bit world, being keen to take advantage of the larger address space of that architecture. Indeed it does, as we shall show here.

## Tuesday, 12 June 2012

### So just how expensive is Marshaling?

One complication in using the NAG C Library from .NET is callback functions which, in C, have arrays in their parameter lists. Take for example the optimization algorithm e04nfc(nag_opt_qp) for quadratic programming problems. This NAG function requires a callback function qphess of the following C prototype:

In C# the corresponding delegate is

public delegate void NAG_E04NFC_QPHESS (int n, int jthcol,

## Thursday, 24 May 2012

### ISC'12 Hamburg Preview

I will be attending, along with several of my NAG colleagues. Will you be attending? What will you be looking to learn? I will be listening out for these five key topics.

**GPU vs MIC vs Other**

As at ISC'11 last year (and SC11), I think there will be a strong fight for attention in the key area of manycore/GPU devices - and a matching search for evidence of real progress. So far the loudest voice has been NVidia and CUDA, especially following NVidia's successful GTC event recently. However, interest in Intel's MIC (Knights Corner) is strong and growing - MIC has often been a big discussion topic in workshops, conferences and meetings over the last year. As the MIC product launch gets closer, people will be making obvious comparisons with NVidia's Kepler announced at the GTC.

What about others - will anyone else develop a strong voice in this manycore world? AMD Fusion? ARM? DSP-based products? Will we talk with the same energy about the software issues of manycore, or just the hardware choices?

**What is happening with Exascale?**

The quest to attain exascale computing rumbles on. I'd expect exascale to take a big share of the debate and agenda at ISC'12. What is happening with the exascale programs around the world? How are the budgets in the weak global economy affecting the exascale ambition? How are the various national pride (I mean national competitiveness) efforts towards exascale progressing? Will the software challenges get the level of investment needed? What new technologies will emerge to be studied as candidates to solve part of the problems getting to exascale?

Is exascale so old hat - do we need to move to discussion of zettaFLOPS to be trendy now? Will zettaFLOPS be impossible (

*Sterling*) or inevitable (

*Barr*)? Maybe we should spare some discussion for making multi-petaFLOPS work properly first?

## Monday, 21 May 2012

### An Affair with the Java Native Access (JNA)

I was recently speaking to a colleague about my first couple projects here at NAG. The first project was learning to call the Library from Python using c-types (thanks to Mike Croucher’s blog which helped immensely). Next, was a project using the Java Native Interface (JNI), which I had difficulty using. After hearing the above two pieces of information, my colleague recommended I look into Java Native Access (JNA) as it was very similar to c-types in Python. Thus began a brief love affair! I say ‘love affair’ because my experience the JNA was a bit of a roller coaster of highs and lows. In the beginning, the JNA and I got along great. As time went on, I was left sitting at the computer screen wondering what to do next, hoping for the JNA to fix things.

Background of JNI

NAG already has a thorough technical report on our website for calling the NAG Library using the Java Native Interface. This includes creating header files, compiling java files, compiling the interface library, and running the program. Seems like lots of work, even for simple functions. I was hoping the JNA would be easier.

First date with the JNA

To start using the JNA you just need to go to download the .jar file from https://github.com/twall/jna. Download the file and then move it to the directory you will be working in. Unzip it to create a **com** folder and you’re done! You can now start using it. Whenever you need to use a package from the JNA, just import it at the top of your Java file.

My first impression of the JNA started off on a high note. I found it extremely easy to start using the interface. Looking at code for calling the Bessel function (routine s17acc):

## Friday, 11 May 2012

### Optimization, statistics, big data and business analytics

The NAG Library contains a variety of optimization routines (for both local and global minimization) - along with, of course, a wide range of solvers for other types of problems in analytics (such as statistical analysis, correlation and regression modelling and time series analysis) and in a variety of other numerical areas. At the conference, we presented the results of some consultancy work performed for a client who was using NAG routines to solve a large-scale constrained optimization problem arising from activities such as price promotion (see abstract 4 on this list for more details).

The remainder of the conference consisted of a couple of plenary talks (from Google and eBay), an entertaining panel discussion on the perennial topic of Big Data, and a collection of contributed talks which were arranged in fifteen parallel sessions on topics like

*The Analytics Process*,

*Decision Analytics*,

*Analytics Around Us*, etc. I found the standard of the presentations to be very high; a few personal highlights were:

## Friday, 13 April 2012

### NAG on the Cloud

I began with Microsoft's Cloud Numerics; a .NET analytical library that can easily be scaled out to Windows Azure for large computations. Cloud Numerics provides a library of about 400 Mathematical and Statistical functions that the user can call (in this case, from C#). Since NAG supplies the library in a .NET framework, I decided this was a good way to start.

Getting an account and all the correct software downloaded can be a challenge. I actually found this example quite useful for installation, setup, and deployment of Cloud Numerics on Azure.

To start calling NAG functions from the MSCloudNumerics example program, just add the NAG .NET dll under references and include the namespace NagLibrary. When you are ready to deploy the application to the cloud, right-click the 'AppConfigure' tab in the Solution Explorer and select 'Set as StartUp Project'. Then put in your Azure account information, create a cluster, and deploy!

## Tuesday, 3 April 2012

### How To: Call Brent's Root-Finding Algorithm From C#

*As a Senior Technical Consultant for NAG, I answer many customer questions covering many topics. I thought I’d write up one such question I recently received from a NAG C Library user, as the answer may be useful to others.*

**Q:**In looking through the C# associated info, I found many examples of InteropService calls from C# to the C Library (CLW3209DA_nag.dll). Have any examples been posted for the "c05" functions, e.g. nag_zero_cont_func_brent_bsrch(c05agc)?

**A:**I'm glad you asked! By the time you reach the end of this post, there will be one. J

## Tuesday, 20 March 2012

### How to worry about everything (and nothing)

You'll recall that I challenged you (and me) to get our e-mail inboxes empty at least once per week in a systematic way, as follows:

Go through you inbox one message at a time and ask yourself "Is there an action required?"

## Thursday, 1 March 2012

### Adding functionality to Excel using the NAG Library for .NET

^{®}(and similar packages such Octave), LabVIEW and Maple, and programmers working in languages like Java, Python and Visual Basic (along with, of course, more traditional languages such as C and Fortran) have all been making use of NAG algorithms to enhance their applications and solve numerical problems for a long time.

Microsoft Excel

^{®}users can easily access NAG routines from both the NAG Fortran Library and the NAG C Library, because they are distributed as Dynamic Link Libraries (DLLs). For example, my colleague Marcin Krzysztofik has recently described how to solve a nonlinear least-squares problem in Excel using the nag_opt_nlin_lsq (e04unc) routine from the NAG C Library.

## Thursday, 23 February 2012

### Upgrades - hotels, airlines and HPC

**upgrades!**- but it is equally important for other businesses to consider.

In the contexts of High Performance Computing (HPC) and of software, upgrades are a part of the routine. This covers both upgrades to newer or more powerful hardware (e.g. see the recent upgrades to the Cray supercomputers at HECToR - the UK's national supercomputing service - run by partners including NAG for CSE Support); and software upgrades for new features etc. However, these are all expected and planned upgrades - whilst they do deliver more to the customers, they are not a "over-delivery". And of course, for the service teams, upgrades mean hard work installing, testing, benchmarking and documenting the updated system.

But the key point of the linked blog was not upgrades, rather it was about managing (and meeting) customer expectations - and about over-delivery.

## Friday, 17 February 2012

### Self Improvement - An Algorithm for getting to "empty"

One of my favorite ways to get to know someone is to ask how they use their inbox. It's almost a litmus test for personalities. So, what does yours say about you? Is yours:

The Black Hole: E-mail gets sucked in but never leaves, a filing cabinet with one gigantic drawer and two folders labelled "In" and "Sent". Periodically, either due to an inspired desire to get organized or "intervention" from a systems administrator, the inbox gets purged and the cycle begins anew.

The Formula One Pit: E-mail comes racing in and the pit crew (you) frantically tries to dash off a response that nominally addresses or acknowledges the item. It could be a "holding" response (thinking about it, promise to get back to you later) or a delegating response (passing it on or telling the writer to see some one else). The key feature, like the pit crew, is to get it out as fast as possible. This is a difficult personality to maintain, especially when you aren't connected to the network or need to sleep;-)

The Swiss Army Knife: This inbox, like its multi-tool namesake, can do most everything. It carries information for later reference, tasks that need to be done, dates/times for meetings, etc. It does it all. Stuff gets thrown out or filed elsewhere occasionally when it has been taken care of but it remains the ultimate "nerve center" of work life.

## Wednesday, 15 February 2012

### How to solve a NLLS problem using SQP method in Excel?

## Thursday, 2 February 2012

### Girls, Geeks, Twitter and Me.

A Girl Geek Tea Party |

When I was at school, maths wasn’t my best subject, well actually and I’m going to be completely honest with you, it was my worst subject. My fear of all things mathematical started after being made to stand in front of the class reciting times tables. So it’s somewhat ironic that I found myself working at a numerical software company a few years ago, albeit in the marketing department. Had the 'Girl Geeks' been around in the 80s and had visited my school it might have made maths and science a bit more alluring for me. It’s a sad truth that girls are still way in the minority in choosing technical and science options at GCSE, A’level and degree level*.

NAG want to help in some way to reverse this trend. I think I can speak for NAG in saying that we want to see more women achieving prominence in our organisation and in scientific computing in general.

Anyway, back to my reason for blogging. After reading about the great work that Manchester Girl Geeks are involved in we decided to support them by way of sponsorship. I’m writing today to raise awareness of their mission as we feel it’s really worthwhile. We have some ideas for other ways in which NAG can assist their goals in the future.

What other ways can we as an organisation make a positive difference?

*around 16% in 2009 of students in undergraduate computer science degrees are female.

## Thursday, 19 January 2012

### Cloud computing or HPC? Finding trends.

**Enable innovation and efficiency in product design and manufacture by using more powerful simulations. Apply more complex models to better understand and predict the behaviour of the world around us. Process datasets faster and with more advance analyses to extract more reliable and previously hidden insights and opportunities.**

All ambitions that will probably resonate with those seeking scientific advances, commercial innovation, industrial growth and more cost-effective research. Underpinning all of the above is the use of more powerful computing methods and technologies. Faster and more capable computers - but equally important - more advanced and better performing algorithms and software implementations.

It's a pretty convincing story for those who take the time to listen - whether business leaders, governments, or research funders. Even in these challenging economic times, it has led to investments from industry and governments for this reason - the potential return is well documented and significant. It is even enticing enough to interest the media and the public - especially when we use emotive descriptions like "

*world's fastest supercomputer*", "

*international competitiveness in digital economy*", "

*personal supercomputing*", and so on.

And it is this last thought that cause me to diverge from the grand theme to explore names and attention. I will come back to the main theme later (a future blog), as it is both important and timely. But on to my side topic.

## Thursday, 5 January 2012

### Self-Improvement

My custom is to use the time away from work at the end of the year to think about what I want to do differently in the year ahead. Among the topics that came up was e-mail, the bane of my life and perhaps yours as well. I get hundreds every day (and that doesn't include the SPAM).

Being a lifelong optimist I've decided to make e-mail less of a pain in my life, both work and personal. Being an occasional realist, I recognize that I have a limited number of options and they must focus on what I can do.

- Experiment with three-sentence emails when I need an answer from someone. (improve the likelihood that someone will actually read and answer).
- Start e-mails with action I want, Don't leave the reader guessing until the end.
- Use subject lines that intrigue the reader and actually invite them to open and read the e-mail.
- Take disagreements offline. There are volumes that could be written on this.
- Don't "reply all" unless everyone needs to be involved. How do you feel when you are one of 92 people copied on an e-mail that doesn't interest or pertain to you? Resist the urge, it's probably illegal.
- If you need to make several points and expect a response, use numbers for reference to reduce length, opportunity for confusion.

More on managing e-mail in a future installment.

## Tuesday, 3 January 2012

### Question one: Where's my phone?

Setting quiz questions in the connected age - particularly for a technical-savvy band such as the employees of NAG - can present a few challenges, however. For example, anyone with internet access (via, say, a smartphone's web browser) would be able to find the answers to the questions indicated above in a matter of moments. More direct questions, such as

- Who wrote "A Child's Christmas In Wales"?
- What kind of logs did Good King Wenceslas ask for?
- When was "Merry Xmas Everybody" number 1 in the UK?

are even easier to answer, though some skill and judgement may still be required in the selection of the correct response to an ambiguous or ill-posed question such as the first one on this list (does it refer to Dylan Thomas's prose piece or John Cale's song?). Whilst this is clearly a valid and imaginative use of technology, I wondered whether it would provide an unfair advantage over those participants who wouldn't be using their phones in this fashion (or texting more knowledgeable friends for answers) and started wondering about ways to obviate their effectiveness.