Friday, 28 June 2013

A quick but dirty way to use the NAG C Library Mark 23 from Python

I get requests about using NAG from python from time to time, and NAG strives to make its algorithms as accessible as possible from a wide variety of languages and platforms. Python is no exception.

While we don’t yet provide a fully documented python API replete with examples and so on, there is a quick but dirty way to get NAG C Library python bindings covering every library algorithm, struct, and enum.

My starting point was the old PyNAG, which is excellent as far as it goes. It is also for the older Mark 8 library -- which none of you out there in userland should be using.

PyNAG also looks a bit labor-intensive to produce for the whole library, which would entirely disrupt my Maynard G. Krebs Cycle. There has to be better way, I thought.

And there is! At least on Linux. For this exercise I used our NAG C Library Mark 23 implementation for Linux and gcc, LP64 model, hereafter abbreviated as cll6a23dgl.

Here is the method that I used: 

Step 1 -- Get the Tools

For this exercise we'll need two tools. The first is gccxml, the extension to gcc that produces an xml description of the gcc compiler's internal representation of a compiled code. The second tool is the python extension ctypeslib. Both are available from the Ubuntu repositories (for example.)

 

Step 2 -- Collect the C Headers

Make a file "nag4py.h" that includes all of the relevant headers:

echo "#include <nag.h>" > nag4py.h

find /opt/NAG/cll6a23dgl/include -name "nag[a-z][0-9][0-9].h" | sort | sed -e "s/^.*\//#include </" | sed -e "s/$/>/" >> nag4py.h

echo "#include <nags.h>" >> nag4py.h

 

Step 3 -- Produce the XML

h2xml -I. -I/opt/NAG/cll6a23dgl/include nag4py.h -o nag4py.h.xml

 

Step 4 -- Produce the Python

xml2py nag4py.h.xml -o nag4py.py -l/opt/NAG/cll6a23dgl/lib/libnagc_nag.so

 

Step 5 (Optional) -- edits for clear C library concordance / compatibility with PyNAG

Edit the top of nag4py.py to read:

from ctypes import *

 

#human edit

E04_DEFAULT = None

NAGCOMM_NULL = None

NAGERR_DEFAULT = None

NE_NOERROR = 0

#end human edit

 

and edit the bottom of nag4py.py to read:

           'Nag_HashTable', 'Nag_SparseNsym_CheckData',

#human edit

'E04_DEFAULT','NAGCOMM_NULL','NAGERR_DEFAULT',’NE_NOERROR’]

#end human edit

 

Then with very minor modifcations the PyNAG examples will run. For many the only change needed is to change the import, from “PyNAG” to “nag4py”.

Another necessary change is that Numpy arrays must be passed slightly differently than with PyNAG, e.g.

g01alc(n, x.ctypes.data_as(POINTER(c_double)), …

  

instead of

g01alc(n, x.ctypes.data_as(c_void_p), …

Nag C Library enum types are translated nicely and may be used like one normally would, e.g.

f07aac(Nag_RowMajor,n,nrhs,…

and callback function typing is taken care of for us, e.g.

def py_obj_func(n,xc,objf,comm):

     objf[0] = exp(xc[0]) * (xc[0] * 4.0 * (xc[0]+xc[1]) + …

     return

c_obj_func = NAG_E04CCC_FUN(py_obj_func)

While the module nag4py.py generated in this way is immediately usable on Linux, it is specific to the local installation. To get it working on anything else a small amount of work remains to be done. First, we have to remove the extraneous debris from gcc that finds its way into nag4py.py. Second, we must omit some symbols exposed by the NAG C Library on Linux that are not exposed by all NAG C Library Mark 23 implementations.

Once that’s done, we get a module suitable for Linux, Windows, and OS X, available here. Updated PyNAG examples are here.

End Notes:

On Linux, if you’re instead using the NAG C Library ILP64 implementation CLL6A23DH, use this alternative module.

On Windows, I tested with the 64-bit Anaconda python distribution, which comes with all of the python bits needed to run the examples, including the graphical Rosenbrock example. An applicable NAG C Library implementation is CLW6I23DA. I found I had to add the folder containing the NAG DLLs to the system PATH environment variable.

On OS X, I tested with python 2.7.5 from python.org. A suitable NAG C Library implementation is CLMI623DG. In order to run the Rosenbrock example, I also needed to install numpy-1.7.1, matplotlib-1.2.1, and XQuartz , since I’m running Snow Leopard.

In all cases I’ve assumed the NAG C Library has been installed in the default location. If on your system it’s installed elsewhere, you’ll need to edit nag4py.py accordingly.

Thanks to my colleague Brian Spector for testing on OS X Lion and for some suggested example changes. Thanks also to my colleague Mike Hooper, whose gentle questions suborned my inner Krebs.

Jul 17 Update: Some example cleanup, included three new graphical examples, and changed the module name.

3 comments:

  1. Nice work. I can retire PyNAG it seems. I only left it un-updated because no one appeared to be using it. I only received one piece of user feedback in the three years since writing it and that user went on to use pure python instead.

    ReplyDelete
  2. Have you considered using the cffi library? Much easier to use than ctypes, in my experience...

    ReplyDelete
  3. Hi Anon. I have run across cffi, and indeed I might use it for new code, or if I needed to use a handful of C algorithms.

    ReplyDelete

NAG moderates all replies and reserves the right to not publish posts that are deemed inappropriate.