Self Improvement - An Algorithms for getting to "empty"

Whether we like it or not, at NAG and many other organizations, we live in an "e-mail" culture meaning that e-mails are how we communicate, receive and retain information. For many of us, e-mails also document both what we have done and what we still have to do. If you are like most in this culture, your e-mail inbox is the hub of your work life. I'm going to suggest an inbox "experiment" for you but first, a little fun.

One of my favorite ways to get to know someone is to ask how they use their inbox. It's almost a litmus test for personalities. So, what does yours say about you? Is yours:

The Black Hole: E-mail gets sucked in but never leaves, a filing cabinet with one gigantic drawer and two folders labelled "In" and "Sent". Periodically, either due to an inspired desire to get organized or "intervention" from a systems administrator, the inbox gets purged and the cycle begins anew.

The Formula One Pit: E-mail comes racing in and the pit crew (you) frantically tries to dash off a response that nominally addresses or acknowledges the item. It could be a "holding" response (thinking about it, promise to get back to you later) or a delegating response (passing it on or telling the writer to see some one else). The key feature, like the pit crew, is to get it out as fast as possible. This is a difficult personality to maintain, especially when you aren't connected to the network or need to sleep;-)

The Swiss Army Knife: This inbox, like its multi-tool namesake, can do most everything. It carries information for later reference, tasks that need to be done, dates/times for meetings, etc. It does it all. Stuff gets thrown out or filed elsewhere occasionally when it has been taken care of but it remains the ultimate "nerve center" of work life.

There are others but I think you get the idea. So, you ask, what's the experiment you want me to try? In essence, I want you to get your inbox emptied at least once each week but I want you to do it in a way that is functionally different than the Formula One Pit. Here's the algorithm:

Set aside an hour at a quieter time each week (early morning, late afternoon, whatever works)

Go through you inbox one message at a time and ask yourself "Is there an action required?"

1. If the answer is "No", then either delete as trash, file it elsewhere or reference or file it in your "great things I'd like to do someday but don't know when" folder. Be ruthless.
2. If the answer is "Yes" then answer the question "what's the action required?" and one of the following four things happens to it
• If you can take the needed action in 2 minutes, do it now and either delete the e-mail or file it in another folder for later reference.
• If somebody else needs to do what's needed, forward it and delegate
• If you need to act on it on a specific day or day and time (e.g., a meeting) put it on your calendar
• If you need to act on it but the it's not time or day specific, put it on your task list. By the way, if it's really a project with multiple steps, put it on your task list as a project and just note the very next step.
3. You're done! Everything that was in your inbox is now in the trash, filed for later reference, delegated to someone else, on your calendar or on your task list.

How do you feel? Can you focus better now on what you need to do without that feeling of dread you get when you look at an inbox with 50 messages (or 150, 250)? Let me know what you think.

Next time: Thinking outside the Inbox and an unattributed quote to puzzle over: "If you worry about everything, then you don't have to worry about anything."

How to solve a NLLS problem using SQP method in Excel?

NLLS stands for nonlinear least-squares and SQP is sequential quadratic programming. So essentially this is an optimization problem, and everyone knows that NAG Library's chapter e04 is the best place to look for optimization solvers. The appropriate NAG routine in our C Library is nag_opt_nlin_lsq (e04unc).

A few weeks ago one of our users contacted NAG and asked for an example program of using e04unc in Excel. NAG and Excel page has quite a few examples and guidelines about using NAG Library in Excel, but we didn't have this particular one.

I wrote this example and now it is available for download on the Excel page. I encourage readers of this blog to download it and play with it on your own. It wasn't difficult to create it, but there was one issue that caused me a nasty headache. Some routines that have callback functions (just as e04unc does) where a vector or matrix is passed to/from a subroutine require usage of Windows API subroutine RtlMoveMemory.

Since the underlying NAG Library is a C library, array arguments are simply declared as the appropriate type. VB6 passes arguments, by default, by reference (ByRef). Hence we have access to a pointer to the array. In the case of input array arguments, the appropriate amount of storage has to be copied to a VB array before it can be used. At the end of the function, output arrays must be copied back to the pointer. For more information please have a look at our Introduction to using NAG C Library in VBA.

Well, I guess it all sounds easy. Nevertheless I had some problems with RtlMoveMemory- I couldn't pass the data to and from a callback without some sort of memory violation error or if it didn't crash I kept on getting incorrect results.

The trick is not to use the default declaration of RtlMoveMemory, but actually have two versions of it: one for passing memory to a callback function via a pointer, and the second for copying memory from an array in the callback function back to a pointer. They differ slightly in declarations:

Declare Sub CopyMemFromPtr Lib "kernel32" Alias "RtlMoveMemory"( _

ByRef hpvDest As Any, ByVal hpvSource As Long, _

ByVal cbCopy As Long)

Declare Sub CopyMemToPtr Lib "kernel32" Alias "RtlMoveMemory"( _

ByVal hpvDest As Long, ByRef hpvSource As Any, _

ByVal cbCopy As Long)

OK, but why is this so important in this example? e04unc has 2 callback functions.

• Objfun, which returns the value of the objective function and its Jacobian.
• Confun, which returns the values of constraint functions and their respective Jacobians.

Both of them take a vector of variables x(n) on input. In practice it means that the actual input is a pointer x_rptr to x. The user then uses CopyMemFromPtr to fill the vector x(n) with values that x_rptr points to. We start copying memory to the first element of x, from pointer x_rptr, and the full amount of memory copied is the length of the vector times the number of bytes required to store a single variable. Here's how it looks in the code:

Call CopyMemFromPtr(x(1), x_rptr, n * Len(x(1)))

So at this point we have vector x with the input values, so we can calculate the value of the objective function (and optionally its Jacobian). Once we have done it we have a vector f(m) that contains the function values. In order to get these values back to the main function we need to use CopyMemToPtr.

Call CopyMemToPtr(f_rptr, f(1), m * Len(f(1)))

This call means that we focus on the first element of vector f, take a specific amount of memory and make sure that pointer f_rptr will point at this particular vector. A similar approach applies to constraint function and the Jacobians.

This is essentially how RtlMoveMemory Windows API function is used with NAG C Library routines.

I decided not to put the whole VBA code here in order to encourage you, dear reader of this blog, to download the mentioned example and check the code on your own. In order to run it you need the 32-bit Windows implementation of NAG C Library. You can obtain a trial licence key for the Library from support@nag.co.uk.

Please let us know if you found the example useful and if you like us to create example programs for other NAG routines!

Girls, Geeks, Twitter and Me.

When I get to work one of the first things that I do each morning is check out what’s happening on my Twitter timeline. One Thursday, a couple of weeks ago, one particular tweet caught my eye. It lead me to a great blog 'Girls can love computing; someone just needs to show them how' about the Manchester Girl Geeks. They are a group who are trying to encourage more girls and women to be interested in maths, science and technology. Being a girl myself, (OK, a woman really), and working for a mathematical software company, the article sparked a real interest.

A Girl Geek Tea Party

When I was at school, maths wasn’t my best subject, well actually and I’m going to be completely honest with you, it was my worst subject. My fear of all things mathematical started after being made to stand in front of the class reciting times tables. So it’s somewhat ironic that I found myself working at a numerical software company a few years ago, albeit in the marketing department. Had the 'Girl Geeks' been around in the 80s and had visited my school it might have made maths and science a bit more alluring for me. It’s a sad truth that girls are still way in the minority in choosing technical and science options at GCSE, A’level and degree level*. 

NAG want to help in some way to reverse this trend. I think I can speak for NAG in saying that we want to see more women achieving prominence in our organisation and in scientific computing in general.

Anyway, back to my reason for blogging. After reading about the great work that Manchester Girl Geeks are involved in we decided to support them by way of sponsorship. I’m writing today to raise awareness of their mission as we feel it’s really worthwhile. We have some ideas for other ways in which NAG can assist their goals in the future.

What other ways can we as an organisation make a positive difference?


*around 16% in 2009 of students in undergraduate computer science degrees are female.

Cloud computing or HPC? Finding trends.

Enable innovation and efficiency in product design and manufacture by using more powerful simulations. Apply more complex models to better understand and predict the behaviour of the world around us. Process datasets faster and with more advance analyses to extract more reliable and previously hidden insights and opportunities.

All ambitions that will probably resonate with those seeking scientific advances, commercial innovation, industrial growth and more cost-effective research. Underpinning all of the above is the use of more powerful computing methods and technologies. Faster and more capable computers - but equally important - more advanced and better performing algorithms and software implementations.

It's a pretty convincing story for those who take the time to listen - whether business leaders, governments, or research funders. Even in these challenging economic times, it has led to investments from industry and governments for this reason - the potential return is well documented and significant. It is even enticing enough to interest the media and the public - especially when we use emotive descriptions like "world's fastest supercomputer", "international competitiveness in digital economy", "personal supercomputing", and so on.

And it is this last thought that cause me to diverge from the grand theme to explore names and attention. I will come back to the main theme later (a future blog), as it is both important and timely. But on to my side topic.

Self-Improvement

Winston Churchill once said "The pessimist sees difficulty in every opportunity. The optimist sees the opportunity in every difficulty." (for you confirmed pedants, it may have been L.P. Jacks)

My custom is to use the time away from work at the end of the year to think about what I want to do differently in the year ahead. Among the topics that came up was e-mail, the bane of my life and perhaps yours as well. I get hundreds every day (and that doesn't include the SPAM).

Being a lifelong optimist I've decided to make e-mail less of a pain in my life, both work and personal. Being an occasional realist, I recognize that I have a limited number of options and they must focus on what I can do.

So, here's my plan for 2012 (with acknowledgement to Scott Belsky and Stever Robbins who supplied some of the ideas and got me thinking). See Disrupt Your Inbox and What you should never say in an e-mail

• Experiment with three-sentence emails when I need an answer from someone. (improve the likelihood that someone will actually read and answer).
• Start e-mails with action I want, Don't leave the reader guessing until the end.
• Use subject lines that intrigue the reader and actually invite them to open and read the e-mail.
• Take disagreements offline. There are volumes that could be written on this.
• Don't "reply all" unless everyone needs to be involved. How do you feel when you are one of 92 people copied on an e-mail that doesn't interest or pertain to you? Resist the urge, it's probably illegal.
• If you need to make several points and expect a response, use numbers for reference to reduce length, opportunity for confusion.

What's your plan?

More on managing e-mail in a future installment.

Question one: Where's my phone?

The end of the year at NAG is always celebrated with a rather splendid Christmas lunch, which is generously paid for by the company as an acknowledgement of the hard work its employees have put in over the previous twelve months. Accordingly, it provides an occasion for some much-needed relaxation and refreshment before the Christmas break. It's also the time for the NAG Christmas Quiz which, whilst not necessarily contributing to the participants' relaxation, usually provides some entertainment or diversion for those who care about such arcane matters as the number of hearts an octopus has or Paul McCartney's middle name. One of those tortured souls is the present author, who cleverly realized some time ago that the only way to be sure of knowing all the answers to the questions was to set them.

Setting quiz questions in the connected age - particularly for a technical-savvy band such as the employees of NAG - can present a few challenges, however. For example, anyone with internet access (via, say, a smartphone's web browser) would be able to find the answers to the questions indicated above in a matter of moments. More direct questions, such as

• Who wrote "A Child's Christmas In Wales"?
• What kind of logs did Good King Wenceslas ask for?
• When was "Merry Xmas Everybody" number 1 in the UK?

are even easier to answer, though some skill and judgement may still be required in the selection of the correct response to an ambiguous or ill-posed question such as the first one on this list (does it refer to Dylan Thomas's prose piece or John Cale's song?). Whilst this is clearly a valid and imaginative use of technology, I wondered whether it would provide an unfair advantage over those participants who wouldn't be using their phones in this fashion (or texting more knowledgeable friends for answers) and started wondering about ways to obviate their effectiveness.

Coffee and Filters

Given the origins of NAG and our mission, it’s natural for us to take the “long view” in giving back to the communities in which we operate. In the US, we’ve just finished our second year as sponsor of the DemandTec Retail Challenge (DTRC) scholarship competition for high school students in the Chicago area. DemandTec is one of our earliest and strongest software company partners who incorporate NAG components into their software products, providing demand management software for major retailers around the world.

The DTRC puts 2-person teams of high school seniors in the role of category managers for a retail store in a a 2-week computer simulation where each day represents a week in the “real world”. The students are given many weeks of data showing the price, inventory, unit sales, promotions used and the profit earned on each product they are managing. In the contest, they are responsible for two brands of coffee, one brand of tea and coffee filters. During the competition, they must analyze prior results to set the new price of each item, decide whether to run promotions and decide how much inventory to purchase. The simulation creates an interaction both with consumers and with the other teams in the competition. Ultimately, the three teams with the highest profit at the end of two weeks advance to the regional finals here in Chicago.

At the Chicago Regional finals held November 10th, teams from Glenbard West High School and Wheaton Warrenville South High School gave presentations of their data analysis, strategy formulation and how they worked as a team as they adapted their strategy in the course of the contest. The judges for the contest were experienced professionals in retail analytics from DemandTec and other local companies. Our winners were team “Price Lords” consisting of Peter Ericksen and Greg Grabarek. Each of them received $2500 toward their colleges expenses next year from NAG and our co-sponsor software company Informatica. Our third sponsor was the Network of Executive Women (NEW), which supports the education and advancement of women in leadership roles.

This contest is a labor of love for those of us involved. We have cultivated relationships with teachers of mathematics and statistics at local schools and have made presentations on the contest to a number of high school classes. It not only engages us with the students and teachers but it also helps us educate the next generation of applied scientists who might one day be NAG users. We wish Peter and Greg good luck as they compete in the national semi-finals in early December and are already making plans or next year’s contest.

Calling NAG routines from R

R is a widely-used environment for statistical computing and data analysis. It is one of the modern implementations of the S programming language (i.e. much code written in S will run unaltered in R) although its underlying semantics are derived from Scheme. R is Free Software.

The capabilities of R can be extended through the use of add-on packages and a large number of these are available from the Comprehensive R Archive Network (CRAN). Some users have expressed an interest in calling NAG Library routines from within R; accordingly, we have recently created a package which provides access to some of NAG's numerical functionality. More specifically, the NAGFWrappers R package contains the local and global optimization chapters of the NAG Fortran Library, together with a few nearest correlation matrix solvers and some other simpler routines. The package incorporates documentation in the so-called Rdoc style that will automatically produce help pages within the R system (see figure below), and also in HTML and PDF - for example, here is the full list of NAG routines that are contained in the package.

Help for the NAG routine e04uc, as displayed in R

For completeness, and to help R users further, we have also published more general instructions about how to use the R extension mechanisms to access any NAG routine from within R.

The original version of NAGFWrappers has been available since mid-2011; we have just updated it to use Mark 23 of the Fortran Library, and are releasing R binary packages for Windows 32 bit and Windows 64 bit, along with the R source package which can be used on other platforms (for example, we have built and run it on 64 bit Linux).

It should perhaps be noted that this is a preview release of the package, which is aimed at obtaining user feedback. Although it has been built and run on the platforms mentioned above, it is not a NAG product. We are keen to receive user feedback, and will respond to technical queries and problem reports via support@nag.co.uk so that we can further refine this package and make it still more useful to the R community.

SC11 diary catch up

I posted here a week or two ago about my diary leading up to the year's biggest supercomputing event - SC11 in Seattle. I though it would be handy to give a quick summary of the diary entries so far for those who haven't been reading along.

If you recall, I said: "On my twitter stream (@hpcnotes), I described it as: "the world of #HPC in one week & one place but hard to find time for all of technical program + exhibition + customer/partner/supplier meetings + social meetings + sleep!" To follow the news about SC11 on twitter, follow @supercomputing and/or the #SC11 hashtag."

"Any hope of "live" blogging or actively tweeting during the week of SC11 itself is almost nil - the week is just too busy with the day job. Even simply consuming the relevant news, comment and gossip is a challenge."

"So instead I am going to try to write a diary of the lead up to SC11."

If you've missed them, here are the 8 SC11 blogs so far:

1. "The big HPC event of the year - lots of news, people & meetings. Busy week."
2. "schedule certainty, locations, spare time & hard work"
3. "SC11 news deluge, the missing HPC world"
4. "how do you do it?"
5. "navigation, rope and choosing wisely"
6. "fog and sports events"
7. "not everyone will be in Seattle"
8. "ppt, professionals, preparation & precipitation"

Along the way, I have briefly alluded to a few things NAG will be doing at SC11. One of my colleagues will be along shortly to post here about our activities at SC11, but in the meantime, plan to visit us on booth 2622, or get in touch to arrange a conversation.