Saturday, December 04, 2010

Installing Rmpi under Linux on Ranger at TACC

Rmpi is a package for the R language to allow it to use MPI to run in parallel. Installation on Ranger is complicated by Ranger's selection of compilers, MPI libraries, and default installed packages. I got it to work, so here are hints.

R's default compiler options for installed packages are the same as those used to compile R. The Ranger copy of R was compiled with gcc. MPI libraries on Ranger are compiled with either PGI or Intel compilers, and there can be incompatibilities loading shared libraries from PGI and Intel into gcc code (especially the version of PGI on Ranger). An easy way to get started healthily is to recompile R with the Intel compilers, so "module swap pgi intel" and configure a home directory installation of R using mpicc, mpicxx, and mpif90 with the Known Good Intel options for Ranger, "-O2 -xW -fPIC".

The Rmpi library depends on the R wrapper for the SPRNG library which, of course, depends on SPRNG itself. It also needs gmp. Ranger's gmp library is just fine, so "module add gmp". If you want to know includes and libs for any Ranger library, use "module help ". The SPRNG library has a problem, though, because it wasn't compiled with -fPIC, which is necessary to build shared libraries on this platform.

Rmpi needs SPRNG 2.0, no other version. The Ranger sprng2.0 library is compiled without -fPIC, so we make our own. Download it with "curl|tar zxf -". Build the MPI version with the Intel compilers, again, using the Known Good Intel Options, "-O2 -xW -fPIC" and mpicc, mpicxx, mpif90, but add another option Fortran flags to help Fortran compatibility, "-assume 2underscores". This comes from a little bug in the SPRNG autoconf, but it only affects Fortran builds. You have to set this information in sprng2.0/make.CHOICES and sprng2.0/src/make.INTEL.

SPRNG doesn't make a shared executable. No worries. We can rebuild it as a shared library. Go to the directory of libsprng.a, and repack it with a reference to the gmp library.

ar -x libsprng.a
icc -shared -fPIC -Wl,-rpath,$TACC_GMP_LIB -Wl,-zmuldefs *.o -L${TACC_GMP_LIB} -lgmp -o

Were you to forget to add the reference to -lgmp, you would later see that cannot be loaded by R because it cannot find a function __gmp_cmp.

To build rsprng, first download the tar file. Then invoke R's installer with hints about the location of sprng. Mine looked as follows.

~/R/bin/R CMD INSTALL --configure-vars='CFLAGS="-I/opt/apps/gmp/4.2.4/include" LDFLAGS="-L/opt/apps/gmp/4.2.4/lib"' --configure-args='--with-sprng=/share/home/00933/tg459569/sprng' rsprng_1.0.tar.gz

If you get the paths correct, that should build. Now the Rmpi library wants the same attention as Rsprng. First download it. Then let R build it.

~/R/bin/R CMD INSTALL --configure-vars='CFLAGS="-O2 -xW -fPIC" LDFLAGS="-O2 -xW -fPIC"' --configure-args='--with-Rmpi-type=MPICH' Rmpi_0.5-9.tar.gz

When R installs, it will fail its first test because miprun won't work on a Ranger login node. Just go to and try the tutorial in a batch job.


Friday, September 03, 2010

A Standard Way to Use Python's Logging Module

The documentation for Python's logging module shows you its great features, but it doesn't mention standard usage if you are writing a little application or a library.

Make a separate logger for each file, and name the loggers to match the hierarchy of modules. For the file acert/hfesr/, it starts with:

import logging
logger = logging.getLogger('acert.hfesr.Runner')
class Runner(object):
def __init__(self):
logger.debug('Initializing Runner')

That's quite simple. Then you can enable and disable logging by file or by module.

The second tip is that, if you have written a library, don't use logging.basicConfig() in that library because it makes logging handlers that are difficult for subsequent client applications to quiet.


/dev/null for C++ ostream

I often make C++ classes that write to some stream given to them:

Bear::Save(std::ostream& out) {
out << "Fur is " << color << std::endl;
out << "Age is " << age << std::endl;

Editing somebody's code today, I needed an ostream equivalent of /dev/null, some stream into which a class could write without printing anything. This can be done by creating a stream buffer that never prints.

class nullbuf : public std::basic_streambuf
virtual int overflow(int c) { return c; }

class nullstream : public std::basic_ostream > {
nullbuf _streambuf;
nullstream() :
std::basic_ostream &gt(&_streambuf)
{ clear(); }


Using it looks like any other stream:

nullstream dev_null;
dev_null << "Nobody will ever see this.";
ostream* output = new nullstream();
(*output) << "This will be invisible.";

As usual, hope this helps.