Installing the data.table package on Amazon Linux – LEMRS v0.6

This week I’ve upgraded the server to LEMRS v0.6, which adds three R packages for meta-analysis, esc, metafor, and robumeta.

The esc package requires another, much larger R package called data.table. That package, however, wouldn’t install. Like many R packages, data.table required the installation to do an automatic compilation of code written in C, but the C compiler couldn’t find some of the files it needed. This is because Amazon Linux uses non-standard folder locations for many of those files.

I did a search to find the symbolic links to the files the compiler needed and quickly found others with the same problem on this thread on the Amazon Web Services discussion forums. However, the messages didn’t indicate how to solve the problem.

Back in my post on Installing and configuring a database engine like MySQL or MariaDB on Amazon Lightsail, I hacked out the procedure, but I didn’t document it very well. So today I hacked out the required symbolic links for installing the data.table package and added them to the AWS thread.

As I explain there, the general solution is to carefully read the installation output in R to figure out exactly what file the C compiler can’t find. Then quit() R and find that file in the Linux directory structure. The Linux find command can be helpful in this pursuit (look it up to see how it works). I find find‘s output pretty weird; in general, it will instantly scroll the thousands of filenames it’s looking through up your screen, then end with a message that says it couldn’t find your file. However, if you look at the line immediately above the error message, often, there’s your file. The scrolling stopped when it was found. Other times, however, this doesn’t work until you’ve guessed a near parent folder of the folder the file is in.

To construct a working symbolic link to the file, you not only need to know where the file is, but you also need to know where the compiler is looking for it. This Linux command is helpful for that (replace filename with the actual name of the file):

sudo ld -filename --verbose

This will give you the list of folder locations the compiler is looking in, but not finding, your file. Now create the symbolic links you need like this:

sudo ln -s /folder/where/the/file/actually/is/filename /folder/where/the/compiler/is/looking/for/it/filename

Now you just have to repeat this entire procedure over and over until you’ve created links to all of the files the compiler can’t find. For the data.table package on Amazon Linux 2018.03.0, the needed links are:

sudo ln -s /usr/lib/gcc/x86_64-amazon-linux/6.4.1/include/omp.h /usr/local/include/
sudo ln -s /usr/lib/gcc/x86_64-amazon-linux/6.4.1/libgomp.spec /usr/lib64/libgomp.spec
sudo ln -s /usr/lib64/ /usr/lib64/

The script for LEMRS v0.6 is almost exactly the same as LEMRS v0.5, with the addition of those three links in the symbolic links section and the addition of esc, metafor, and robumeta to the list of R packages to be installed.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.