The Tidyverse is a collection of related R packages, many of which I use on Open-Meta. I recently added the entire Tidyverse to my Windows-based development system, which meant I had to do the same to move that code to my Amazon Lightsail instance, which I did yesterday.
It didn’t go as smoothly as I thought it would. First, the Tidyverse wouldn’t load on a Lightsail instance with 1 gigabyte of memory. I had to upgrade to a 2G instance. Second, as has become typical with new LEMRS stack installs, there were some required Linux packages that were installed but that the R installer couldn’t find.
Fortunately, this time, someone else had already encountered this problem, had figured out the needed symbolic links, and had put them on StackOverflow.com.
Here’s the Lightsail install script I used this time. Most of it is the same as LEMRS v0.2.
# LEMRS launch script
# For Amazon Lightsail, Amazon Linux instance
# v0.3 May 16, 2018 - Amazon Linux 2017.09.01
# Changed R packages
# update yum
yum update -y
# folder for storing files needed to setup this instance
mkdir /home/ec2-user/instance-setup-files
chown ec2-user:ec2-user /home/ec2-user/instance-setup-files
chmod 755 /home/ec2-user/instance-setup-files
# install missing Linux packages we'll need
#yum install git.x86_64 -y
yum install libcurl-devel.x86_64 -y
yum install openssl-devel.x86_64 -y
yum install libxml2-devel.x86_64 -y
yum install libgfortran.x86_64 -y
yum install libquadmath.x86_64 -y
# install E-R-M part of the stack
yum install nginx -y
yum install R -y
yum install mysql57-server.x86_64 -y
yum install mysql57-devel.x86_64 -y
# start nginx and MySQL at reboot
chkconfig nginx on
chkconfig mysqld on
# symbolic links needed to install the rJava package required by mailR
ln -s /usr/lib/gcc/x86_64-amazon-linux/6.4.1/libgomp.spec /usr/lib64/libgomp.spec
ln -s /usr/lib/gcc/x86_64-amazon-linux/6.4.1/libgomp.a /usr/lib64/libgomp.a
ln -s /usr/lib64/libgomp.so.1.0.0 /usr/lib64/libgomp.so
# symbolic link needed to install the RMariaDB package
ln -s /usr/lib64/mysql57/libmysqlclient.so.1020 /usr/lib64/libmysqlclient.so.1020
# symbolic links needed to install Tidyverse package
ln -s /usr/lib/gcc/x86_64-amazon-linux/6.4.1/libgfortran.so /usr/lib64/
ln -s /usr/lib/gcc/x86_64-amazon-linux/6.4.1/libquadmath.so /usr/lib64/
# add packages to R
# Rcpp
# devtools
# shiny
# tidyverse
# pool
# RMariaDB
# bcrypt
# DT
# mailR
# RefManageR
# v0.2: 'shiny','htmltools','dplyr','stringr','lubridate','DT','DBI','pool','RMariaDB','bcrypt', 'mailR'
# first create a script and save it
printf "install.packages('Rcpp', repos='http://lib.stat.cmu.edu/R/CRAN/'); install.packages('devtools', repos='http://cran.rstudio.com/'); install.packages(c('shiny','tidyverse','pool','RMariaDB','bcrypt','DT','mailR','RefManageR'), repos='http://lib.stat.cmu.edu/R/CRAN/')\n" > /home/ec2-user/instance-setup-files/install.R.packages
# next run the script
Rscript /home/ec2-user/instance-setup-files/install.R.packages
# install Shiny Server
wget -O /home/ec2-user/instance-setup-files/shiny-server.rpm https://download3.rstudio.org/centos5.9/x86_64/shiny-server-1.5.6.875-rh5-x86_64.rpm
yum install --nogpgcheck /home/ec2-user/instance-setup-files/shiny-server.rpm -y
# install turns it on; turn it back off
stop shiny-server
# fix some folder ownerships for shiny
chown shiny:shiny /var/lib/shiny-server
chown shiny:shiny /opt/shiny-server
# Users!
# Add ec2-user to the shiny and nginx groups and vice-versa
usermod -a -G shiny ec2-user
usermod -a -G nginx ec2-user
usermod -a -G ec2-user shiny
usermod -a -G ec2-user nginx
# Web server root directories!
# permissions should be 775 = rwx rwx r-x so we can ftp in files
# fix /srv owner and permissions
chown ec2-user:ec2-user /srv
chmod 775 /srv
# this is the root for standard shiny apps
mkdir /srv/shiny-site-dir
chown shiny:shiny /srv/shiny-site-dir
chmod 775 /srv/shiny-site-dir
# move what shiny-server auto installs
mv /srv/shiny-server/sample-apps /srv/shiny-site-dir
mv /srv/shiny-server/index.html /srv/shiny-site-dir
rm -r /srv/shiny-server # -r because it's a directory
# this is the root for the shiny app that routes requests
mkdir /srv/shiny-app-dir
chown shiny:shiny /srv/shiny-app-dir
chmod 775 /srv/shiny-app-dir
# this is the nginx root
mkdir /srv/nginx/
chown nginx:ec2-user /srv/nginx
chmod 775 /srv/nginx
# copy default nginx files from std nginx root folder to new root at /srv/nginx
cp /usr/share/nginx/html/404.html /srv/nginx/
cp /usr/share/nginx/html/50x.html /srv/nginx/
cp /usr/share/nginx/html/index.html /srv/nginx/
cp /usr/share/nginx/html/nginx-logo.png /srv/nginx/
cp /usr/share/nginx/html/poweredby.png /srv/nginx/
rm -rf /usr/share/nginx/html
# put a symbolic link in the old location
ln -s /srv/nginx/ /usr/share/nginx/html
# configs!
# add folder in /srv for configs
mkdir /srv/configs
chown ec2-user:ec2-user /srv/configs
chmod 755 /srv/configs
# "Move" configuration files to /srv/configs
chown nginx:nginx /etc/nginx/ # update nginx folder owner
chmod 775 /etc/nginx/ # and permissions
cp /etc/nginx/nginx.conf /srv/configs/original-nginx-conf # save old configuation file
mv /etc/nginx/nginx.conf /srv/configs/nginx.conf # put nginx configuration file here
chown ec2-user:ec2-user /srv/configs/nginx.conf # take ownership
chmod 644 /srv/configs/nginx.conf # make writable
ln -s /srv/configs/nginx.conf /etc/nginx/nginx.conf # replace with a link to the new location
chown ec2-user:ec2user /etc/nginx/nginx.conf # clarify how this link happened
# repeat for shiny configuration folder
chown shiny:shiny /etc/shiny-server
chmod 775 /etc/shiny-server
cp /etc/shiny-server/shiny-server.conf /srv/configs/original-shiny-conf
mv /etc/shiny-server/shiny-server.conf /srv/configs/shiny-server.conf
chown ec2-user:ec2-user /srv/configs/shiny-server.conf
chmomd 644 /srv/configs/shiny-server.conf
ln -s /srv/configs/shiny-server.conf /etc/shiny-server/shiny-server.conf
chown ec2-user:ec2user /etc/shiny-server/shiny-server.conf
# shiny init configuration is a little different
chmod 775 /etc/init
cp /etc/init/shiny-server.conf /srv/configs/original-shiny-init ### shiny won't start if you delete this
chmod 775 /etc/init/shiny-server.conf ### even with a symbolic link replacement
# logs!
# add folder in /srv for logs
mkdir /srv/logs
chown ec2-user:ec2-user /srv/logs
chmod 777 /srv/logs
# make the REAL nginx log folders readable (shiny's are ok)
# 755 because programs need to write to log folders and I don't care who reads them
chown ec2-user:ec2-user /var/log
chmod 755 /var/log
chown nginx:nginx /var/log/nginx
chmod 755 /var/log/nginx
# symbolic links to actual log folders from /srv/logs
ln -s /var/log/nginx /srv/logs/var-log-nginx
ln -s /var/log/shiny-server /srv/logs/var-log-shiny
ln -s /var/log/ /srv/logs/var-log
# and vice-versa
ln -s /srv/logs/ /var/log/srv-logs
ln -s /srv/logs/ /var/log/nginx/srv-logs
ln -s /srv/logs/ /var/log/shiny-server/srv-logs
# Commands!
# set up ~/bin folder
mkdir /home/ec2-user/bin
chown ec2-user:ec2-user /home/ec2-user/bin
chmod 755 /home/ec2-user/bin
# this adds the file "add-cmds" to the ~/instance-setup-files folder
printf "printf 'top' > /home/ec2-user/bin/see.mem.space\nprintf 'df' > /home/ec2-user/bin/see.disk.space\nprintf 'sudo service nginx restart' > /home/ec2-user/bin/restart.nginx\nprintf 'sudo service nginx start' > /home/ec2-user/bin/start.nginx\nprintf 'sudo service nginx stop' > /home/ec2-user/bin/stop.nginx\nprintf 'sudo stop shiny-server\nsudo start shiny-server' > /home/ec2-user/bin/restart.shiny\nprintf 'sudo start shiny-server' > /home/ec2-user/bin/start.shiny\nprintf 'sudo stop shiny-server' > /home/ec2-user/bin/stop.shiny\nprintf 'sudo ps -ely | more' > /home/ec2-user/bin/whats.running\nprintf 'sudo chkconfig --list | more' > /home/ec2-user/bin/whats.autostart\nprintf 'sudo yum history' > /home/ec2-user/bin/yum.history\nprintf 'sudo yum list | more' > /home/ec2-user/bin/yum.installed\nprintf 'sudo yum search \0441' > /home/ec2-user/bin/yum.search\nprintf 'sudo stop shiny-server\nsudo cp /srv/configs/shiny-logs-debug.init /etc/init/shiny-server.conf\nsudo start shiny-server' > /home/ec2-user/bin/switch-shiny-logs-to-debug\nprintf 'sudo stop shiny-server\nsudo cp /srv/configs/shiny-logs-info.init /etc/init/shiny-server.conf\nsudo start shiny-server' > /home/ec2-user/bin/switch-shiny-logs-to-info\nprintf 'sudo stop shiny-server\nsudo cp /srv/configs/shiny-logs-warn.init /etc/init/shiny-server.conf\nsudo start shiny-server' > /home/ec2-user/bin/switch-shiny-logs-to-warn\nchown -R ec2-user:ec2-user /home/ec2-user/bin\nchmod -R 755 /home/ec2-user/bin # change on all files in the directory\n" > /home/ec2-user/instance-setup-files/add-cmds
# This runs the "add-cmds" file
chown ec2-user:ec2-user /home/ec2-user/instance-setup-files/add-cmds
chmod 755 /home/ec2-user/instance-setup-files/add-cmds
cd /home/ec2-user/instance-setup-files
./add-cmds
# Additional commands for MySQL
printf 'sudo service mysqld restart' > /home/ec2-user/bin/restart.mysql
printf 'sudo service mysqld start' > /home/ec2-user/bin/start.mysql
printf 'sudo service mysqld stop' > /home/ec2-user/bin/stop.mysql
chown -R ec2-user:ec2-user /home/ec2-user/bin
chmod -R 755 /home/ec2-user/bin # change on all files in the directory
# create links to these commands from /srv/configs
ln -s /home/ec2-user/bin/see.disk.space /srv/configs/see.disk.space
ln -s /home/ec2-user/bin/see.mem.space /srv/configs/see.mem.space
ln -s /home/ec2-user/bin/restart.nginx /srv/configs/restart.nginx
ln -s /home/ec2-user/bin/restart.shiny /srv/configs/restart.shiny
ln -s /home/ec2-user/bin/start.nginx /srv/configs/start.nginx
ln -s /home/ec2-user/bin/start.shiny /srv/configs/start.shiny
ln -s /home/ec2-user/bin/stop.nginx /srv/configs/stop.nginx
ln -s /home/ec2-user/bin/stop.shiny /srv/configs/stop.shiny
ln -s /home/ec2-user/bin/switch-shiny-logs-to-debug /srv/configs/switch-shiny-logs-to-debug
ln -s /home/ec2-user/bin/switch-shiny-logs-to-info /srv/configs/switch-shiny-logs-to-info
ln -s /home/ec2-user/bin/switch-shiny-logs-to-warn /srv/configs/switch-shiny-logs-to-warn
ln -s /home/ec2-user/bin/whats.autostart /srv/configs/whats.autostart
ln -s /home/ec2-user/bin/whats.running /srv/configs/whats.running
ln -s /home/ec2-user/bin/ /srv/configs/drop-d-here-and-chmod-755-it
ln -s /home/ec2-user/bin/restart.mysql /srv/configs/restart.mysql
ln -s /home/ec2-user/bin/start.mysql /srv/configs/start.mysql
ln -s /home/ec2-user/bin/stop.mysql /srv/configs/stop.mysql
After creating the new Lightsail instance, you need to:
-
- copy over non-redundant files in /srv/configs, /srv/nginx/, /srv/shiny-app-dir, and /srv/shiny-site-dir to the same folders on the new instance and run the following commands:
start.mysqlmysql_secure_installationrestart.nginxrestart.mysqlrestart.shiny
- copy over non-redundant files in /srv/configs, /srv/nginx/, /srv/shiny-app-dir, and /srv/shiny-site-dir to the same folders on the new instance and run the following commands: