I’ve been having problems with the web version of the Open-Meta app crashing with a C stack usage
error. I entered an issue on the Shiny Server Github page and with the help of others who have had this problem, now suspect that the error is caused by the rJava package, which is required by mailR, the package I’ve been using to send email.
I’ve been wanting to switch to directly using the Amazon Web Services api for its Simple Email Service rather than using mailR and its SMTP connection to Amazon’s email service anyhow, because mailR takes about 10 seconds to send an email. It turns out this delay is mostly related to the time it takes to load and initialize the rJava package! My difficulty has been figuring out how to build the encrypted signature Amazon requires to use its api interface.
It turns out there’s a package for that, aws.signature! In order to switch over to the AWS SES api and away from mailR and rJava, I’ve created a new instance, v0.5. Here’s the script:
# LEMRS launch script # For Amazon Lightsail, Amazon Linux instance # v0.3 May 16, 2018 - Amazon Linux 2017.09.01 # v0.4 Jul 11, 2018 - Amazon Linux 2018.03.0 # v0.5 Nov 07, 2018 - Amazon Linux 2018.03.0 # update yum yum update -y yum install epel-release # Gets more recent version of R # folder for storing files needed to setup this instance mkdir /home/ec2-user/instance-setup-files chown ec2-user:ec2-user /home/ec2-user/instance-setup-files chmod 755 /home/ec2-user/instance-setup-files # install missing Linux packages we'll need #yum install git.x86_64 -y yum install libcurl-devel.x86_64 -y yum install openssl-devel.x86_64 -y yum install libxml2-devel.x86_64 -y yum install libgfortran.x86_64 -y yum install libquadmath.x86_64 -y # install E-R-M part of the stack yum install nginx -y yum install R -y yum install mysql57-server.x86_64 -y yum install mysql57-devel.x86_64 -y # start nginx and MySQL at reboot chkconfig nginx on chkconfig mysqld on # symbolic link needed to install the RMariaDB package ln -s /usr/lib64/mysql57/libmysqlclient.so.1020 /usr/lib64/libmysqlclient.so.1020 # symbolic links needed to install Tidyverse package ln -s /usr/lib/gcc/x86_64-amazon-linux/6.4.1/libgfortran.so /usr/lib64/ ln -s /usr/lib/gcc/x86_64-amazon-linux/6.4.1/libquadmath.so /usr/lib64/ # add packages to R # v0.2: Rcpp, devtools, 'shiny', 'htmltools', 'dplyr', 'stringr', 'lubridate', 'DT', 'DBI', 'pool', 'RMariaDB', 'bcrypt', 'mailR' # v0.4: Rcpp, devtools, 'shiny', 'tidyverse', 'pool', 'RMariaDB', 'bcrypt', 'DT', 'mailR', 'RefManageR', 'RCurl' # v0.5: Rcpp, 'shiny', 'tidyverse', 'pool', 'RMariaDB', 'bcrypt', 'DT', 'RefManageR', 'RCurl', 'rvest', 'aws.signature' # first create a script and save it printf "install.packages('Rcpp', repos='http://lib.stat.cmu.edu/R/CRAN/'); install.packages(c('shiny','tidyverse','pool','RMariaDB','bcrypt','DT','RefManageR','RCurl','rvest','aws.signature'), repos='http://lib.stat.cmu.edu/R/CRAN/')\n" > /home/ec2-user/instance-setup-files/install.R.packages # next run the script Rscript /home/ec2-user/instance-setup-files/install.R.packages # install Shiny Server wget -O /home/ec2-user/instance-setup-files/shiny-server.rpm https://download3.rstudio.org/centos6.3/x86_64/shiny-server-1.5.9.923-x86_64.rpm yum install --nogpgcheck /home/ec2-user/instance-setup-files/shiny-server.rpm -y # install turns it on; turn it back off stop shiny-server # fix some folder ownerships for shiny chown shiny:shiny /var/lib/shiny-server chown shiny:shiny /opt/shiny-server # Users! # Add ec2-user to the shiny and nginx groups and vice-versa usermod -a -G shiny ec2-user usermod -a -G nginx ec2-user usermod -a -G ec2-user shiny usermod -a -G ec2-user nginx # Web server root directories! # permissions should be 775 = rwx rwx r-x so we can ftp in files # fix /srv owner and permissions chown ec2-user:ec2-user /srv chmod 775 /srv # this is the root for standard shiny apps mkdir /srv/shiny-site-dir chown shiny:shiny /srv/shiny-site-dir chmod 775 /srv/shiny-site-dir # move what shiny-server auto installs mv /srv/shiny-server/sample-apps /srv/shiny-site-dir mv /srv/shiny-server/index.html /srv/shiny-site-dir rm -r /srv/shiny-server # -r because it's a directory # this is the root for the shiny app that routes requests mkdir /srv/shiny-app-dir chown shiny:shiny /srv/shiny-app-dir chmod 775 /srv/shiny-app-dir # this is the nginx root mkdir /srv/nginx/ chown nginx:ec2-user /srv/nginx chmod 775 /srv/nginx # copy default nginx files from std nginx root folder to new root at /srv/nginx cp /usr/share/nginx/html/404.html /srv/nginx/ cp /usr/share/nginx/html/50x.html /srv/nginx/ cp /usr/share/nginx/html/index.html /srv/nginx/ cp /usr/share/nginx/html/nginx-logo.png /srv/nginx/ cp /usr/share/nginx/html/poweredby.png /srv/nginx/ rm -rf /usr/share/nginx/html # put a symbolic link in the old location ln -s /srv/nginx/ /usr/share/nginx/html # configs! # add folder in /srv for configs mkdir /srv/configs chown ec2-user:ec2-user /srv/configs chmod 755 /srv/configs # "Move" configuration files to /srv/configs chown nginx:nginx /etc/nginx/ # update nginx folder owner chmod 775 /etc/nginx/ # and permissions cp /etc/nginx/nginx.conf /srv/configs/original-nginx-conf # save old configuation file mv /etc/nginx/nginx.conf /srv/configs/nginx.conf # put nginx configuration file here chown ec2-user:ec2-user /srv/configs/nginx.conf # take ownership chmod 644 /srv/configs/nginx.conf # make writable ln -s /srv/configs/nginx.conf /etc/nginx/nginx.conf # replace with a link to the new location chown ec2-user:ec2user /etc/nginx/nginx.conf # clarify how this link happened # repeat for shiny configuration folder chown shiny:shiny /etc/shiny-server chmod 775 /etc/shiny-server cp /etc/shiny-server/shiny-server.conf /srv/configs/original-shiny-conf mv /etc/shiny-server/shiny-server.conf /srv/configs/shiny-server.conf chown ec2-user:ec2-user /srv/configs/shiny-server.conf chmomd 644 /srv/configs/shiny-server.conf ln -s /srv/configs/shiny-server.conf /etc/shiny-server/shiny-server.conf chown ec2-user:ec2user /etc/shiny-server/shiny-server.conf # shiny init configuration is a little different chmod 775 /etc/init cp /etc/init/shiny-server.conf /srv/configs/original-shiny-init ### shiny won't start if you delete this chmod 775 /etc/init/shiny-server.conf ### even with a symbolic link replacement # logs! # add folder in /srv for logs mkdir /srv/logs chown ec2-user:ec2-user /srv/logs chmod 777 /srv/logs # make the REAL nginx log folders readable (shiny's are ok) # 755 because programs need to write to log folders and I don't care who reads them chown ec2-user:ec2-user /var/log chmod 755 /var/log chown nginx:nginx /var/log/nginx chmod 755 /var/log/nginx # symbolic links to actual log folders from /srv/logs ln -s /var/log/nginx /srv/logs/var-log-nginx ln -s /var/log/shiny-server /srv/logs/var-log-shiny ln -s /var/log/ /srv/logs/var-log # and vice-versa ln -s /srv/logs/ /var/log/srv-logs ln -s /srv/logs/ /var/log/nginx/srv-logs ln -s /srv/logs/ /var/log/shiny-server/srv-logs # Commands! # set up ~/bin folder mkdir /home/ec2-user/bin chown ec2-user:ec2-user /home/ec2-user/bin chmod 755 /home/ec2-user/bin # this adds the file "add-cmds" to the ~/instance-setup-files folder printf "printf 'top' > /home/ec2-user/bin/see.mem.space\nprintf 'df' > /home/ec2-user/bin/see.disk.space\nprintf 'sudo service nginx restart' > /home/ec2-user/bin/restart.nginx\nprintf 'sudo service nginx start' > /home/ec2-user/bin/start.nginx\nprintf 'sudo service nginx stop' > /home/ec2-user/bin/stop.nginx\nprintf 'sudo stop shiny-server\nsudo start shiny-server' > /home/ec2-user/bin/restart.shiny\nprintf 'sudo start shiny-server' > /home/ec2-user/bin/start.shiny\nprintf 'sudo stop shiny-server' > /home/ec2-user/bin/stop.shiny\nprintf 'sudo ps -ely | more' > /home/ec2-user/bin/whats.running\nprintf 'sudo chkconfig --list | more' > /home/ec2-user/bin/whats.autostart\nprintf 'sudo yum history' > /home/ec2-user/bin/yum.history\nprintf 'sudo yum list | more' > /home/ec2-user/bin/yum.installed\nprintf 'sudo yum search \0441' > /home/ec2-user/bin/yum.search\nprintf 'sudo stop shiny-server\nsudo cp /srv/configs/shiny-logs-debug.init /etc/init/shiny-server.conf\nsudo start shiny-server' > /home/ec2-user/bin/switch-shiny-logs-to-debug\nprintf 'sudo stop shiny-server\nsudo cp /srv/configs/shiny-logs-info.init /etc/init/shiny-server.conf\nsudo start shiny-server' > /home/ec2-user/bin/switch-shiny-logs-to-info\nprintf 'sudo stop shiny-server\nsudo cp /srv/configs/shiny-logs-warn.init /etc/init/shiny-server.conf\nsudo start shiny-server' > /home/ec2-user/bin/switch-shiny-logs-to-warn\nchown -R ec2-user:ec2-user /home/ec2-user/bin\nchmod -R 755 /home/ec2-user/bin # change on all files in the directory\n" > /home/ec2-user/instance-setup-files/add-cmds # This runs the "add-cmds" file chown ec2-user:ec2-user /home/ec2-user/instance-setup-files/add-cmds chmod 755 /home/ec2-user/instance-setup-files/add-cmds cd /home/ec2-user/instance-setup-files ./add-cmds # Additional commands for MySQL printf 'sudo service mysqld restart' > /home/ec2-user/bin/restart.mysql printf 'sudo service mysqld start' > /home/ec2-user/bin/start.mysql printf 'sudo service mysqld stop' > /home/ec2-user/bin/stop.mysql chown -R ec2-user:ec2-user /home/ec2-user/bin chmod -R 755 /home/ec2-user/bin # change on all files in the directory # create links to these commands from /srv/configs ln -s /home/ec2-user/bin/see.disk.space /srv/configs/see.disk.space ln -s /home/ec2-user/bin/see.mem.space /srv/configs/see.mem.space ln -s /home/ec2-user/bin/restart.nginx /srv/configs/restart.nginx ln -s /home/ec2-user/bin/restart.shiny /srv/configs/restart.shiny ln -s /home/ec2-user/bin/start.nginx /srv/configs/start.nginx ln -s /home/ec2-user/bin/start.shiny /srv/configs/start.shiny ln -s /home/ec2-user/bin/stop.nginx /srv/configs/stop.nginx ln -s /home/ec2-user/bin/stop.shiny /srv/configs/stop.shiny ln -s /home/ec2-user/bin/switch-shiny-logs-to-debug /srv/configs/switch-shiny-logs-to-debug ln -s /home/ec2-user/bin/switch-shiny-logs-to-info /srv/configs/switch-shiny-logs-to-info ln -s /home/ec2-user/bin/switch-shiny-logs-to-warn /srv/configs/switch-shiny-logs-to-warn ln -s /home/ec2-user/bin/whats.autostart /srv/configs/whats.autostart ln -s /home/ec2-user/bin/whats.running /srv/configs/whats.running ln -s /home/ec2-user/bin/ /srv/configs/drop-d-here-and-chmod-755-it ln -s /home/ec2-user/bin/restart.mysql /srv/configs/restart.mysql ln -s /home/ec2-user/bin/start.mysql /srv/configs/start.mysql ln -s /home/ec2-user/bin/stop.mysql /srv/configs/stop.mysql
As before, I couldn’t get the script to run on a 1 gigabyte instance, so I used the next size up, 2GB. After the script runs, you need to follow this process:
- Use PuTTY to log in to the new instance
- run
yum.history
to make sure all the programs installed correctly - run
R
, note version number, then enterinstalled.packages()[,1:3]
and make sure all the R packages installed correctly.quit()
R.
- run
- Use an FTP program to copy over non-redundant files in /srv/configs, /srv/nginx/, /srv/shiny-app-dir, and /srv/shiny-site-dir to the same folders on the new instance.
- Run the following commands:
start.mysql
mysql_secure_installation
restart.nginx
restart.mysql
restart.shiny