Skip to content

Installing RNAdetector on the cloud

Jump to bottom
apulvirenti edited this page Apr 23, 2021 · 4 revisions
RNAdetector

Introduction

RNAdetector uses a client-server architecture for flexibility and easy deployment. The server can be easily installed on a local or remote cluster, and the GUI will provide all its functions seamlessly. At present, RNAdetector can be installed on a Kubernetes cluster (on local infrastructure or through a cloud-based service such as Google Cloud, Amazon AWS, or Microsoft Azure) or a local server through docker-compose. In the following sections, we will detail the procedures needed to deploy RNAdetector on such architectures and connect the GUI to a remote cluster.

Docker-compose deployment

First, you need to have a docker environment already installed on your server and a user account allowed to deploy docker containers. To verify if docker is available for your user account, you can run the following command:

$ docker run --rm hello-world

The command downloads a test image and runs it in a container. When the container runs, it prints an information message and exits:

docker run --rm hello-world

If Docker is not available in your system, please install it by following the procedure for your Linux distribution.

You will also need docker-compose for easy deployment of RNAdetector. You can test if it is already installed in your system by running the docker-compose --version command:

$ docker-compose --version
docker-compose version 1.29.1, build 1110ad01

If docker-compose is not available, please follow the installation procedure at the Docker Docs website.

Now, to deploy RNAdetector, you can perform the following procedure:

  1. Download and extract the content of the RNAdetector repository from the following URL: https://github.com/alessandrolaferlita/RNAdetector/archive/refs/heads/master.zip 
$ wget https://github.com/alessandrolaferlita/RNAdetector/archive/refs/heads/master.zip && unzip master.zip
  1. Enter the RNAdetector-master/cloud/docker-compose/ 
$ cd RNAdetector-master/cloud/docker-compose/
  1. Edit the deploy.conf file with your preferred text editor. This file is fundamental for the deployment and the security of your installation. You will need to set the APP_PORT variable to a free network port on your machine. The GUI uses the port to connect to your server. The APP_URL variable is used to set a user-friendly URL for your machine. If you don't know, leave the default value. The APP_KEY variable is fundamental for the session encryption. You can generate a random value at the following link https://rnadetector.atlas.dmi.unict.it/key-generate.php. Finally, the DB_PASSWORD variable is the password used for the internal database. A strong and complicated password is the best choice for security and privacy.

docker run --rm hello-world

  1. Now you can perform the deployment using the deploy.bash script. First run the ./deploy.bash mysql command:

docker run --rm hello-world

  1. To check whether the database server is running you can run the ./deploy.bash logs command. When the database is started you will get the following output:
$ ./deploy.bash logs
....
....
mysql_1        | 2021-04-18T18:20:57.524724Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.23'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.
  1. Now, you can run the ./deploy.bash app command to start the RNAdetector server.

docker run --rm hello-world

  1. After a few minutes, the RNAdetector server will be available at the URL http://YOUR_IP_ADDRESS:APP_PORT/. Where YOUR_IP_ADDRESS is the IP address (or URL) of your machine, and APP_PORT is the number you specified in the deploy.conf file.

  2. To conclude the server setup, you can follow the procedure detailed in the section Setting up RNAdetector server at the bottom of this page.

Kubernetes deployment

RNAdetector can be deployed in any Kubernetes cluster using the kubectl utility. We provide two configurations: single instance or scalable instance. The first configuration is suitable for a single user or when the Kubernetes cluster has a single node. For use with multiple users, the second configuration is recommended since it provides a scalable environment.

Each node in the Kubernetes cluster should have at least 8 cores and 16Gb of RAM for use with Salmon or Hisat2. For STAR, we recommend at least 32Gb of RAM per single instance. A minimum of 2Tb of disk space is suggested.

Since many cloud providers have a Kubernetes service, it is also possible to deploy RNAdetector to these facilities. Our configuration files are also suitable for cloud deployment. To create a cluster, refer to each provider's guide:

Installing kubectl depends on your operating system. Refer to the Kubernetes docs for more details:

You will also need to configure kubectl to connect to your cluster. Please refer to your provider for more details. For cloud providers, you might use the following resources:

Now, you are ready to deploy RNAdetector on a Kubernetes cluster. The following procedure will give a provider-agnostic example.

  1. Download and extract the content of the RNAdetector repository from the following URL: https://github.com/alessandrolaferlita/RNAdetector/archive/refs/heads/master.zip 
$ wget https://github.com/alessandrolaferlita/RNAdetector/archive/refs/heads/master.zip && unzip master.zip
  1. If you wish to perform a single-instance deployment, enter the RNAdetector-master/cloud/kubernetes/single-instance/ directory. For a scalable instance deployment, please use the RNAdetector-master/cloud/kubernetes/scalable-instance/. The rest of this guide will use the scalable instance example.
$ cd RNAdetector-master/cloud/kubernetes/scalable-instance/
  1. Test the connection with the Kubernetes cluster with the kubectl get svc command

docker run --rm hello-world

  1. Now you can store the RNAdetector key and password in the secret store of the kubernetes cluster with:
$ kubectl create secret generic rnadetector --from-literal=mysql_password="[MYSQL_PASSWORD]"--from-literal=rnadetector_key="[APP_KEY]"

The APP_KEY variable is fundamental for the session encryption. You can generate a random value at the following link https://rnadetector.atlas.dmi.unict.it/key-generate.php. The DB_PASSWORD variable is the password used for the internal database. A strong and complicated password is the best choice for security and privacy.

  1. After storing the secrets, you can deploy the internal database with:
$ kubectl apply -f 01-mysql-volume.yml && kubectl create -f 02-mysql-container.yml && kubectl create -f 03-mysql-service.yml

You can verify that the server is running by using the kubectl get pods command. If the pod is going to be created, you will see an output similar to this:

$ kubectl get pods
NAME                    READY   STATUS              RESTARTS   AGE
mysql-b69697994-w8jhc   0/1     ContainerCreating   0          20s

When the pod has been created, the output will be the following:

$ kubectl get pods
NAME                    READY   STATUS    RESTARTS   AGE
mysql-b69697994-w8jhc   1/1     Running   0          28s
  1. As soon as the MySQL pod is running, you can create the RNAdetector instance with:
$ kubectl apply -f 04-rnadetector-volume.yml && kubectl create -f 05-rnadetector-container.yml

You can verify that the server is running with the following kubectl get pods command. If the pod is going to be created, you will see an output similar to this:

$ kubectl get pods
NAME                          READY   STATUS              RESTARTS   AGE
mysql-b69697994-w8jhc         1/1     Running             0          14m
rnadetector-cd64b8db7-j4tz4   0/1     ContainerCreating   0          2s
rnadetector-cd64b8db7-pmsr8   0/1     ContainerCreating   0          2s
rnadetector-cd64b8db7-zpd22   0/1     ContainerCreating   0          2s

Please notice that only one rnadetector container will show if you are creating a single instance. When the pod has been created the output will be the following:

$ kubectl get pods
NAME                          READY   STATUS              RESTARTS   AGE
mysql-b69697994-w8jhc         1/1     Running             0          16m
rnadetector-cd64b8db7-j4tz4   1/1     Running             0          109s
rnadetector-cd64b8db7-pmsr8   1/1     Running             0          109s
rnadetector-cd64b8db7-zpd22   1/1     Running             0          109s
  1. When creating a scalable instance, you also need to create a scaler that manages load balancing with:
$ kubectl create -f 06-rnadetector-scaler.yml
  1. Finally, you can expose the RNAdetector service using:
$ kubectl create -f 07-rnadetector-service.yml

To determine the IP address (or URL) to use for connecting to the RNAdetector service, you can use the kubectl get svc command and look at the "External IP" column corresponding with the rnadetector service.

$ kubectl get svc
NAME          TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)        AGE
kubernetes    ClusterIP      10.8.0.1      <none>          443/TCP        32m
mysql         ClusterIP      10.8.13.157   <none>          3306/TCP       19m
rnadetector   LoadBalancer   10.8.13.65    35.188.21.199   80:31732/TCP   48s

In this example, you will connect by using the hostname 35.188.21.199 and the port 80.

  1. To conclude the server setup, you can follow the procedure detailed in the section Setting up RNAdetector server at the bottom of this page.

Setting up RNAdetector server

After the RNAdetector container has been started, a public web server is activated and a default user account is created. Therefore, the first step for setting up the server is creating a new user account and removing the default user.

  1. You should access the URL of the RNAdetector server
  2. Click on the Login button at the top-right corner of the web page.

docker run --rm hello-world

  1. You can enter the control panel using the default credentials. E-mail address: admin@admin Password: password

  2. By clicking on the Manage users button on the dashboard you can add or remove users.

docker run --rm hello-world

  1. Now you can create a new user account with the New User button. The new user must be an administrator with a strong password.

docker run --rm hello-world

docker run --rm hello-world

  1. By clicking on the Create user button, you will create a new user account, and you will be redirected to the users list.

docker run --rm hello-world

  1. You can now click on the Delete beside the admin user to remove its user account. As soon as you click the button, you will be disconnected from the dashboard and you will have to use the new user account to login back.

  2. The server is now ready!! You can easily create other user account following the procedure detailed in steps 4-7. We suggest to create only one admin account.

Connecting RNAdetector GUI with your remote server

  1. Access the URL of the RNAdetector server and login with your user account

  2. Click on the Reset access token button to create a new secret access key for the GUI authentication. Copy the value displayed in the green box and store it safely. Anyone with this key will be able to access your account. Please remember that any time you click the Reset access token a new key will be generated and all the old ones will not be valid anymore.

docker run --rm hello-world

  1. Download the RNAdetector GUI for your operating system and install the application

  2. As soon as the application starts, click on the Custom Setup button to start the configuration procedure.

docker run --rm hello-world

  1. Deactivate the option Use a local docker installation? and click on the Next button.

docker run --rm hello-world

  1. In the Connection parameters step, fill the API Hostname with the IP address or the URL of the RNAdetector server. Furthermore, you should specify the API Port field. If you are using docker please refer to the deploy.conf file. For a Kubernetes cluster, if you are using the default values please specify the port 80. Finally, put in the API Key field, the key generated previously. Finally, click on the Next button to proceed with the next step.

docker run --rm hello-world

  1. Now click on the Install button to complete the setup process. RNAdetector will check the parameters and reload the interface as soon as the connection is established.

docker run --rm hello-world

  1. Congratulations! You are ready to use our cloud-based RNAdetector!
Clone this wiki locally