Installing QAnswer on AWS EC2 instance

Getting an instance

The first step is to start up an EC2 instance with the appropriate resources.

For the below guide, we will assume that you’ve chosen to use Ubuntu Server 24.04 LTS (HVM) with the g6e.xlarge instance.

Instance Size	GPU	GPU Memory (GiB)	vCPUs	Memory (GiB)	Storage (GB)	Network Bandwidth (Gbps)	EBS Bandwidth (Gbps)
g6e.xlarge	1	48	4	32	250	Up to 20	Up to 5

warning

We provision a total of 282GB (250GB + 32GB) of storage for the g6e.xlarge instance, consisting of a 32GB SSD volume combined with the original 250GB SSH.

Handling 250GB SSD Mounting for g6e.xlarge

For AWS instances of type g6e.xlarge, the 250GB SSD is not mounted by default under /dev/nvme0n1. This requires manual mounting to enable the download of Docker images.

Steps to Mount the SSD:

1. Run the lsblk command to list all block devices and identify where the unmounted volume is located.

lsblk

2. Format the volume using the mkfs command with the XFS file system:

sudo mkfs -t xfs /dev/nvme1n1

3. Create a directory to serve as the mount point and mount the volume to the directory you just created.

sudo mkdir /home/$USER/.local
sudo mount /dev/nvme1n1 /home/$USER/.local

4. Use the lsblk -f command to display the UUID of the mounted volume. Note this value for later steps (e.g., e4bc66e3-c793-4b54-ad00-e5a1e11c80ef).

lsblk -f

---

Make the Mounting Permanent:

1. Edit the /etc/fstab file:

sudo vim /etc/fstab

2. Add the following entry at the end of the file, replacing UUID and username with your actual values:

UUID=e4bc66e3-c793-4b54-ad00-e5a1e11c80ef  /home/ubuntu/.local  xfs  defaults,nofail  0  0

Make the Mounted Volume accessible by $USER

sudo chown -R $USER:$USER /home/$USER/.local

Check volume mounted without problems with df -h

You will see something similar to:

Filesystem       Size  Used Avail Use% Mounted on
/dev/root         30G  1.7G   29G   6% /
tmpfs             16G     0   16G   0% /dev/shm
tmpfs            6.2G  1.1M  6.2G   1% /run
tmpfs            5.0M     0  5.0M   0% /run/lock
efivarfs         128K  3.6K  120K   3% /sys/firmware/efi/efivars
/dev/nvme0n1p16  881M   76M  744M  10% /boot
/dev/nvme0n1p15  105M  6.1M   99M   6% /boot/efi
tmpfs            3.1G   12K  3.1G   1% /run/user/1000
/dev/nvme1n1     233G  4.5G  229G   2% /home/ubuntu/.local

---

Configuring Container Runtime Storage:

Docker

To set the location for Docker images:

sudo ln -s /home/ubuntu/.local /var/lib/docker

Podman

To set the location for Podman images:

1. Create the configuration directory and open the storage.conf file:

mkdir -p ~/.config/containers/
vim ~/.config/containers/storage.conf

2. Add the following content to the file:

[storage]
driver = "overlay"
runroot = "/run/user/1000"
graphroot = "/home/ubuntu/.local/share/containers/storage"

Install NVIDIA Driver and NVIDIA Container Toolkit

If the server doesn't have nvidia driver installed (verify using nvidia-smi command), we need to install the driver first in order to use GPU on the machine.

1. Go to NVIDIA Driver Finder, Enter the GPU model, in our case L40S to select the right distribution in the dropdown list.

2. Hit Find button, Nvidia will present you the recommended driver version for you to download, we have 550.127.08 for the model L40S.

3. So, we can download the nvidia-driver-550 (550.127.08) via:

sudo apt update

sudo apt install nvidia-driver-550
# may take few minutes to build
# Progress: [ 92%] [###########################################.........]

4. install the nvidia-container-toolkit, to ensure GPU to be used in the container, check documentation for help if needed.

# Configure production repo:
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Update the packages list from the repository:
sudo apt-get update

# Install the NVIDIA Container Toolkit packages:
sudo apt-get install -y nvidia-container-toolkit

# Generate nvidia-.yaml (optional for docker, necessary for podman)
sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
nvidia-ctk cdi list

5. After rebooting, we should be able to see the similar output via command nvidia-smi:

Installing a Container Runtime

Depending on your preference, you can choose to install either Docker or Podman as your container runtime. Follow the respective instructions below:

Installing Docker

If Docker is not yet installed on your machine, follow the official Docker installation guide to set it up.

Installing Podman (Alternative to Docker)

If Podman is not yet installed on your machine, you can install it by following the official Podman installation guide.

Installing podman-compose

To use Podman with docker-compose-like functionality, you’ll need to install podman-compose. Refer to the official podman-compose documentation for detailed installation steps.

Example Installation Command:

sudo apt install -y podman podman-compose

Once installed, verify that both Podman and podman-compose are properly set up by running:

podman --version
podman-compose --version

Solve the pending issue of rootless containers in podman, skip if you are using root.

sudo loginctl enable-linger $USER

Install & Deploy QAnswer

QAnswer is provided as a docker-compose file. You can clone the repository by running the following command

git clone https://gitlab.the-qa-company.com/qanswer-app/qanswer-bundle.git

To deploy QAnswer, navigate to the qanswer-bundle directory. Depending on your container runtime (Docker or Podman), follow the steps below:

Docker

Steps for Docker:

1. Run our setup script, it will pull all the container images we need. You need to put here the credentials we will provide to you:

QANSWER_REGISTRY_USER=<user-we-provide> \
QANSWER_REGISTRY_PASSWORD=<password-we-provide> \
VERSION=main source ./setup.sh --container_runtime docker --no_documentation

2. Start the QAnswer bundle using docker compose:

Run the docker command:

QANSWER_KEY=<qanswerKey-we-provide> \
VERSION=main docker compose up -d

Podman

Steps for Podman:

1. Run our setup script, it will pull all the container images we need. You need to put here the credentials we will provide to you:

QANSWER_REGISTRY_USER=<user-we-provide> \
QANSWER_REGISTRY_PASSWORD=<password-we-provide> \
VERSION=main source ./setup.sh --container_runtime podman --no_documentation

2. Start the QAnswer bundle using podman-compose:

Run the docker command:

QANSWER_KEY=<qanswerKey-we-provide> \
VERSION=main podman-compose up -d