docs/validator_setup.md

Running Validator

---

Machine Requirements

To achieve optimal results, we recommend the following setup:

• Operating System: Ubuntu 24.04 LTS or higher Learn more about Ubuntu 24.04 LTS
• GPU: NVIDIA RTX A6000 with 48GB VRAM and 30 CPU Cores

---

Install PM2 (Process Manager)

PM2 is used to manage and monitor the validator process. If you haven’t installed PM2 yet, follow these steps:

1. Install npm and PM2:

   sudo apt update
   sudo apt install npm -y
   sudo npm install pm2 -g
   pm2 update
   

2. For more details, refer to the PM2 Documentation.

---

Install Redis

1. Install 'redis'

   sudo apt update
   sudo apt install redis-server
   sudo systemctl start redis
   sudo systemctl enable redis-server
   sudo systemctl status redis
   

Install Project Dependencies

Prerequisites

• Python: Version 3.10 or higher
• pip: Python package manager
• virtualenv (optional): For dependency isolation

---

1. Clone the Repository

Clone the project repository to your local machine:

git clone https://github.com/vidaio-subnet/vidaio-subnet.git
cd vidaio-subnet

---

2. Set Up a Virtual Environment (Recommended)

Create and activate a virtual environment to isolate project dependencies:

python3 -m venv venv
source venv/bin/activate  

---

3. Install the Package and Dependencies

Install the project and its dependencies using pip:

pip install -e .

---

4. Configure Environment Variables

To configure environment variables, follow these steps:

1. Create a .env file in the project root directory by referencing the provided .env.template file:

   cp .env.template .env
   

2. Set up a bucket in cloud storage. The base miner code utilizes MinIO to connect with cloud storage services, so you'll need to prepare your bucket using a platform that supports MinIO integration, such as Backblaze. Alternatively, you can modify the code to suit your specific requirements. IMPORTANT: Note that currently the region of the storage is hardcoded, and must be adjusted in vidaio_subnet_core/utilities/storage_client.py for corresponding storage, such as AWS.

3. Add the required variables to the .env file. For example:

   BUCKET_NAME="S3 buckent name"
   BUCKET_COMPATIBLE_ENDPOINT="S3 bucket endpoint"
   BUTKET_COMPATIBLE_ACCESS_KEY="S3 bucket personal access key"
   BUCKET_COMPATIBLE_SECRET_KEY="S3 bucket personal secret key"
   PEXELS_API_KEY="Your Pexels account api key"
   WANDB_API_KEY="Your WANDB account api key"
   

4. Ensure that the bucket is configured with the appropriate permissions to allow file uploads and enable public access for downloads via presigned URLs.

5. Create your Pexels API key and replace it. (https://www.pexels.com/)

6. Once the .env file is properly configured, the application will use the specified credentials for S3 bucket, Pexels and Wandb.

---

Install FFMPEG

FFMPEG is required for processing video files. Install it using the following commands:

sudo apt update
sudo apt install ffmpeg -y

For more details, refer to the FFMPEG Documentation.

---

Install VMAF

To enable video quality validation, install VMAF by following the steps below to set up a clean virtual environment, install dependencies, and compile the tool.

---

Clone the VMAF repository into the working root directory of your vidaio-subnet package. If the vidaio-subnet virtual environment is currently active, deactivate it first:

git clone https://github.com/vidAio-subnet/vmaf.git
cd vmaf

---

Step 1: Set Up a Virtual Environment in VMAF directory

1. Install venv if it’s not already installed:

   python3 -m venv vmaf-venv
   

2. Activate the virtual environment:

   source vmaf-venv/bin/activate
   

---

Step 2: Install Dependencies

1. Install meson:

   pip install meson
   

2. Install system dependencies:

   sudo apt-get update
   sudo apt-get install nasm ninja-build doxygen xxd
   

   For Ninja, verify whether the package name is ninja or ninja-build before running the install command.

---

Step 3: Compile VMAF

1. Set up the build environment:

   cd libvmaf
   meson setup build --buildtype release -Denable_avx512=true
   

2. Optional flags:

   • Use -Denable_float=true to enable floating-point feature extractors.
   • Use -Denable_avx512=true to enable AVX-512 SIMD instructions for faster processing on supported CPUs.
   • Use -Denable_cuda=true to build with CUDA support (requires nvcc and CUDA >= 11).
     Note: To enable CUDA successfully, ensure nvcc and the CUDA driver are installed. Refer to the CUDA and NVCC setup guide.
   • Use -Denable_nvtx=true to enable NVTX marker support for profiling with Nsight Systems.
   • Recommendation: We recommend adding -Denable_avx512=true to enhance validation speed. If CUDA is available, include the flag -Denable_cuda=true But At present, VMAF does not include support for CUDA integration.

3. Build the project:

   ninja -vC build
   

---

Step 4: Test the Build

Run tests to verify the build:

ninja -vC build test

---

Step 5: Install VMAF

Install the library, headers, and the command-line tool:

ninja -vC build install

---

Step 6: Generate Documentation

Generate HTML documentation:

ninja -vC build doc/html

Step 7: Deactivate vmaf-venv, activate project venv

deactivate
cd ..
cd ..
source venv/bin/activate

Running the Validator with PM2

To run the validator, use the following command:

pm2 start run.sh --name vidaio_v_autoupdater -- --wallet.name [Your_Wallet_Name] --wallet.hotkey [Your_Hotkey_Name] --subtensor.network finney --netuid 85 --axon.port [port] --logging.debug

Parameters:

• --wallet.name: Replace [Your_Wallet_Name] with your wallet name.
• --wallet.hotkey: Replace [Your_Hotkey_Name] with your hotkey name.
• --subtensor.network: Specify the target network (e.g., finney).
• --netuid: Specify the network UID (e.g., 85).
• --axon.port: Replace [port] with the desired port number.
• --logging.debug: Enables debug-level logging for detailed output.

---