Upgrade Splunk UBA prerequisites

Upgrading to Splunk UBA version 5.4.5 requires Splunk UBA version 5.4.4. See How to install or upgrade to this release of Splunk UBA for upgrade path information.

  • If you are running a version lower than 5.1.0, you must first upgrade to version 5.1.0, then upgrade to version 5.2.0 or 5.3.0, then upgrade to version 5.4.0, then upgrade to version 5.4.1, then upgrade to version 5.4.2, then upgrade to version 5.4.3., then upgrade to version 5.4.4, and then upgrade to version 5.4.5.
  • If you are running a version lower than 5.0.5, you must first upgrade to version 5.0.5, then upgrade to version 5.1.0, then upgrade to version 5.2.0 or 5.3.0, then upgrade to version 5.4.0, then upgrade to version 5.4.1, then upgrade to version 5.4.2, then upgrade to version 5.4.3, then upgrade to version 5.4.4, and then upgrade to version 5.4.5.

Pre-upgrade tasks

Before you upgrade, perform the following tasks:

Note: To connect to Splunk using the connector.splunk.protocol, the only valid value is TLSv1.2 . If you have a different value in place for that field, it is ignored and the Splunk connection configuration uses TLS v1.2.
  1. In RHEL Linux environments:
    • Ensure that Splunk UBA has access to RHEL repositories.
    • When installed on RHEL 8.x operating systems, Splunk UBA uses a 2048 bit RSA encryption key. The Splunk platform that communicates with Splunk UBA must also use a 2048 bit encryption key. See Red Hat Enterprise Linux 8.x cryptographic policies.
  2. Review the Known issues as well as What's new for this release in the Release Notes manual for issues, enhancements, and changes that might need to be addressed before upgrading.
  3. The software update contains two archive files approximately 1.1G and 4.8G. The total file size after extraction is 9.1G. Verify that you have enough free space in /home/caspida to store the extracted installer files.
  4. Backup your system. See Prepare to backup Splunk UBA in Administer Splunk User Behavior Analytics.
  5. Make sure your system is running normally by using the uba_health_check.sh shell script.
    PLAINTEXT 
    /opt/caspida/bin/utils/uba_health_check.sh

    See Check system status before and after installation for more information about the script.

  6. When you upgrade Splunk UBA to version 5.4.5, the Splunk UBA Monitoring App as present on UBA nodes is also upgraded to the latest app version of 1.1.5. This monitoring app upgrade overwrites files in the /opt/splunk/etc/apps/Splunk_UBA_Monitor/default/ directory.

    If you made any changes to these files, backup your changes before starting the UBA upgrade process. You can later apply those changes back.

    Note: As a best practice, make changes in the local directory (/opt/splunk/etc/apps/Splunk_UBA_Monitor/local/) and not the default directory.
  7. If your caspida account user has restricted sudo access, follow the steps in Enable all sudo access for the caspida account to temporarily provide sudo access privileges.

Verify Postgres Collate and Ctype values

Perform the following steps to verify the values of Collate and Ctype:

Note: Run these commands on the Postgres node. For a 20-node deployment, that is node 2. For non-20-node deployments, that is node 1.
  1. Check the Collate and Ctype values in a database. Confirm the values are en_US.UTF-8.

    Run the following command:

    PLAINTEXT 
    psql -d caspidadb -c "\l"
  2. If the values are correct, you are done. Do not run migrate_psql_collate_ctype.sh or any of the numbered migration steps in the following subsection. Continue with the standard UBA upgrade prerequisites and upgrade instructions.

    The following image shows an example of correct Collate and Ctype values:

    This image shows the results from running the command in this step. A series of columns and rows is displayed including columns for Collate and Ctype. The value under both these columns is en_US.UTF-8.
If your Collate and Ctype configuration is not set to en_US.UTF-8, perform the following steps before proceeding with a UBA upgrade.
Note: If your installed UBA version or package layout does not match this topic, contact Splunk Support for the correct script and steps before proceeding
Taking these steps avoids potential data corruption issues in the Postgresql database on your current UBA version:
Note: The following steps apply to the affected version of UBA only when Collate and Ctype are not set to en_US.UTF-8 for caspidadb as shown by psql -d caspidadb -c "\l".
  1. Run the following command to stop UBA and all services on the management node of the version of UBA where the Postgres Collate and Ctype configuration is not set to en_US.UTF-8:
    PLAINTEXT 
    /opt/caspida/bin/Caspida stop-all

    1. Keep UBA stopped on the environment while you perform the database migration steps through backup validation and restore, unless Splunk Support documents a different sequence for your deployment.

  2. Perform the following steps to extract and use the data migration script from the UBA 5.4.5 package on the database node as per your deployment:
    Note: Before copying or running the script, confirm you are on the PostgreSQL database host. Check database.host in /opt/caspida/conf/deployment/caspida-deployment.conf
    1. Download and extract UBA 5.4.5 upgrade package in /home/caspida/:
      PLAINTEXT 
      $ tar xvzf /home/caspida/splunk-uba-software-upgrade-package_545.tgz
Splunk-UBA-Platform-5.4.5-20260227-34401448.tgz
Splunk-UBA-Platform-5.4.5-20260227-34401448.tgz.md5sum
uba-ext-pkgs-5.4.5.tgz
uba-ext-pkgs-5.4.5.tgz.md5sum
    2. Extract the UBA-Platform-5.4.5 build into a temporary folder in /home/caspida/ directory:
      PLAINTEXT 
      mkdir -p /home/caspida/tmp/
tar xfvz Splunk-UBA-Platform-5.4.5-20260227-34401448.tgz -C /home/caspida/tmp/
    3. Copy the migrate_psql_collate_ctype.sh script from the above tmp directory into /opt/caspida/bin/ directory:
      Note: Perform cp / chmod and subsequent script steps using the same supported execution context throughout . This is typically the caspida user, with sudo only where required. Avoid alternating users without a clear, documented reason.
      PLAINTEXT 
      cp /home/caspida/tmp/bin/migrate_psql_collate_ctype.sh /opt/caspida/bin/
    4. Provide executable permission to the migrate_psql_collate_ctype.sh script for data migration:
      PLAINTEXT 
      chmod u+x /opt/caspida/bin/migrate_psql_collate_ctype.sh
  3. Update locale settings on your system:
    System Steps
    RHEL/OEL 1. Update the /etc/locale.conf file with the following configuration:
    PLAINTEXT 
    LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"

    2. Run the following command to source /etc/locale:

    PLAINTEXT 
    source /etc/locale
    Ubuntu 1. Update the /etc/default/locale file with the following configuration:
    PLAINTEXT 
    LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"

    2. Run the following command to source /etc/default/locale:

    PLAINTEXT 
    source /etc/default/locale
    1. (Conditional procedure) Run migrate_psql_collate_ctype_sh only when Postgre SQL Collate or Ctype is not en_US.UTF-8, as shown in the following code: psql -d caspidadb -c "\l".

      If both are already en_US.UTF-8 skip this procedure and continue with the standard UBA upgrade steps.

      1. Run this procedure on the database host only.
      2. Follow the documented order: Run the migration script as written, validate the backup file, then restore.
      3. Do not rerun the backup step multiple times unless instructed by Splunk Support.
    2. Common issues to avoid:

      1. Running this procedure when Collate and Ctype value are already correct.

      2. Skipping backup validation before restore.

      3. Running migration steps repeatedly without confirming each step succeeded.

      4. Mixing execution contexts or users in an uncontrolled way. For example alternating sudo and the caspida user without a clear, consistent method.

  4. Once the UBA 5.4.5 migration script is inside the /opt/caspida/ directory, run the migration script (backup and built-in post-backup steps).

    On the database host, run the following:
    PLAINTEXT 
    $ /opt/caspida/bin/migrate_psql_collate_ctype.sh -b
    This creates a backup file under/home/caspida/named similar topsql-backup-<DATE-TIME>. The script also performs the post-backup processing required for the collate/ctype migration, as implemented by this script.
    Note: Let the command finish and do not assume it is only copying data.
    Expected result: The command completes without errors.
    CAUTION: Do not continue to restore if this step fails or if you are unsure it completed successfully.
  5. (Required) Validate backup before restore.
    CAUTION: Backups from this flow are plain SQL from pg_dumpall. Do not use pg_restore --list.
    1. Pick one backup file under /home/caspida/(psql-backup-*) that matches the successful -b run you are using. If several exist,the newest is not always correct. Match time to your run, then follow that path for steps B and C and for -r.
    2. Check that the file exists and that the size is plausible but not empty:
      PLAINTEXT 
      ls -lrth /home/caspida/ | grep psql-backup 
wc -c /home/caspida/<your-backup-file>
    3. Check that the content looks like SQL and mentions caspidadb. It must not look empty or like a template. For example: head -n 50 or grep caspidadb
    Note: If anything fails, stop and contact Splunk Support. Run -r with the same path you just validated.
  6. Restore from the validated backup:
    CAUTION: Only restore after step 5 succeeds.
    PLAINTEXT 
    $ /opt/caspida/bin/migrate_psql_collate_ctype.sh -r /home/caspida/<your-backup-file>
  7. Then verify locale settings:
    PLAINTEXT 
    psql -d caspidadb -c "\l"
    1. Confirm that Collate and Ctype values are en_US.UTF-8.
    2. Operator notes:
      1. -b runs the script's backup path andi ts built-in post-backup processing for this migration. It is not a "copy-only" manual step.

      2. -r <backup-file> restores from the backup file validated in step 5.

      3. Do not run multiple -b runs in a row unless Splunk Support instructs you to.

      4. Always use the same backup file you validated when you run -r.

  8. After confirming that the configuration values are correct, start UBA and verify any potential issues with the system:
    PLAINTEXT 
    /opt/caspida/bin/Caspida start-all

Validate the UMASK value

Ensure the UMASK value of the root user is set to 0002 or 0022, or grant read permissions for newly created files and directories to the caspida user.

Complete the following steps to validate the UMASK value:

  1. Check the UMASK value of the root user by running the following command. The value must be 0002 or 0022:
    PLAINTEXT 
    umask
  2. Verify the UMASK value in the /etc/login.defs file:
    PLAINTEXT 
    grep -i "^UMASK" /etc/login.defs
    Note: The umask value specified in /etc/login.defs applies as the default for all users.
  3. Validate the permissions for new files and directories:
    1. As the caspida user, create a new file or directory using sudo to observe the permissions:
      PLAINTEXT 
      sudo touch testfile.txt 
sudo mkdir testdirectory
    2. Next, check the permissions of the created files and directories. The read permission for caspida(other) users is required:
      PLAINTEXT 
      ls -l testfile.txt 
ls -ld testdirectory

To set the required umask value, edit the /etc/login.defs file and set the UMASK value to 022.

CAUTION: If the caspida user does not have read permissions, update the UMASK value accordingly. Failure to provide the required permission to the caspida user will result in a UBA installation or upgrade failure.

Upgrade multiple Splunk UBA clusters that are using warm standby

If you have two Splunk UBA clusters running in a warm standby configuration, perform the following tasks to upgrade both clusters. Links to documentation in the Administer Splunk User Behavior Analytics manual are provided. In this example, the original primary system is called System A and the standby system is called System B.

  1. Verify that both the System A and System B are configured for warm standby and are running as expected. See Verify that the primary and standby systems are synchronized .
  2. Manually trigger a sync between System A and System B. See Synchronize the primary and standby systems on-demand.
  3. Perform a failover from System A to System B. See Failover to a standby Splunk UBA system.
  4. Switch the roles of both systems to reflect the failover. See Change the role of both systems to switch the primary and standby systems.
  5. Failover from System B back to System A. See Failover to a standby Splunk UBA system.
  6. Switch the roles of both system again to reflect the second failover operation. See Change the role of both systems to switch the primary and standby systems.
  7. Run the uba_health_check.sh script. See Check system status before and after installation in the Install and Upgrade Splunk User Behavior Analytics manual.
  8. Use the health monitor to verify that both Splunk UBA systems are up and running.
  9. Upgrade the primary system (System A) to this release. Follow the upgrade instructions for your operating system.
  10. Upgrade the standby system (System B) to this release. Follow the upgrade instructions for your operating system.
  11. Check /var/log/caspida/UpgradeStatus-<release>.properties on both systems to verify that the upgrade succeeded. See Verify a successful upgrade of Splunk UBA in the Install and Upgrade Splunk User Behavior Analytics manual.