Built by Maven

Making RapidDeploy Highly Available

High Availability infrastructures come in many shapes and sizes, for example:

  • Active/Active using 1 cell/1 cluster across 2+ data centres with both horizontal and vertical scaling. It may have node redundancy (2+ nodes) in each data centre, or not (1 node per data centre)
  • Active/Active using 2 cells (1 per data centre) again with or without node redundancy in each data centre.
  • Active/Passive single cell across 2 data centres
  • Active/Passive 2 cell across 2 data centres
  • Active/Passive 1 cell on 1 data centre with file-system takeover

    Depending on which of these (or another option) you use, the potential design might look different.

Considerations when choosing a high availability infrastructure

  • Network bandwidth and latency between data centres
  • Shared filesystems via NAS, NFS etc - is this supported in your organisation?
  • SCM tool availability (or again filesystem) between data centres
  • Update policy (deploying new versions of RapidDeploy if in different cells)

RapidDeploy specifics

RapidDeploy version 3.5+ has HA support. A job is executed from a single JVM, but an executing job can be viewed from any JVM, even if in a different cluster. The real-time logs can also be viewed in any of the JVMs. The main requirement is that each JVM must be configured to talk to the same RapidDeploy database, and the internal Hypersonic database cannot be used for this, although any of the supported enterprise databases, such as Oracle or Oracle RAC can be used.

RapidDeploy uses a quartz scheduler, which will only allow 1 JVM in a set of JVMs (HA cluster) to execute a scheduled job.

General Requirements:

  • All JVM�s must access the same back-end RapidDeploy Oracle (RAC) Database,
  • The buildstore filesystem (if using file based artefact repository) will need to be mounted at the same mount point across all nodes hosting a RapidDeploy JVM in the HA cluster.
  • The logs filesystem will need to be mounted at the same mount point across all nodes hosting a RapidDeploy JVM in the HA cluster.
  • The snapshots filesystem will need to be mounted at the same mount point across all nodes hosting a RapidDeploy JVM in the HA cluster.
  • If using the filesystem based SCM plugin, this filesystem will also need mounting across all nodes hosting a RapidDeploy JVM in the HA cluster.
  • Each node in the cluster hosting a RapidDeploy instance will need its own rapiddeploy_default.properties file and rapiddeploy.properties file in its own $MV_HOME directory

Risks/failures

A job is still executed from one JVM. If that JVM terminates or becomes unresponsive, the job may fail, as the connection is made from within that JVM to the target, either by SSH or agent. We do not currently advise using session replication due to the potentially large size of RapidDeploy user sessions As we do not recommend session replication is enabled, the loss of a JVM will result in all users connected to that JVM losing their session and needing to log back in. Some failure scenarios relate to specific plugins. These can be discussed on a per-plugin basis.

Summary

This document provides a high level view of some HA scenarios and considerations. MidVision is always happy to work with customers to provide additional consultancy around HA for RapidDeploy, and to work with you to create the right architecture for your requirements.