Five minute introduction
This site is intended to provide technical documentation for MidVision's application release automation solution.
Please visit the MidVision home page for all other information.
This documentation is intended for implementers of the solution and may include java developers, technology administrators, technical architects, technical project managers, DevOps professionals and release managers.
Overview
This page gives an overview of how the MidVision automated deployment product, RapidDeploy, packages a project into a point-in-time zip file containing deployable artifacts (deployment package) and or their configuration that can be deployed to any of the targets (for which the package is appropriate) in your estate.
RapidDeploy Project (also known as a blueprint)
The project is at the heart of the RapidDeploy application. Typically it will have a project root directory, below which artifacts and configuration are stored. A project may be directly related to a specific development project, such as a small web application project, or may be the basis for just one deployable component of a much larger software development effort. Typically a project will be the basis of one or more version locked deployable components, for example one or more script, ear, war or jar files, that form a Unit of Deployment.
A large development project with many independent components, such as web applications, web servers, scripts and database objects, may have many RapidDeploy projects (Units of Deployment), which may be run independently or as part of a Job Plan.
Each project has exactly one orchestration task flow, which determines the task steps to run for each job this project runs. This task flow is largely target independent. The task flow is edited through the UI and stored in the project as an xml file.
Each task in the task flow allows the setting of multiple data dictionary parameters (keys), in place of setting literal values. The data dictionary keys you define are automatically discovered whenever you save the project. You can set/edit the values for these keys on the orchestration data dictionary panel. This panel defines the 'Project Scoped' data dictionary values which will be used if the corresponding key value is not set at the target scope.
The Project scoped data dictionary parameters may be overridden at the target scope, to allow target differences such as database passwords to be set for each target this project deploys to.
The storage location of configuration files, whether in the fileystem or in an SCM repository, is determined from the project root directory (or working copy directory).
You can point multiple projects to the same root directory, but supply different orchestration file names, so that each project can perform different actions on the same set of configurations and deployable artifacts.
Project Orchestration
The orchestration file is located in the project root directory and by default is called midvision-deploy-orchestration.xml. It may be edited through the RapidDeploy GUI on the project Orchestration tab. The orchestration defines the task steps to run on the target servers. You can select tasks from a palette of base tasks and additional plugin-dependent tasks. Each task will normally define all of the configuration required to perform the task, in the form of fields known as task parameters. Most tasks also support one or more outputParameter fields. These typically contain the outputs from the task or true/false results.
outputParameters may be used as inputs in any fields of any subsequent tasks in the task flow.
The task steps are usually, but not always, target neutral. Configuration differences between targets are specified using Data Dictionary values.
As well as the standard tasks, there are some special tasks listed below:
- Select 'Conditionals and Loops' to add basic programming constructs into your orchestration such as 'If', 'While' and 'ForEach' statements.
- Select 'Add Target Specific' to add tasks that run only on the specified target(s), for example to check a change management tool in a production target only.
- Select 'Add RD Job Task' to add a 'Linked Deployment' which enables the current task flow to callback into RapidDeploy to invoke another job on this or a different target, either as a synchronous (wait) or asynchronous (parallel) call. For example, in between application stop/start on the current target, call out synchronously to a Database deploy project to make changes to a database.
Project Data Dictionary
The dictionary file is located in the project root directory and is called 'data-dictionary.xml'. It may be edited through the RapidDeploy GUI on the project 'Orchestration' -> 'Data Dictionary' tab.
Data dictionary parameters are key/value pairs of the following form:
@@DATA_DICTIONARY_KEY@@ Data-Dictionary-Value
The Data Dictionary defines the differences between one target and another target. The (target specific) dictionary values are injected into the deployment orchestration at deployment time.
Project, Target and Job scopes are supported.
- Project Scope - Any Data Dictionary values set at project scope are used by all targets, unless overridden by the target scope.
- Target Scope - Overrides Data Dictionary values at the project scope for a particular target.
- Job Scope - Overrides Data Dictionary values at the project and target scopes. Can be set when a job is called via the web services interface. User is prompted to enter Job Scoped Data Dictionary values in the UI if the value is empty at the Project and Target scope. This is called 'Late Property Injection'.
Project Deployable Resources
The projects deployable resources can be stored locally under the project root directory, or pulled in at package time from an external resource library location. They tend to be target neutral resources such as scripts, Ear files, Jar files etc. Typically they represent the 'payload' of the job and are often the output of a continuous integration or build process.
Local resources
The projects deployable resources can be stored locally under the project root directory under the 'resources' folder. You can add resources by uploading them through the RapidDeploy GUI on the Projects Files tab or via the command line in the filesystem.
External resources
Created packages can contain one or more external resources that are added to the package during the package creation. These resources are selected on the project main 'General' tab by selecting from the list of available resources.
The selected resources are stored in a 'resources.xml' file in the project root directory.
New resources can be defined on the resources panel, by defining a resource based on one of the external resource types (Maven, SCP, FileSystem, HTTP (wget) etc). All defined resources can be consumed by any project.
Targets
A 'Target' is defined as a 'Server' and one of its contained 'Installations'. A project can define many deployment targets. For example a project might define several development, test, UAT and production targets, each of which can be deployed to independently or as part of a release 'Pipeline'. Each deployment target can override the Data Dictionary values discovered and set at Project scope.
You can click on the 'Server' and 'Installation' links on the 'Targets' list panel to go to the Server or Installation definition respectively.
Deployment Package
The deployment package is a point in time, zipped copy of the project root directory that is stored outside SCM, normally on the filesystem or in an artifact repository such as Maven. It will also contain any external resources as required by the project (see above). The package will be named according to the label in the Scm repository. This package contains all of the configuration and artifacts to completely build (from scratch) and/or update ALL of the target that this package relates to.
This package can be created the RapidDeploy application itself using the Projects Package Wizard or through a Job Plan by selecting 'NEW VERSION' on the version selector. In the latter case, a new package is created in the project as the first step each time the pipeline is invoked, and used for all subsequent targets in the pipeline for this project.
RapidDeploy Server
The server panel defines target servers that the RapidDeploy application can communicate with. The Server 'Display Name' is the logical name used throughout the RapidDeploy Application to refer to this server definition.
A logical server definition in RapidDeploy may also refer to a group of servers (cluster). Multiple IP addresses or host names may be added in the 'Hostnames' field by using the '+' button. If the Server uses a cloud plugin, the cloud plugin will allow the determination of the number of hosts (physical IP addresses) managed buy this logical definition.
- On the 'General' tab you configure a display name used throughout the GUI, a target hostname/DNS or IP address. You must also specify a temporary location copy deployable artifacts to on the remote server.
- The 'Transport' tab is where you configure one or more transports used to connect to the target.
You select the transport using the 'Select Transport Mechanism' droplist. You can choose between 'Ssh' and 'Remote Agent'.
- If using 'Ssh', You can specify environment variables to set for use by the deployment process on this target server.
- You may specify the location of the Java runtime that is installed on the target server.
- You may also use these panels to deploy a version of Java to the target server and thereafter manage it, if you so decide.
- On the 'Cloud' tab you can specify a cloud provider to use. The available cloud providers will depend on the plugins you have installed. If you are using a cloud provider, you can specify the number of physical hosts that this logical definition will manage.
A server may have zero or more installations that are installed upon it.
RapidDeploy Installation
The Installations panel defines a logical target installation, on a specific server or cluster. This construct allows for the definitions of multiple targets on the same server. For example, we might want to run the same script, but with different parameters on the same set of servers. The combination of server/installation, a target, allows us to do this.
There are two tabs on the installation panel.
- The 'General' tab specifies the parent Server object, the Owner, the type of installation and if deployments are currently enabled to this installation. You can also add installation Notes. The Environment allows the logical grouping of installations. For example several installations may comprise the 'Internet banking development' environment. Similarly for 'Internet banking production'.
- The 'Snapshot' tab allows you to select from the snapshots that have been taken for this installation. A snapshot is a configurable point-in-time view of the target. The snapshots are stored as zip files and can be compared against each other on this panel.
The Deployment Process
Simple Case
In its simplest form, the deployment process allows you to select a project defining your set of version locked components, a target installation and a point-in-time deployment package (version of components, configuration or both) and execute them.
This performs the following actions:
- Copy the package to the target server on which the installation resides (or deployment manager if it resides on multiple servers such as a WebSphere Cell)
- Unpack the package in the temporary location
- Use the dictionary file to replace any placeholders in the orchestration file and any other configuration files (to make them target specific) if necessary
- Run through the tasks in the orchestration file in turn to perform the individual deployment tasks
- Logging is streamed back to the RapidDeploy UI in real-time for monitoring and debugging purposes.
Linked Deployments
RapidDeploy supports calling one or more downstream project jobs from a currently running job, either synchronously or asynchronously by using the 'Linked Deployment' task, which makes a Rest Web Services call back to the RapidDeploy framework server. This task can invoke any target on any project. This method can therefore be used to create dependencies between project targets (such as in a route to live scenario) or between projects (such as where deploying multiple components to create a complete target environment).
Job Plan
You can combine projects, targets and deployment package versions into a job plan, which is a higher level construct above the project level, enabling dependencies between project jobs to be created. The jobs plan is composed of pipelines, steps and jobs. Pipelines are based on steps and every step can have multiple parallel jobs. This means that within each pipeline jobs and be executed serially and/or in parallel, depending on which configuration is set. Additionally every pipeline can be scheduled to run at any time and steps auto executed or manually triggered. Dependencies can be set up between deployments and Human task steps can be introduced so that you can for example deploy all web servers in parallel, wait for them all to complete and then deploy the web application, perhaps requiring a human task after the web application has deployed to confirm success or order a backout.
Additionally, a pipeline step may make a REST call to any external service and wait (poll) for a specific success or failure response. A pipeline step may also call another pipeline, either synchronously or asynchronously.
Export / Import
The project can be exported as a single package to be imported in the future via a wizard page. If the user has the granted role, he is able to download the project as a single zip file in the project details page on edit mode. This single project zip file can be uploaded from project list page if the user is allowed to perform this operation. Once the file is uploaded the user is redirected to import project wizard page. The project import operation includes project settings, resources and target files. If any resource does not already exist, they will be restored at the same point when they were exported.
In this way projects and all their dependencies can be shared between different RapidDeploy instances for the purposes of backup, promotion, testing, upgrading and debugging.
When performing upgrades or backups, there is a facility within the tool to export/import all defined projects to/from a users local filesystem.