RapidDeploy 5.2.3 may fail to start, returning HTTP 404 for every page

Why the installation looks healthy

Every check other than the HTTP status suggests the server is fine:

systemctl is-active rapiddeploystart     ->  active
ss -ltn | grep :9090                     ->  listening
curl http://localhost:9090/MidVision/    ->  404      <-- the only real signal

Tomcat itself starts correctly and binds its port; only the RapidDeploy application context is missing. The service unit stays active because it waits for a start-up line in the application log that is never written. The service unit is behaving correctly - do not modify it.

The quickest way to confirm this problem is therefore the HTTP status, not the service state:

curl -s -o /dev/null -w '%{http_code}\n' http://localhost:9090/MidVision/

In $MV_HOME/logs/rapiddeploy-web-app.log the failure appears as an error creating the persistence layer:

ERROR org.springframework.web.context.ContextLoader - Context initialization failed
  BeanCreationException: Error creating bean with name 'entityManagerFactory'
  Caused by: java.lang.NoSuchMethodError:
    'java.lang.Object org.jboss.logging.Logger.getMessageLogger(...)'

Am I affected

The outcome depends on the order in which the file system returns the contents of the application's library directory. That order is a property of the file system itself, fixed when it was created, so:

• the result is the same on every start for a given machine - a 5.2.3 installation that starts correctly will keep starting correctly, and one that fails will fail every time;
• the same installation package can succeed on one machine and fail on another. Both were observed from a byte-identical package;
• reinstalling, redeploying or updating does not change the outcome. If 5.2.3 fails to start on a machine, repeating the installation will not help.

  Because a working 5.2.3 installation is working only by luck of the file system layout, we recommend upgrading even where the problem has not appeared.

Cause

The CDI runtime shipped inside 5.2.3 was packaged as a single all-in-one archive that also contained its own, older copy of the JBoss logging library. Two copies of one class were therefore present in the application, and whichever the server loaded first determined the result. The older copy does not contain a method that the version of Hibernate introduced in 5.2.3 requires, so when it was loaded first, the persistence layer failed to start.

5.2.4 takes the CDI runtime as separate components instead, so the logging library is present exactly once. The CDI runtime itself is unchanged.

Resolution

Upgrade to RapidDeploy 5.2.4 or later, following the standard upgrade procedure.

Interim workaround for an installation already deployed

If upgrading immediately is not practical, the older duplicate can be removed from the deployed application. This survives restarts and re-deployments of the same package.

Run as the user that owns the RapidDeploy installation:

WEBAPPS=$MV_HOME/web-apps/tomcat/webapps
J=WEB-INF/lib/weld-servlet-shaded-5.1.5.Final.jar

systemctl stop rapiddeploystart

cp -p $WEBAPPS/MidVision.war $WEBAPPS/MidVision.war.bak
mkdir -p /tmp/weldfix && cd /tmp/weldfix
unzip -q $WEBAPPS/MidVision.war "$J"
zip -q -d "$J" 'org/jboss/logging/*'
zip -q $WEBAPPS/MidVision.war "$J"
chown --reference=$WEBAPPS/MidVision.war.bak $WEBAPPS/MidVision.war

rm -rf $WEBAPPS/MidVision

systemctl start rapiddeploystart

Then confirm the application answers:

curl -s -o /dev/null -w '%{http_code}\n' http://localhost:9090/MidVision/

Notes on the procedure:

• removing $WEBAPPS/MidVision forces the server to extract the corrected archive on the next start;
• the chown step preserves file ownership. The --reference option requires GNU coreutils, as found on Linux; set the owner and group explicitly on other platforms;
• keep MidVision.war.bak. It is the unmodified 5.2.3 archive;
• removing the duplicate is safe because the remaining copy of the logging library is a strict superset: it provides both the older methods the CDI runtime uses and the newer one Hibernate requires;
• this is a workaround, not a substitute for the upgrade. Applying a different package later - an upgrade, or a re-install from the 5.2.3 media - will restore the original archive and the problem with it.