How to handle vCenter Orchestrator logs?

vCenter Orchestrator uses a powerful log service. This article explains the basics of this service and possible configuration changes.

This article will identify the log files currently in use for a typical vCO server installation. Additionally, it will address the following areas of interest:

• Server Log rollover control: MaxFileSize and MaxBackupIndex
• Enable REST call logging in the vCloud Director Plug-in
• Enable logging to a SYSLOG server
• Setting the server and all components to debug mode

The vCenter Orchestrator application server log files are located in install_directory\app- server\server\vmo\log

The server.log file the most relevant file for auditing the vCenter Orchestrator application server components, the plug-in adapters and the scripts contained in workflows, policies and actions. For convenience the script-logs.log only contains logs produced by the scripts.

Log levels and persistence

Each of the vCenter Orchestrator application server components, plug-in adapter provides logs in different levels including fatal, error, warning, info and debug.

During normal operations it is recommended to use the default info level to maximize the server performance while keeping quite detailed level information. Using the debug level for a single component or for all server components is recommended for troubleshooting.

A vCenter Orchestrator script in an action, workflow or policy can generate an error, warning, info and debug log by invoking the System.log(), System.warn(), System.error() and System.debug() methods with a single log content string parameter.

A vCenter Orchestrator script in an action, workflow or policy can generate an error, warning, info and debug log and an action, workflow or policy event by invoking the Server.log(), Server.warn(), Server.error() and Server.debug() methods with an event description string parameter and an event content string parameter.

The events will be saved in vCenter Orchestrator database and stay attached to their action, workflow or policy until deleted by the purge daemon when reaching a total of 5000 events or more than 100 runs for a given workflow. These values can be changed in the vmo.properties configuration file using the max-log-events and max-workflow-tokens values. Substantial increases will lead to database performance decrease requiring database admin regular maintenance operations such as re-indexing the event and workflow token tables.

Log configuration

vCenter Orchestrator uses Apache log4j allowing to configure and enable granular logging at runtime without modifying the application. The target of the log output is a configured by default to a set files but can be changed to different output targets such as a remote Unix Syslog daemon.

The log configuration file is named log4j.xml and is located in install_directory\app- server\server\vmo\conf.

Changes made by editing the configuration file manually may be lost when using vCenter Orchestrator web configuration to perform log configuration changes. This should be avoided once manual changes have been provided.

Common configuration changes use cases

The server log files may be rolling over in a short period of time when the log verbosity is increased substantially. This can be prevented by changing the default size and number of files as follows (Here multiplying the MaxFileSize and the MaxBackupInded by four times the default value).

1
2

<param name="MaxFileSize" value="8192KB"/>
<param name="MaxBackupIndex" value="16"/>

When troubleshooting vCenter Orchestrator with vCloud Director setting the vCloud Director Plug-in in debug mode logs the REST calls and responses. This can be changed in the following section as follows:

1
2
3

<category additivity="true" name="com.vmware.vmo.plugin.vcloud">
    <priority value="DEBUG"/>
</category>

Using log centralization to a Unix Syslog Daemon for all the cloud management applications is recommended. This can be done by adding the following section:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

  <!-- ============================== -->
  <!-- Append messages to the syslog-->
  <!-- ============================== -->
  <appender name="SYSLOG" class="org.apache.log4j.net.SyslogAppender">
    <param name="SyslogHost" value="X.X.X.X"/> 
    <param name="Facility" value="USER"/> 
    <param name="FacilityPrinting" value="true"/> 
    <layout class="org.apache.log4j.PatternLayout">
      <param name="ConversionPattern" value="%t %5r %-5p %-21d{yyyyMMdd HH:mm:ss,SSS} %c{2} [%x] %m %n"/> 
    </layout> 
  </appender>

X.X.X.X must be changed to the Syslog server and USER replaced with the right log facility for your specific environment.

Scroll down to the bottom of the file and locate the following section:

1
2
3
4
5
6
7
8
9

  <!-- ======================= -->
   <!-- Setup the Root category -->
   <!-- ======================= -->
   <root>
      <priority value="INFO"/>  
         
      <appender-ref ref="CONSOLE"/>
      <appender-ref ref="FILE"/>          
   </root>

Next, add the following in that empty line between INFO and CONSOLE:

1

  <appender-ref ref="SYSLOG"/>

Enabling DEBUG mode

During overall server troubleshooting it is recommended to set the server and all the components in debug mode. This can be done by changing the log level in the three following sections.

In the section:

1

<appender class="org.jboss.logging.appender.RollingFileAppender" name="FILE">

Update this line as follows:

1

<param name="Threshold" value="DEBUG"/>

In the section:

1

<appender name="CONSOLE">

Update this line as follows:

1

<param name="Threshold" value="DEBUG"/>

In the section:

1

<!-- VMware vCO -->

Update this section as follows:

1
2
3

<category additivity="true" name="ch.dunes">
    <priority value="DEBUG"/>
</category>

Putting a vCO server in debug mode slow down considerably the server. It is recommended to use DEBUG mode temporarily.