Jeremy Goodrum from NetApp has released the NetApp OnCommand Workflow Automation (WFA) package for vCenter Orchestrator.

Also included are a How-to video for using the OnCommand Workflow Automation package for vCenter Orchestrator and a solution video for creating new Multi-Tenant vCloud Director environment using WFA and vCenter Orchestrator.

Find more about this here.

In this article (Which Virtual Hardware Versions (VM Compatibility) Are Supported In vSphere? ), William gave a simple example using a PERL script to retrieve the desired information for a given Cluster. I just came across this today, viewed the source of the .pl file and thought - hey that looks easy enough, I should be able to replicate the results in vCenter Orchestrator (vCO).

Prerequisites:

vCenter Orchestrator should have the vCenter plug-in correctly configured.

Test/Dev:

1. Create a new workflow named "Get Supported Virtual Hardware Version" and add a single Scriptable Task to it.
2. Add a single input named "cluster" of type "VC:ClusterComputeResource" to the workflow and bind it to the scriptable task
3. Add the following code to the scriptable task:
   {code}
   var virtualHWs = cluster.environmentBrowser.queryConfigOptionDescriptor();
   for each (vhw in virtualHWs){
       System.log(vhw.key + " -- " + vhw.description);
   }
   {/code}
4. (Optional) - Go to the Presentation tab of the workflow, select the input and add a new property: Show in Inventory -- this will allow you to right-click a Cluster in the vCO Inventory and run this workflow.
   
5. Save and Close the workflow
6. Run the workflow
7. Select the desired cluster you wish to run the workflow against, then hit Submit

Upon submitting the workflow, the logs tab should display something along these lines (Your results will vary depending on your environment):

[2013-05-03 14:05:24.984] [I] vmx-03 -- ESX 2.x virtual machine
[2013-05-03 14:05:24.985] [I] vmx-04 -- ESX 3.x virtual machine
[2013-05-03 14:05:24.986] [I] vmx-07 -- ESX/ESXi 4.x virtual machine
[2013-05-03 14:05:24.987] [I] vmx-08 -- ESXi 5.0 virtual machine
[2013-05-03 14:05:24.987] [I] vmx-09 -- ESXi 5.1 virtual machine

Attached is the solution workflow (Right Click and Choose Save As):
 Get Supported Virtual Hardware Version.workflow

Next Step for you: Take the code from this script and add it to an action. Action should take the same single input, but return an array of strings containing the vmx-## values. You would then be able to use this action as a source for future workflows that need the Hardware Version as an input. The action can be specified as the source for a "Predefined List of elements" property for the input.

The people working on vCenter Orchestrator have created a page for it on Facebook. If you have a facebook account you can visit the page and click on like.

vCenter Orchestrator 5.1 provided a lot of new great features at the workflow designer level and integration with the vSphere web client I have detailed in this article. Still there were a few things that were missing in the plug-ins installed with the server. It has been addressed in the 5.1.1 release.

IMAP client

A typical use case for integration with third parties applications is to trigger workflows based on email messages retrieved in a mail box.  With vCO 5.1 you were limited to the POP protocol. While this is a widely available protocol IMAP may be the standard protocol in your company.

vCO 5.1.1 (or 5.1 U1)  provides an IMAP client. As with the POP one the IMAP client is located in the NET plug-in. It provides low level scripting API for retrieving messages from mail server using IMAP protocol. Here are the properties and methods it supports:

In addition to make the management of emails easier the mail plug-in has been extended with several new objects that can be use either with the IMAP or POP client for:

• Retrieving messages
• Reading details of the retrieved messages as well as file attachments
• Searching messages
• Deleting messages

Here are the objects with their properties and methods.

And to make your workflow implementer life easier an example workflow called "Retrieve messages (via mailClient)" demonstrate the uses of these new objects.

Add new vCenter and ESX hosts using workflows

If you have installed vCenter Orchestrator as part of the Windows based vCenter installer it will be configured automatically with a single vCenter host to orchestrate. You can use the web-based configuration to add other hosts.

There are a few use cases for automating this process as a workflow. For example if you work for a large enterprise or a service provider you could automate the deployment of a new vCenter and make it manageable by a vCO server. A more frequent use case is the requirement to orchestrate directly an ESX host. This may be that it is not yet added to a vCenter server or that you need to access functionalities not available through vCenter. A functionality not available is changing the ESX root password. If you have hundreds of ESX servers you may not want to configure these by hand in the configuration interface, you may just create a workflow that for each ESX will add it to vCO configuration, change the password and remove it.

It also makes the vCenter plug-in consistent with the other ones such as vCloud Director, AMQP, REST and so on which allow to add hosts to the vCO inventory.

This functionality is available through the VcPlugin.registerVcenterServerInstance which despite the name should also work with ESX hosts and is also available in configuration workflows that have been added to the vCenter plug-in. They also handle importing the host certificate which is a required step.

REST authentication

vCO 5.1 brought us two interesting features with Single Sign On and the REST API web service. Authenticating with the REST API using basic authentication is very simple when vCO uses a LDAP directory service as authentication provider but is complex when using SSO. vCO 5.1.1 remediates this by allowing to authenticate with SSO with basic authentication.

Other things

There are other nice addition and fixes:

• I want my beans back : In a previous article I covered different ways to monitor vCO, including using JMX. Unfortunately the MBeans presented in vCO 4.X were disabled in 5.1. They are now back.
• If you had issue using the REST API with JSON (XML was fine) it is now fixed.
• You can now copy and paste all the workflow attributes, including the input and output ones.
• The zoom feature for the schema has been enhanced so it zooms on the selected element(s), it is in the contextual menu and has a shortcut.

Conclusion

While 5.1 was a game changer, vCO 5.1.1 really adds the polish with filling some gaps and with making the end user experience smoother. This is the first .1 update I have applied right away on my main vCO server ever.

For further information check the vCO 5.1.1 release notes.

You can find vCO 5.1.1 documentation here

You can download vCO 5.1.1 from this page. (The Windows version is included in vCenter otherwise you can download a separate appliance).

The only disadvantage of this amazing documentation set is the large number of pages, more than 500, which is the average size of a popular novel. Well, I have to admit that reading manuals might not be as amusing as reading novels and for sure, it might take a long time and patience.

If manuals are the “novels” in technical documentation, then quick references might be the “poetry”. Poems are shorter, easier to read, and they don’t tell complicated stories, exactly as the following three Quick References for Orchestrator:

 vCO Client Quick Reference

 vCO Configuration Quick Reference

 vCO Developer's Quick Reference

Dependencies

vCloud Automation Center (vCAC)

• Configured with at least one vCO Endpoint
• Configured with at least one Windows Template
• Design Center installed/configured

vCenter Orchestrator (vCO)

• Active Directory Plug-in for vCenter Orchestrator (tested with version 1.0.2 and 1.0.3 of the plug-in) Release Notes
  
• vCO Package file imported (FILE:  com.vmware.coe.ad.deletedisablecomputer.package)

Before getting started, we'll need to create a vCO workflow that gets triggered by vCAC. For the input, we'll just want the name and a true/false to indicate whether the computer account should be deleted or just disabled (default).

vCO Workflow - Delete or Disable ComputerAD by Name

The workflow shown above is contained in the attached package file. It consists of a custom action that gets an AD:ComputerAD object from the computer name string that is passed into it. It then, checks the "deleteComputer" input variable to see whether the workflow should Delete or simply Disable the computer account. Once the action is determined, it is logged and performed. In the case of an error during the action, System and Server logging takes place and an exception is thrown.

Here's a look at the content of the getComputerADbyName action:
{code}
var computers = ActiveDirectory.getComputerADRecursively(computerName);
//System.log("Computer count: "+computers.length);
for each (pc in computers){
    //System.log("Checking computer: "+pc.name);
    if (computerName.toLowerCase() == pc.name.toLowerCase()){
        //System.log("Found Computer: "+pc.name);
        return pc;
    }
}

// Nothing found so throw exception:
throw "No matching Computer found: " + computerName;
{/code}

As you can see, the script performs a recursive search of Active Directory to locate the computer account. It then checks each result's lowercase value to against the input computerName's lowercase value to ensure a match. Once the match is found, it is returned to the calling workflow. If no match is found, an exception is thrown and logged.

Modify vCAC Workflow Stub to execute vCO Workflow

You'll now need to open the vCAC Designer (Start -> Programs -> VMware -> vCAC Designer) so that the workflow stub can be modified. Since we want this particular workflow to execute upon decommission, we'll need to modify the WFStubMachineDisposing workflow. (You could optionally use the WFStubUnprovisionMachine instead of WFStubMachineDisposing)
After you Open the vCAC Designer:

1. Click on Load in the Menu bar
2. Select the WFStubMachineDisposing workflow.
3. Click OK

Double-Click "Machine Disposing"

Double-Click "Custom Code"

The first thing you'll need to get before you can call the vCO workflow is the Machine information. We'll need a variable to store the computerName we extract from the workflow:
At the bottom of the Custom Code Window:

1. Click Variables
2. Now Click the Create Variable line directly under the mgmtContext variable, and name the new variable computerName (This variable will be used later and it IS Case sensitive). Leave the Variable type as String and Scope as Custom Code

So, Drag the GetMachineName activity and position it below the Start

1. Now connect Start to GetMachineName by hovering over start until you see the little connectors show up, then click and drag down to GetMachineName
2. Double-click the GetMachineName activity and fill in the Machine Id and Machine Name as follows.

Fill in the boxes as follows:
Note: Case Sensitive

1. Machine Id to "virtualMachineId"
2. Machine Name = computerName
3. Now Click Custom Code at the top of the screen to return to the custom part of the workflow

1. Click and Drag the Invoke vCO Workflow activity onto the workflow and connect it to the GetMachineName activity
2. Click the elipsis button to select the Workflow

1. Start typing "Delete or Disable" in the top filter box
2. Select the Delete or Disable ComputerAD by Name workflow
3. Click OK

1. Enter the variable we created earlier into the computerName field... note that our variable name is also "computerName" and is case sensitive
2. Enter "True" for the delete Computer field. If you prefer to simply Disable the computer account in AD instead of Delete, then enter False in this box
3. Enter virtualMachineId in the VirtualMachineId proprties field in the right pane

Confirm that there are no Errors in the workflow. Errors will be indicated by a red exclamation point icon in the top right of workflow activities. Once you have confirmed there are no errors, click the Send button in the toolbar.

Enter a brief description in the box provided and click OK

Click OK on the Success message window

Configure vCAC to use the call-out to vCO

In order for the custom work to actually be used by vCAC we need to configure a custom property that tells it to enable one of the stubs. In our case, this is the MachineDisposing stub. You may refer to page 18 of the vCAC 5.1 Extensibility guide for a table listing each of the stubs and the corresponding property.

To enable this particular call-out, add the following property to a blueprint that is to be added to a domain:
ExternalWFStubs.MachineDisposing

Note: no value is required for the property above, the property simply has to exist and be bound to the blueprint in order for it to be used.

Now, go ahead and provision a VM using that blueprint and confirm that a domain account was created in the Computers container or whatever OU it was configured to be placed in. Once you have confirmed that an account exists, Use the vCAC interface to Destroy the computer. This should trigger the MachineDisposing stub that we have created, resulting in the AD account beling deleted (or disabled) depending on how you configured your workflow.

VMware education has updated the vCenter Orchestrator instructional videos with new items. See the complete list below.

These short instructional videos, between 5 and 20 minutes in length and presented by VMware Senior Technical Trainer Brian Watrous, cover the following topics:

• Introducing vCenter Orchestrator Overview : Introduction of the features and benefits of vCenter Orchestrator.
• Introducing vCenter Orchestrator Demo : This video demonstrates using a vCenter Orchestrator workflow to gather information.
• Installing and Configuring vCenter Orchestrator : This video covers the different ways to install and configure vCenter Orchestrator.
• Creating a Simple Workflow : Learn to develop a simple workflow using vCenter Orchestrator.
• Supplying Input Parameters to a Workflow : Learn how to supply input parameters to a workflow in vCenter Orchestrator.
• Understanding Variables and Variable Binding Overview : Learn how to set up a variable binding for a workflow in vCenter Orchestrator.
• Understanding Variables and Variable Binding Demo : Learn how to set up variable binding for a workflow in vCenter Orchestrator
• Configuring Input Presentation : Learn how to configure input presentations in vCenter Orchestrator.
• Invoking Actions : Learn how to invoke actions from within vCenter Orchestrator workflows.
• Creating Actions : Learn how to create actions within vCenter Orchestrator workflows
• Wrapping Workflows : Learn how to wrap one workflow within another in vCenter Orchestrator.
• Documenting Workflows : Learn how to document vCenter Orchestrator workflows.
• Integrating with vSphere Overview : Learn how to integrate vCenter Orchestrator workflow with vSphere Infrastructure.
• Integrating with vSphere Demo Part 1 : Learn to create a workflow to build vCenter inventory programmatically demo.
• Integrating with vSphere Demo Part 2 : Learn to create a workflow to build vCenter inventory programmatically demo

Pre-requisites:

• vCO 5.x Server installed and configured (using separate DB from vCenter)
  • vCO may be installed on the same server as vCenter as long as it is not using the same database (IE: Simple Install of vCO 5.1) as this configuration has issues with the JDBC configuration we'll be working with
  • vCO may be installed on a separate Windows server
  • The vCO Appliance may be used
• You must have the ability to download and install jar files to the vCO server (and set ownership/permissions)
• vCO must be configured for read/write access for a local directory (http://pubs.vmware.com/vsphere-51/topic/com.vmware.vsphere.vco_install_config.doc/GUID-7C2CFC22-519B-4D38-84F9-00380CEB15CF.html  )
• Sample Excel Spreadsheet (attached at end of article) - place the excel-vco-demo.xls file in a folder readable by the vCO service account. If using appliance, be sure to set ownership and permissions as outlined below.

Sample Package and Excel Spreadsheet:

excel-vco-demo.xls - the second worksheet (Build Info) looks like this:

com.vmware.coe.sqlsheet.package:

• Workflows (\PSO\Library\excel\)
  • Create Excel Example File
  • Process Excel vCO demo xls
  • Read complete worksheet from Excel file
  • Read certain columns of a worksheet from Excel file
  • Invoke Script readExcelVcoDemoXls
• Actions
  • com.vmware.coe.sqlsheet.getColumnNames
  • com.vmware.coe.sqlsheet.readDataForColumns
  • com.vmware.coe.demo.readExcelVcoDemoXls

JDBC Instructions

In order to get started, be sure that you have met the configuration requirements mentioned above in the Pre-Requisites section. Before getting started down this path, you should be aware of its limitations. Then you'll need to get some files and put them on your vCO server. Special thanks to Joerg Lew for his assistance in getting all the syntax correct in the core workflows here.

Limitations

Brief description from the project page:
Simple, POI based JDBC driver for XLS/XLSX files. Currently supports basic SELECT ( 'select *' only, w/o where clause), CREATE, INSERT operations. The driver is pure Java - no native Windows components are required. From version 6.1 streaming implemented to process large files.

Please refer to the documentation on http://code.google.com/p/sqlsheet/issues/

• The SQLSheet JDBC driver does not process WHERE clauses in SQL statements. Therefor you always get the complete table of a worksheet, and have to implement your own filter logic manually.
  The provided Action “com.vmware.coe.readDataForColumns” provides an example how to filter the result set of the SQL Select query only to return data for specified columns.
• It seems that the SQLSheet JDBC driver stops processing the Excel worksheet at the first empty row. So make sure all the data you want to read is stored in consecutive rows in the worksheet. (Example spreadsheet has a blank line after the tenth record to demonstrate that records 11-20 are not processed in the workflow)
• The SQLSheet JDBC driver expects the first rows containing the column names.

Download SQLSheet and Apache POI

This solution relies on the sqlSheet JDBC driver for MS Excel. The project page can be found here: http://code.google.com/p/sqlsheet/

The specific files to download are as follows:

• sqlsheet-6.3.jar and jsqlparser-0.7.1.jar from: http://code.google.com/p/sqlsheet/downloads/list
  
• poi-bin-3.8-20120326.zip (or .tar.gz) from http://archive.apache.org/dist/poi/release/bin/
  

Once the files have been downloaded, unzip the poi*.zip file. The archive file contains many different files. The only ones that are needed to support xls and xlsx file formats are:
NOTE: some of the file names listed below will have build dates in their filename

• poi-3.8.jar
• poi-ooxml-3.8.jar
• poi-ooxml-schemas-3.8.jar
• ooxml-lib/dom4j-1.6.1.jar
• ooxml-lib/stax-api-1.0.1.jar
• ooxml-lib/xmlbeans-2.3.0.jar

Install files and restart service

Now that you have all the jar files (sqlsheet, jsqlparser, and the 6 poi/ooxml files), they need to be copied to the vCO server.

All of these .jar files should be copied to the %vCO Install Directory%/app-server/server/vmo/lib/ext

Assuming default folders, the specific path would be:

• vCO Installed as part of vCenter install: c:\program files\vmware\infrastructure\orchestrator\app-server\server\vmo\lib\ext
• Standalone vCO Windows installation: c:\program files\vmware\orchestrator\app-server\server\vmo\lib\ext
• vCO Virtual Appliance: /opt/vmo/app-server/server/vmo/lib/ext

When using the vCO Appliance, make sure the new .jar files have proper permission settings and is owned by the vco user/group:

-rw-r-----

This permission may be set by executing the following two command in the folder: /opt/vmo/app-server/server/vmo/lib/ext

chown vco.vco *.jar
chmod 640 *.jar

Now that the files are in place with permissions set, restart the vCenter Orchestrator Server service (in some cases, WIndows based vCO servers may require a full reboot.

Install and use the example workflow package

Using the vCenter Orchestrator Client, import the provided package. The package will provide you with sample workflows and actions to help you get started reading/writing XLS files!

NOTE: Always make sure that the vCO Service has the proper permissions to read & write the Excel file - especially when using the vCO Appliance!

Example

Make sure the Excel file is in a location readable by the vco service account. For my test here, I have updated my js-io-rights.conf file to give permission to the /orchestrator/xls folder of my appliance and have placed the file there.

Run the "Process excel vco demo xls" workflow:

The expected results are shown in the "Logs" tab of the workflow execution and should look like this:

[2013-03-22 09:50:36.691] [I] Processed: null
[2013-03-22 09:50:36.693] [I] Hostname: VM-005
[2013-03-22 09:50:36.694] [I] Domain: demo.lab
[2013-03-22 09:50:36.695] [I] FQDN: VM-005.demo.lab
[2013-03-22 09:50:36.696] [I] IP Address: 192.168.1.105
[2013-03-22 09:50:36.697] [I] Memory (GB): 4
[2013-03-22 09:50:36.698] [I] CPU: 2
[2013-03-22 09:50:36.798] [I] Processed: null
[2013-03-22 09:50:36.799] [I] Hostname: VM-006
[2013-03-22 09:50:36.800] [I] Domain: demo.lab
[2013-03-22 09:50:36.802] [I] FQDN: VM-006.demo.lab
[2013-03-22 09:50:36.804] [I] IP Address: 192.168.1.106
[2013-03-22 09:50:36.805] [I] Memory (GB): 1
[2013-03-22 09:50:36.806] [I] CPU: 1
[2013-03-22 09:50:36.921] [I] Processed: null
[2013-03-22 09:50:36.921] [I] Hostname: VM-007
[2013-03-22 09:50:36.923] [I] Domain: demo.lab
[2013-03-22 09:50:36.924] [I] FQDN: VM-007.demo.lab
[2013-03-22 09:50:36.925] [I] IP Address: 192.168.1.107
[2013-03-22 09:50:36.927] [I] Memory (GB): 4
[2013-03-22 09:50:36.928] [I] CPU: 2
[2013-03-22 09:50:37.102] [I] Processed: null
[2013-03-22 09:50:37.104] [I] Hostname: VM-008
[2013-03-22 09:50:37.105] [I] Domain: demo.lab
[2013-03-22 09:50:37.106] [I] FQDN: VM-008.demo.lab
[2013-03-22 09:50:37.107] [I] IP Address: 192.168.1.108
[2013-03-22 09:50:37.109] [I] Memory (GB): 4
[2013-03-22 09:50:37.111] [I] CPU: 2
[2013-03-22 09:50:37.222] [I] Processed: null
[2013-03-22 09:50:37.223] [I] Hostname: VM-009
[2013-03-22 09:50:37.224] [I] Domain: demo.lab
[2013-03-22 09:50:37.225] [I] FQDN: VM-009.demo.lab
[2013-03-22 09:50:37.226] [I] IP Address: 192.168.1.109
[2013-03-22 09:50:37.227] [I] Memory (GB): 1
[2013-03-22 09:50:37.228] [I] CPU: 1
[2013-03-22 09:50:37.348] [I] Processed: null
[2013-03-22 09:50:37.349] [I] Hostname: VM-010
[2013-03-22 09:50:37.350] [I] Domain: demo.lab
[2013-03-22 09:50:37.352] [I] FQDN: VM-010.demo.lab
[2013-03-22 09:50:37.354] [I] IP Address: 192.168.1.110
[2013-03-22 09:50:37.355] [I] Memory (GB): 2
[2013-03-22 09:50:37.357] [I] CPU: 1

PowerShell Instructions

Please note that I have very limited experience with PowerShell so my guidance here will not be as detailed as the JDBC instructions above. I do want to include this information though as it looks like it would be a far better and more flexible approach to working with Microsoft's files.

If choosing this approach, be sure to have the PowerShell plug-in installed and configured with a PowerShell Host. The following links can help provide with guidance on getting this setup

PowerShell Plug-in documentation / vCO PowerShell plug-in blog post

Special thanks to my colleague Bill Call for pointing me in this direction.

For this solution, you'll need to install the Microsoft Access Database Engine 2010 Redistributable package on your PowerShell Host. Since I used my vCO Server also as the PowerShell host, I downloaded the 64-bit executable since my server is Windows 2008 R2 x64.

http://www.microsoft.com/en-us/download/details.aspx?id=13255

That package will allow PowerShell to make calls to the needed Providers for the following code to work. An idea of the .ps1 script I tested with is below. It was modified after being imported as an action to allow the workflow to have an input that specifies the Excel version as 2003 (Excel 8) or 2007+ (Excel 12):

{code brush="powershell"}#Change these settings as needed $filepath = 'C:\orchestrator\excel-vco-demo.xlsx' #Comment/Uncomment connection string based on version #Connection String for Excel 2007: $connString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=`"$filepath`";Extended Properties=`"Excel 12.0 Xml;HDR=YES`";" #Connection String for Excel 2003: #$connString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=`"$filepath`";Extended Properties=`"Excel 8.0;HDR=Yes;IMEX=1`";" $qry = 'select * from [Build Info$]' $DataTable = New-Object "System.Data.DataTable" $OleDbAdapter = New-Object System.Data.OleDb.OleDbDataAdapter $qry,$connString $RowsReturned = $OleDbAdapter.Fill($DataTable) ForEach ($DataRec in $DataTable) { Write-host "Hostname=$($DataRec.Hostname) Domain=$($DataRec.Domain) FQDN=$($DataRec.fqdn) IP Address=$($DataRec.'IP Address') Memory=$($DataRec.'Memory (GB)') CPU Count=$($DataRec.cpu) Processed=$($DataRec.processed)" }{/code}

Example

Run the "Invoke Script readExcelVcoDemoXls" workflow, select your PowerShell Host and click Next.
Fill in the form using the correct path to your copy of the excel-vco-demo.xlsx file

Here's te expected output in the Logs tab of the workflow execution:

[2013-03-22 16:10:28.933] [D] Invoke command in session fa9092e4-137c-4435-a14a-7d80d4760f57
[2013-03-22 16:10:30.460] [I] Hostname=VM-001 Domain=demo.lab FQDN=VM-001.demo.lab IP Address= Memory=2 CPU Count=1 Processed=Y
Hostname=VM-002 Domain=demo.lab FQDN=VM-002.demo.lab IP Address= Memory=2 CPU Count=1 Processed=Y
Hostname=VM-003 Domain=demo.lab FQDN=VM-003.demo.lab IP Address= Memory=2 CPU Count=1 Processed=Y
Hostname=VM-004 Domain=demo.lab FQDN=VM-004.demo.lab IP Address= Memory=4 CPU Count=2 Processed=Y
Hostname=VM-005 Domain=demo.lab FQDN=VM-005.demo.lab IP Address= Memory=4 CPU Count=2 Processed=Y
Hostname=VM-006 Domain=demo.lab FQDN=VM-006.demo.lab IP Address= Memory=1 CPU Count=1 Processed=
Hostname=VM-007 Domain=demo.lab FQDN=VM-007.demo.lab IP Address= Memory=4 CPU Count=2 Processed=
Hostname=VM-008 Domain=demo.lab FQDN=VM-008.demo.lab IP Address= Memory=4 CPU Count=2 Processed=
Hostname=VM-009 Domain=demo.lab FQDN=VM-009.demo.lab IP Address= Memory=1 CPU Count=1 Processed=
Hostname=VM-010 Domain=demo.lab FQDN=VM-010.demo.lab IP Address= Memory=2 CPU Count=1 Processed=
Hostname= Domain= FQDN= IP Address= Memory= CPU Count= Processed=
Hostname=VM-011 Domain=demo.lab FQDN=VM-011.demo.lab IP Address= Memory=2 CPU Count=1 Processed=
Hostname=VM-012 Domain=demo.lab FQDN=VM-012.demo.lab IP Address= Memory=2 CPU Count=1 Processed=
Hostname=VM-013 Domain=demo.lab FQDN=VM-013.demo.lab IP Address= Memory=4 CPU Count=2 Processed=
Hostname=VM-014 Domain=demo.lab FQDN=VM-014.demo.lab IP Address= Memory=4 CPU Count=2 Processed=
Hostname=VM-015 Domain=demo.lab FQDN=VM-015.demo.lab IP Address= Memory=4 CPU Count=2 Processed=
Hostname=VM-016 Domain=demo.lab FQDN=VM-016.demo.lab IP Address= Memory=1 CPU Count=1 Processed=
Hostname=VM-017 Domain=demo.lab FQDN=VM-017.demo.lab IP Address= Memory=1 CPU Count=1 Processed=
Hostname=VM-018 Domain=demo.lab FQDN=VM-018.demo.lab IP Address= Memory=1 CPU Count=1 Processed=
Hostname=VM-019 Domain=demo.lab FQDN=VM-019.demo.lab IP Address= Memory=2 CPU Count=1 Processed=
Hostname=VM-020 Domain=demo.lab FQDN=VM-020.demo.lab IP Address= Memory=4 CPU Count=2 Processed=

Sample Files:
 com.vmware.coe.excel.package
 excel-vco-demo.xlsx

I have written a solution to do it a long time ago but never had the time to release it publicly. It is now done and here are the features.

Features

This package comes with a rich set of features.

Catalog comparison and replication

Replicate  a catalog from one source catalog to a destination organization. If a  catalog with the same name exists in the destination it will be modified  to have the same content as in the source catalog. If not a new catalog  with the same name will be created. The replication is one way (source  to destination).
The actions taken are:

• Delete catalog items in destination not present in source
• Update name and description of previously copied catalog items that would have been changed in the destination
• Copy new source catalog item to destination (download once to vCO, upload to one or more vCD)

Each  of these are optional. Destination VDC is also optional. If  provided  all the elements to be copied will have for destination the  provided  vDC. If not the workflow will find destination VDCs with  matching  names. If not the workflow will raise an exception.

Connected replication

When  using the "Replicate catalog" workflow a single vCO server is used to  download  the vApp Templates from the vCloud Director hosting the source  vApp  Templates and upload these to the other vCloud Director. This  requires  to have the two vCloud hosts configured in vCO.

vCloud Director source -----------> vCO -----------> vCloud Director Destination

While  the vCO server could be located on any network having access to both  vCloud servers the vCO server should be located to optimize the network  speed for example if the upload speed between the source and destination  is a lot slower then the download speed the vCO server should be on the  same network as the destination.

Several instances of  the same workflow can run on the same vCO server  with different  destinations (there is a locking system to avoid uploading a file being  downloaded). Below an example of a vCO server replicating a catalog from  one cloud source to two cloud destinations.

Replication from filesystem

When  using the "Export catalog to filesystem" a first vCO server is used to  download the vApp Templates from the vCloud Director hosting the source  vApp  Templates. XML files describing the catalog and the catalog items  are also downloaded.

You can then use the "Replicate catalog from  filesystem" workflow to compare the file based catalog to the one on the  destination server.

The main idea behind this two  steps replication is to optimize the replication process by using a  third party replication using deduplication, compression, safe  transfers.

vCloud Director source -> vCO 1 ->  Third party replication system source -----------> Third party  replication system destination -> vCO 2 -> vCloud Director  Destination

A first vCO server is located on the same  network as the source vCloud Director to download the vAppTemplates as  fast as possible to a filesystem hosted on a replication system (this  requires having this filesystem accessible from vCO) . The replication  system copies the changed blocks to the destination replication system.  Once the replication finished the destination vCO upload the new files  to the vCloud DIrector on the same network.

Parallel operations

The  workflows use and abuse asynchronous sub workflows to make several  operations running at the same time. and also copies the VMDK files  within a vApp Templates in parallel. The level of parallelism is  configurable.

The "Replicate catalog" workflow ran.

It started one to many "Copy catalog item" workflows in parallel

The "Copy catalog item" ran a single "Copy vApp Template multithreade" workflow

Which in turn ran one to many "Copy file across vAppTemplates" workflows in parallel

Which in turn runs:

• "Download  single file from vApp Template" workflow (not shown on the picture  since in this case it was already downloaded previously and the workflow  optimize the process with not downloading the vApp Template again)
• Upload  single file to vApp Template which will run several "Upload File chink  to vAppTemplate" in serial (parallel upload of the chunks is not  supported, they need to arrive in the right order).

Safer uploads

Sending  over a multi GB file over http may cause reliability issues. If the  upload fails you have to restart again. To avoid this the workflows  uploads the files in chunks of a configurable size. If a chunk fails to  be uploaded the workflow retries to upload it hence avoiding the need to  restart completely. The chunk size is configurable.

Here  I have run the same replication as before. At 14:35 I have unpluged my  vCO server network connection to make the file chunk being uploaded  failed. I have then plugged it again to simulate a temporary network  issue. As you can see below the upload was resumed. The file chunk that  was uploaded starting at 14:36:32 seconds was the same as the previous  one (this can be checked by clicking on the workflow run and check the  variables tab) . This is why we had 11 chunks uploaded in the previous  replication and 12 in this one.

The  other advantage is since vCO is an orchestration engine supporting  checkpointing if the replication workflow would be stopped because of  vCO server maintenance (i.e Windows update rebooting) the workflow would  resume from the point where it uploaded the last chunk (as long as the  maintenance time is not longer then the timeout vCloud Director has for  waiting on the vApp Template update which is one hour according to my  observation).

vApp Template Metadata

The  vApp Templates metadatas are copied as part of the catalog replication  (connected or from filesystem). If the metadatas are copied from a  vCloud Host using a System admin organization and credentials to a host  connection using a specific organization then the hidden and read only  metadata will be created as read / write (this is the only option when  connecting as org admin).

Not included

There are a lot of things that  are doable and other that were not because of lack of API support in  vCLoud Director or lack of time to add more features:

• Does not replicate medias : There is no API yet to download a media so no replication possible.
• Does  not reconnect the vApp Template to the networks of the target cloud.  This requires deploying the template, delete it from the catalog match  the network to the ones in th e new environment, capture the modified  vApp as a vApp Template. You can do this manually (one time operation)  or create your workflow to do it (there is something similar already  implemented in the mass VM import workflow in communities). The updated  vAppTemplate will not be considered as a new one and will not be deleted  / overwritten by the replication mechanism.
• Does not clean up the downloaded files. A separate workflow (scheduled or run by policy) could do this.
• The update of the catalog item does not update metadatas. I will consider adding it if this is something people ask for.

Documentation

Import the replication package using the vCO client.

The  "Catalog Replication Settings" saves all the configuration in a  configuration element. Open either the vSphere Web client or the vCO  client and search for the "Catalog Replication Settings" workflow.

Catalog Replication Settings

Click on it, then right click / "Run a workflow".

General configuration

The first step is to define a path where vCO will download the vApp Templates. The pre-requesites are:

• There  is enough disk space in this path (if you use the appliance you will  have to mount an NFS share or create an additional drive)
• You have set proper share / file permissions
• The user running the vCO service has access to this path (by default this is not the case if this is a network share)
• You have changed the vCO access rights as documented here.

Replicate a catalog settings

The  next step is for configuring the vApp Template transfer. This are the  settings used when running the "Replicate a Catalog" workflow. You can  set the number of parallel copies (one copy = 1 download + 1 upload) and  a time out. When timing out the copy will be canceled. This is to avoid  a catalog replication to ait forever on a copy that would be stalled.

The  next setting is to set the number of parallel copies within a single  vApp Template. For example if a vApp Template has several VMs with  several disks these can be copied concurently.

Export a catalog to a filesystem

These  settings apply for the "Export a catalog to a filesystem" workflow.  These are similar settings as above except it has an option to keep some  vApp Template settings during the (lossless) download. If this option  is selected it is not possible to download files concurently within a  vApp Template.

If the lossless option was not selected it is possible as before to provide the number of concurent downloads.

Replicate a catalog from the filesystem

The same settings as above but this time when uploading the vApp Templates.

And  the same settings for the the vApp template file. The last one is for  specifying the size of each chunk when uploading. This allows to have  safer uploads that can be resumed in case of a network issue or vCO host  maintenance.

Operations

Now that the replication configuration is done it is possible to start the replication workflows.

Replicating a catalog using a single vCO server

To replicate a catalog using a single vCO orchestrating a source and destination cloud run the "Replicate a catalog" workflow.

The first parameter is the source catalog.

The  next step is to provide a destination organization and a destination  VDC. If the catalog does not exist in the destination organization it is  created. If it exists it will be updated. A one time full copy will be  required before the workflow can snychronize the changes.

The  next step defines what are the changes to apply. Beware that if you  already have catalog items in the destination catalog the workflow will  delete these on the first run and then copying the ones from the source  catalog if the copy and delete options are selected.

You  have the option to run the synchonization now or to Schedule it.  Scheduling a recurrent task is a good way to automate the  synchronization. Another way is to trigger the workflow using the vCloud  Director notification on catalog changes.

The workflow will  syncronize the catalog.

Replicating a catalog to and from the filesystem

To  optimize the replication speed / bandwith usage you may want to use a  third party system leveraging deduplication, compression and other  mechanisms. To do this you need to run the "Export a catalog to a  filesystem" workflow on the vCO server local to your source vCloud  Director. You need to provide the catalog that you want to replicate.

Then you need to provide the export path. As a default it will use the one you have configured previously.

As before you can run the workflow immediately or schedule it.

Once the export completed and your third party replication is completed.

Run  "Replicate a catalog from the filesystem" on the  vCO server local to  your destination cloud. This vCO server must have access to a copy of  the files that were exported. Provide the full path to the catalog file  (The catalog file name is the catalog name + ".xml").

Select the destination organization and VDC.

Select the synchronization options.

And as always you can run it right away or schedule it.

Download

You can find the package on the vCenter Orchestrator Communities

vCloud Director 5.1 introduced a new feature called API extensions which basically allows a cloud provider to extend the vCloud Director API with custom services with leveraging VMware or third party applications. In this article I discuss why someone would want to leverage this feature and explain the "service builder" solution I have created around it.

The first question you may ask is "Why would you want to extend vCloud Director API ?"

As the vCloud Director Administrator I need to provide a feature not available in the vCloud API to cloud API consumers (i.e. the portal development team)

• It is a combination of several vCloud API calls, and or API calls with predetermined parameters I can define
• It is included in another VMware or third party API
• It is a mix of both

OK, but then why not use the existing APIs instead of extending vCloud Director's one ?

Lets see some of the common portal / cloud implementations.

The first one is the old fashioned portal calling the vCloud and other APIs. Here are two variations exposing or not the vCloud API. Exposing the vCloud API allows to leverage all the applications consuming the vCloud API. For example a service provider will want his tenants to use vCloud Connector to copy back & forth vApps from / to vSphere / vCloud. This design is IMHO flawed in several aspects:

The portal must include all the automation you would typically put in an orchestration solution. This means re-inventing the wheel to re-create a lot of facilities offered by an orchestration engine, making these operation only usable by the portal or creating a custom web service when thousands of customers already used standard orchestration APIs. This will be a solution difficult to support that will be tied with the scripting language used to consume the underlying cloud and third parties APIs. These APIs will support different types of web services, different type of authentications.

- Please note -  I have used a vCloud Automation Center picture to represent the portal. This could be any portal. vCloud Automation Center is not only a portal but also a workflow engine.

The second approach is that orchestration rules them all ! The portal calls in the orchestrator calling the underlying systems. The two designs below are using this approach with keeping the orchestration API private and either exposing the vCloud REST API or not. The Orchestrator APi could ahve been made public to expose the additional services but there are some serious workflows / resources access control considerations required.

This approach was a good one when there was no cloud API and that the target system was vSphere / vCenter. The Orchestrator would expose a subset of the functionality with using several API calls chained together to perform the desired automation. Using orchestration has several benefits including supportability, modularity, use of hundreds of supported workflows, API abstraction, caching, high availability and many other things I detail in this article.

However this has a few flaws as well. An orchestration engine is meant to automate complex operations. A lot of the API calls started from the portal will be about almost real time operations such as refreshing a list of vApps, presenting information in a panel and so on. I have written about this in details in this article : Building your custom cloud portal - Knowing when to use the vCloud API and the vCenter Orchestrator web service

Several successful cloud implementation involving using custom services I have seen have implemented the Cloud & orchestration design below in the left. While this resolve some of the issues mentioned previously it still make the custom services only available to the portal. With vCloud Director 1.5 blocking tasks and notifications allowed to customize existing vCloud Director operations, hence diminishing the number of calls to be made from the portal to the orchestration engine but direct calls to orchestration are most often required for operations required that cannot be started by vCloud blocking tasks and notifications. With vCloud Director 5.1 you can use the bottom right design.

The reasons for prefering this design on the right are the following: As a cloud administrator I do not want to give direct access to my cloud backend applications:

• Access: I provide only one single point of access : the vCloud Director API and I want to publish as a service only a given operation, not give access to all the API of any of my cloud back-end components.
• Security: The only authentication, session management is the vCloud one.
• Consistency : My customers only use the vCloud API which reduces their development effort (avoid using different web services and different authentications)

Giving API access for the functionality exposed in the portal is also an important differentiator between different cloud providers. This provides ways to develop client applications for specific uses, build automation. At VMware API parity with functionality exposed in the UI is an important goal for engineering for these reasons.

OK but what for ?

There are several use cases for providing functionality that is not out of the box for example:

• vApp custom provisioning or decommissioning
• vApp backup
• vApp load balancing
• vApp fault tolerance
• vApp updates
• vApp VMs security hardening
• vApp VMs affinity / anti-affinity rules
• vShield security groups
• Install agents, inject parameters in vApp VMs
• Database as a service
• Update user password in Active Directory
• iSCSI target for MSCS Clusters

A lot of these are achievable with the components of the vCloud Suite. There are a lot more use cases with integrating third party systems such as management systems. You can "cloudify" the whole datacenter if needed.

How ?

My colleagues at VMware have done an excellent job exploring the API documentation and explaining vCloud API extensions. Here are some references I have used to create "service builder". If you are not affraid of going deep technical I definitely recommend going through these:

• Blog articles written by Christopher Knowles:

• Introduction to vCloud API extensions
  
• Registering an API extension service
  
• Registering API Service Links
  
• Storage as a Service Extension Overview
  

• Blog articles written by Thomas Kraus:vCloud Director API Extensions (implementation example)

• VMworld 2013 session OPS-CSM1379

I do not have the time to invest a lot of time in these articles and create my services manually. How can I get started quickly ?

With using service builder. Service Builder is:

• A wizard based workflow to create a custom service and service operations (called service links in the vCloud extensibility API)
• Expose workflow as service operations (/api/myservice/myOperation)
• Expose this workflow as well in the "Workflow run service" so they can be managed (/api/workflow/workflowID/executionId)

In addition service builder creates a "workflow run service" that allows vCloud Director API to manage vCenter Orchestrator workflows.

And I forgot to mention it is free.

How do I get started with service builder  ?

I have put together an online presentation explaining in details :

• What is "vCloud service builder" and "vCloud workflow run service"
• Pre-requisites
• How it works animations
• Installation
• Usage
• Examples for calling the newly created APIs
• Custom operations monitoring with vCloud Director UI
• Troubleshooting
• Download link

Note : The presentation requires your Email and organization. This is solely for me to better understand who has interest in this tool and eventually to contact you in case I need further information on your submitted comments. This information will not be used for anything else and not be transmitted.

If you want to go directly to the download page you can find it here.

VMware education has released a series of vCenter Orchestrator instructional videos.

These short instructional videos, between 5 and 20 minutes in length and presented by VMware Senior Technical Trainer Brian Watrous, cover the following topics:

• Introducing vCenter Orchestrator Overview : Introduction of the features and benefits of vCenter Orchestrator.
• Introducing vCenter Orchestrator Demo : This video demonstrates using a vCenter Orchestrator workflow to gather information.
• Installing and Configuring vCenter Orchestrator : This video covers the different ways to install and configure vCenter Orchestrator.
• Creating a Simple Workflow : Learn to develop a simple workflow using vCenter Orchestrator.
• Supplying Input Parameters to a Workflow : Learn how to supply input parameters to a workflow in vCenter Orchestrator.
• Understanding Variables and Variable Binding Overview : Learn how to set up a variable binding for a workflow in vCenter Orchestrator.
• Understanding Variables and Variable Binding Demo : Learn how to set up variable binding for a workflow in vCenter Orchestrator
• Configuring Input Presentation : Learn how to configure input presentations in vCenter Orchestrator.
• Invoking Actions : Learn how to invoke actions from within vCenter Orchestrator workflows.
• Creating Actions : Learn how to create actions within vCenter Orchestrator workflows
• Wrapping Workflows : Learn how to wrap one workflow within another in vCenter Orchestrator.
• Documenting Workflows : Learn how to document vCenter Orchestrator workflows.
• Integrating with vSphere Overview : Learn how to integrate vCenter Orchestrator workflow with vSphere Infrastructure.
• Integrating with vSphere Demo Part 1 : Learn to create a workflow to build vCenter inventory programmatically demo.
• Integrating with vSphere Demo Part 2 : Learn to create a workflow to build vCenter inventory programmatically demo

First thing is there is no out of the box workflow that download a lossless vApp Template but there is one to download a vApp Template. On the top right window search for "Download a vApp template"

This will find a single Workflow. Select it.

Now that we have the original workflow wa re going to duplicate it (Right Click / Duplicate workflow)

Enter a meaning less new name and a workflow folder.

Right click / Edit the workflow. Go on the schema tab. The action that performs the download is at the bottom right corner. Right Click / Change Action.

Since the 5.1 plug-in all the actions are automatically generated from the API. This means that if a workflow is missing in the base library you can build your own using actions. Enter downloadLosslessOvf in the search field and select.

Save your workflwow and you are done. You now have a workflow to download loss less vApp Templates.

The vCloud DIrector plug-in has built in workflows for managing custom properties on vApp and VMs (storing these in the productSection if the OVF) but these are not the same as the metadatas. The metadatas can be applied to a lot more objects such as VDC, storage profile, catalog, catalog item, disk, Org VDC network, media, external network).

This article shows how to use these in vCloud Director 1.5 and in 5.1.

The functionality we will be looking for is simple: an action to set a metadata string entry on an objet and one to get it back.

Inputs for the setMetadataStringEntry action:

• object of type "any"
• key "string"
• value "string"

Inputs for the getMetadataStringEntry action:

• object of type "any"
• key "string"

vCloud Director 1.5.

vCloud Director 1.5 has a very simple metadata API limited to string values

setMetadataEntry action

{code; class="auto-links:false"}var metadata = object.getMetadata();
return metadata.updateEntry(key, value);{/code}

Which returns a vCloud task.

getMetadataEntry action

{code; class="auto-links:false"}var metadata = object.getMetadata();
var entries = metadata.getEntries();
return entries.get(key);{/code}

Which returns the value if present.

vCloud Director 5.1

vCloud Director has extended the metada with typed entries (number, string, boolean, date) and also with access control. You can read all the details in the vCloud API programming guide.

All these new features come with a drawback : the API has changed and the code above will not work. This is what happened to me and I thought it would be worth sharing how I adapted it to work with vCloud Director 5.1.

We will assume we still want to use a string as the value. If you need one of the other type it will be easy to adapt.

setMetadataStringEntry action

{code; class="auto-links:false"}

var metadata = object.getMetadata();

var vclMetadataStringValue = new VclMetadataStringValue() ;
vclMetadataStringValue.value = value;

var vclAbstractValueObject = new VclAbstractValueObject();
vclAbstractValueObject.setValue(vclMetadataStringValue);

var vclMetadataEntry = new VclMetadataEntry();
vclMetadataEntry.key = key;
vclMetadataEntry.typedValue = vclAbstractValueObject;

System.log("Setting key " + key + " with value " + value);
return metadata.updateTypedEntry(vclMetadataEntry);

{/code}

Which returns a vCloud task.

getMetadataStringEntry action

{code; class="auto-links:false"}var metadata = object.getMetadata(); //All objects that support metadata return something, even if there are no metada yet

var metadataEntries = metadata.getTypedEntries().enumerate();
var value = null;
for each (var metadataEntry in metadataEntries) {
    if (metadataEntry.key == key) {
        value = metadataEntry.typedValue.getValue(new VclMetadataStringValue).value;
        System.log("Got key : " + key + " with value : " + value);
        break;
    }   
}

return value;{/code}

Which returns the value if present.

Testing the actions

To test these actions you can create a workflow with as input an object (for example a vAppTemplate), a key and a value. You can either drag and drop the two actions or create a workflow per action. As long as you set the metadata before you get it it should work. Do not forget to link the inputs and create the attributes. Also do not forget to add a "Wait for a task" workflow right after setting the metadata. It should look like this:

The output of the getMetadata action should of course return the same of what you entered in the workflow input.

Package

The actions above are available as an instllable package:

• Package containing the two actions for vCloud Director 1.5
• Package containing the two actions for vCloud Director 5.1

If you were at VMworld, during Steve Herrod (VMware CTO) session you could answer in real time to some questions. One of them was Over the next 3 years, how do you plan to manage your hardware infrastructure for your VMware environment ? Primarily from:

• vCenter : 62%
• A converged hardware management tool: 32 %
• Existing element management tools: 6%

It is a good thing most of you want to, it is even better that the vSphere 5.1 Web client enables you to manage pretty much anything you want (and not just physical). Here are a few examples.

vSphere Web client has a great integration with vCenter Orchestrator and since vCenter Orchestrator can manage a lot of your VMware and non VMware cloud, virtual and physical infrastructure and any kind of applications supporting a web service this makes the Web client extensible. This is a lot different than the fixed set of functionality you get when a third party creates vSphere Web plug-in: now you can add the functionality you need either as a standard single step operation or as a multiple steps, automated, multi integration workflow. No need to wait for a vendor commitment on providing a given feature and once you have them still have to spend your day nagivating in each vendor user interface within vSphere web client.

Using vSphere Web Client to run out of the box VMware Workflows

For example if I want to add a catalog to my vCloud Organization from the vSphere web client I woud follow these steps:

On the home page I just type "add a catalog" in the search field and the available workflows show up. I click on the workflow I want and this brings me to the next page.

Here I have all I need to run and monitor this workflow. I right click and run it. Note that I could also have scheduled it.

The workflow is presented with the same wizard based presentation as multi-steps vCenter operations.

When selecting my organization I can choose it in a regular tree view OR I can use the filter tab which will show the list of all the organizations in all the clouds I have. I really love this filter feature as it avoids a lot of drilling into the tree view.

Filling the last mandatory input and hit finish and I am done without opening another browser tab, authenticating in vCloud Director, navigate to the right tab and go through a similar process.

vSphere web client will notify me this was successful.

Something that is not visible on these screenshot is the single sign on feature. I authenticated once in the vSphere client and I am automatically authenticated in vCO and if I configured vCloud director for single sing on as well I can be automatically authenticated in vCloud Director as well (If vcloud Director is not setup for single sign on then vCo will use the user configured for the vCloud Shared session).

OK I have demonstrated I can use the vSphere Web client to operate vCloud with the hundreds of out of the box workflows included in each vCO plug-in. Now I can do something more exotic.

Using vSphere Web Client to run out of the box Partners Workflows

VMware is not the only one writing the vCO plug-ins. Several partners do as well.

Need to drive your F5 load balancers from vSphere ? Let's add a member to a local load balancer pool.

.

I select a pool in the tree view.

Hit finish.

Done. I did not have to open a new browser window, type in the hostname of the F5, authenticate, navigate through the menus, add the member. The other good side of this is that I am able to do what has been defined in the workflow that has been made available to me in vSphere and I cannot mess up the configuration of the load balancer.

Using vSphere Web Client to run custom Workflows

Since by definition a good system admin is someone who does something manually once, twice, and automate it the third time I may want further integration / automation. Typing an IP in a field is not the most exiting thing to do and it may be a source of errors.

By combining a workflow that returns the IP of a VM and the F5 one I can add a VM IP to a load balancer pool just by selecting the VM. Since the vSphere Web client can be extended with any custom workflow and start them in context.

Another example is the following.:

I need to migrate some of my VMs from being managed by vCenter to be managed by vCloud Director. I can do this with the vCenter and vCD User Interface, a notepad, one at a time and a lot of time and possible errors. The major steps are:

• Take a note on which network the VM is connected.
• Shut it down.
• If the network is not in vCloud Director create an external network and an organization vDC network connected to this external network
• Import the VM
• Reconnect it to the Org VDC network

Or I can run a workflow I wrote that does it (and can import VMs in parallel) just like this:

The possibilities are endless. You can combine the out of the box thousands existing plug-ins workflows from VMware (vCenter, vCloud, Active Directory, Chargeback, Powershell, Configuration manager, AMQP, update manager, SQL, Cisco UCS, auto-deploy) and partners (Infoblox , F5, EMC, Radware, uptime) and a lot more in the pipeline. Vendors can use the newly released plug-in SDK which make it a lot easier to create a plug-in. If there is no specific vCO plug-in you can use the REST / SOAP / Powershell generic plug-ins (Here is an example to orchestrate twitter).

The VMware vCloud suite was made available a few hours ago. While most of its components have been bumped only by a dot one it supports a LOT of new features including two major items :

• The new vSphere Web client : The single pane of glass user interface on which you can extend functionality on your own by running your own custom workflows automating and integrating VMware and third party applications.
• The new vCenter Orchestrator platform : The Nirvana of cloud Orchestration. Did I mention it is coming free of charge with vCenter Server ?

If this is the right time for you to try these check this out ...

First things first vSphere 5.1 is the core of your cloud. You can get it here. If you install it on Windows it will automatically configure vCenter Orchestrator so you can use it OOTB from the vSphere web client.

Now if you prefer to use the vCenter Virtual Appliance and vCenter Orchestrator Virtual Appliance you can click on the picture below linking to the following guide which gives some tips on their integration.

VMware has just made available its Orchestration plug-in leveraging vCloud Director 5.1 latest and greatest features. This is the most complete automation solution for vCloud Director and is a free download !

To put things in perspective the 1.0 version of the plug-in was released months after vCloud Director 1.0, the 1.5 release weeks after vCloud Director 1.5, the 5.1 version is released together with vCloud Director 5.1. Kudos to the VMware engineering team who have automated the plug-in build process so much that the plug-in could accomodate the changes in pre-release vCloud Director version so quickly.

Please note that the vCenter Orchestrator Plug-in for vCloud Director requires using vCloud Director 5.1 and vCenter Orchestrator 5.1. vCO 4.X is not supported since this plug-in takes advantage of some specific vCO 5.1 functionality. Taking advantage of the new vCloud Suite makes a lot of sense since its components have a high level of integration bringing a lot of new features such as being able to run your custom workflows (including vCloud) from the vSphere web client (i.e Right click on a resource pool / Import these VMs in vCloud Director). If your vCenter Orchestrator server is already running the 1.5 plug-in it is recommended to perform a new installation of vCO 5.1 and the vCD 5.1 plug-in instead of performing an upgrade. If you wrote custom workflows that you run in production a vCloud 1.5 to 5.1 workflow compatibility guide will be published soon to validate these in a 5.1 environment. To make it clear: Do not upgrade your production vCO from the 1.5 to the 5.1 plug-in.

If you have vCenter Orchestrator 4.1.x or 4.2.x and vCloud Director plug-in 1.x, and you upgrade to Orchestrator 5.1 and vCloud Director plug-in 5.1, some actions and workflows from the plug-in's previous version might remain in the library, but cannot work with version 5.1.

Here are the list of new features and improvements:

• Objects / methods of the plug-in, now auto generated and cover the entire 5.1 API.
• Actions are also auto generated so you should be able to author automation workflows without using a single line of script.
• Library workflows ported to 5.1. 40 new workflows added to the library.
• API documentation has been improved a lot.
• SSO Authentification: Authenticate using SSO in vCO or in vSphere Web Client and the credentials are used to authenticate in vCLoud DIrector as well.
• Error messages are more verbose.
• Stability and performance improvements.

Links:

• vCenter Orchestrator Plug-in for vCloud Director 5.1 release notes.
• vCenter Orchestrator Plug-in for vCloud Director 5.1 documentation.
• vCenter Orchestrator Plug-in for vCloud Director 5.1 download.

As part of the Cloud Infrastructure and Management Center of Excellence Team who worked on this project we are glad to announce the release of the new vCloud Architecture Toolkit. Amongst many enhancements, this version covers vCenter Orchestrator best practices that have been proven in real-world enterprise and service provider cloud deployments. It also includes a Workflow examples implementation guide and its workflows.

Quoting the vCAT page:

The VMware vCloud Architecture Toolkit (vCAT) is a set of reference documents designed to help you construct a superior working cloud solution leveraging current technologies, recommended practices and innovative tools that have been proven in real-world enterprises. Written by VMware cloud architects and experts, vCAT provides enterprise IT managers and architects recommended design and support solutions attested, validated and optimized – they represent the most efficient examples and help you make the right choices for your business.

We have expanded the scope across the current 700+ pages to include the following sections:

Core Documents:

• Introduction
• VMware vCloud Service Definitions
• Architecting a VMware vCloud
• Operating a VMware vCloud
• Consuming a VMware vCloud
• Implementation Examples

Add-ons and Tools:

• Workflow Examples
• Software Tools
• Cloud Bursting

The workflows mentioned in the Workflow examples section are located in the Orchestrator community

To learn more, and download the Toolkit, visit the vCAT Toolkit page.

VMware released the most important upgrade of vCenter Orchestrator. vCenter Orchestrator 5.1 is providing important enhancements geared towards the end user.

vCenter Orchestrator, formely VMO (VMware Orchestrator) and (VS-O) Dunes Virtual Service Orchestrator) was packaged as part of vCenter Server since vSphere4.

As part of this was a rebranding effort and an exhaustive vCenter plug-in synched with the vCenter server release. A year later the 4.1 version brought in scalability with moving to a 64 Bits JVM, performance improvements and reliabilty improvements.

The next 4.X releases focused on stabilizing the platform further with bug fixing, security enhancements and small improvements.  A major effort was done in creating 13 new plug-in adapters to orchestrate a lot more VMware and non VMware applications and the release of the vCenter Orchestrator Virtual appliance improved a lot the configuration experience for first time users.

While all of these improvement in the platform and adapters were necessary the end user experience did not change much. The newly released vCO 5.1 is the most dramatic update in terms of features and end user improvements. This article covers some of these changes.

Create your own workflows in a completely revisited studio

The vCenter Orchestrator client has recieved a lot of additional features / tabs going from version 1 to version 3. it is not as complex and as complete as a typical development studio such as eclipse but it is a lot more end user friendly as these and as the 4.X version. With the 5.1 release the user interface got a newer look and feel consistent with the new generation VMware products but has also been reorganized for better end user experience.

vCO 4.1

vCO 5.1

As you can see on these screen shots the 5.1 version is making better use of the screen space.

Views and reorganized tabs

The 5.1 client use different views corresponding to the different roles. Each view has a set of icons

• Run : My Orchestrator, Scheduler, Policies, Workflows, Inventory 

• Design :Workflows, Actions, Resources, Configurations, Packages, Inventory

• Administer : Inventory, Policy Templates, Authorizations, Web views, Packages

Selecting an icon list the relevant objects as a tree view or list on the full height of the left pane compared to the vCO 4.X fractioned one. This is saving a lot of the scrolling.

New icon set

Another update beside the look of the user interface is the update of the icons used for the palette elements that you can drag and drop on your workflow schema. vCO 4.X was quite minimalist: most of the items were boxes resembling to each other.

vCO 4.1

The same workflow in vCO 5.1

Also you can now associate any custom icon to your workflow.

New palette elements

Having nice icons is good, having new workflow elements is IMHO better.

One of the common workflow is to first call another workflow and depending on its output branch the flow. In vCO 4.X this requires using an attribute linked to the output of the workflow followed by a decision element.

In vCO 5.x there is a new elment callled a "Decision activity". A decision activity combines a workflow and branching depending on the workflow results. This saves an element and an attribute and a few clciks.

Another common use of the elements is to build workflow loops to iterate an operation from an array of objects. Workflow loops are not the easiest thing to do when you begin with vCO but it is often a first use case. So much I did a tutorial about it here that got more than 3000 views.

A typical vCO 4.X Workflow loop looks like this. It requires many steps and using attributes to set the current object the index of the loop. The beginner may forgot one of these steps or not set the decision properly.

With the vCO 5.1 foreach element this tutorial consists on drag and dropping a foreach element on the schema, choosing the "Create a snapshot" workflow and fill the promotion parameters like follows (input promotion is covered next).

The important thing to see on this screenshot is the first column. I have highlighted the iterator on the vm input so the foreach element will iterate through an array of VMs. Here is the result.

The new palette elements are powerful and save a lof of time. They have a single down side: if you include these in your workflows these will not load properly on previous versions of vCO.

Input Promotion

Most of the time when you create a workflow you use other workflows and actions as building blocks of yours. Each of these workflow and action inputs and outputs need to be bound to your workflow inputs, attributes and outputs. This can take a lot of clicks,

With vCO 5.1 all these clicks can be saved. When adding a workflow or action to your workflow this banner appears:

Clicking on the setup is popping up the Promote Workflow Input / Output

In this single window you can quickly set the bindings for your workflow including setting fixed values. Once promoting the input and attributes that do not exist will be automatically created.

Automatic linking

With using vCO 4.x, you drag and drop your elements on the schema then link the elements in order as  a process flow. With using vCO 5.1 a new workflow has already a Start and  End elements linked.

With drag and dropping palettte elements on the link these are automatically added with moving the other elements kepping consistent space between them. For example just by drag and dropping a scriptable task and a Custom decision on the blue link you get this workflow.

Popup windows

vCO 4.X User Interface consists in a lot of panes to display the different object properties. In vCO 5 the default mode is to open object properties in a popup window. This is for example valid for editing the bindongs of an item, editing a scriptable task, opening the API explorer. You can also inspect a workflow you want to add in your main workflow while editing it. In vCO 4.X you have to open multiple windows or vCO clients before opening each item. If you have only one window open you must close the workflow you edit to inspect another one.

The advantage of the popup windows is that you can place them and size them as you want. On my dual monitor setup this is a huge benefit and it saves me a lot of scrolling.

vCO 4.X inspector

vCO 5.1 popup window

Multiselection

In vCO 4.x binding a workflow element input parameters to the main workflow input parameter and attribute requires poping up a window and clicking on the right item for each item. In vCO 5.1 you can select all of them with Ctrl A or by using the mouse pointer.

Preferences

There are a number of new features that can be set or unset in the preferences, but also some that existed and were not set by default. Here are the differences.

vCO 5.1 defaults are to:

• Auto edit a new element. On vCO 4.x , the default is to create a workflow and then edit it after.
• Check usage of an element when deleting it. This was disabled in vCO4.x. For example when you delete a workflow or action vCO will warn you if it is being used by another workflow.
• Edit workflow in a popup window. This option did not exist in vCO 4.x
• Display grid in the schema editor. This is not set on 5.1 and was set in vCO 4.x
• Validate a workflow before saving it. This is a new feature set by default on vCO 5.1. In vCO 4.x the validation was set to happen before running the workflow (still the case as default in 5.1).
• Increase the workflow version when saving and closing.This is a new feature set by default on vCO 5.1.
• Pop up a workflow user interaction form. This is the default on vCO 5.1. On vCO 4.x this required right clicking on the workflow run and answer user interaction.
• Use contextual menu in inventory. This allows to right click / run workflows that have a matching object type (for this you need to use the "show in inventory" presentation property). This is default on 5.1 and was not set in vCO 4.x

vCO 5.1 is more user friendly with these new settings.

Versioning

The feature increasing the workflow version when saving shows a new important feature.

in vCO 4.x the version history was informational. For each version a date, author and comment of the changes entered by the user when changing the workflow.

In vCO 5.1 there is version control of the workflows. If you incremented the versions when changing the workflows you can show the difference against the current version and decide to revert to a given version.

A diff on a workflow will open a split view of the current and selected version of the workflow requiring to check visualy what has changed.

A diff on an action works like diff tools by highlighting the changes:

Another benefit of the versioning is that you can edit a workflow without risk while it is running. In vCO 4.x this could possibly create a problem.

Workflow documentation

You have done a lot of workflows and it is time to document these. Necessary but boring. Since you should better spend your time not doing brong things you can right click on a workflow or a workflow category to generate a PDF documentation of your workflows.

The documentation lists the worklow inputs, outputs, attributes, workflow schema, scriptable tasks.

Consume Workflows from anywhere

Making it easier to develop new workflows is great but then making these as well easier to consume is better.

vCO 4.X offers different ways to run a workflow:

• Use the vCO Client (Usualy the vCO admin running batch jobs)
• Use the vCO web views (Web operator, perspectives, custom webview)
• Use the vCO SOAP web service (programatically from a custom portal or a management application)
• Use the vCO scheduler (for recurrent tasks)
• Use the vCO policies (such as starting a workflow when getting an SNMP trap or an AMQP message or a vCenter threshold)

This is a lot but the vCO client look and feel of vCO 4.x is outdated and not the best place to have operators running the workflows. The web-views are outdated as well and not alway compatible with the latest browsers.

vCO client

As seen above the vCO client got some major look and user experience improvements. This is not specific to the workflow designer: Running workflows was improved as well.

The workflow presentation has been redesigned and made consistent with VMware next generation products. The input validation seems to be a lot faster as well.

vCO installed, configured and integrated with vCenter Server

vCO 4.X is silently installed with vCenter server but it is not configured and it does not have any plug-in for the vSphere .NET client. As a result most vCenter customers do not even know they have a fully licensed cloud orchestration platform.

vCO 5.1 is installed and configured with vCenter Server (Windows version.) No need to mess up with LDAP and databases connection strings and authentication. This is nice as it provides the same level of ease of deployment than the vCO virtual appliance but it is not all.

vCenter 5.1 comes with a jewel called the "vSphere Web Client". Forget about the previous incarnations that were providing minimal functionality over the web. This one is fully featured and includes features not available in the .NET vSphere client. One of these great featurea is the native vCenter Orchestrator integration.

Once you authenticate in vCenter Server the main menu is showing:

With clicking on vCenter Orchestrator you get to the vCO home:

You can browse the workflows you have on your vCO server, including your custom ones and start them directly from the vSphere client. For example the clone workflow I started previously in the vCO 5.1 client can be started.

Now if you think about it you can extend the vSphere Web client features as much as your workflow creativity allows ! The workflows are not limited to manage vCenter objects. They can handle anything vCO can orchestrate. For example you can use the vCloud Director library workflows to create organizations, VDCs, catalogs, ... from vSphere client. If you have third party plug-ins you can for example manage your Active Directory, UCS system, a F5 load balancer, an EMC or NetApp storage array from the vSphere client. Add to that the generic plug-in such as the SQL, SOAP, REST, SSH, SNMP and you can control most of your datacenter operations as workflows from the vSphere Client !

Now if you think you reached the Nirvana, the single pane of glass you have been told about all these years it is not all. There are some other interesting features:

• The workflow user interactions (i.e a technical approval) are popping up and get listed in the vSphere client so you can interact with the workflow.
• The  workflow tasks are listed and you can control them.
• You can schedule a workflow
• You can create contextual workflows

A contextual workflow is a workflow that can be started using as input an object of the vCenter Server inventory. When you right click on a vCenter inventor object you have a list of operations you can run on this object. In the vSphere web client there is a sub menu called "All vCenter Orchestrator Actions". These actions are vCO workflow taking as input a given object type (i.e VM). The vSphere Client comes up preconfigured with some contextual workflows but you can also add your own. In the example I have added the clone with customization workflow and my own import (vCenter) VMs to (vCloud Director) VDC workflow !

It is possible to add your own contextual workflows using the Manage tab of the vCenter Orchestrator home. Here you can add any workflows available in one of your vCO servers, associate an input type and optionally enable multi selection.

The new vCO back-end features

The integration with the vSphere Web client required the implementation of some new fnctionality in the vCO platform.

Single Sign-On (SSO)

When you log in the vSphere Web client you authenticate yourself. These credentials are used to authenticate you in vCO as well. If you open the vCO configuration page you can notice it is set up for SSO authentication, registred with the SSO server provided in vCenter Server and configured with a given group. Here is my setup between my vCO 5.1 Appliance connecting to my vCenter 5.1 Appliance. Using the vSphere Client to run vCO workflows requires using SSO.

vCO 5.1 has a new API for the vCO plug-ins so they can authenticate in the Orchestrated solutions using SSO. For example the vCloud Director 5.1 plug-in can authenticate the user using SSO: You login with a credential in vSphere client, start a workflow managing vCloud objects. In the background SSO authenticates you in vCO and in vCoud Director.

vCO REST API

vCO 4.X has a SOAP web service.

vCO 5.X still has the SOAP web service but also supports a brand new REST one.

Apart from the advantages specific to REST vs SOAP technology the REST one covers a lot more.The SOAP API was designed mainly to start a workflow whereas the REST API is designed to have most of the vCO functionality available as a service.

The REST API is available on https://vCOHostname:8281/api . This API offers an extensive list of services:

• Actions Service
• Catalog Service
• Category Service
• Configuration Service
• Content Service
• Inventory Service
• Notification Service
• Packages Service
• Resource Service
• Service Descriptor Service
• Task Service
• User Service
• Workflow Presentation Service
• Workflow Run Service
• Workflow Service
• Workflow User Interaction Presentation Service

What does it mean? It means a web portal or a management tool can do a lot more than passing parameters to start a workflow. They can leverage the extensive vCO inventory or use the new catalog service and implement things like the workflow presentation layer: This is the ground work to have for example Flex based or HTML5 based web-view equivalent.

You may think this is a v 1.0 API and is not proven but there is more to it: The REST API is what drives vCO integration from the vSphere Client. As such several  / men - years have already been spent on making it work smoothly. This API will be the prime candidate for all the other VMware and third party application integration.

IPV6 Support

vCO 5.1 including the vCO client, server and configurator can be configured with IP V6 addresses.

Other goodies

I am still missing a few of the 5.1 release features.

vCenter 5.1 Plug-in

The 5.1 plug-in has got some updates:

• Performance improvement for the inventory leveraging the vCenter Query Service (this was available in the vCenter 5.0 plug-in but was not as fast).
• New workflows, notably guest operations: The replacement of the VIX functionality using the vSphere new APIs to copy a file to a VM guest OS and execute it.
• Updated workflows, notably the clone workflows to make use of DVS (this was long due).

SQL Plug-in

The SQL plug-in is now part of the default installation and includes the JDBC connection scripting classes that were part of vCO 4.X "database" plug-in.

vCO 5.1 will be released with vSphere 5.1

This is version 2.0 of the package. It contains:

• Enhancements and Bug fixes in the original (version 1) workflows
• New workflows to automate OS provisioning and Windows Software distribution functions

New workflow package and associated documents can be downloaded from here

Quoting the communities post:

•   vCO-Powershell-Plug-in  -1.0.1 - kerberos support added
•   vCO-SQL-Plug-in  -1.1.0  - two related plugins combined with auto-generation as well as advanced SQL scripts. A few important issues fixed.
•   vCO-vCenter-Plug-in  - 5.0.1.1 - Critical update. Fixes a major issue with performance and losing attributes value. Update should be performed!
•   vCO-VUM-Plug-in  - 5.0. - The new version of the plugin that supports the VUM 5.0 version of the product.
•   vCO-SNMP-Plug-in  - 1.0.1 - Minor fix in relation to vCO policies listening for SNMP traps.

Download links located at:

https://my.vmware.com/web/vmware/details?downloadGroup=VCO_POWERSHELL_PLUGIN_101&productId=229

https://my.vmware.com/web/vmware/details?downloadGroup=VCO_VUM_PLUGIN_50&productId=229

https://my.vmware.com/web/vmware/details?downloadGroup=VCO_SNMP_PLUGIN_101&productId=229

https://my.vmware.com/web/vmware/details?downloadGroup=VCO_SQL_PLUGIN_110&productId=229
https://my.vmware.com/web/vmware/details?downloadGroup=VCO_VC50_PLUGIN_5011&productId=229

vCO doc landing pages:

https://www.vmware.com/support/pubs/vco_plugins_pubs.html

Release Notes:

https://www.vmware.com/support/orchestrator/doc/powershell-plugin-101-release-notes.html

https://www.vmware.com/support/orchestrator/doc/snmp-plugin-101-release-notes.html

https://www.vmware.com/support/orchestrator/doc/sql-plugin-11-release-notes.html

https://www.vmware.com/support/orchestrator/doc/vum-plugin-50-release-notes.html

https://www.vmware.com/support/orchestrator/doc/vcenter-server-plugin-5011-release-notes.html

Most of the plugins could be found fromt the product page:

http://www.vmware.com/products/datacenter-virtualization/vcenter-orchestrator/plugins.html

All plugins (including plugins from partners) could be found here:

https://solutionexchange.vmware.com/store/categories/21

As referenced in the "Onyx Output Conversion" thread linked at the end of this article, one way to get started creating new workflows is to use Onyx to capture actions you perform in your vCenter client. This will help you identify the necessary objects and values for the various inputs of the methods you execute against those objects. Be sure to spend some time figuring out this tool and getting to know the API Explorer in vCO.

In order to get things started, create a new workflow named something like "Update HostSystem Service" or whatever you like. 

Once you've created the workflow, place a single Scriptable task named "Apply Action" in there and make all your connections.

Now, let's identify what we'll need for this workflow:

• ESXi server (VC:HostSystem) <- very important input 
• Service Name we wish to take action against
• Service action - what we want to do to the service
• Policy action - how we want the service to act when the ESXi server starts

Let's get started with editing the Apply Action Scriptable Task!

INPUT PARAMETER: First of all, add a new input to your scriptable task named "host". The type should be "VC:HostSystem". This will be the ESXi server that we will manage the service of.

Now that we have our HostSystem object, let's start coding!

{code}

// Get the hostServiceSystem object from the host:

var hostServiceSystem = host.configManager.serviceSystem;

// refresh services info to make sure all properties are fresh:

hostServiceSystem.refreshServices();

{/code}

We now have our hostServiceSystem object. This object is the one that has the methods that allow us to start/stop/restart the services so we're off to a good start. Next up, we need to make sure we get the service object that we wish to manage.

INPUT PARAMETER: Add another input to your scriptable task named "serviceName" - this should be a String.

{code}

// Get Service:

var serviceObj = null;

var services = hostServiceSystem.serviceInfo.service; // Retrieve a list of available services on this host

for each (svc in services){ // now try to match the service key with the service name we're passing in

     //System.log("Checking "+svc.key+" / "+serviceName); // Optionally uncomment the beginning of this line for additional logging

    if(svc.key == serviceName){

        System.log("Service Found! "+svc.label);

serviceObj = svc;

        break;

    }

}

if (serviceObj == null){ // Make sure we got the service object we are trying to manipulate - if null, throw exception

    throw "unable to locate service: "+serviceName+" on host: "+host.name;

}

{/code}

Okay, great. We found the service. Now let's do something with it.

Use the API Explorer to look at the "VcHostServiceSystem" object. This is the object we obtained in the first script snippet above. Take a look at the methods available (indicated by solid block dots).

Based on the API, it looks like we can: restart, start, stop, or uninstall a service. We can also update the Service Policy.

We'll need to account for each of these possible actions. As part of this, we'll need an input to prompt what we want to do. Additionally, we should pre-define our values so they cannot be mis-typed.

INPUT PARAMETER: Add another input to your scriptable task named "serviceAction" - this should be a String

ATTRIBUTE: Add a new attribute named "serviceActions" - this should be an Array of Strings with the following values:

• start
• stop
• restart
• uninstall

We'll use this attribute in our presentation to present a dropdown list of service actions. This allows us to prevent a user from typing an invalid value.

Now paste the following code at the bottom of the scriptable task.

{code}

switch(serviceAction){
    case "start": // Only run the startService method if the service is not running
        // before trying to start, make sure running is not true
        if (serviceObj.running != true){
            hostServiceSystem.startService(serviceObj.key);
        }
        break;

    case "stop": // Only run the stopService method if the service is running
        if (serviceObj.running == true){
            hostServiceSystem.stopService(serviceObj.key);
        }
        break;

    case "restart":
        hostServiceSystem.restartService(serviceObj.key);
        break;

    case "uninstall": // Not all services can be uninstalled so we use a try/catch here
        try{
            hostServiceSystem.uninstallService(serviceObj.key);
        }catch(err){
            System.error("Error uninstalling service "+serviceObj.key+" ("+err+")");
            Server.error("Error uninstalling service "+serviceObj.key,serviceObj.key);
        }
        break;

    default: // We should never get here so Provide some warning logging and optionally throw an exception if appropriate for your environment
        System.warn("Invalid service action selected");
        Server.warn("Invalid service action selected",serviceAction);
}

{/code}

Finally, we need to provide the ability to specify how the service starts up. 
INPUT PARAMETER: Add a new input to your Scriptable Task named "policyAction" - this should be a String

The updateServicePolicy takes two parameters - the id of our service object and the policy. It took a little playing with Onyx for me to find that the possible values for the policy are on, off, or automatic. We don't always want to take action on a policy so we'll be adding an extra value to our choice: No Change.

ATTRIBUTE: Add a new attribute named "policyActions" - this should be an Array of Strings with the following values:

• No Change
• on
• off
• automatic

We'll use this attribute in our presentation to present a dropdown list of policy actions. This allows us to prevent a user from typing an invalid value.

{code}

if(policyAction != null && policyAction != "" && policyAction != "No Change"){
    // only valid policy actions are: on, off, automatic
    try{
        hostServiceSystem.updateServicePolicy(serviceObj.key,policyAction);
    }catch(err){
        System.error("Error updating service policy "+serviceObj.key+" ("+err+")");
        Server.error("Error updating service policy "+serviceObj.key+" ("+err+")",serviceObj.key);
    }
}

{/code}

At this point, the workflow has 1 object input and 3 string inputs. All of which are passed in to the Scriptable Task.

Go ahead and try running your workflow right now...

See a problem? Right, you could mis-type the service name and may not even know which actions are available. Earlier in the article, we created a pair of Attributes that we store an array of strings containing a static set of values. We don't want to do this for the services since the services available on one host may not be available on other hosts and we want this workflow to be very reusable. So, we'll create an Action that retrieves an array of Service Names running on the given HostSystem.

Create a new Action named "getHostSystemServiceKeys". I will create this in a module named com.vmware.pso

The action should have an input named "host" of type "VC:HostSystem". I have also added a "logging" input of type "boolean" to allow for some System.log entries to take place if I need them.

For the Return Type, set that to "Array/string" since we'll be returning the names of all the services available to the HostSystem.

Now, for the script:

{code}

var hostServiceSystem = host.configManager.serviceSystem;
hostServiceSystem.refreshServices();
var services = hostServiceSystem.serviceInfo.service; // Return the array of service objects
var serviceKeys = new Array();
for each (svc in services){
    if(logging == true){
        System.log("Key: "+svc.key+" -- Label: "+svc.label+" -- Policy: "+svc.policy);
    }
    serviceKeys.push(svc.key); // Since this object type cannot be passed as an output/input, we'll just pass the names into the array
}
return serviceKeys;

{/code}

Okay, all of our code is in place. The only thing left is to cleanup our presentation so that our inputs are pre-populated with appropriate content after we have chosen our HostSystem.

Click on the Presentation tab and arrange as shown in the following screenshot:

Make the following changes on the properties tab of each input:

• (VC:HostSystem) host
  • Mandatory Input: Yes
• (string) serviceName (On General tab, under description, set value to "Service Name")
  • Predefined list of elements (Click the action icon, choose "getHostSystemServiceKeys". For first parameter, enter "#host". For second parameter, enter false)
• (string) serviceAction (On General tab, under description, set value to "Service Action")
  • Predefined list of elements (Enter #serviceActions for the value)
• (string) policyAction (On General tab, under description, set value to "Policy")
  • Predefined list of elements (Enter #policyNames for the value)
  • Default value (Set to "No Change")

Save and Close! Now try it out!

Solution/Package:

Solution Package File - Download Here

Reference/Inspiration:

Onyx Output Converstion

Turning on SSH Service and Using SSH plug-in

With the release of vCloud director 1.5 a new important API feature was introduced : the query service. Quoting the "What's new" whitepaper:

VMware vCloud Director 1.5 also introduces a VMware vCloud API query service,
which can significantly improve developer efficiency, by minimizing the number of API requests and the amount
of data transferred for an API client to obtain needed information. Example query parameters include sorting
and ordering, pagination, filtering, projection, and expressions.

Following the development of vCloud Director prior to its first release I have begged for having this feature as a public API on several occasions. I now use it on a regular basis in my workflows. This article explains the reasons you may have to leverage the query service and gives an example on how to use it.

First you may wonder why you should use it. The short answer is because it is more efficient for finding objects based on their properties.

The longuer answer is that with using the vCloud API:

You need to drill down through the hierarchy of objects to find a particular object and its properties and need to handle things such as sorting and filtering in your code.

If you need to get to a list of objects such as vApps your vCLoud API client will issue an HTTP GET per object. Each vApp XML object representation can be a few KBytes with its OVF envelope. Multiply this by thousands of objects and you can imagine how many requests and how much data can be transfered just to get a simple information for a few vApps you want to filter on a certain property.

The query service is done server side. Your client sends a simple query which is executed on the server and sending back only the requested, filtered information. The information is sent in pages, meaning that you can at any time stop to get the next pages if you have the information needed.

Using the query service means less complex operations for the developer, and much better performance for getting the information.

Now let's see a practical example using the query service with vCO.

I needed to delete a vApp Template but then I needed to remove it from the catalogs it was in. If you are familiar with the vCloud workflow library you may know there is a "Delete Catalog Item and linked item". This is fine when you have a catalog item you want to delte with its vApp Template but it does not work the other way around.

First solution: Using the vCloud API

{code}

var catalogItemsOut = new Array();
var org = vAppTemplate.parent.parent;
var catalogs = org.getCatalogs()

for each (var catalog in catalogs) {
    var catalogItems =  catalog.getCatalogItems();
    for each (var catalogItem in catalogItems) {
        if (catalogItem.entity.href == vAppTemplate.getReference().href) {
            System.log(catalogItem.name + " : " + catalogItem.entity.href);
            catalogItemsOut.push(catalogItem);
        }
    }
}   
return catalogItemsOut;

{/code}

From the vAppTemplate getting the parent VDC, then the parent Organization. From there getting all the catalogs, all their catalog items and check if they reference the vApp Template.

Second solution: Using the query service

{code}
var catalogItems = new Array();

var vcdHost = vAppTemplate.getHost();
var queryService = vcdHost.getQueryService();
var expression = new VclExpression(VclQueryCatalogItemField.ENTITY, vAppTemplate.getReference().href , VclExpressionType.EQUALS);
var filter = new VclFilter(expression);
var params = new VclQueryParams();
params.setFilter(filter);
 
var resultSet = queryService.queryRecords(VclQueryRecordType.CATALOGITEM, params);

while (resultSet != null)  {
    var records = resultSet.getRecords(new VclQueryResultCatalogItemRecord());
    for each (var record in records) {
        var catalogItemRef = new VclReference();
        catalogItemRef.href = record.href;
        catalogItemRef.name = record.name;
        catalogItemRef.type = record.type;
          var catalogItem = (vcdHost.getEntityByReference(VclFinderType.CATALOG_ITEM, catalogItemRef));
        catalogItems.push(catalogItem);
    }
    resultSet = resultSet.getNextPage();
}
return catalogItems;

{/code}

First get the vCloud Director host from the vApp Template, then create an expression looking for the catalog item entity having an HREF equals the one of the vAppTemplate. For each record create a reference to the catalog item, get the object form the reference and add it to the array of catalogItems.

I have a small remote vCloud Director Cloud test environment with two catalogs and a total of 16 catalogs. Does using the query service makes a difference in such a small environment ?

Running a workflow looking for the same Template using the two solutions it takes:

• About 7 seconds and about 400 lines of XML to get all the objects with the vCloud API (1 VDC, 1 Organization, catalogs and their catalog items)
• About 1 second and 12 lines of XML (QueryResultRecords)

To see the XML exchanged the Debug mode is activated on the vCloud Director plug-in (as described in this article)

The catalog and catalog items objects are a lot smaller than objects such as the vApps so you can imagine how much difference it can do when querying in an environment with thousands of vApps.

Now you may think the clear winner is always the Query service but it is not always the case. If you run the workflow using the vCloud API a second time it runs in less than a second and no XML in the logs. In fact the vCloud Director plug-in cache is coming to the rescue. The objects are already in the vCO server allocated memory and no exchange need to be done with the vCloud Server. This is very useful but it has a couple of drawbacks:

• The heavy lifting of getting the necessary objects must have been done at least once in the last 30 minutes (cache TTL).
• The objects state does not get update from the vCloud server until you refresh these objects, which mean downloading them again using the updateInternalState() method. For example any vCD library workflow will update the changed objects but other calls to the vCD server won't (vCloud Director UI or API).

Here are a few query service scripts I have contributed on VMware communities.

• Get a list of all vApps (sorted)
• List VMs in vApp
• Get a VM Managed Object Reference
• Get external networks

Also, be sure to check out another perspective to using queries in this article by William Lam: 

http://blogs.vmware.com/vsphere/2012/04/quickly-finding-objects-using-the-vcloud-api-query-service.html

This is the first common release for the Windows (Build 555) and Appliance (Build 612841) version.

This release contains several improvements and bug fixes such as:

• The app-server/server/vmo/log/script-logs.log now automatically include the workflow the log is coming from using the format:
  {code}<Date> <Severity> [SCRIPTING_LOG] [<Workflow> (<Execution start date>)] <Message>{/code}
• SSH plug-in does not fail anymore transfering files.
• Issues fixed in library workflows.

Also note he that for security reason the vCenter Orchestrator 4.2.1 and its subsequent releases no longer support the default remote login to the Orchestrator configuration interface. The remote login ability can be restored if needed.

Please check the vCenter Orchestrator 4.2.1 release notes for further information.

Download the vCenter Orchestrator 4.2.1 Virtual Appliance

Quoting the product page

VMware vCenter Configuration Manager, a key part of the VMware vCenter Operations Management Suite, automates configuration management across virtual, physical and cloud environments. Enterprises can use vCenter Configuration Manager to continuously audit the configurations of VMware Infrastructure (including VMware ESX, ESXi, vCenter, vCloud Director and vShield) as well as Windows, Linux and Unix operating systems. Easily maintain configuration compliance against internal standards, security best practices, vendor hardening guidelines and regulatory mandates.

Now, how does it relate to Orchestration ?

Functionaly VCM provides a piece that was missing in the VMware orchestrated products portfolio: the configuration management engine delivering compliance to the physical devices, the hypervisors, the virtual infrastructure, the cloud infrastructure and the guest Operating systems. An important item on the feature page is the following :

vCenter Configuration Manager API coupled with VMware vCenter Orchestrator supports integration with other VMware and 3rd party products

Technicaly VCM is exposing a REST API. vCenter Orchestrator has a REST plug-in, VCM ships with vCO workflows will be available on the VMware solution exchange market place to automate patching and compliance assessment of vCenter templates, offline and online VMs. It has also a set of specific VCM operation workflows for assessing compliance, operating patches, monitoring operations, running remote commands.

Using the API it is possible to extend these operations to vCloud Director VM patching and software installation. Eventually these will come as an out of the box feature in a future version of the workflows or as part of the workflow library shipping with a dedicated vCM plug-in.

It is a good thing that most of the VMware cloud and management applications are now released with a build in integration with vCenter Orchestrator. This definitely helps to build agile, orchestrated clouds.

For more information check vCenter Configuration Manager product page.

If you like the VMware vCenter Orchestrator (vCO) Appliance because it is easy to deploy, configure and use then you may want to improve it with enabling additional features.

I like very much the vCO appliance but I recognize it has a few shortcomings:

• The vCO web start client does not work well on Mac since it does not preserve the client preferences.
• I am missing local shell control and live logs.
• I do not like to author my workflows as one of the hundreds "vcoadmin" out there. The appliance is missing user management.

Let's address these. Warning: The following is not documented, not supported and may not work in future vCO versions. It is intended for lab / educational usage.

First make sure your appliance is started. Note the appliance IP / hostname and remember the password you set to login as root from the console.

Installing the vCO client from the appliance

The appliance is not exposing all its gems. It is hiding a collection of vCenter Orchestrator clients for windows, Linux and MacOS.

If you open a browser and type in http://appliance/vco-client-install/ you will end ip on this page:

You can download and install the client for Windows (exe), Linux (bin). Or MacOS (zip).

Make sure you install it on a system that will only need to connect to the vCO server of the appliance and not any other version of vCO (i.e any Windows base version). The system it is going to be installed on may not support any upgrade when installing a newer version of vCO.

Remote shell and remote logs

If like me you are not a fan of using the VM local console for administration purposes and find the web access to be too limited you may want to have a remote shell.

A simple way to enable remote shell is to permit the root login for ssh. To do so on the appliance command line type:

{code}sed -i 's/PermitRootLogin no/PermitRootLogin yes/g' /etc/ssh/sshd_config{/code}

And then:

{code}service sshd restart{/code}

Now you can use your favorite ssh client to connect to the appliance (ssh root@appliance)

There are several interesting things to do in command line on the vCO server but one is very important when developping / testing workflows : have a live view of the vCO server log file. To do so type:

{code}tail /opt/vmo/app-server/server/vmo/log/server.log -f{/code}

You will see the last lines of the log file. If you try to run a workflow you will see it scrolling live.

Basic user management

The appliance has a simplified configuration using a built-in database and directory service. The directory service has a vcoadmins and vcousers group to allow to author and run workflows. In each of these groups there is a single member (vcoadmin and vco user). There are two issues with that. The first one is security. Someone may use the vcoadmin user name with the default vcoadmin password. The second one is that if everyone use the vcoadmin account to create content they loose the benefit of knowing or letting know who really created the workflows.

For the first issue Jörg Lew wrote a detailed article here. Changing the vCO Admin password is nice, adding new vCO admin would be better. It would even be better if you could do it with a workflow. This is what I did with providing this package. You can find the package in the VMware communities HERE.

Once imported you have a few options to manage your users:

These workflows use SSH to connect remotely. Make sure you enable remote shell.

First run the configuration workflow.

Enter the hostname or IP of the appliance, the user to SSH remotely (default = root), the password (the one you set for the appliance), the LDAP Admin user (default = vcoadmin) and its password (default = vcoadmin).

Make sure you have the credentials to access to the vCO web configuration since you will need to change the password there as well.

Run the "change user password" workflow.

Select the vcoadmin (use the filter) and pick up a password (do not mistype since I did not use password check). Remember it !

Once submitted and successful (if not you have certainly wrongly entered the configuration in the previous step)  you can close the client and open the vCO web configure (htps://appliance:8283).

You will notice some red warnings.

Requiring to update the password in this tab:

Apply the change. The LDAP tab should go back to green.

Now the Plug-in tab has still a warning.

Update the password there as well:

Since your vCO server may still be authenticating in LDAP using the previous credentials you need to restart the service.

Now that your vCO server is secure you can authenticate as vcoadmin / your new password using the client. Now it is time to create a new user for you.

Run the "Create a new user" workflow.

Fill with your username and display name, pick up a password. Submit.

The last step is to run the workflow "Add user to group".

Select your new user and the vcoadmins group. Submit. Now close the client and log back in with your newly created user.

You can now create your workflows ussing your user.

Another new VMware plug-in release for vCenter Orchestrator, this time for vCenter Chargeback Manager.

You can use the plug-in to run Orchestrator workflows that automate vCenter Chargeback processes. In this release, the plug-in contains sample workflows.

You can also create custom workflows that implement the plug-in API to automate tasks in your vCenter Chargeback Manager environment.

The vCenter Chargeback Manager plug-in API provides various Javascript objects such as cost models, fixed costs, and hierarchies, that are created in you vCenter Chargeback Manager instance. You can also create and manage reports and reporting schedules in your vCenter Chargeback Manager instance by using the plug-in API.

For more information pleae check the VMware vCenter Orchestrator SQL Plug-In: release notes, documentation and download.

These operations include:

• Check Org for vApp name already existing in any vDCs, if so append number until unique
• Instantiate, but do not deploy or power on immediately
• Apply custom naming convention to each VM in the vApp
• Base VM name on vApp name
• Customize the Guest OS Customization script
• Specify IP for VM
• Create Computer Account in specific OU of Active Directory
• Add specified domain\user to local administrators group
• Connect VM to Network
• Change Hard Disk Size
• Add extra Disk to VM
• Modify the VM CPU count
• Modify the VM Memory
• Deploy vApp

The package and custom workflow requires a functional vCO server with the vCloud Director 1.5 plugin installed and configured. Additionally, the Active Directory plugin should be installed and configured to facilitate the Computer Account creation in AD. A functional vCD environment should be available with 1 or more vApp Templates containing at least one Windows VM. All of which should be configured as discussed throughout this document. (Note: step-by-Step installation/configuration is out of scope here)

While this workflow has been developed and tested successfully, it should always be executed and tested using the vCO Client before attempting to call via the SOAP API. This allows for determining appropriate inputs and faster troubleshooting. Once the workflow has completed successfully in the target environment, it is safe to begin integration with other systems.

Find the workflow package and more information on the VMware vCenter Orchestrator communities.

The virtual Insanity blog has a good article describing the VMware virtual Customer Lab (vCL) consisting in a fully automated cloud solution where users can checkout VMware software solutions for 14 days of testing and training/education.

The article explains what is the vCL and how it works. A part of the solution involves a vCO powered request / approval portal. Read more about it here.

VMware Solution Exchange (VSX) Enables Customers to Discover, Evaluate and Expedite Buying Process for Joint Solutions through Direct Engagement with Partners and Developers. This are important news for vCenter Orchestrator. Here is what these involve:

• vSphere customers will now find all the vCenter Orchestrator plug-ins in a single location. vCenter Orchestrator is bundled with some plug-ins but VMware and partners have provided several additional ones. The solution exchange will feature the latest version of all VMware and all VMware partners vCenter Orchestrator plug-ins.

• For VMware partners it will be significantly easier to publish their plug-ins to make them available to the hundreds of thousands VMware vSphere Customers that are entitled to use vCenter Orchestrator.

• The VMware Solution eXchange improves the customer experience with providing rich, multi-media content, white papers, user guides, blog postings and search tools allowing to engage with partners and developers to help ease the process of finding the right solution for their businesses.

• The VMware Solution eXchange provides customers with the opportunity to use a five-star rating system and provide a written review about their experience with the solution.

For more information:

• Visit the VMware Solution Exchange home page. 
• Visit the vCenter Orchestrator section.
• Read the official press release.
  
• The VMware vCenter Orchestrator blog
  
• Twitter hashtag #VMwareVSX

If so, don't worry - the session was recorded! See below for details:

• Thomas Corfmat - vCO Product Manager
• Hemant Gaidhani - vCO Product Marketing Manager

with bonus commentary from one of VMware's PowerShell gurus: Alan Renouf

Learn more about vCO and the new support/integrations with PowerShell.

Links/Resources:

• PowerScripting Blog Article: http://powerscripting.wordpress.com/2012/01/16/episode-172-vcenter-orchestrator/
• Video: http://static.bambuser.com/r/player.swf?username=PowerScripting
• PodCast: http://feeds.feedburner.com/PowerScripting

If you are interested in a possible career change to work in the most exciting, most ground breaking company in the software industry here is a chace to do it !

 Integration Orchestration Development Engineers

VMware is seeking several Integration Orchestration Development engineers and they must have strong infrastructure integration scripting knowledge using scripting languages like JavaScript, Perl, Python, PowerShell, Ruby, Bash, MSDOS, etc… This is a backend developer role with little to no client interfacing responsibilities. VMware is looking for candidates that have experience in tying together several technology point products into a cohesive solution, particularly through automation, scripting or custom modifications. The individuals will work under direction of VMware Architects, and take direction from on-site consulting teams that are collecting and coordinating customer requirements.

Integration Orchestration Practice Manager

VMware is seeking an Integration Orchestration Practice Manager that will supervise a team performing custom integration & orchestration (i.e. Automation). Candidates should come from a strong software development lifecycle (SDLC) background and ideally have managed teams consisting of skill sets within SOAP, REST, vendor SDK interfaces and strong infrastructure scripting languages such as JavaScript, Perl, Python, PowerShell, Ruby, Bash, MSDOS, etc... The candidate should have experience as an engineer or architect with Virtualized Datacenter design and implementations.

Learn ore about this here.

The goal of publishing this package is to give a good starting point implementation with working examples that can be adapted for other requirements. This include setting all the AMQP configuration automatically for listening for a particular blocking task notification, converting the AMQP message to vCloud Director objects, extracting the necessary information from this object, including using the vCloud Director query service, and calling back vCloud Director to resume / cancel / fail the task.

In addition to this package we provided a walk through video.

http://www.youtube.com/watch?v=OXg2GmPGRW0

If you ever wanted to integrate your workflows with a database with having your database records available as part of the inventory then this plug-in will help you to do so.

Chances are that at some point you will need to read or write information from an SQL database as part of your workflows. vCenter Orchestrator provides some scripter / developper capabilities such as the JDBC plug-in and the integration with the hybernate framework. These requires quite some development and test to have a well integrated solution.

With the vCenter Orchestrator SQL plug-in allows to:

• expose in the vCO enventory your database records and to perform read, create, update, and delete operations without writing SQL statements.
• generate workflows from these operations.

For more information pleae check the VMware vCenter Orchestrator SQL Plug-In: release notes, documentation and download.

The plug-in exposes an inventory and CRUD opratons for:

• ESXi depots
• Host profiles
• Rule sets

There are workflows to manage answer files and to reprovision a host using answer files, host profiles, new image.

For more information check the VMware vCenter Orchestrator Plug-In for vSphere Auto Deploy release notes, documentation and download

If you are designing enterprise wide Orchestration you may need to consider more than a single instance of vCenter Orchestrator. The vCenter Orchestrator Multi-Node Plug-in can help you doing so with offering the ability of a vCenter Orchestrator server to remote control other vCenter Orchestrator servers.

What this plug-in provides is:

• Publish the remote vCenter Orchestrator library elements  (Configurations, Packgaesm Workflows, Actions and Resources) in the master vCenter Orchestrator inventory
• Publish the remote vCenter Orchestrator inventory elements (vCenter, vCloud Director, Microsoft AD, Cisco UCS, ...) in the master vCenter Orchestrator inventory
• Provide remote operations to start, stop, cancel remote workflows
• Provide remote operation to synchronize packages

These are advanced remoting capabilities providing a lot more flexibility than the existing nested remote workflow capability. Despite the name it is not a clustering capability.

For more information check the VMware vCenter Orchestrator Multi-Node Plug-In: release notes, documentation and download

If you have an investment in PowerShell cmdlets such as VMware vSphere PowerCLI but also snapins from other vendors the VMware vCenter Orchestrator plug-in for Microsoft Windows PowerShell can help you to integrate these as part of workflows steps in vCenter Orchestrator.

Citing the release notes:

The plug-in offers many capabilities to workflow developers, such as:

• Invoking unmodified scripts by copying and pasting them into workflows
• Invoking external scripts and passing workflow parameters as script inputs
• Generating a new Orchestrator action from a PowerShell script
• Generating a new Orchestrator action for a PowerCLI cmdlet
• Browsing snap-ins and their associated cmdlets in the Orchestrator workflow editor
• Invoking scripts from the Orchestrator JavaScript API
• Converting vCenter Server objects in Orchestrator workflows to PowerShell objects and the reverse

The VMware vCenter Orchestrator plug-in for Microsoft Windows PowerShell 1.0 release runs on VMware vCenter Orchestrator 4.2. The plug-in is compatible with Windows PowerShell 1.0 and 2.0, and requires WinRM 2.0 or OpenSSH

For more information please check the VMware vCenter Orchestrator Plug-In for Microsoft Windows PowerShell: release notes, documentation and download

VCE, the company formed by Cisco and EMC with investments from VMware and Intel is showing a demonstration of their vCenter Orchestrator powered infrastructure provisioning for vCloud Director. This is done with using EMC Unified Infrastructure Manager (UIM) which is the single pane of glass managing the stand alone Vblock Element Management applications.

If you followed the vCenter Orchestrator plug-in releases you must have noticed the UCS Manager release. So, how is a UIM plug-in different ? It covers some of the functionality in the UCS Manager plug-in but also Cisco MCS and Nexus, EMC Unisphere & SMC and some vCenter functionality. The advantage of orchestrating the top management application is that all the integration between the different parts is already included and can be automated from vCO. The exception is for vCloud Director which is orchestrated directly from vCO vCloud Director Plug-in. This integration is done by a demo workflow provided with the UIM plug-in by the VCE folks and has the following steps:

• Instantiate a Service from a UIM service offering using Vblock resources
• Reserve and activate the service from the Service offering
• Synchronize vCenter with the service
• Add a provider Virtual Data Center to vCloud Director

As a result the vCloud Director administrator get additional resources he can allocate to be consumed by the different organizations Virtual Data Centers.

The UIM plug-in exposes an inventory listing:

• Service offerings
• Services
• tasks
• vBlocks
• vCenters
• vCenter Clusters

Watch the full workflow demonstration video here:

http://www.youtube.com/watch?v=pckoF0s3nHA

My team and I have helped customers to build private and public clouds using vCenter Orchestrator for the last 5 years. Adding vCloud Director in the picture tremendously helps with providing customers a scalable standardized cloud solution. Most people will think at orchestration as a single entry point on top of the cloud when in fact for several use cases it is recommended to have vCloud Director and vCenter Orchestrator running side by side.

If you are running out of the box vCloud Director user interface or API you can use vCloud notifications and blocking tasks to delegate underlying processes to vCenter Orchestrator as shown in this article. This works well when there is a requirement to extend existing vCloud Director functionalities.

If you are building your own customized portal using a web framework or a Service Management solution such as VMware Service Manager you may wonder when you should consume the vCloud Director API and when you should consume the vCenter Orchestrator web service.

Consuming the vCloud API should be preferred for:

• All close to real time interactive operations such as:
  • Listing Cloud objects and their properties (For example a Web portal user interface or data extraction)
  • Running vCloud single step operations (For example Start / Stop vApp)
  • Operations requiring direct connection from a client to the cloud API such as:
    • Uploading of resources to the cloud.
    • The ones being initiated by the cloud ecosystem.

Consuming the vCenter Orchestrator Web Service, which in turn can calls the vCloud API with using the vCloud Director plug-in for vCenter Orchestrator should be preferred for operations :

• That should not or cannot be triggerred with extending the vCloud API / UI operations using AMQP:
  • A workflow requiring input parameters not available in the vCloud API (For examplee vApp placement based on project code input).
  • Operations that have no equivalent in the cloud API (For example : "Backup vApp")

• Involving multiple steps, long running operations and computed decisions that should not affect regular operations in vCloud Director
  
  • For example a workflow for the end users including vApp approval, vApp placement, pre-provisioning (i.e load balancer), provisioning, post provisioning (i.e adding the computer account in Microsoft Active Directory) that should not apply for the other applications leveraging the vCloud API.
• Leveraging operations not covered in the vCloud API
  • Preparing the physical infrastructure for the Virtual Infrastructure (For Example provisioning a Cisco UCS blade).
  • Setting up resources in vCenter for vCloud Director
• Requiring auditing
  • Each step of the workflow, each decision is recorded so process compliance can be observed.
  • vCenter Orchestrator communicate back and forth with an external Change Management application.
• Requiring resiliency
  • A process running in a vCO workflow can be on vCO host shut down or failure stopped and resumed on the same or a different vCO server.
  • Workflows can trap error and apply remediation

On the same subject, at a higher level my colleague Massimo Re Ferre wrote a good article here.

The vCenter Orchestrator plug-ins are coming with a lot of built in workflows and action scripts. The first thing to do when looking for functionality is to browse the library or use the search filter to find these.

If you do not get exactly what you are looking for, there are chances that some of the existing workflows and actions will contain close enough examples. If you do not find any of these you can still use the API explorer to create your own.

Today there was a community post about getting VM Guest information using the vCloud Director plug-in for vCO. While I answered there I thought it would be helpful to publish it here. Here is how I do it:

For the sake of simplicity I am assuming starting with a vApp input in the workflow set as an input of a scriptable box.

First I need to get the VMs within this vApp.

In the API explorator look at the vApp methods & properties:

You can see there is vApp.getChildrenVms();

To avoid doing typos press ctrl espace after the "." : This will bring the completion window like this:

Now that you have got the VMs in the vApp you need to get their guestCustomization:

Once you have identified the right method to do it, you need to check VclGuestCustomizationSection to know what properties are available.

 So if for example we need the Guest OS computer name we can use the computername property.

So to get guest information:

{code}

var vms = vApp.getChildrenVms();

for each (var vm in vms) { //iterating the vms
    var getGuestCustomizationSection = vm.getGuestCustomizationSection();
    System.log(getGuestCustomizationSection.computerName);
}

{/code}

The methodology above applies to any vCenter Orchestrator plug-in.

to be re-usable, versionable this peace of code can be embedded in an action that would return an array of vm names (strings)

{code}

var vmNames = new Array();

var vms = vApp.getChildrenVms();

for each (var vm in vms) { //iterating the vms
    var getGuestCustomizationSection = vm.getGuestCustomizationSection();
    vmNames.push(getGuestCustomizationSection.computerName);
}

return vmNames;

{/code}

I will re-iterate what I have said about nine months ago : Today VMware released the most complete automation solution for vCloud Director as a free download ! This is a plug-in for vCenter Orchestrator allowing to orchestrate vCloud Director 1.5 with a large collection of out of the box workflows. The previous version of this plug-in (1.02) is used at several service providers and enterprise customers.

As before in terms of functionality the plug-in allows:

• Tenants to automate catalogs and vApp provisioning and decommissioning operations within their organization.
• Service providers to automate the configuration and management of the tenant organizations, their virtual data centers and their resources.
• Service providers to automate the addition and carve up of new Virtual Infrastructure resources such as Provider vDC, External networks and network pools.

Technically the plug-in implements the User, Admin, and Admin Extensions APIs using the latest release of the vCloud Java SDK. The vCloud Java SDK communicates with vCloud Director REST API. All this complexity happens in the background by being abstracted for the vCO Admin. The plug-in provides a total of 138 building block workflows and 250 actions. Some high level workflows are also provided and are great examples of the combination of these.

So what is new ?

• The blocking tasks and notifications. Using this plug-in in combination with the AMQP plug-in allows vCO to dynamicaly respond to vCloud Director tasks and notifications.
• The query service. The query service allows to gather information on a lot of objects very quickly.
• 47 new out of the box workflows and 142 new actions allowing to develop complex automation in a lot less time than with any other solution.
• The support of the 1.5 API adds a LOT of new objects (close to 500 instead of about 160 before !!!).

Network related workflows	VM related workflows

There were also a lot of other improvements such as better cache handling / object fetching, new objects in the inventory and several fixes.

The vCenter Orchestrator virtual appliance can be deployed in minutes for being used in combination with this plug-in to automate your cloud and integrate it with other solutions.

If you are new to vCenter Orchestrator, you can leverage the video tutorials to create your own workflows. Then you can share your experience, ask questions on the VMware vCenter Orchestrator Communities.

The VMware vCenter Orchestrator plug-in for vCloud Director 1.5 release runs on VMware vCenter Orchestrator 4.1 and the subsequent update releases, as well as VMware vCenter Orchestrator 4.2.

For more information, check the vCenter Orchestrator Plug-in for vCloud Director release notes.

The vCenter Orchestrator Plug-in for vCloud Director is is available for download here and the user guide for the plug-in is here.

The vCO appliance makes it very easy to deploy vCenter Orchestrator in minutes. My colleague William Lam went a bit further with providing a script that let you pre-define a lot of the configuration settings.

The script leverage the OVF format to set a number of parameters such as the appliance network settings, placement on the virtual infrastucture datastores and network.

You can find out more about it here.

I am looking forward to see more vCO articles from William since he has a very successfull VMware scripting blog with a lot of interesting articles. I encourage you to browse it if you did not knew it yet.

If you are in EMEA and would like to be trained on using vCenter Orchestrator to create custom workflows here is your chance.

VMware is providing not one but 10 instructor lead training classes in EMEA in the next months !

I can attest the training content is really good since I have given this training recently. Also the EMEA trainer is the person who developped the content and has several years of experience with implementing vCO solutions at enterprise customers.

To find out all the details please visit vcoportal.

This is a nice solution for monitoring throughput, latency, hit rate and error rates, it provides code level diagnostics and a nice user interface to fix problems quickly.The automation expert reading here certainly expect more and will be pleased to know it also offer built-in automation and policy-based alerting and remediation.

If you go on the product page features you can see that this is the kind of flexible remediation we want to see in such a product:

How is this integration implemented you may wonder ?  A screenshot is worth a thousand words !

vCenter Orchestrator provides the most advanced, the most flexible VMware cloud orchestration and as such the most capapble remediation automation extension for vFabric Application Performance Manager !

For more information on vFabric Application Performance Manager please visit the product page.

Previously we showed you how to hot clone vCloud Director vApps. We even did a video tutorial for it. Now it is time to do something else the vCloud Director User Interface will not let you do : rename an organization.

My test org has my name so my colleagues know it is mine. For a demo I needed to rename it. I opened the organization properties and saw the read only organization name.

In less then a minute I created a new workflow just doing this:

Ran it.

And voilà. Org renamed.

Now there may be very good reasons not to do this on a production org such as the change of the URL to access this org.

Hopefully this can be usefull and can add one more bullet point to this article. All the item listed there and a lot more have been done with vCO already.

This article will identify the log files currently in use for a typical vCO server installation. Additionally, it will address the following areas of interest:

• Server Log rollover control: MaxFileSize and MaxBackupIndex
• Enable REST call logging in the vCloud Director Plug-in
• Enable logging to a SYSLOG server
• Setting the server and all components to debug mode

The vCenter Orchestrator application server log files are located in install_directory\app- server\server\vmo\log

The server.log file the most relevant file for auditing the vCenter Orchestrator application server components, the plug-in adapters and the scripts contained in workflows, policies and actions. For convenience the script-logs.log only contains logs produced by the scripts.

Log levels and persistence

Each of the vCenter Orchestrator application server components, plug-in adapter provides logs in different levels including fatal, error, warning, info and debug.

During normal operations it is recommended to use the default info level to maximize the server performance while keeping quite detailed level information. Using the debug level for a single component or for all server components is recommended for troubleshooting.

A vCenter Orchestrator script in an action, workflow or policy can generate an error, warning, info and debug log by invoking the System.log(), System.warn(), System.error() and System.debug() methods with a single log content string parameter.

A vCenter Orchestrator script in an action, workflow or policy can generate an error, warning, info and debug log and an action, workflow or policy event by invoking the Server.log(), Server.warn(), Server.error() and Server.debug() methods with an event description string parameter and an event content string parameter.

The events will be saved in vCenter Orchestrator database and stay attached to their action, workflow or policy until deleted by the purge daemon when reaching a total of 5000 events or more than 100 runs for a given workflow. These values can be changed in the vmo.properties configuration file using the max-log-events and max-workflow-tokens values. Substantial increases will lead to database performance decrease requiring database admin regular maintenance operations such as re-indexing the event and workflow token tables.

Log configuration

vCenter Orchestrator uses Apache log4j allowing to configure and enable granular logging at runtime without modifying the application. The target of the log output is a configured by default to a set files but can be changed to different output targets such as a remote Unix Syslog daemon.

The log configuration file is named log4j.xml and is located in install_directory\app- server\server\vmo\conf.

Changes made by editing the configuration file manually may be lost when using vCenter Orchestrator web configuration to perform log configuration changes. This should be avoided once manual changes have been provided.

Common configuration changes use cases

The server log files may be rolling over in a short period of time when the log verbosity is increased substantially. This can be prevented by changing the default size and number of files as follows (Here multiplying the MaxFileSize and the MaxBackupInded by four times the default value).

{code class="brush: xml; gutter: false;" }
<param name="MaxFileSize" value="8192KB"/>
<param name="MaxBackupIndex" value="16"/>
{/code}

When troubleshooting vCenter Orchestrator with vCloud Director setting the vCloud Director Plug-in in debug mode logs the REST calls and responses. This can be changed in the following section as follows:
{code class="brush: xml; gutter: false;" }
<category additivity="true" name="com.vmware.vmo.plugin.vcloud">
<priority value="DEBUG"/>
</category>
{/code}

Using log centralization to a Unix Syslog Daemon for all the cloud management applications is recommended. This can be done by adding the following section:
{code class="brush: xml; gutter: false;" }

<!-- ============================== -->
   <!-- Append messages to the syslog-->
   <!-- ============================== -->
   <appender name="SYSLOG" class="org.apache.log4j.net.SyslogAppender">
     <param name="SyslogHost" value="X.X.X.X"/> 
     <param name="Facility" value="USER"/> 
     <param name="FacilityPrinting" value="true"/> 
     <layout class="org.apache.log4j.PatternLayout">
           <param name="ConversionPattern"
                 value="%t %5r %-5p %-21d{yyyyMMdd HH:mm:ss,SSS} %c{2} [%x] %m %n"/> 
     </layout> 
   </appender>
{/code}

X.X.X.X must be changed to the Syslog server and USER replaced with the right log facility for your specific environment.

Scroll down to the bottom of the file and locate the following section:
{code class="brush: xml; gutter: false;"}

<!-- ======================= -->
   <!-- Setup the Root category -->
   <!-- ======================= -->
   <root>
      <priority value="INFO"/> 
        
      <appender-ref ref="CONSOLE"/>
      <appender-ref ref="FILE"/>         
   </root>
{/code}
Next, add the following in that empty line between INFO and CONSOLE:
{code class="brush: xml; gutter: false;" }
<appender-ref ref="SYSLOG"/>   
{/code}

Enabling DEBUG mode

During overall server troubleshooting it is recommended to set the server and all the components in debug mode. This can be done by changing the log level in the three following sections.

In the section:
{code class="brush: xml; gutter: false;" }
<appender class="org.jboss.logging.appender.RollingFileAppender" name="FILE">
{/code}

Update this line as follows:
{code class="brush: xml; gutter: false;" }
<param name="Threshold" value="DEBUG"/>
{/code}

In the section:
{code class="brush: xml; gutter: false;" }
<appender name="CONSOLE">
{/code}

Update this line as follows:
{code class="brush: xml; gutter: false;" }
<param name="Threshold" value="DEBUG"/>
{/code}

In the section:
{code class="brush: xml; gutter: false;" }
<!-- VMware vCO -->
{/code}

Update this section as follows:

{code class="brush: xml; gutter: false;" }
<category additivity="true" name="ch.dunes">
    <priority value="DEBUG"/>
</category>
{/code}

Putting a vCO server in debug mode slow down considerably the server. It is recommended to use DEBUG mode temporarely.

Concepts covered during the course of the series include:

Part 1
http://www.youtube.com/watch?v=_J5w7Nvv9PQ

• Create a workflow
• Defne input parameters
• Create the schema

Part 2
http://www.youtube.com/watch?v=EcOVTg236ag

• Bind workflow elements with input parameters and attributes

Part 3
http://www.youtube.com/watch?v=TOcGYTsh6Eg

• JavaScript code for the scriptable task elements
• Create the workflow presentation
• Validate the workflow

Need vCO to fetch some data from a database for integration purposes ? There is an easy solution with creating a plug-in ...

Jöerg Lew wrote an excellent article on the different options to integrate vCO with a database here. Creating a hibernate based plug-in is definitely the best way to map a database as vCO objects but it is a lot of effort and can get quite complex.

Since there is another much simpler way to create a database plug-in (with read only rights on the database) I thought I should share it.

First thing is to create your plug-in structure. Create a directory with a name and a "dar" extension.

In this directory create two directories: VSO-INF and resources. In VSO-INF create a text file name vso.xml. This is the file defining the plug-in interface.

Optionaly add a 32x32 pixel picture in the resources directory. This is the picture that will be used in the vCO client.

Now edit the vso.xml file as follows.

This is an example on how to create your database plug-in:
{code class="brush:xml"}
<module name="LogEventDB" display-name="Log event database plug-in" version="1.0.0" build-number="001" image="db-32x32.png" xsi:noNamespaceSchemaLocation="http://www.vmware.com/support/orchestrator/plugin-4-1.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<description>This is an example on how to create your database plug-in</description>
<enumerations>
<dynamic-enumeration type="logevent">
<datasource type="jdbc" driver-class="net.sourceforge.jtds.jdbc.Driver" connection-url="jdbc:jtds:sqlserver://localhost:1433/vco;instance=sqlexpress" username="vco" password="vmware" refetch-timeout="300">
<datasource-select>SELECT severity, logtimestamp, shortdescription, longdescription, originatorusername FROM vmo_logevent ORDER BY logtimestamp</datasource-select>
<datasource-id id-element="shortdescription"/>
<datasource-entry name="severity" element-name="severity" display="Severity"/>
<datasource-entry name="logtimestamp" element-name="logtimestamp" display="Log Timestamp"/>
<datasource-entry name="shortdescription" element-name="shortdescription" display="Short Description"/>
<datasource-entry name="longdescription" element-name="longdescription" display="Long Description"/>
<datasource-entry name="originatorusername" element-name="originatorusername" display="Originator Username"/>
</datasource>
<description>Dynamic list from logevent</description>
</dynamic-enumeration>
</enumerations>
</module> {/code}

Some explanations on the code:

• Module name: The prefix of all the objects in this plug-in.
• Display name: The name that will show up in vCO configuration.
• version, build: Used by the vCO configuration to update on new version.
• Image: The image you have in the resources directory, used as the icon for the plug-in.
• Description: Will show up in the API explorer.
• Dynamic Enumeration logEvent: This is an object of the plug-in.
• Datasource type="jdbc" ... : This is where your object will be fetched from. Here it is with using the JDBC driver that allows to connect to Microsoft SQL Server, Oracle, PostgreSQL, mySQL and other databases (drivers must be downlaoded separately). You will need to change the connection-url. You can generate this URL automatically and test it on your database with running the "JDBC URL generator" workflow in the workflow library. Make sure to set up your username and password as well.
• Datasource-select : This defines the data source for your object properties in the form of a SELECT statement
• datasource-id : This defines the ID of the object. this will be the one showing as a label once you have selected an object. It does not necessarely have to be an unique ID.
• datasource-entry: This defines the name of a property of an object, the matching column from the SELECT statement and the display name shown to the end user.

As you can see this is not very complicated and in my example I am fetching the vCO log events.

Once you have completed this copy the JDBCPlug-in.dar in [your vCO install directory]/app-server/server/vmo/plugins

Open your vCO web configure and go in the plug-ins tab. You should see this.

Restart your vCO service to enable the plug-in.

Open the vCO client. If you look in the inventory you will not see the plug-in since we have not created an inventory in the vso.xml. Create a new workflow, edit it, add a scriptable box. Click on Search API, enter logeventdb or the custom name you gave to your plug-in:

it will find the LogEventDB:logevent object. Double click on it.

It will show you the plug-in with the nice icon you had set in the first step and the LogEventDB:logevent object.

Now let's use it as an input parameter. Create a new workflow with a scriptable task. In the workflow inputs tab, add an input called "log", search for logEvent. You will find 2 objects: vCO native LogEvent and the one from your plug-in. Select yours.

Edit your scriptable task box to add the log input and enter the following script:

System.log(log.logtimestamp + " : " + log.shortdescription);

Link the elements, validate, save.

Start your workflow. Enter something in the filter (here I look for my username). You will see all the records from your SELECT statement listed. Select one and submit.

It will display the Log Timestamp and the short description as defined in the script. You could use any of the information in these columns to feed information into a workflow.

EMC released a Compute as a Service white paper documenting design considerations leveraging vCenter Orchestrator and its vCloud Director, AMQP and REST plug-in

With these components EMC created a workflow running the following operations:

• Provision storage from EMC Symmetrix VMAX or EMC VNX based on the vCenter High Availability cluster.
• Create the datastore.
• Create the provider virtual datacenter.
• Create the organization virtual datacenter within the provider virtual datacenter.
• Create the catalog on the organization datacenter

The whitepaper shows the workflow design, a simple custom web portal to start it and the workflow execution record.

For more information check the EMC Compute as a Service white paper. vCenter Orchestrator is covered from page 27 to page 34.

You have now no more excuses for not using vCenter Orchestrator: you can now have it up and running in a few minutes.

With the vCenter Orchestrator Virtual Appliance the configuration is now reduced to the bare minimum (on the Windows version there is quite a lot of configuration to do and important pre-requisites including a directory service and database).

This is ideal for loading a vCO dev / test environment (including a mobile lab on your laptop) either by importing the provided OVF in vCenter or in Workstation or Fusion (For fusion you will need to use  ovftool in command line).

Here is my Fusion lab after importing the vCO 4.2 VA:

Once you will have imported and started it you will be greated by a "To manage this appliance browse to [hostname]" message. Once you open this in your browser you have a new page with links to everything you need to operate the appliance and get to useful online resources.

Configuration

The first thing you should do is to go to Orchestrator Configuration.

You will be prompted twice to set the Appliance configuration & root password and the access to the web configuration. You will also have the usual browsers certificate warnings. Apart from the orchestrated technologies the appliance is free from external dependencies with embedding OpenLDAP, postgreSQL and sendmail servers. You will see these configured.

The appliance is coming with the same plug-ins as bundled with vCO 4.2 such as vCenter Server 5. There are a few things that still may require configuration:

• To make the vCenter 5 plug-in active a vCenter host needs to be configured. You can add either a vCenter 5 VA or a Windows based vCenter 5.
• For other plug-ins you will need to download them from here (or the resource link on the appliance main page), install and configure them through the vCO web configuration.
• The appliance is coming with a 90 days license. If you have a vCenter license you can enter the key or point to your vcenter to get unlimited usage granted.
• if you want to use an appliance for production use you will certainly need to set an external directory (openLDAP, Microsoft AD, Novell eDirectory or Sun java Directory) and database (PostgreSQL, Oracle or Microsoft SQL server).

One great thing about the appliance is that the vCenter Orchestrator service restarts very quickly. On my setup it is done in close to 20 seconds whereas on a similar Windows setup it requires between 2 and 3 times more.

The appliance configuration allows to change IP & timezone from the UI & to perform future appliance updates.

vCenter Orchestrator Client

Apart from the convenience provided by the super easy configuration there is a feature I really like about this appliance: When you click on "Start Orchestrator Client" the browser will download a JNLP file which will be loaded by java webstart. The first time you will be promped for a certificate, then the client will start almost instantanously. The great thing is the client will start on any JAVA enabled platforms. I am running the client on MacOS 10 but it can run on Linux or Windows as well !

Once prompted by the client you have to enter the hostname or the IP of the server you want to connect to and one of the default account set in openLDAP (user: vcoadmin, password / vcoadmin). There is no user interface driven option offered to change manage openLDAP users and their passwords. You can always do it in command line if you have experience with handling LDIF files. here is the client running on MacOS and for the first time available to VMware customers.

Get it now !

• vCenter Orchestrator Virtual Appliance
• vCenter Orchestrator Virtual Appliance Release notes
• vCenter Orchestrator virtual Appliance User guide

Here is yet another vCenter Orchestrator (vCO) plug-in. This time to enable a set of discovery and monitoring features.

Citing the CIM plug-in summary page:

ESX includes a CIM Object Manager (CIMOM) that implements a set of server discovery and monitoring features. With the VMware CIM SMASH/Server Management API, clients that use industry-standard protocols can do the following:

• Enumerate system resources
• Monitor system health data

The plugin is general enough to support other CIM compliant services and is not limited only to ESX. However the primary goal for developing the plugin was to expose ESX CIMOM in the vCenter Orchestrator. This affected the API design.

Features

Enables the following use-cases without being limited to them.

• Reporting Manufacturer, Model, and Serial Number
• Reporting the BIOS Version
• Monitoring State of All Sensors
• Monitoring State of All Sensors By Using Only the Implementation Namespace
• Reporting Fan Redundancy
• Reporting CPU Cores and Threads
• Reporting Memory Slots

For more information and access to the download please visit the CIM plug-in page on VMware labs.