vCloud Director has a nice feature allowing to organize vApps into catalogs. If like me you need to replicate these catalog across organization or even different vCloud hosts you may find your solution here.

I have written a solution to do it a long time ago but never had the time to release it publicly. It is now done and here are the features.

Features

This package comes with a rich set of features.

Catalog comparison and replication

Replicate  a catalog from one source catalog to a destination organization. If a  catalog with the same name exists in the destination it will be modified  to have the same content as in the source catalog. If not a new catalog  with the same name will be created. The replication is one way (source  to destination).
The actions taken are:

  • Delete catalog items in destination not present in source
  • Update name and description of previously copied catalog items that would have been changed in the destination
  • Copy new source catalog item to destination (download once to vCO, upload to one or more vCD)

Each  of these are optional. Destination VDC is also optional. If  provided  all the elements to be copied will have for destination the  provided  vDC. If not the workflow will find destination VDCs with  matching  names. If not the workflow will raise an exception.

Connected replication

When  using the "Replicate catalog" workflow a single vCO server is used to  download  the vApp Templates from the vCloud Director hosting the source  vApp  Templates and upload these to the other vCloud Director. This  requires  to have the two vCloud hosts configured in vCO.

vCloud Director source -----------> vCO -----------> vCloud Director Destination

Catalog replication.png

While  the vCO server could be located on any network having access to both  vCloud servers the vCO server should be located to optimize the network  speed for example if the upload speed between the source and destination  is a lot slower then the download speed the vCO server should be on the  same network as the destination.

Several instances of  the same workflow can run on the same vCO server  with different  destinations (there is a locking system to avoid uploading a file being  downloaded). Below an example of a vCO server replicating a catalog from  one cloud source to two cloud destinations.

multiple_replications.png 

Replication from filesystem

When  using the "Export catalog to filesystem" a first vCO server is used to  download the vApp Templates from the vCloud Director hosting the source  vApp  Templates. XML files describing the catalog and the catalog items  are also downloaded.

You can then use the "Replicate catalog from  filesystem" workflow to compare the file based catalog to the one on the  destination server.

The main idea behind this two  steps replication is to optimize the replication process by using a  third party replication using deduplication, compression, safe  transfers.

vCloud Director source -> vCO 1 ->  Third party replication system source -----------> Third party  replication system destination -> vCO 2 -> vCloud Director  Destination

A first vCO server is located on the same  network as the source vCloud Director to download the vAppTemplates as  fast as possible to a filesystem hosted on a replication system (this  requires having this filesystem accessible from vCO) . The replication  system copies the changed blocks to the destination replication system.  Once the replication finished the destination vCO upload the new files  to the vCloud DIrector on the same network.

Parallel operations

The  workflows use and abuse asynchronous sub workflows to make several  operations running at the same time. and also copies the VMDK files  within a vApp Templates in parallel. The level of parallelism is  configurable.

The "Replicate catalog" workflow ran.

-1- Replicate catalog.png

It started one to many "Copy catalog item" workflows in parallel

-2- Copy catalog item.png

The "Copy catalog item" ran a single "Copy vApp Template multithreade" workflow

-3- Copy vApp Template multithreaded.png

Which in turn ran one to many "Copy file across vAppTemplates" workflows in parallel

-4- Copy file across vApp Templates.png

Which in turn runs:

  • "Download  single file from vApp Template" workflow (not shown on the picture  since in this case it was already downloaded previously and the workflow  optimize the process with not downloading the vApp Template again)
  • Upload  single file to vApp Template which will run several "Upload File chink  to vAppTemplate" in serial (parallel upload of the chunks is not  supported, they need to arrive in the right order).

Safer uploads

Sending  over a multi GB file over http may cause reliability issues. If the  upload fails you have to restart again. To avoid this the workflows  uploads the files in chunks of a configurable size. If a chunk fails to  be uploaded the workflow retries to upload it hence avoiding the need to  restart completely. The chunk size is configurable.

Here  I have run the same replication as before. At 14:35 I have unpluged my  vCO server network connection to make the file chunk being uploaded  failed. I have then plugged it again to simulate a temporary network  issue. As you can see below the upload was resumed. The file chunk that  was uploaded starting at 14:36:32 seconds was the same as the previous  one (this can be checked by clicking on the workflow run and check the  variables tab) . This is why we had 11 chunks uploaded in the previous  replication and 12 in this one.

Safe-upload.png

The  other advantage is since vCO is an orchestration engine supporting  checkpointing if the replication workflow would be stopped because of  vCO server maintenance (i.e Windows update rebooting) the workflow would  resume from the point where it uploaded the last chunk (as long as the  maintenance time is not longer then the timeout vCloud Director has for  waiting on the vApp Template update which is one hour according to my  observation).

vApp Template Metadata

The  vApp Templates metadatas are copied as part of the catalog replication  (connected or from filesystem). If the metadatas are copied from a  vCloud Host using a System admin organization and credentials to a host  connection using a specific organization then the hidden and read only  metadata will be created as read / write (this is the only option when  connecting as org admin). 

Not included

There are a lot of things that  are doable and other that were not because of lack of API support in  vCLoud Director or lack of time to add more features:

  • Does not replicate medias : There is no API yet to download a media so no replication possible.
  • Does  not reconnect the vApp Template to the networks of the target cloud.  This requires deploying the template, delete it from the catalog match  the network to the ones in th e new environment, capture the modified  vApp as a vApp Template. You can do this manually (one time operation)  or create your workflow to do it (there is something similar already  implemented in the mass VM import workflow in communities). The updated  vAppTemplate will not be considered as a new one and will not be deleted  / overwritten by the replication mechanism.
  • Does not clean up the downloaded files. A separate workflow (scheduled or run by policy) could do this.
  • The update of the catalog item does not update metadatas. I will consider adding it if this is something people ask for.

Documentation

Import the replication package using the vCO client.

The  "Catalog Replication Settings" saves all the configuration in a  configuration element. Open either the vSphere Web client or the vCO  client and search for the "Catalog Replication Settings" workflow.

Catalog Replication Settings

Screen Shot 2012-10-15 at 3.32.36 PM.png

Click on it, then right click / "Run a workflow".

Screen Shot 2012-10-15 at 3.33.07 PM.png

General configuration

The first step is to define a path where vCO will download the vApp Templates. The pre-requesites are:

  • There  is enough disk space in this path (if you use the appliance you will  have to mount an NFS share or create an additional drive)
  • You have set proper share / file permissions
  • The user running the vCO service has access to this path (by default this is not the case if this is a network share)
  • You have changed the vCO access rights as documented here.

Screen Shot 2012-10-15 at 5.19.49 PM.png

Replicate a catalog settings

The  next step is for configuring the vApp Template transfer. This are the  settings used when running the "Replicate a Catalog" workflow. You can  set the number of parallel copies (one copy = 1 download + 1 upload) and  a time out. When timing out the copy will be canceled. This is to avoid  a catalog replication to ait forever on a copy that would be stalled.

Screen Shot 2012-10-15 at 3.35.25 PM.png

The  next setting is to set the number of parallel copies within a single  vApp Template. For example if a vApp Template has several VMs with  several disks these can be copied concurently.

Screen Shot 2012-10-15 at 3.36.05 PM.png

Export a catalog to a filesystem

These  settings apply for the "Export a catalog to a filesystem" workflow.  These are similar settings as above except it has an option to keep some  vApp Template settings during the (lossless) download. If this option  is selected it is not possible to download files concurently within a  vApp Template.

Screen Shot 2012-10-15 at 3.36.29 PM.png

If the lossless option was not selected it is possible as before to provide the number of concurent downloads.

Screen Shot 2012-10-15 at 3.36.56 PM.png

Replicate a catalog from the filesystem

The same settings as above but this time when uploading the vApp Templates.

Screen Shot 2012-10-15 at 3.37.18 PM.png

And  the same settings for the the vApp template file. The last one is for  specifying the size of each chunk when uploading. This allows to have  safer uploads that can be resumed in case of a network issue or vCO host  maintenance.

Screen Shot 2012-10-15 at 3.37.30 PM.png

Operations

Now that the replication configuration is done it is possible to start the replication workflows.

Replicating a catalog using a single vCO server

To replicate a catalog using a single vCO orchestrating a source and destination cloud run the "Replicate a catalog" workflow.

Screen Shot 2012-10-17 at 2.10.33 PM.png

The first parameter is the source catalog.

Screen Shot 2012-10-17 at 2.13.59 PM.png

The  next step is to provide a destination organization and a destination  VDC. If the catalog does not exist in the destination organization it is  created. If it exists it will be updated. A one time full copy will be  required before the workflow can snychronize the changes.

Screen Shot 2012-10-17 at 2.14.57 PM.png

The  next step defines what are the changes to apply. Beware that if you  already have catalog items in the destination catalog the workflow will  delete these on the first run and then copying the ones from the source  catalog if the copy and delete options are selected.

Screen Shot 2012-10-17 at 2.16.14 PM.png

You  have the option to run the synchronization now or to Schedule it.  Scheduling a recurrent task is a good way to automate the  synchronization. Another way is to trigger the workflow using the vCloud  Director notification on catalog changes.

Screen Shot 2012-10-17 at 2.16.43 PM.png

 

The workflow will  synchronize the catalog.

Replicating a catalog to and from the filesystem

To  optimize the replication speed / bandwith usage you may want to use a  third party system leveraging deduplication, compression and other  mechanisms. To do this you need to run the "Export a catalog to a  filesystem" workflow on the vCO server local to your source vCloud  Director. You need to provide the catalog that you want to replicate.

Screen Shot 2012-10-17 at 3.04.06 PM.png 

Then you need to provide the export path. As a default it will use the one you have configured previously.

Screen Shot 2012-10-17 at 3.04.49 PM.png

As before you can run the workflow immediately or schedule it.

Screen Shot 2012-10-17 at 3.05.03 PM.png 

Once the export completed and your third party replication is completed.

 

Run  "Replicate a catalog from the filesystem" on the  vCO server local to  your destination cloud. This vCO server must have access to a copy of  the files that were exported. Provide the full path to the catalog file  (The catalog file name is the catalog name + ".xml").

Screen Shot 2012-10-17 at 3.24.28 PM.png

Select the destination organization and VDC.

Screen Shot 2012-10-17 at 3.25.28 PM.png

Select the synchronization options.

Screen Shot 2012-10-17 at 3.25.48 PM.png

And as always you can run it right away or schedule it.

Screen Shot 2012-10-17 at 3.26.08 PM.png

 

Download

You can find the package on the vCenter Orchestrator Communities