Providing customized workflows to an internal or an external customer is a process following a typical software development lifecycle with specific orchestration requirements and best practices for setting up the environments and managing the content lifecycle.
1.1 Workflow Development life cycle
This section summarizes briefly the different steps to observe to deliver custom workflows (or any custom application).
1.1.1 Requirement gathering
Gathering requirement consists of interviewing the customer on project characteristics such as planning, budget, scope, prioritization and constraints but also particular specific technical requirements such as:
- The operations to be automated (must be defined, documented, reliable, repeatable)
- The system environment, the external system to be integrated, their interfaces
- The data flows
- The users and roles
1.1.2 Functional specifications and Effort estimate
Functional specifications and level of effort consists of a restatement of the requirements with a matching high-level workflows based implementation and an effort estimate for each resulting task. The tasks may be broken down into different milestones to adjust to the aforementioned project characteristics. This should be documented and signed off by the customer. In the case of an external customer effort will be translated to cost and be included with the functional specifications in a Statement of Work.
Upon agreement between the customer and the delivery team the solution is designed following the high level implementation defined in the functional specification. This consists of architecting how the different elements of the solution work together with the external systems, the use of existing components such as plug-ins and workflows, and the development of custom ones.
The development consists of breaking down the different development tasks and assigning them to available development resources that will create the elements of the solution.
The test consists of checking that the solution is reliable and conforms to the functional specifications. Testing requires setting up an environment that simulates the target environment.
The solution implementation consists of installing and configuring the solution in a pre-production and / or production environment and demonstrating that it works as expected.
The solution support consists of handling support requests, checking that the solution conforms to the specification, and if not troubleshooting and providing bug fixes to the customer.
1.2 Orchestration Content life cycle
The orchestration content life cycle is the process of staging the elements of the solution from development to test, test to pre-production and pre-production to production.
Figure 1. Orchestration Content Life Cycle
Packages are used for exporting content from one Orchestrator server and importing them to another server. Packages can contain workflows, policies, actions, web views, configurations, and resources. Packages and/or their individual elements can also be synchronized directly from one server to another one as long as they are interconnected.
At creation time, Packages manage dependencies between package elements by adding automatically the missing elements. During a package import the server analyzes and displays differences giving control to the admin on which element to import or not. Packages use X509 certificates to monitor which users export and redistribute elements.
After meeting the required quality criteria for a release the package should be exported and stored either on a backed up file system or on a repository vCenter Orchestrator server (a server used specifically for storing packages). At export time to the file system there are options to:
- Encrypt the package.
- Not export the elements version history.
It is also possible to use the synchronization feature to synchronize the packages from one vCenter Orchestrator server to another one.
vCenter Orchestrator configuration elements should be used for all the attributes that have dependencies on the environment. This allows moving the orchestration content from an orchestration server to another one without having to edit the workflows to modify the attribute values. It is recommended to provide configuration workflows with the solution to set these once and to update the configuration attribute values when needed.
1.3 Orchestrated Cloud Environments
There are specific environments for some of the solution development life cycle. Here the orchestrated environment are clouds but these could be virtual iinfrastructure as well.
Figure 2. Orchestrated Cloud Environments
1.3.1 Developer environment
The recommended development environment is as follows:
- One development server per developer. It is not recommended to share a single development server between different developers for the following reasons:
- The server may have to be restarted to install or upgrade plug-ins under development thus creating downtime.
- The server may have to be configured for a particular environment for a given developer.
- Changing the same element (For example: editing a workflow) at the same time is not supported.
- It is recommended to use the developer workstation as the vCO development server:
- Integration within the development tool chain (For example the plug-in development environment).
- The vCenter Orchestrator client used to develop the workflows relies on its connection to the server. If there is a network disconnection the changes done since the last save are lost.
- For unit testing purposes the developer server should be configured in a cloud development environment plus additional integrations when required. This can either be a simple small environment per developer or shared amongst developers. Ideally this environment should be local but can be remote. For example a vCenter orchestrator Client and server can be installed on a laptop and used to connect over the Wide Area Network to vCloud Director.
- The vCenter Orchestrator client is not optimized for connecting over the Wide Area Network to a vCenter Orchestrator server and not working through a firewall. If remote access is required to a vCenter Orchestrator server than remote desktop should be used to start a vCenter orchestrator client install on the same server as the vCenter Orchestrator server.
1.3.2 Test Environment
There should be at least a vCenter Orchestrator server dedicated for testing:
- This server is mainly use for testing the solution and report bugs back to the development team.
- This should be a newly installed vCenter Orchestrator server with the same specifications as the target production vCenter Orchestrator server. This can be deployed from a template, reverted to snapshot when decommissioning an existing test environment or by creating a new vCenter Orchestrator Database.
- The cloud environment should be different than the developer environment and as close as possible in configuration to the production environment.
1.3.3 Pre-production environment
For validating the solution with the customer there should be a vCenter Orchestrator test server in a pre-production environment (connected to the production environment but not orchestrating business critical items or orchestrating copies of these). This server will be used for final validation testing and demonstration to the customer.
1.3.4 Production Environment
If the end customer is a cloud provider and the orchestration solution applies to multiple organizations workflows should be used as a cloud back-end extension with respecting the organization multi-tenancy.
If the end customer is an organization tenant and the orchestration solution is to automate and integrate with his specific environment than a vCenter Orchestrator should be deployed within his organization and connected to an external network having access to the vCloud Director API.
1.3.5 Support Environment
In order to provide support in a timely manner it is necessary to stand up the customer environment quickly. While this could be rebuilt from the customer delivered packages and plug-ins archives, keeping a copy of the customer environment as a vApp allows resuming the specific environment quickly.
VMware added Cluster support to vCenter Orchestrator 5.5 and newer. It is important to note that it is not supported to connect the client to an Orchestrator Cluster, only to individual node(s). The Load Balanced address (VIP) is only supported for use via Orchestrator's API. For example, pointing vRealize Automation or vSphere Web Client to the Load Balanced Address for specifying the default Orchestration server.
Getting content onto nodes may be accomplished as follows:
- Export package from Test/Dev
- Connect Orchestrator Client to one node of cluster
- Import package to node
- Right Click Package and select Synchronize
- Specify a node of the cluster as host, provide credentials, and login
- Review differences and Submit
- Configure the Multi-Node plug-in on your Test/Dev server to point to your Production Cluster
- Use multi-node workflows to deploy package from Test/Dev server to production cluster
- Use the REST API to import the package to the cluster
- PowerShell Example available on Github
- Packages only need to be imported to single node
- Plug-ins must be loaded on EACH NODE. For example, to add or update the vCloud Director plug-in on a vCO 5.5.x Cluster:
- Install vCO vCD Plug-in and import all vCD hosts SSL certificate in all vCO nodes
- Stop vCO services on all vCO nodes except one
- Add/Configure all vCD hosts on the vCO node that has a running vCO services
- Copy vCloud.xml file through all vCO nodes with stopped vCO services (for vCO VA the files is located at /etc/vco/app-server/plugins)
- Start all vCO services on all vCO nodes that are with previously stopped vCO services
- Plug-in configuration workflows likely need to be run on EACH NODE - confirm by checking inventory on node to see if config workflow(s) must be run