In order to remain within the purchased license entity count, it is necessary to filter the entities that will be imported from vCenter into TrueSight Capacity Optimization (TSCO).

We are looking for a best practice recommendation to filter the entities to be imported.



 

There are two separate sections below depending on your TSCO release and patch level.

You can watch this video to see How to enable VMware ETL Blacklist/Whitelist for Clusters and Standalone hosts via Run configuration:


You can watch this video to see how to perform the creation of Whitelist/Blacklist file:



There is an Import filter configuration section in the Advanced settings.
  
  Enhanced filtering options for the vCenter Extractor Service
  Legacy design VMware guest filtering functionality by UUID 

Enhanced filtering options for the vCenter Extractor Service

In the TrueSight Capacity Optimization (TSCO) vCenter ETL there is entity whitelist functionality available that performs a whitelist match against the HOSTNAME lookup of the VMware guest to determine if it should be imported into TSCO.  A configuration option within the ETL Run Configuration can be used to change the comparison to be done against the NAME lookup (the VMware guest name visible within the tree in vCenter).

The design requires two configuration steps when your whitelist contains VMware guest entities or VMware ghost entities contained within a cluster.

If you want to whitelist VMware guests then you need to do the following:
  (1) In the "Filtering for clusters and standalone hosts" enter the names of the VMware clusters that contain the VMware guests that you want to import into TSCO.
  (2) In the 'Blacklist file path' section specify the VMware guest list regular expression whitelist file

The editable field is called 'Blacklist file path' in the TSCO UI because a field that supported older VMware guest blacklist by UUID functionality has been reused for the newer whitelist functionality.

The whitelist file itself supports two different formats.

In both whitelist file formats the top line of the file must be this:

#FORMAT=v2;TYPE=VM_WHITELIST

Next, you can either specify a list of regular expressions (with one regular expression per line) or you can specify one long regular expression that includes all of the computers to be selected by the whitelist.

So, if you have a regular expression list like this:
  ^guestabc.*|^guest123.*|^guesta1b2c3.*|…

Then that can be specified with all the VMware guests listed on a single line as a large regular expression like this:
  (^guestabc.*|^guest123.*|^guesta1b2c3.*|…)

Alternately, each line can list a separate VMware guest hostname regular expression, like this:

#FORMAT=v2;TYPE=VM_WHITELIST
^guestabc.*
^guest123.*
^guesta1b2c3.*
...

In the vCenter Extractor Service (but not the VMware History Extractor), when VMware guest entities are included in the whitelist  that will trigger the collection of the VMware guests, hosts on which those guests reside, and clusters in which those hosts reside.

Whitelist filtering against the NAME lookup versus the HOSTNAME lookup

By default the vCenter Extractor Service compares the VMware guest entity HOSTNAME lookup against the specified whitelist to determine if an entity passes the whitelist.  The whitelist comparsion can be run against the VMware guest NAME lookup field instead via an ETL Run Configuration change.

To change the vCenter Extractor Service to filter based upon the NAME rather than the HOSTNAME set the following property within the vCenter Extractor Service ETL configuration:
  extract.vmware.use.vm.name.for.whitelist = true

To do that:

(1) Edit the vCenter Extractor Service Run Configuration (Administration -> ETL & System Tasks -> ETL Tasks -> Select your ETL from the list -> Click the "edit this run configuration" button (within the 'Run configurations' section of the ETL) for the 'Active' run configuration.

(2) Click the "this page" link at the bottom of the Edit run configuration screen in the "You can manually edit ETL properties from this page." text.

(3) In the 'Add new property' edit box at the bottom of the page, add the following new property:
  extract.vmware.use.vm.name.for.whitelist

(4) Click the 'Add' button.  This will add the property to the list.

(5) In the property list, find the new extract.vmware.use.vm.name.for.whitelist property and change the set value from "#INSERT VALUE#" to "true".  Click the 'Save' button to save your change.

(6) You must restart the vCenter Extractor Service ETL.  Your vCenter Extractor Service should now be configured to apply VMware guest filtering on the NAME field rather than the HOSTNAME field.

Q: When using the vCenter Extractor Service VMware guest whitelist, what entities will be enabled for data collection from vSphere?

Once a VMware guest is whitelisted it actives its parent cluster in the whitelist.  So that activates the collection of data for the cluster in which it resides once a cluster is activated all of the entities directly associated with it will also be activated for collection in that ETL.  So the whitelist controls which VMware guests will have their data imported into TSCO (although on the backend all the VMware guest data is still being collected since it needs to be in order to aggregate metrics up to the cluster level), and additionally all of the other objects associated with the cluster in which those VMware guests reside will also be imported into TSCO (VMware hosts, datastores, datastore clusters, and so on).

Legacy design VMware guest filtering functionality by UUID

NOTE: For almost all use cases the VMware whitelist functionality described above is the best choice.  This section covers the old VMware guest blacklist by UUID functionality that still exists in the ETL configuration but would almost never be the optimal way to control the set of VMware guest entities for collection in an environment.

An important note

Some cluster metrics are computed by the ETL aggregating data of the VMs.
If you make a blacklist/whitelist at VMs level, some metrics at cluster level will report incorrect values.

Using vCenter Extractor Service properties

There is Whitelist/Blacklist functionality within the vCenter Extractor Service.

In order to enable this filter, you have to edit the Run Configuration of a vCenter ETL, click on Advanced and open the tab Import Filter
Here you can select:

• None (default): no filter
• WHITELIST - Import only list of clusters and standalone hosts
• BLACKLIST - Do not import list of clusters and standalone hosts

You have to fill the list of clusters and standalone hosts (hosts not under a cluster) with the name of the entities, separated with semi-column.
 
Note: that the filtering is done at the cluster level and standalone hosts. You can't filter specific hosts below a cluster.
 
A common use case, is the data import only for host related to production environment, excluding the ones related to test environment.
If these hosts are under clusters and you have clusters divided by environments, it is possible blacklist the clusters you don't need.
For example, you have one cluster with all the production hosts and one cluster with all the test hosts. In this case, the filtering is simply.
But, if the hosts are distributed across clusters, there is no way to operate the filter.
For example, you have Cluster_1 that contains test_host_1 and prod_host_1 and Cluster_2 that contains test_host_2 and prod_host_2. In this use case, there is no way to import only prod_host_1 and prod_host_2

In the form, there is also the property Blacklist file path
With this property, you can specify a list of VMs and hosts that you want exclude from the import.
This list is based on the UUID, and not on the name, as the previous ones.

The file format is: SYSTEM_TYPE;ENTITY UUID

For example:
For VMware Host,
vh:vmw;44454c4c-4600-1054-8052-cac04f525231

For VMware Guest,
gm:vmw;4208badb-6a91-23d1-c6b5-061745b2c8d9

The vCenter Extractor Service doesn't support a VMware guest level whitelist or blacklist using the guest's hostname, but one can exclude VMware guests from import by UUID using the blacklist file functionality.

This way has two problems:

1. You need to know the UUID of the entities you want filter
2. You need to know these UUIDs before the entities will be imported the first time. Since this is a blacklist mechanism, if you don't list in the file the UUID before the ETL runs, it is possible that VMs that you don't want import are created into CO

The UUID of the entities are stored in the Lookup, and the best approach  to get this information is by using the SQL against the public view pv_ent_lookup.
This Knowledge Article  gives additional additional Background.
  000130306: TSCO - How can I get information regarding the Entity Lookup fields for a system outside of the TSCO console?

WARNING: If the blacklist file contains an empty line then the rest of the file will not be processed by the ETL
 

Using a separate ETL to define entities for import

A possible solution if you need the whitelist of the VMs is create a CMDB ETL that create the list of the VMs you want import.
Next, you have to configure your vCenter ETL in order to use the entity catalog of this ETL and configure the loader of the vCenter ETL to skip entity creation


Typically this design is used when there is a CMDB that acts as an authoritative source of which entities should have data imported for them, but it would also be possible to implement this type of a custom whitelist ETL by running a custom query against the vCenter to build the entity list and then apply the necessary filtering based upon the entity hostname.
The benefit of this design over the blacklist file is that it would prevent the possibility of entities from being mistakenly created in CO because they hadn't yet been added to the blacklist.

Please look at this article for the sharing of entity catalog between a vCenter ETL and a CMDB ETL
https://bmcsites.force.com/casemgmt/sc_KnowledgeArticle?sfdcid=000025065