Discovery: How to restart the appliance if DISK_FREE_REQUIRE

Knowledge Article

Article Number

000390337

Old Article Number

Article Type

FAQ/Procedural

Title

Discovery: How to restart the appliance if DISK_FREE_REQUIRE_MB was set to an unreasonable value?

Summary

Discovery: How to restart the appliance if DISK_FREE_REQUIRE_MB was set to an unreasonable value?

Product

BMC Discovery

Component

BMC Discovery

Applies to

BMC Discovery and ITOM On Prem All versions

Question

The settings below are free disk space thresholds.

DISK_FREE_REQUIRE_MB (db and db transaction logs partitions)
DISK_FREE_REQUIRE_MB_LOG (log partition)

They were set to an unreasonable value with tw_options while appliance was running. Afterwards, the appliance shut down and can't be restarted.

In this case, tw_svc_cluster_manager.log contains errors similar to this:

cluster_manager.resourcemonitor: ERROR: Initiating shutdown: free space of the file system mounted on <partition name> now: <number> MB (minimum is <number>)

As the appliance is down, tw_options can't be executed to change these settings.

How to restart the appliance in this case?

Answer

Option 1:

In case of a cluster, run the following procedure on each of the cluster members:

1- Connect to the appliance via ssh

2- In the line below, replace "myPassword" by the password for the Discovery account "system password":

tw_service_control --start model security && tw_options -p system -u myPassword DISK_FREE_REQUIRE_MB=1024 && tw_options -p system -u myPassword DISK_FREE_REQUIRE_MB_LOGS=100

Option 2:

Add the storage expected by Discovery. Review the error message from tw_svc_cluster_manager.log and extract from it:

the amount of free disk space expected by Discovery for this partition. If it is not possible to add a disk meeting this expectation, this option is not feasible.
the partition affected by this issue

If the partition is /usr, add a new disks and move partitions to it until /usr has more than the expected free disk space. If the datastore and/or the datastore transaction logs are still on this partition (default configuration), the free disk space have to be more than DISK_FREE_REQUIRE_MB.

If this is /mnt/addm/logs, add a new disk with more free disk space than the expected free disk space and move the debug logs logs into it.

If this is /mnt/addm/db_logs, add a new disk with more free disk space than the expected free disk space and move the db transactions logs into it.

If this is /mnt/addm/db_data, add a new disk with more free disk space the expected free disk space and move db transactions logs into it. Note that this partition can also be extended with the procedure documented here.

Option 3:

In standalone appliance case: if any available, restore a Discovery backup with tw_restore. This is not possible in cluster case.

Option 4:

In cluster case: if VM snapshots are available, and that they were taken:

- at the same time for all members

- while the appliances were all shutdown during each other snapshots

Then restore the snapshots on all members, and start the appliances one they all have been restored.

Attachment(s):