The BDSSA ETL job fails quickly with the following error even though no BSA Cleanup job us running at the time:

Error Description: ODI-1227: Task CHECK_BSA_DB_CLEANUP (Procedure) fails on the source ORACLE connection BSA_phy_om_svr_site1. 
Caused By: java.sql.SQLException: ORA-20001: Online DB Maintenance is running so cannot start ETL.Please wait till completion 
ORA-06512: at line 64 

Running the following SQL Query on the Bladelogic schema confirms the flag is indeed set to "Yes" even though BSA Cleanup is not running:

SELECT Name, Value FROM system_property WHERE name =  'OnlineDBMaintenanceInProgress';

OnlineDBMaintenanceInProgress 
Yes

Root Cause 1 - BSA Cleanup Defect QM002231132 - BSA online cleanup does not reset OnlineDBMaintenanceInProgress flag back to No when "Deletable Threshold rows limit" is exceeded

The cleanupHisotricalData commands first check the backlog of rows which need to be hard-deleted and, if the number of rows is detected to be higher than the default threshold of 100 million rows, a message such as the following is printed:

Info,02/07/2017 07:36:34,[Tue Feb 07 07:36:34 PST 2017] Begin dbm_online.prevalidate
Info,02/07/2017 07:36:34,[Tue Feb 07 07:36:34 PST 2017] Projected deletable rows using computed statistics are - 259050150. Deletable Threshold rows limit is - 100000000
Info,02/07/2017 07:36:34,[Tue Feb 07 07:36:34 PST 2017] End dbm_online.prevalidate
Info,02/07/2017 07:36:34,[Tue Feb 07 07:36:34 PST 2017] Projected DBM Online delete volume exceeds threshold limt which is "100000000". Use blasadmin command "set cleanup ForceDBMOnline true" to run DBM Online Forcefully but Recommendattion is to run DBM Online first
Info,02/07/2017 07:36:34,[Tue Feb 07 07:36:34 PST 2017] Finished Executing stored procedure: ComplianceResult

Defect QM002231132 tracks an issue where, after the above message is displayed and cleanup exits, the OnlineDBMaintenanceInProgress flag is not set back to 'No' so the next ETL run will fail.
If the highlighted text above does not appear in the DB cleanup job which preceded the failing ETL, or if you are running BSA 8.9.01 where this issue is resolved, this is not the root cause of your occurrence so proceed to Root Cause 2 below.

To workaround the issue, the following options exist:

1) Take an outage and run the BSA DBM Offline cleanup
2) Run the following blasadmin command to have online cleanup process the backlog. This may result in some lengthy cleanup runs:

blasadmin -a set cleanup ForceDBMOnline true

3) If a quick solution is required to get ETL to run immediately, the following SQL update can be run on the bladelogic db schema before rerunning ETL

Update system_property set Value = 'No' where name =  'OnlineDBMaintenanceInProgress';
commit;

This will allow the next ETL run to execute but the issue will likely return until either #1 or #2 above are completed.

Defect QM002231132 is resolved in BSA 8.9.01.


Root Cause 2 - QM002312403 DB cleanup - SharedData stored procedure is not unsetting OnlineDBMaintenanceInProgress flag when duration is encountered

In this case, the issue is caused by some of the BSA Cleanup procedures exiting due to a max duration being hit and not unsetting the 'OnlineDBMaintenanceInProgress' flag. In his case, the output will be as follows:

Run at 05/22/2017 04:00:40,Info,05/22/2017 04:10:40,[Mon May 22 04:10:40 UTC 2017] Executing stored procedure: SharedData
Run at 05/22/2017 04:00:40,Info,05/22/2017 11:08:04,[Mon May 22 11:08:04 UTC 2017] The task has been interrupted due to duration timeout.
Run at 05/22/2017 04:00:40,Info,05/22/2017 11:08:04,[Mon May 22 11:08:04 UTC 2017] Finished Executing stored procedure: SharedData
Run at 05/22/2017 04:00:40,Info,05/22/2017 11:08:04,[Mon May 22 11:08:04 UTC 2017] Cleanup execution expired (maximum duration 415 minutes).
Run at 05/22/2017 04:00:40,Info,05/22/2017 11:08:04,[Mon May 22 11:08:04 UTC 2017] Executing available patches.
Run at 05/22/2017 04:00:40,Info,05/22/2017 11:08:37,[Mon May 22 11:08:37 UTC 2017] Successfully executed available patche

Not all cleanup procedures have this problem. Defect QM002312403 has been created to resolve those with the problem and is targeted for BSA 8.9.02.

If you encounter this situation and a quick solution is required to get ETL to run immediately, the following SQL update can be run on the bladelogic db schema before rerunning ETL

Update system_property set Value = 'No' where name =  'OnlineDBMaintenanceInProgress';
commit;

If a longer term solution is required, contact BMC Customer Support and an updated dbm_online.sql script can be provided for your BSA version and DB platform to resolve the issue.