Discovery: error "Windows proxy overloaded" reported in the credential page and in the DiscoveryAccess

Knowledge Article

Article Number

000207874

Old Article Number

000090905

Article Type

Solutions to a Product Problem

Title

Discovery: error "Windows proxy overloaded" reported in the credential page and in the DiscoveryAccess

Summary

Product

BMC Discovery

Component

BMC Discovery

Applies to

all versions

Problem

The message "overloaded" or "Windows Proxy overloaded testing credential" appears in the credentials, DiscoveryAccess and/or SessionResults

The query below returns some recent entries

search SessionResult where message has subword 'overloaded' traverse :::DiscoveryAccess

Most of the time, the related DiscoveryAccess fail with NoAccess. The message does not appear in the proxy logs.

Cause

Solution

Note:

The proxy uses a maximum number of concurrent threads that is considered an overload threshold.
When this threshold is reached, the proxy rejects scan requests and this is reported as a "windows proxy overloaded" error.
If the request from the appliance to the proxy times out, then the proxy will be reported as overloaded (rather than reporting a CORBA timeout exception).
If the "Windows proxy overloaded" error is reported by the proxy or by the appliance, this may not cause any CPU/RAM saturation on the proxy
Proxies can be configured to send an "overload notification" (see Overload Notification).

Extract: if you do not select this option, BMC Discovery will wait for all the requests in progress to finish, which can take a long time if the Windows proxy is overloaded.

Root cause 1: The proxy tried to exceed the maximum number of scan threads (if the notification is on)

Workaround: Disable the "Overload Notification" option
Solution: add more proxies in the pool or upgrade the hardware of the proxies

Root cause 2: The hardware of the proxy is overloaded

Solution: see the root cause 1

Root cause 3: The network reported many opened ports 80 on ips that are not endpoints. It leads discovery to open many wmi connections. Sometimes, it exceeds the max number of proxy theads.

To confirm:

If the event logs of the proxy host logs 1000+ errors/day like the one below, the root cause is possible.

Error Microsoft-Windows-DistributedCOM 10009 None DCOM was unable to communicate with the computer <ip> using any of the configured protocols.

If the ips mentioned in the error are non-windows hosts with port 135 closed, the root cause is possible.

When this issue happen, some proxy worker processes may look stuck.

Workaround: In Administration > Discovery Configuration, disable "Use HTTP(S) HEAD request to identify OS" and rescan. The affected DiscoveryAccess should end with "Unsupported Device" afterwards.

Root cause 4: Event log about an exception in kernebase.dll (part of the Windows OS running the proxy)

To confirm:

If the proxy accumulates an unreasonable number of tw_proxy_worker processes (> 20) that can't be manually killed, the root cause is probable.
If the cpu of the proxy is idle, this root cause is possible.
If the error below is found in the event log of the proxy, the root cause is confirmed.

Workaround: Reboot the proxy host

Solution: Engage Microsoft Customer Support to fix the kernebase.dll error below.

Attachment(s):