This knowledge article may contain information that does not apply to version 21.05 or later which runs in a container environment. Please refer to Article Number 000385088 for more information about troubleshooting BMC products in containers.
Option 1: Enable HTTP Wire Logs (Recommended option)
Option 2: Setup a Proxy in AR Server
How to Read Wire Logs
Additionally for HTTPS (HTTPS Troubleshooting)
The wire log is used to log all data transmitted to and from servers when executing HTTP requests. The wire log uses the org.apache.http.wire logging category. This log should only be enabled to debug problems, as it will produce an extremely large amount of log data.
Steps:
1.- Modify the AR Server file: logback_server.xml located under: <ARSystem>/Conf dir.
(Take a backup just in case).
2.- Add the following code after line number 3 (after the <configuration> tag) depending on your AR Server version
For AR Server 9.1.04 and 18.05:
<!-- Web service log configuration -->
<appender name="webServicelog" class="com.bmc.arsys.logging.ReconfigurableRollingFileAppender">
<file>${com.bmc.arsys.homedir}/${com.bmc.arsys.server.dbdir}/webservice.log</file>
<append>true</append>
<param name="Encoding" value="UTF-8" />
<rollingPolicy class="com.bmc.arsys.logging.StoredRollingPolicy">
<fileNamePattern>${com.bmc.arsys.homedir}/${com.bmc.arsys.server.dbdir}/webservice.log.%i</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>8</maxIndex>
</rollingPolicy>
<triggeringPolicy class="com.bmc.arsys.logging.ARSizeBasedTriggeringPolicy">
<maxFileSize>128MB</maxFileSize>
</triggeringPolicy>
<!-- Extra param to say log file creation at startup -->
<needPreLoading>true</needPreLoading>
<encoder>
<pattern>%d{EEE MMM dd HH:mm:ss.SSS yyyy} [%thread] %-5level %logger{36} %m%nopex%n</pattern>
</encoder>
</appender>
<logger name=" org.apache.http"
level="debug" additivity="false">
<appender-ref ref="webServicelog" />
</logger>
For AR Server 18.08 to 19.08:
<!-- Web service log configuration -->
<appender name="webServicelog" class="com.bmc.arsys.logging.ReconfigurableRollingFileAppender">
<file>${com.bmc.arsys.homedir}/${com.bmc.arsys.server.dbdir}/webservice.log</file>
<append>true</append>
<param name="Encoding" value="UTF-8" />
<rollingPolicy class="com.bmc.arsys.logging.StoredRollingPolicy">
<fileNamePattern>${com.bmc.arsys.homedir}/${com.bmc.arsys.server.dbdir}/webservice.log.%i</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>8</maxIndex>
</rollingPolicy>
<triggeringPolicy class="com.bmc.arsys.logging.ARSizeBasedTriggeringPolicy">
<maxFileSize>128MB</maxFileSize>
</triggeringPolicy>
<!-- Extra param to say log file creation at startup -->
<needPreLoading>true</needPreLoading>
<encoder>
<pattern>%d{EEE MMM dd HH:mm:ss.SSS yyyy} [%thread] %-5level %logger{36} %m%nopex%n</pattern>
</encoder>
</appender>
<logger name="com.bmc.arsys.ws.client" level="debug" additivity="false">
<appender-ref ref="DebugLog"/>
<appender-ref ref="webServicelog" />
</logger>
<logger name="org.apache.commons.httpclient" level="debug" additivity="false">
<appender-ref ref="DebugLog"/>
<appender-ref ref="webServicelog" />
</logger>
<logger name="com.bmc.arsys.server.domain.xml.MappingXMLParserImpl" level="debug" additivity="false">
<appender-ref ref="DebugLog"/>
<appender-ref ref="webServicelog" />
</logger>
<logger name="com.bmc.arsys.server.domain.runtime.impl.actions.SetFieldsFromWebServiceActionExecutor" level="debug" additivity="false">
<appender-ref ref="DebugLog"/>
<appender-ref ref="webServicelog" />
</logger>
<logger name="com.bmc.arsys.ws.client.WebServiceSoapClientImpl" level="debug" additivity="false">
<appender-ref ref="DebugLog"/>
<appender-ref ref="webServicelog" />
</logger>
<logger name="org.apache.http" level="debug" additivity="false">
<appender-ref ref="webServicelog" />
</logger>
For 20.02 and forward:
<!-- Web service log configuration -->
<appender name="webServicelog" class="com.bmc.arsys.logging.ReconfigurableRollingFileAppender">
<file>${com.bmc.arsys.homedir}/${com.bmc.arsys.server.dbdir}/webservice.log</file>
<append>true</append>
<param name="Encoding" value="UTF-8" />
<rollingPolicy class="com.bmc.arsys.logging.StoredRollingPolicy">
<fileNamePattern>${com.bmc.arsys.homedir}/${com.bmc.arsys.server.dbdir}/webservice.log.%i</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>8</maxIndex>
</rollingPolicy>
<triggeringPolicy class="com.bmc.arsys.logging.ARSizeBasedTriggeringPolicy">
<maxFileSize>128MB</maxFileSize>
</triggeringPolicy>
<!-- Extra param to say log file creation at startup -->
<needPreLoading>true</needPreLoading>
<encoder>
<pattern>%d{EEE MMM dd HH:mm:ss.SSS yyyy} [%thread] %-5level %logger{36} %m%nopex%n</pattern>
</encoder>
</appender>
<logger name="httpclient.wire" level="debug" additivity="false">
<appender-ref ref="webServicelog" />
</logger>
<logger name="com.bmc.arsys.ws" level="debug" additivity="false">
<appender-ref ref="webServicelog" />
</logger>
3.- Once these settings are applied you will need to reload the configuration changes on log back server, This can be done from the AR System Administration Server Information form on each AR server:
Reload Configuration > Apply
Important Note*:
AR Server defect SW00557654 may cause that when hitting the Reload Log Conf File your log doesn't get generated.
If your webservice.log is not generated, restart your AR Service.
Steps:
1.- First, we have to enable the Proxy settings according to your AR Server version:
For version 9.1.04:
Proxy settings are JVM Options in arserver.config file, you have to add the following:
jvm.option.25=-Dhttp.proxyPort=8888
jvm.option.26=-Dhttp.proxyHost=xxx.xx.xxx.xxx
Important Note:
(Make sure to follow the number sequence in your file)
For versions 1805 or higher:
1.1) Modify the file
C:\Program Files\BMC Software\ARSystem\Conf\axis\axis2.xml
1.2) Before the last </axisconfig> tag add
<parameter name="Proxy">
<Configuration>
<ProxyHost>xxx.xx.xxx.xxx</ProxyHost>
<ProxyPort>8888</ProxyPort>
<ProxyUser>EXAMPLE\test</ProxyUser>
<ProxyPassword>test</ProxyPassword>
</Configuration>
</parameter>
For 19.08:
arserver.config file, as below:
jvm.option.32=-Dhttp.proxySet=true
jvm.option.33=-Dhttp.proxyHost=prod.mydomain.com
jvm.option.34=-Dhttp.proxyPort=8080
jvm.option.35=-Dhttp.proxyUser=itsm
jvm.option.36=-Dhttp.proxyPassword=*******************
jvm.option.37=-Dhttp.nonProxyHosts="localhost|127.0.0.1|10.*.*.*|*.mydomain.com"
2.- Then, use any HTTP Capture tool as a Proxy such as Fiddler with these options:
The previous logger will create an output of HTTP wire logs, the log will be stored in the DB directory such as C:\Program Files\BMC Software\ARSystem\Arserver\Db\webservice.log
The content of the log: (truncated for readability)
DEBUG org.apache.http.wire >> "POST http://midtierserver:8080/arsys9103/services/AR...
DEBUG org.apache.http.wire >> "Content-Type: application/soap+xml; charset=UTF-8; ac
DEBUG org.apache.http.wire >> "Host: midtierserver[\r][\n]"
DEBUG org.apache.http.wire >> "User-Agent: Axis2[\r][\n]"
DEBUG org.apache.http.wire >> "Content-Length: 469[\r][\n]"
DEBUG org.apache.http.wire >> "Proxy-Connection: Keep-Alive[\r][\n]"
DEBUG org.apache.http.wire >> "[\r][\n]"
DEBUG org.apache.http.headers >> POST http://midtierserver:8080/arsys9103/services/A
DEBUG org.apache.http.headers >> Content-Type: application/soap+xml; charset=UTF-8; a
DEBUG org.apache.http.headers >> Host: midtierserver
DEBUG org.apache.http.headers >> User-Agent: Axis2
DEBUG org.apache.http.headers >> Content-Length: 469
DEBUG org.apache.http.headers >> Proxy-Connection: Keep-Alive
DEBUG org.apache.http.wire >> "<?xml version='1.0' encoding='UTF-8'?><soapenv:Envelo
DEBUG org.apache.http.wire >> " <ns0:userName>Demo</ns0:userName>[\n]"
This will also show information on how the axis client is configured as
Wed Sep 26 03:44:33.798 2018 [pool-Fast-thread-11] DEBUG o.a.a.t.h.i.h.HTTPSenderImpl Configuring HTTP proxy.
Wed Sep 26 03:44:33.841 2018 [pool-Fast-thread-11] DEBUG o.a.h.i.c.PoolingClientConnectionManager Connection request: [route: {}->http://midtierserver:8080][total kept alive: 0; route allocated: 0 of 200; total allocated: 0 of 200]
Wed Sep 26 03:44:33.851 2018 [pool-Fast-thread-11] DEBUG o.a.h.i.c.PoolingClientConnectionManager Connection leased: [id: 0][route: {}->http://midtierserver:8080][total kept alive: 0; route allocated: 1 of 200; total allocated: 1 of 200]
Wed Sep 26 03:44:33.853 2018 [pool-Fast-thread-11] DEBUG o.a.h.i.c.DefaultClientConnectionOperator Connecting to midtierserver:8080
This information is needed to verify if the proxy settings are enforced
In this snippet, the proxy is enabled:
Wed Sep 26 04:05:11.090 2018 [pool-Fast-thread-9] DEBUG o.a.a.t.h.i.h.HTTPSenderImpl Configuring HTTP proxy.
Wed Sep 26 04:05:11.136 2018 [pool-Fast-thread-9] DEBUG o.a.h.i.c.PoolingClientConnectionManager Connection request: [route: {}->http://xxx.xx.xxx.xxx:8888->http://midtierserver:8080][total kept alive: 0; route allocated: 0 of 200; total allocated: 0 of 200]
Wed Sep 26 04:05:11.146 2018 [pool-Fast-thread-9] DEBUG o.a.h.i.c.PoolingClientConnectionManager Connection leased: [id: 0][route: {}->http://xxx.xx.xxx.xxx:8888->http://midtierserver:8080][total kept alive: 0; route allocated: 1 of 200; total allocated: 1 of 200]
Wed Sep 26 04:05:11.148 2018 [pool-Fast-thread-9] DEBUG o.a.h.i.c.DefaultClientConnectionOperator Connecting to xxx.xx.xxx.xxx:8888
Modify file C:\Program Files\BMC Software\ARSystem\arserver.config
add
jvm.option.27=-Djavax.net.debug=all
Consider that number 27 should be adjusted accordingly
This will produce logs to troubleshoot SSL handshake errors. This information will be available on the arerror.log
eg:
Wed Sep 26 03:00:13.436 2018 trustStore is: C:\Program Files\Java\jre1.8.0_161\lib\security\cacerts
Wed Sep 26 03:00:13.439 2018 trustStore type is : jks