Knowledge Article
ARLogAnalyzer Version 3 for ARSystem versions 2x.xx, 19.xx, 18.xx, 9.1.xx
ARLogAnalzyer Version 3 is now available for all supported versions of ARSystem.
Remedy AR System Server
AR System
How can I analyze my API, SQL, Filter, and escalation logs from ARSystem in order to identify performance issue or just to understand the work that ARServer is performing? |
BMC is providing ARLogAnalyzer version 3 to provide detailed analysis of your API, SQL, Filter, and Escalation logs Version 3 replaces all prior versions and is provided as a Java program. It is designed to read logs files from ARSystem 9.1.x and above but will likely read logs as far back as 8.x. Announcement: Version 3.1 available Nov. 18, 2020. See ‘What’s new in 3.1’ below To learn more, watch the Connect with BMC Helix ITSM and Remedy – Leveraging the New Java AR Log Analyzer Webinar ARLogAnalyzer reads an AR server log file (or multiple log files) and prepares a report containing breakdowns of server activities reported in the log(s), including longest-running API, SQL, Filter, and Escalation calls, longest-queued API calls, and thread summaries. The report is used to highlight performance trouble spots and to help guide AR System administrators in performance tuning. Some of the key features of the ARLogAnalyzer are:
Installation: To install, simply unzip the attached package into a directory of your choice such a C:\ARLogAnalyzer (Windows) or /opt/ARLogAnalyzer (Unix) For ease of use, you can add this directory to your PATH environment variable. Or you can simply include the full path when you execute ARLogAnalyzer Note: It is assumed that the Java directory is in your path. Usage: There are multiple ways to run the utility: 1. Drag and Drop a single file or a folder onto the appropriate batch file provided (Windows). 2. Execute one of the batch files or shell scripts according to your needs (Windows or Unix) 3. Execute the java command line to run ARLogAnalyzer.jar with the appropriate options The utility provides the following batch files and shell scripts
Drag and Drop In Windows, you can open 2 windows in Windows Explorer, one with the ARLogAnalyzer folder and one with the directory containing your log file(s) or directories containing log files. If you are analyzing a single log file, drag and drop the log file from the log folder onto the one of the provided batch files. For text-only, an output file of the form "<log file name>.ARLogAnalyzer.log" will be created in the same directory as the log file.
For web output, the results will be contained in a folder with the name "<log file name> analysis" in the same directory as the log file.
If you are analyzing one or more individual files, drag and drop the log files from the log folder onto the one of the provided batch files.
For text-only, an output file of the form "multiple.ARLogAnalyzer.log" will be created in the same directory as the log file.
For web output, the results will be contained in a folder with the name "multiple analysis" in the same directory as the log file.
If you will be analyzing a folder with multiple log files, drag and drop the folder onto one of the batch files that processes all files in a provided directory.
For text-only, an output file of the form "<log file name>.ARLogAnalyzer.log" will be created at the same directory tree level as the directory containing the logs..
For web output, the results we be contained in a folder with the name "<log file name> analysis" at the same directory tree level as the directory containing the logs.
If you will be analyzing multiple folders with one or more log files in each folder, drag and drop the folders onto one of the batch files that processes all files in a provided directory.
For text-only, an output file of the form "multiple.ARLogAnalyzer.log" will be created in the same directory as the log file.
For web output, the results will be contained in a folder with the name "multiple analysis" in the same directory as the log file.
Executing the batch file or shell script Each batch file or shell script requires one or more input parameters; either the single filename, the directory name that contains multiple files, or multiple files or directories. For example: Windows -
CD \LogRepository
C:\ARLogAnalyzer\analyzeFolder.bat arsql.log
Unix -
cd /tmp/logrepository
/opt/ARLogAnalyzer/analyzeFolder.sh arsql.log Executing the Java command The basic command is java -jar ARLogAnalyzer.jar [options] [logfilename] The complete listing and definition of all options is documented in the webTemplates/ARLogAnalyzer.html file included in the package. For example: Windows -
CD \LogRepository
java -jar C:\ARLogAnalyzer\ARLogAnalyzer.jar -w myanalysis arapisql.log
Unix -
cd /tmp/logrepository
java -jar /opt/ARLogAnalyzer/ARLogAnalyzer.jar -w myanalysis arapisql.log Large Log Sets If you will be analyzing a large amount of logs, you may want to add a couple of Java options to help utility memory more efficiently. The ARLogAnalyzer may use Java heap space in the amount of 3.5 to 4 times the size of the log files. For example, if you are analyzing 5 GB of logs, it may use up to 20 GB of heap. The utility effectively uses swap space so you will not run out of heap. But it may run faster if you specify a heap size and use the ConcMarkSweep Garbage Collection method. You can edit the batch file or shell scripts or simply run from the command line and add the following: -Xmx20g -XX:+UseConcMarkSweepGC For example: Windows -
CD \LogRepository
java -Xmx20g -XX:+UseConcMarkSweepGC -jar C:\ARLogAnalyzer\ARLogAnalyzer.jar -w myanalysis arapisql.log
Unix -
cd /tmp/logrepository
java -Xmx20g -XX:+UseConcMarkSweepGC -jar /opt/ARLogAnalyzer/ARLogAnalyzer.jar -w myanalysis arapisql.log Accessing the Report If you used text-only output you can simply open the output file in a text editor/viewer. The report will have the following sections within the file, depending on which log-types you analyzed: The top section is the General Statistics
LONGEST RUNNING INDIVIDUAL API CALLS LONGEST QUEUED INDIVIDUAL API CALLS API CALL AGGREGATES grouped by Form sorted by descending AVG execution time API CALL AGGREGATES grouped by Client sorted by descending AVG execution time API CALL AGGREGATES grouped by Client IP sorted by descending AVG execution time API THREAD STATISTICS BY QUEUE API EXCEPTION REPORT LONGEST RUNNING INDIVIDUAL SQL CALLS SQL CALL AGGREGATES grouped by Table sorted by descending AVG execution time SQL THREAD STATISTICS BY QUEUE SQL EXCEPTION REPORT LONGEST RUNNING INDIVIDUAL FLTR MOST EXECUTED FLTR MOST FILTERS PER TRANSACTION MOST EXECUTED FLTR PER TRANSACTION MOST FILTER LEVELS IN TRANSACTIONS LONGEST RUNNING INDIVIDUAL ESCALATION CALLS ESC CALLS THAT GOT DELAYED Escalation CALL AGGREGATES grouped by Form sorted by descending AVG execution time Escalation CALL AGGREGATES grouped by Pool sorted by descending AVG execution time ESC CALLS THAT ERRORED OUT If you used one of the provided batch or script filers with Web Output, a folder will have been created with the original folder name + " analysis" In this folder will be an index.html file. Open index.html to view the report. The Report will contain the following sections depending on which log-types you analyzed: General
API Aggregates SQL Aggregates Escalation Aggregates Filter Statistics Reading the Report
All of the information in the text-only report is also in the web report. The web report also allows you to drill down into the actual log lines.
So this article will focus on reading the web report. General Statistics A good place to start is the General Statistics. You can ensure that the log analysis looks appropriate. Some key items to look at are:
Start Time, End Time, Elapsed Time to ensure that these log contain the correct time frame.
If you analyzed separate logs for API, SQL, Filter, and/or Escalation, you can see how well they align timewise by looking at the Logging Activity Much of the analysis refers to the log File Number and the log line. This can be referenced from the Input Filenames. API Aggregates
If your focus is end users slowness or other performance issues that impact end-users, a good place to start is the Top 50 API calls (based on the -n option used in the command)
This will show the duration, Queue Name, API call, and other information about all the longest running API calls You can click on a line number to drill down into the API call to find out what made it take long to complete. If you included SQL and Filter logging, you can attempt to isolate a long running filter or SQL call. There are several other API reports available to help understand API and client behavior. SQL Aggregates
You can analyze the database performance by looking at the SQL statistics, starting with the Top 50 Longest SQL calls (based on the -n option used in the command)..
Like the Top 50 API calls, you can click on a line number to drill down. This will take you to the per-thread log file information related to that SQL call. There are several other SQL reports available to help understand the ARSystem and database SQL behavior. Escalation Aggregates You can analyze Escalation behavior by looking at Top 50 (based on the -n option used in the command) and Delayed escalations. This can help understand why escalations take long to complete or do not run at the expected time.
There are several other Escalation reports available to help understand the behavior of your escalations. Filter Aggregates If you have long running API calls or Escalations that create or modify records, you may have to analyze your Filters. You can use the Filter statistics to help understand how Filters are affecting performance.
Localization Note:
The ARLogAnalyzer attempts to auto-detect the locale of the date/time fields and handle them properly. If this does not work properly, you can provide the 2-digit locale using the -l (lower case l) command-line optiion.
It was noted late in the development cycle that some non-English ARServer language features don't allow processing filter data properly. This is being worked on for a later release but for now, Analysis for non-English ARServers may not provide proper results. What's New in 3.1 Version 3.1, available Nov 18, 2020, adds new features and fixes minor defects New Features include:
Log Gaps show if there is a timeframe in which nothing was written to the logs. This may be caused by resource issues on the OS Thread Gaps show if a thread unexpectedly waited for a long period of time.
If you are trying to solve a Server Group issue, you can analyze logs from multiple servers and report on the entire group. Simply provide the names of each folder (one per server) that contain the logs. Version Updates 3.1.4 6/7/2021 Added better locale and language handling See also: Best FAQ on AR System and BMC Helix ITSM Performance Issues |
Attachment(s):