Skip to main content
Sign in

Logging

Brendan Gibbs
  • Updated

There are three main Log4J loggers configured for Fusion Application Server:

  • SIP call logger

  • HTTP logger (for LBs only)

  • Root logger

The SIP and HTTP loggers each have their own log category. If you need a different logging level for those types of log, you should change the log level for the appropriate category. There are a number of other categories that each have their own logging level, but these do not normally need to be changed.

Increasing logging levels will affect performance; we recommend that you change the logging level back to the default as soon as you have resolved your problem.

For any packages that do not have a specific logger category defined with a logging level, the level set for the root logger is used.

By default, SIP and HTTP logging is size-based, which means that log files rotate when the log file reaches a specified size (which is 100 MB by default). When the active log reaches the maximum size, FAS backs it up with a suffix of .1 and creates a new log file. When the new log file reaches the maximum size, the log with a suffix .1 changes to have a suffix of .2, and the most recent log becomes the backup with a suffix of .1. The log files rotate in this way up to the specified maximum backup index (which is 5 by default), after which FAS deletes older logs.

You can change the logging from rotating when the log file reaches a specified size to rotating after a specified time period, for example each day (see the Changing to Periodic Logging section in this article).

Configuring SIP Call Logging

The SIP call logger writes log items to the calls.log file, which is in the <install dir>/domain/servers/<server process name>/log directory, where <server process name> is the name of the AS or LB server process.

SIP call logging is size-based; log files rotate when they reaches a specified size, which by default is 100 MB; when the number of backup files reaches the maximum number (5 by default) the oldest is deleted. You can edit these values as described in the To Configure the Log Files for SIP Call Logging section in this article.

With call logging enabled, the ASs produce a SIP message log entry for each SIP message handled. The log entries include the following information:

  • Timestamp

  • Call-ID

  • Method or Condition Code

  • CSeq header

  • From header

  • To header

SIP call logging is set to INFO level by default. If you need to change this, do so at the category-level using the sip.calls category. Call logging is extremely useful for tracing faults in a system, and there is only a small negative impact from having it enabled. However, if required, call logging can be turned off by changing the logging level (see the To Change the SIP Call Logging Level section in this article).

To Configure the Log Files for SIP Call Logging

  1. Launch the Management Console - see the Starting the Management Console section in the Management Interfaces article.

  2. From the top right menu, select Profiles.

  3. Select the profile from the top left menu (you can log SIP calls on either the lb or ha profiles).

  4. From the menu on the left, expand Core and select Logging.

  5. Select the Handler navigation tab.

  6. Select Size in the menu immediately below the tabs.

  7. Select SIP_CALLS_FILE:

image_57.png

  1. Click Edit.

  2. To change the maximum size of the log files, edit the Rotate Size value.

  3. To change the number of backup files that are stored, edit the Max Backup Index value.

  4. Click Save.

Important: There is a logging level defined for the handler, but we recommend that this is not changed here, but at the category level, as described in the To Change the SIP Call Logging Level section in this article.

To Change the SIP Call Logging Level

  1. Launch the Management Console - see the Starting the Management Console section in the Management Interfaces article.

  2. From the top right menu, select Profiles.

  3. Select the profile from the top left menu (you can log SIP calls on either the lb or ha profiles).

  4. From the menu on the left, expand Core and select Logging.

  5. Select the Log Categories navigation tab.

  6. Select the sip.calls category:

image_58.png

  1. Click Edit.

  2. From the Log Level drop down list, select the required logging level.

  3. Click Save.

Configuring HTTP Logging

LBs have an additional log handler that logs HTTP and Web socket traffic passing through the LB. The HTTP logger writes log items to the http.log file, which is in the <install dir>/domain/servers/<lb name>/log directory, where <lb name> is the name of the LB server process.

HTTP logging is size-based; log files rotate when the log file reaches a specified size, which is 100 MB by default; when the number of backup files reaches the maximum number (5 by default) the oldest is deleted. You can edit these values (see the To Configure the Log Files for SIP Call Logging section in this article), selecting the HTTP_FILE handler entry from the lb profile.

HTTP logging is set to INFO level by default. If you need to change it, do so in the same way as described in the To Change the SIP Call Logging Level section in this article, selecting the com.alicecallsbob.lb.http_logs category from the lb profile.

Configuring the Root Logging Level

If there is no specific log handler for a category, log messages are written to the server.log file at <install dir>/domain/servers/<server process name>/log directory; there is one server.log file for each server process.

If there is no category-level logging for a package, it logs messages at the level of the root logger. To change this:

  1. Launch the Management Console - see the Starting the Management Console section in the Management Interfaces article.

  2. From the top right menu, select Profiles.

  3. Select the profile you want to work with from the top left menu. There is an independent root logger for each profile, including the management profile. To change the root logging level for ASs, select the ha profile; to change the root logging level for LBs, select lb.

  4. From the menu on the left, expand Core and select Logging.

  5. Select the Root Logger navigation tab:

image_59.png

  1. Click Edit.

  2. From the Log Level drop down, select the new logging level.

  3. Click Save.

  4. Restart the AS (see the Starting and Stopping Server Processes section in the Managing Cluster Components article) for the changes to take effect.

Configuring a Logging Category

To define a specific level of logging for some messages, create a logging category for those messages. For example, if you do not want a particular type of SIP message to be logged (ACK messages, for instance), you can create a logging category for that message type (sip.calls.ACK) and set the level lower than that of its parent category (sip.calls in this case). You do not need to create a category for every message type; those without a specific category log with the level of their parent category.

To Configure a Logging Category

  1. Launch the Management Console - see the Starting the Management Console section in the Management Interfaces article.

  2. From the top right menu, select Profiles.

  3. Select the profile from the top left menu (e.g. lb or ha).

  4. From the menu on the left, expand Core and select Logging.

  5. Select the Log Categories navigation tab.

  6. Click Add to show the Add Log Categories dialog:

image_60.png

  1. Enter the name of the category.

    • For SIP call message types, the name of each category must start with sip.calls and end with the message type in upper case letters. For instance, sip.calls.BYE creates a category for BYE messages.

    • For informational messages from the FAS Java code, the name of the category should be that of the Java package or class which is logging the message. There are some existing categories defined in this way, but you should not need to create one unless asked to do so by CBA Support.

  1. Select the logging level from the Log Level drop down list, e.g. DEBUG.

  2. Click Save.

Changing to Periodic Logging

By default, the log files rotate when they reach a specified size. You can change this to have them rotate after a specified time period (for example each day).

After changing to periodic logging, you should restart the AS (see the Starting and Stopping Server Processes section in the Managing Cluster Components article).

To Create a Periodic Log Handler

  1. Launch the Management Console - see the Starting the Management Console section in the Management Interfaces article.

  2. From the top right menu, select Profiles.

  3. Select the profile from the top left menu.

  4. From the menu on the left, expand Core and select Logging.

  5. Select the Handler navigation tab.

  6. Select Periodic in the menu immediately below the tabs:

image_61.png

  1. Click Add to bring up the Add Periodic Rotating File Handlers dialog:

image_62.png

  1. Enter a name for the new logging handler (this can be anything, though it should be unique).

  2. In the File Path, enter the name of the file to save the logs in.

  3. In the Suffix field, enter a suffix for backed-up files. This should be a date-time format string of the type used by the java.text.SimpleDateFormat class (see http://docs.oracle.com/javase/7/docs/api/index.html?java/text/SimpleDateFormat.html for details about this class). It should be suitable for the time period you want to use: for example, to create daily logs, use the suffix yyyy-MM-dd.

  4. Click Save.

  5. Select the new file handler, and click Edit.

image_63.png

  1. Select both the Append and Auto Flush checkboxes.

  2. Click Save.

To Use the Periodic Logging Handler for Root Logging

  1. Select the Root Logger navigation tab.

  2. Select the Handlers tab in the Details section:

image_64.png

  1. Click Add to bring up the Add Name dialog:

image_65.png

  1. Select the periodic logging handler from the Name drop down list, and click Save.

The default size based logging handler will still be active. To remove it, select the FILE handler and click Remove.

To Use the Periodic Logging Handler for SIP Call Loging

  1. Select the Log Categories navigation tab.

  2. Select the sip.calls logger category.

  3. Select the Handlers tab in the Details section:

image_67.png

  1. Click Add to bring up the Add Name dialog:

image_68.png

  1. Select the periodic logging handler and click Save.

The default size based logging handler will still be active. To remove it, select the SIP_CALLS_FILE handler in the table, and click Remove.

SNMP Logging

The SNMP Agent in each Host Controller also produces its own log. When it receives an alarm or event from a FAS server process or application, the SNMP Agent not only sends an SNMP trap, but also writes it to the alert.log. The log file is in the <install dir>/domain/log directory.

As with the other Fusion Application Server logs, SNMP Agent logging is size-based.

SNMP Agent diagnostic logging is output to the <install dir>/domain/log/host-controller.log.

Licensing Logging

The License Server which runs in the Management server process also produces its own log file. Every 60 seconds, for each licensed product, it will log the product name, the name of the licensed feature, the number of licenses used and available, and the peak number of used licenses in the last 24 hours:

2016-11-23 10:08:27,576 : product=sa feature=users limit=8 used=0 peakUsed(24h)=0

By default, licensing log entries go to the <install dir>/domain/servers/management/log/licensing.log file. If necessary, you can change this by editing the LICENSING_FILE logger in the Management Console.

Logging for a Specific Period

To help you identify any issues you may experience, FAS provides a script to capture call logs and statistics for a specific period. The logcapture.sh script is installed in the <install dir>/bin directory, and can be used to capture the following information:

  • FASconfiguration

  • vmstat output

  • Java memory

  • Thread dumps

  • Network traffic (in a .pcap file)

The logging script runs until it is stopped, allowing you to reproduce any problem scenarios during this time. When you stop the logging script, the information you require is captured in a series of log files, which are archived into a .tar file:

./logcapture.sh --all --tar-file output.tar

You can define which information is captured by adding a selection of the following arguments when you run the script:

Option Description
-f, ‑‑tar‑file The name of the output .tar file. This option is mandatory.
-c, ‑‑config Includes configuration files
-t, ‑‑threads Includes thread dumps (output of jstack)
-m, ‑‑memory Includes a heap memory dump (output of jmap) for all processes
-n, ‑‑do‑not‑clean Prevents the output directory from being cleaned up
-p, ‑‑capture‑pcap Captures network traffic in a .pcap file
-v, ‑‑vmstat Includes vmstat output
-a, ‑‑all Includes all the options
-F, ‑‑force Forces memory and stack dumps even if a process is hung. Only meaningful if -t, -m, or -a are also included.
-h, ‑‑help Displays online help

To Capture Logs for a Specific Period

  1. Set the logging levels appropriately (see the Configuring the Root Logging Level section in this article).

  2. Run the command:

./logcapture.sh -a -f example.tar

The console will display the message:

\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
\* Capturing files to directory logcapture.remp-LGR \*
\* Press <CTL>-C when ready to tar up captured files \*
\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*

Note: The final three characters of the directory name (LGR in the above example) will change each time the script is run, as this is a temporary directory.

  1. Reproduce any scenarios which were causing issues.

  2. Press Ctrl-C to stop the logging script. The output files will be collected in example.tar, which (for the -a option used above) has the structure:

./vmstat.out

./tcpdump.pcap

./FAS/

./FAS/log/

./FAS/log/alert.log

./FAS/log/process-controller.log

./FAS/log/host-controller.log

./FAS/configuration/

./FAS/configuration/host-master.xml

./FAS/configuration/mgmt-users.properties

./FAS/configuration/application-roles.properties

./FAS/configuration/fas.properties

./FAS/configuration/host.xml

./FAS/configuration/domain.xml

./FAS/configuration/host-slave.xml

./FAS/configuration/logging.properties

./FAS/configuration/application-users.properties

./FAS/servers/

./FAS/servers/loadbalancer-acb-fas-1/

./FAS/servers/loadbalancer-acb-fas-1/log/

./FAS/servers/loadbalancer-acb-fas-1/log/server.log

./FAS/servers/loadbalancer-acb-fas-1/log/boot.log

./FAS/servers/loadbalancer-acb-fas-1/log/http.log

./FAS/servers/loadbalancer-acb-fas-1/log/calls.log

./FAS/servers/loadbalancer-acb-fas-1/heap.bin

./FAS/servers/loadbalancer-acb-fas-1/thread.dump

./FAS/servers/management/

./FAS/servers/management/log/

./FAS/servers/management/log/server.log

./FAS/servers/management/log/boot.log

./FAS/servers/management/heap.bin

./FAS/servers/management/thread.dump

./FAS/servers/appserver-acb-fas-2/

./FAS/servers/appserver-acb-fas-2/log/

./FAS/servers/appserver-acb-fas-2/log/server.log

./FAS/servers/appserver-acb-fas-2/log/boot.log

./FAS/servers/appserver-acb-fas-2/log/calls.log

./FAS/servers/appserver-acb-fas-2/heap.bin

./FAS/servers/appserver-acb-fas-2/thread.dump

./FAS/servers/appserver-acb-fas-1/

./FAS/servers/appserver-acb-fas-1/log/

./FAS/servers/appserver-acb-fas-1/log/server.log

./FAS/servers/appserver-acb-fas-1/log/boot.log

./FAS/servers/appserver-acb-fas-1/log/calls.log

./FAS/servers/appserver-acb-fas-1/heap.bin

./FAS/servers/appserver-acb-fas-1/thread.dump

Creating a Log Archive

If at any point you want to examine the call logs in detail (for example if there is a specific issue that you want to investigate, or want CBA Support to investigate), you can create a log archive by running the log-archiver.sh script, which you can find in the <install dir>/bin directory. The script takes no arguments:

./log-archiver.sh

and puts the following files in an archive called fas-logs.zip:

  • messages.xml

Contains a collection of all of the message.log files found in the AS’s log directory.

  • processingDecisions.log

Contains entries from the AS’s server.log file that contain a message ID and with a logging level of INFO or higher.

  • server.log files for all the server processes (AS, LB, and Management Server) on the node.

  • call.log files for the AS and LB server processes running on the host that the file was created on.

The archive contains most of the logs useful for troubleshooting, and it is more convenient to download the archive than all the individual log files.

Note: In a multi-box setup, to collect the logs for the whole FAS cluster, run the log-archiver script on each host in the cluster.

 

 

Related articles

Comments

0 comments

Please sign in to leave a comment.

Powered by Zendesk