Table of Contents
- 9.1. Logging
- 9.2. Disk Space Management
- 9.3. Transaction Timeout
- 9.4. Handing Undeliverable Messages
- 9.5. Closing client connections on unroutable mandatory messages
- 9.6. Flow to Disk
- 9.7. Consumers
- 9.8. Background Recovery
- 9.9. Message Compression
- 9.10. Connection Limits
- 9.11. Memory
- 9.12. Broker Instrumentation
This section describes the flexible logging capabilities of the Apache Qpid Broker-J.
The Broker is capable of sending logging events to a variety of destinations including plain files, remote syslog daemons, and an in-memory buffer (viewable from Management). The system is also open for extension meaning it is possible to produce a plugin to log to a bespoke destination.
Logging can be dynamically configured at runtime. For instance, it is possible to temporarily increase the logging verbosity of the system whilst a problem is investigated and then revert later, all without the need to restart the Broker.
Virtualhosts can be configured to generate their own separate log, and the Broker is capable of generating a log either inclusive or exclusive of virtualhost events.
Logs are accessible over Management, removing the need for those operating the Broker to have shell level access.
In the remainder of this section you will first find a description of the concepts used in the logging subsystem. Next, you find a description of the default configuration. The section then concludes with a in-depth description of the loggers themselves and how they may be configured.
The logging subsystem uses two concepts:
A Logger is responsible for production of a log. The Broker ships a variety of loggers, for instance, a file logger, which is capable of writing a log file to the file system, a Syslog Logger capable of writing to a remote syslog daemon and console logger capable of writing to stdout or stderr.
Loggers are attached at two points within the Broker Model; the Broker itself and the virtualhosts. Loggers attached at the Broker can capture log events for the system as a whole, or can exclude events related to virtualhosts.
Loggers attached to a virtualhost capture log events relating to that virtualhost only.
The Broker and virtualhosts can have zero or more Loggers. If no loggers are configured, no logging is generated at all.
Inclusion rules govern what appears within a log. Inclusion rules are associated with Loggers. This means it is possible for different Loggers to have different contents.
A Logger with no inclusion rules will produce an empty log.
The default configuration is designed to be suitable for use without change in small production environments. It has the following characteristics:
The Broker generates a single log file
qpid.log. This logfile is rolled automatically when the file reaches 100MB. A maximum history of one file is retained. On restart the the log will be appended to.The log contains:
All operational logging events. See Appendix C, Operational Logging.
Log events from Qpid itself deemed informational or higher.
Log events from Qpid's dependencies (such as Derby or Jetty) that are deemed warning or higher.
The default location for the log file is
${QPID_WORK}/log/qpid.log.The Broker also caches the last 4096 log events in a memory cache. By default, the memory logger logs the same things the file logger does.
The configuration can be customised at runtime using Management. This makes it possible to investigate unusual conditions without the need to restart the Broker. For instance, you may alter the logging level so that a verbose log is produced whilst an investigation is in progress and revert the setting later, all without the need to restart the Broker.
Loggers are responsible for the writing of a log. The log includes log events that match a Logger's inclusion rules.
Loggers are associated with either the Broker or a virtualhost. Virtualhost loggers write only log events related to that virtualhost. Broker Loggers write log events from the Broker as a whole. Optionally a Broker Logger can be configured to exclude log events coming from virtualhosts. These abilities can be usefully exploited together in managed service scenarios to produce separate logs for separate user groups.
Loggers can be added or removed at runtime, without restarting the Broker. However changes to a Logger's configuration such as filenames and rolling options don't take effect until the next restart. Changes to a Logger's inclusion rules take effect immediately.
All loggers allow the log event layout to be customised. Loggers understand Logback Classic Pattern Layouts.
The following sections describes each Logger implementation in detail.
A FileLogger - writes a log file to the filesystem. The name and location of the log file, the rolling configuration, and compression options can be configured.
The roll daily option, if enabled, will cause the log file will be
rolled at midnight local time. The rolled over file will have a suffix in the form
yyyy-mm-dd. In roll daily mode, maximum number of rolled
files controls the maximum number of days to be retained.
Older files will be deleted.
The maximum file size option limits the size of any one log file.
Once a log file reaches the given size, it will be rolled. The rolled over file will have
the numeric suffix, beginning at 1. If the log file rolls again, first
the existing file with the suffix .1 is renamed to .2
and so forth. If roll daily is not in use, maximum number of rolled
files governs the number of rolled files that will be
retained.
Roll on restart governs whether the log file is rolled when the Broker is restarted. If not ticked, the Broker will append to the existing log file until it needs to be rolled.
SyslogLogger - writes a log file to a syslog daemon using the
USER facility. The hostname and port number of the syslog daemon can be
configured.
Log entries can be prefixed with a string. This string defaults to include the word
Qpid and the name of the Broker or virtualhost. This serves to
distinguish the logging generated by this Qpid instance, from other Qpid instances, or other
applications using the USER.
MemoryLogger - writes a log file to a circular in-memory buffer. By default the circular buffer holds the last 4096 log events. The contents of the buffer can be viewed via Management. See Figure 9.3, “Viewing a memory logger”
GraylogLogger - sends log messages to a Graylog server in GELF format via TCP. The hostname and port number of the Graylog server has to be configured. The content of the log messages is also configurable.
The logger is implemented on top of LGPL licenced library de.siegmar:logback-gelf.
The LGPL license is incompatible with Apache License. Thus, the Graylog integration module is not included
into standard broker distribution. It has to be built using option -Dgraylog. The built jar
org.apache.qpid:qpid-broker-plugins-graylog-logging-logback and
de.siegmar:logback-gelf jar should be copied manually under broker
lib folder. The broker restart is required to have GraylogLogger
available for the use.
A Logger has one or more inclusion rules. These govern what appears in the log. A Logger with no inclusion rules will log nothing.
Inclusion rules can be added, removed or changed at runtime. Changes take place immediately.
The Name And Level inclusion rule accepts log events that match a given log event source name and have a level that equals or exceeds the specified value.
The log event source name refers to the fully qualified class name from which the event originates. These names permit a trailing wild card
.*. For instance a source name oforg.apache.qpid.*will match all events from classes in the packageorg.apache.qpidand any sub packages beneath.The Level governs the level of the events that will be included in the log. It may take one of the following values: ERROR, WARN, INFO, DEBUG, TRACE where ERROR is considered the highest and TRACE the lowest. In addition, there are two special values: OFF and ALL, the former excludes all log events whereas the latter will include everything. When considering whether a logging event should be included in the log, the logging event must have a level that matches that of the inclusion rule or be higher, otherwise the log event will not appear in the log.
The logging subsystem can be completely managed from the Web Management Console or the REST API. You can:
Add, remove, or change the configuration of Loggers.
Add, remove, or change the Inclusion Rules.
For FileLoggers, download the log file and rolled log files associated with the Logger.
For MemoryLoggers, view the last
nlog events
The figure that follows shows a FileLogger. The attributes area shows the configuration of the Logger. The inclusion rule table shows the rules that are associated with the Logger. The area towards the bottom of the tab allows the log files to be downloaded to the browser.
The figure below shows the editing of the level of an inclusion rule.
The figure below shows a Memory Logger. Note that the Memory Logger provides access to the cached message via the viewer towards the bottom on the tab.
Apache Qpid, Messaging built on AMQP; Copyright © 2015 The Apache Software Foundation; Licensed under the Apache License, Version 2.0; Apache Qpid, Qpid, Qpid Proton, Proton, Apache, the Apache feather logo, and the Apache Qpid project logo are trademarks of The Apache Software Foundation; All other marks mentioned may be trademarks or registered trademarks of their respective owners