The Broker will, by default, produce structured log messages in response to key events in the lives of objects within the Broker. These concise messages are designed to allow the user to understand the actions of the Broker in retrospect. This is valuable for problem diagnosis and provides a useful audit trail.
Each log message includes details of the entity causing the action (e.g. a management user or messaging client connection), the entity receiving the action (e.g. a queue or connection) and a description of operation itself.
The log messages have the following format:
[Actor] {[Subject]} [Message Id] [Message Text]
Where:
Actor is the entity within the Broker that is
performing the action. There are actors corresponding to the Broker
itself, Management, Connections, Channels, and HA groups. Their format is described in the
table below.
Subject (optional) is the entity within the Broker that is
receiving the action. There are subjects corresponding to the
Connections, Channels, Queues, Exchanges, Bindings, Subscriptions, Message Stores, Ports,
and HA groups. Their format is described in the
table below.
Some actions are reflexive, in these cases the Actor and Subject will be equal.
Message Id is an identifier for the type of message. It has the form
three alphas and four digits separated by a hyphen AAA-9999.
Message Text is a textual description
To illustrate, let's look at two examples.
CON-1001 is used when a messages client makes an AMQP connection. The
connection actor (con) provides us with details of the peer's connection: the
user id used by the client (myapp1), their IP, ephemeral port number and the name of the virtual
host. The message text itself gives us further details about the connection: the client id, the
protocol version in used, and details of the client's qpid library.
[con:8(myapp1@/127.0.0.1:52851/default)] CON-1001 : Open : Destination : AMQP(127.0.0.1:5672) : Protocol Version : 0-10 : Client ID : myapp1 : Client Version : 10.1.1 : Client Product : qpid
QUE-1001 is used when a queue is created. The connection actor
con tells us details of the connection performing the queue creation: the
user id used by the client (myapp1), the IP, ephemeral port number and the name of the virtual
host. The queue subject tells use the queue's name (myqueue) and the virtualhost. The message
itself tells us more information about the queue that is being created.
[con:8(myapp1@/127.0.0.1:52851/default)/ch:0] [vh(/default)/qu(myqueue)] QUE-1001 : Create : "myqueue" : SUCCESS : {name=myqueue}The first two tables that follow describe the actor and subject entities. These are followed by an explanation of message categories without dedicated prefix formats and a catalogue of all supported messages.
Message ID prefixes identify categories of events and do not map one-to-one to actor or subject types. The actor is determined by the execution context, not by the message category. Some categories have no dedicated subject format because their messages provide event details in the message text without a separate subject prefix, or reuse an existing subject format. The table below describes these categories and how their context is represented. Their absence from the actor and subject tables does not indicate missing message categories.
Table C.1. Actors Entities
| Actor Type | Format and Purpose |
|---|---|
| Broker | [Broker] |
|
Used during startup and shutdown | |
| Management |
[mng:userid(clientip:ephemeralport)] |
|
Used for operations performed by the Web Management interfaces. | |
| Connection |
[con:connectionnumber(userid@/clientip:ephemeralport/virtualhostname)] |
|
Used for operations performed by a client connection. Note that connections are numbered by a sequence number that begins at 1. | |
| Channel |
[con:connectionnumber(userid@/clientip:ephemeralport/virtualhostname/ch:channelnumber)] |
|
Used for operations performed by a client's channel (corresponds to the JMS concept of Session). Note that channels are numbered by a sequence number that is scoped by the owning connection. | |
| Group | [grp(/groupname)/vhn(/virtualhostnode
name)] |
|
Used for HA. Used for operations performed by the system itself often as a result of actions performed on another node.. |
Table C.2. Subject Entities
| Subject Type | Format and Purpose |
|---|---|
| Connection |
[con:connectionnumber(userid@/clientip:ephemeralport/virtualhostname)] |
|
A connection to the Broker. | |
| Channel |
[con:connectionnumber(userid@/clientip:ephemeralport/virtualhostname/ch:channelnumber)] |
|
A client's channel within a connection. | |
| Subscription |
[sub:subscriptionnumber(vh(/virtualhostname)/qu(queuename)] |
|
A subscription to a queue. This corresponds to the JMS concept of a Consumer. | |
| Queue | [vh(/virtualhostname)/qu(queuename)] |
|
A queue on a virtualhost | |
| Exchange | [vh(/virtualhostname)/ex(exchangetype/exchangename)] |
|
An exchange on a virtualhost | |
| Binding |
[vh(/virtualhostname)/ex(exchangetype/exchangename)/qu(queuename)/rk(bindingkey)] |
|
A binding between a queue and exchange with the giving binding key. | |
| Message Store |
[vh(/virtualhostname)/ms(messagestorename)] |
|
A virtualhost/message store on the Broker. | |
| Port | [port(port number)] |
|
A network port exposed by the Broker. | |
| HA Group | [grp(/group name)] |
|
A HA group |
Table C.3. Message Categories Without Dedicated Actor or Subject Formats
| Message Categories | Context and Reason for No Dedicated Format |
|---|---|
Access Control (ACL), Key Store (KST),
Trust Store (TST), Authentication Provider (ATH),
and User (USR) |
These messages use the actor associated with the operation and are logged without a separate subject prefix. Event details, such as the affected object's name, operation, or authentication identity, are supplied in the message text as appropriate to the event. |
Network (NET), Sender (SND),
and Resource Limit (RL) |
These messages use the current actor without a separate subject prefix. Network endpoints, sender link and destination details, or resource-limit information are represented in the message text rather than by a dedicated subject format. |
Virtual Host (VHT) |
Lifecycle messages identify the virtual host in the message text, and operation messages identify the operation, without a separate subject prefix. Filesystem warnings reuse the Message Store subject to identify the affected store. There is no dedicated standalone virtual host prefix format for these messages. |
Configuration Store (CFG) |
These messages reuse the Message Store subject format
|
Transaction Store (TXN) |
Transaction recovery messages reuse the Message Store subject format
|
The following tables lists all the operation log messages that can be produced by the Broker, and the describes the circumstances under which each may be seen.
Table C.4. Broker Log Messages
Table C.5. Management Log Messages
Table C.6. Virtual Host Log Messages
Table C.7. Queue Log Messages
| Message Id | Message Text / Purpose |
|---|---|
| QUE-1001 | Create : "queue name" :
outcome : attributes |
|
Indicates that creation of a queue was attempted and reports the outcome and configured attributes. | |
| QUE-1002 | Delete : "queue name" :
outcome |
|
Indicates that deletion of a queue was attempted and reports the outcome. | |
| QUE-1003 |
Overfull : Size : size bytes, Capacity :
capacity, Messages : message count,
Message Capacity : message capacity
|
|
Indicates that a queue has exceeded its permitted capacity when Producer Flow Control overflow policy is used. See Section 4.7.5, “Controlling Queue Size” for details. | |
| QUE-1004 |
Underfull : Size : size bytes, Resume Capacity :
resume capacity, Messages :
message count, Message Capacity :
message capacity
|
|
Indicates that a queue has fallen to its resume capacity when Producer Flow Control overflow policy is used. See Section 4.7.5, “Controlling Queue Size” for details. | |
| QUE-1005 |
Dropped : number messages, Depth : size bytes,
size messages, Capacity : limit bytes,
limit messages
|
|
Indicates that a given number of messages is deleted when Ring overflow policy is used and any of queue capacity limits is breached . See Section 4.7.5, “Controlling Queue Size” for details. | |
| QUE-1006 | Malformed : message identifier :
action |
|
Indicates that a malformed message was detected and reports the action taken by the Broker. | |
| QUE-1016 | Operation : operation name |
|
Indicates that the named operation has been invoked. | |
| QUE-1017 | Update : "queue name" :
outcome : attributes |
|
Indicates that an update of a queue was attempted and reports the outcome and queue attributes. | |
| QUE-1018 | Open : "queue name" :
outcome |
|
Indicates that a recovered queue was opened and reports the outcome. | |
| QUE-1019 | Close : "queue name" |
|
Indicates that a queue has been closed. |
Table C.8. Exchange Log Messages
| Message Id | Message Text / Purpose |
|---|---|
| EXH-1001 | Create : "exchange name" :
outcome : attributes |
|
Indicates that creation of an exchange was attempted and reports the outcome and configured attributes. | |
| EXH-1002 | Delete : "exchange name" :
outcome |
|
Indicates that deletion of an exchange was attempted and reports the outcome. | |
| EXH-1003 | Discarded Message : Name: "exchange name" Routing Key:
"routing key" |
|
Indicates that an exchange received a message that could not be routed to at least one queue. See Section 4.6.4, “Unrouteable Messages” for details. | |
| EXH-1004 | Operation : operation name |
|
Indicates that the named operation has been invoked. | |
| EXH-1005 | Update : "exchange name" :
outcome : attributes |
|
Indicates that an update of an exchange was attempted and reports the outcome and exchange attributes. | |
| EXH-1006 | Open : "exchange name" :
outcome |
|
Indicates that a recovered exchange was opened and reports the outcome. | |
| EXH-1007 | Close : "exchange name" |
|
Indicates that an exchange has been closed. |
Table C.10. Network Log Messages
Table C.11. Connection Log Messages
| Message Id | Message Text / Purpose |
|---|---|
| CON-1001 | Open : Destination : port name(local
address) : Protocol Version : protocol version[
: SSL][ : Client ID : client identifier][ : Client Version :
client version][ : Client Product : client
product] |
|
Indicates that a connection has been opened. The Broker logs one of these message each time it learns more about the client as the connection is negotiated. | |
| CON-1002 | Close[ : reason] |
|
Indicates that a connection has been closed. This message is logged regardless of if the connection is closed normally, or if the connection is somehow lost e.g network error. | |
| CON-1003 | Closed due to inactivity [: reason] |
|
Used when heart beating is in-use. Indicates that the connection has not received a heartbeat for too long and is therefore closed as being inactive. | |
| CON-1004 | Connection dropped |
|
Indicates that a connection has been unexpectedly closed by the peer. This usually occurs if a machine hosting an application fails or the application's process is abruptly terminated. | |
| CON-1005 | Client version "version" logged by validation |
|
Indicates that a connection has been received from client with a version number that is configured to be logged. This feature may help teams manage software currency. | |
| CON-1006 | Client version "version" rejected by validation |
|
Indicates that a connection attempt has been received from client with a version number that is configured to be rejected. This feature may help manage software currency. | |
| CON-1007 | Connection close initiated by operator |
|
Indicates that a connection has been closed by the actions of an operator using management. | |
| CON-1008 | Operation : operation name |
|
Indicates that the named connection operation has been invoked. | |
| CON-1009 | Uncommitted transaction(s) contains size bytes of incoming message data
exceeding size bytes limit. Messages will be flowed to disk. |
|
Warns that the transactions associated with this connection contain so much uncommitted data that a threshold has been breached. The connection responds by flowing the messages already associated with the transactions and any new messages to disk. The connection reverts to normal behaviour once the quantity of uncommitted data falls beneath the threshold. Normally this happens when the transactions commit or rollback. | |
| CON-1010 | Open Transaction : time ms |
|
Indicates that a messaging transaction has been open for longer than that permitted. See Section 9.3, “Transaction Timeout” for more details. | |
| CON-1011 | Idle Transaction : time ms |
|
Indicates that a messaging transaction has been idle for longer than that permitted. See Section 9.3, “Transaction Timeout” for more details. |
Table C.12. Channel Log Messages
| Message Id | Message Text / Purpose |
|---|---|
| CHN-1001 | Create |
|
Indicates that a channel (corresponds to the JMS concept of Session) has been created. | |
| CHN-1002 | Flow state |
|
Indicates whether message flow to a session has started or stopped. | |
| CHN-1003 | Close[ : error code -
reason] |
|
Indicates that a channel has been closed. A forced close includes the error code and reason. | |
| CHN-1004 | Prefetch Size (bytes) size : Count number
of messages |
|
Indicates the prefetch size in use by a channel. | |
| CHN-1005 | Flow Control Enforced (Queue queue name) |
|
Indicates that producer flow control has been imposed on a channel owning to excessive queue depth in the indicated queue. Produces using the channel will be requested to pause the sending of messages. See Section 4.7.5, “Controlling Queue Size” for more details. | |
| CHN-1006 | Flow Control Removed |
|
Indicates that producer flow control has been removed from a channel. See Section 4.7.5, “Controlling Queue Size” for more details. | |
| CHN-1009 | Discarded message : message number as no alternate
binding configured for queue : queue name routing key :
routing key |
|
Indicates that a channel has discarded a message as the maximum delivery count
has been exceeded but the queue has no alternate binding configured. See
Section 9.4.2, “Maximum Delivery Count”
for more details. Note that | |
| CHN-1010 | Discarded message : message number as alternate binding
yields no routes : destination name |
|
Indicates that a channel has discarded a message as the maximum delivery count
has been exceeded but the queue's alternate binding destination yielded no routes.
See
Section 9.4.2, “Maximum Delivery Count”
for more details. Note that | |
| CHN-1011 | Message : message number moved to dead letter queue :
queue name |
|
Indicates that a channel has moved a message to the named dead letter queue | |
| CHN-1012 | Flow Control Ignored. Channel will be closed. |
|
Indicates that a channel violating the imposed flow control has been closed | |
| CHN-1014 | Operation : operation name |
|
Indicates that the named operation has been invoked |
Table C.13. Sender Log Messages
Table C.14. Subscription Log Messages
Table C.15. Message Store Log Messages
| Message Id | Message Text / Purpose |
|---|---|
| MST-1001 | Created |
|
Indicates that a message store has been created. The message store is responsible for the storage of the messages themselves, including the message body and any headers. | |
| MST-1002 | Store location : path |
|
Indicates that the message store is using | |
| MST-1003 | Closed |
|
Indicates that the message store has been closed. | |
| MST-1004 | Recovery Start |
|
Indicates that message recovery has begun. | |
| MST-1005 | Recovered number messages |
|
Indicates that recovery recovered the given number of messages from the store. | |
| MST-1006 | Recovery Complete |
|
Indicates that the message recovery is concluded. | |
| MST-1008 | Store overfull, flow control will be enforced |
|
The store has breached its maximum configured size. See Section 9.2.1, “Disk quota-based flow control” for details. | |
| MST-1009 | Store overfull condition cleared |
|
The store size has fallen beneath its resume capacity and therefore flow control has been rescinded. See Section 9.2.1, “Disk quota-based flow control” for details. |
Table C.16. Transaction Store Log Messages
Table C.17. Configuration Store Log Messages
Table C.18. HA Log Messages
| Message Id | Message Text / Purpose |
|---|---|
| HA-1001 | Create : "node name" :
outcome : attributes |
|
Indicates that creation of an HA node was attempted and reports the outcome and configured attributes. | |
| HA-1002 | Delete : "node name" :
outcome |
|
Indicates that deletion of an HA node was attempted and reports the outcome. | |
| HA-1003 | Added : Node : 'name'
(host:port) |
|
A new node has been added to the group. | |
| HA-1004 | Removed : Node : 'name'
(host:port) |
|
The node has been removed from the group. This removal is permanent. | |
| HA-1005 | Joined : Node : 'name'
(host:port) |
|
The node has become reachable. This may be as a result of the node being restarted, or a network problem may have been resolved. | |
| HA-1006 | Left : Node : 'name'
(host:port) |
|
The node is no longer reachable. This may be as a result of the node being stopped or a network partition may be preventing it from being connected. The node is still a member of the group. | |
| HA-1007 | Master transfer requested : to 'name'
(host:port) |
|
Indicates that a master transfer operation has been requested. | |
| HA-1008 | Intruder detected : Node 'name'
(host:port) |
|
Indicates that an unexpected node has joined the group. The virtualhost node will go into the ERROR state in response to the condition. | |
| HA-1009 | Insufficient replicas contactable |
|
This node (which was in the master role) no longer has sufficient replica in contact in order to complete transactions. | |
| HA-1010 | Role change reported: Node : 'name'
(host:port) : from 'role' to
'role' |
|
Indicates that the node has changed role within the group. | |
| HA-1011 | Minimum group size : new group size |
|
The quorum requirements from completing elections or transactions has been changed. | |
| HA-1012 | Priority : priority |
|
The priority of the object node has been changed. Zero indicates that the node cannot be elected master. | |
| HA-1013 | Designated primary : true|false |
|
This node has been designated primary and can now operate solo. Applies to two node groups only. See Section 10.4.5, “Allow to Operate Solo” | |
| HA-1014 | Diverged transactions discarded |
|
This node is in the process of rejoining the group but has discovered that some of its transactions differ from those of the current master. The node will automatically roll-back (i.e. discard) the diverging transactions in order to be allowed to rejoin the group. This situation can only usually occur as a result of use of the weak durability options. These allow the group to operate with fewer than quorum nodes and therefore allow the inconsistencies to develop. On encountering this condition, it is strongly recommended to run an application-level reconciliation to determine the data that has been lost. | |
| HA-1015 | Update : "node name" :
outcome : attributes |
|
Indicates that an update of an HA node was attempted and reports the outcome and HA node attributes. |
Table C.19. Port Log Messages
Table C.20. Authentication Provider Log Messages
Table C.21. Access Control Log Messages
Table C.22. Key Store Log Messages
Table C.23. Trust Store Log Messages
Table C.25. User Log Messages
Apache Qpid, Messaging built on AMQP; Copyright © 2015 The Apache Software Foundation; Licensed under the Apache License, Version 2.0; Apache Qpid, Qpid, Qpid Proton, Proton, Apache, the Apache logo, and the Apache Qpid project logo are trademarks of The Apache Software Foundation; All other marks mentioned may be trademarks or registered trademarks of their respective owners