Menu Search

Chapter 1.  Running the AMQP Messaging Broker

Table of Contents

1.1. Running a Qpid C++ Broker
1.1.1. Building the C++ Broker and Client Libraries
1.1.2. Running the C++ Broker
1.1.3. Most common questions getting qpidd running
1.1.4. Authentication
1.1.5. Slightly more complex configuration
1.1.6. Loading extra modules
1.1.7. Timestamping Received Messages
1.2. Cheat Sheet for configuring Queue Options
1.2.1. Configuring Queue Options
1.3. Cheat Sheet for configuring Exchange Options
1.3.1. Configuring Exchange Options
1.4. Broker Federation
1.4.1. Message Routes
1.4.2. Federation Topologies
1.4.3. Federation among High Availability Message Clusters
1.4.4. The qpid-route Utility
1.5. Security
1.5.1. User Authentication
1.5.2. Authorization
1.5.3. User Connection and Queue Quotas
1.5.4. Encryption using SSL
1.6. LVQ - Last Value Queue
1.6.1. Understanding LVQ
1.6.2. Creating a Last Value Queue
1.6.3. LVQ Example
1.6.4. Deprecated LVQ Modes
1.7. Queue State Replication
1.7.1. Asynchronous Replication of Queue State
1.8. Active-active Messaging Clusters
1.8.1. Starting a Broker in a Cluster
1.8.2. qpid-cluster
1.8.3. Failover in Clients
1.8.4. Error handling in Clusters
1.8.5. Persistence in High Availability Message Clusters
1.9. Producer Flow Control
1.9.1. Overview
1.9.2. User Interface
1.10. AMQP compatibility
1.10.1. AMQP Compatibility of Qpid releases:
1.10.2. Interop table by AMQP specification version
1.11. Qpid Interoperability Documentation
1.11.1. SASL
1.12. Using Message Groups
1.12.1. Overview
1.12.2. Grouping Messages
1.12.3. The Role of the Broker
1.12.4. Well Behaved Consumers
1.12.5. Broker Configuration
1.13. Active-passive Messaging Clusters
1.13.1. Overview
1.13.2. Virtual IP Addresses
1.13.3. Configuring the Brokers
1.13.4. The Cluster Resource Manager
1.13.5. Configuring rgmanager as resource manager
1.13.6. Broker Administration Tools
1.13.7. Creating replicated queues and exchanges
1.13.8. Client Connection and Fail-over
1.13.9. Security.
1.13.10. Integrating with other Cluster Resource Managers
1.14. Queue Replication with the HA module

1.1.  Running a Qpid C++ Broker

1.1.1.  Building the C++ Broker and Client Libraries

The root directory for the C++ distribution is named qpidc-0.4. The README file in that directory gives instructions for building the broker and client libraries. In most cases you will do the following:

[qpidc-0.4]$ ./configure
[qpidc-0.4]$ make

1.1.2.  Running the C++ Broker

Once you have built the broker and client libraries, you can start the broker from the command line:

[qpidc-0.4]$ src/qpidd

Use the --daemon option to run the broker as a daemon process:

[qpidc-0.4]$ src/qpidd --daemon

You can stop a running daemon with the --quit option:

[qpidc-0.4]$ src/qpidd --quit

You can see all available options with the --help option

[qpidc-0.4]$ src/qpidd --help

1.1.3.  Most common questions getting qpidd running

1.1.3.1.  Error when starting broker: "no data directory"

The qpidd broker requires you to set a data directory or specify --no-data-dir (see help for more details). The data directory is used for the journal, so it is important when reliability counts. Make sure your process has write permission to the data directory.

The default location is

/lib/var/qpidd

An alternate location can be set with --data-dir

1.1.3.2.  Error when starting broker: "that process is locked"

Note that when qpidd starts it creates a lock file is data directory are being used. If you have a un-controlled exit, please mail the trace from the core to the dev@qpid.apache.org mailing list. To clear the lock run

./qpidd -q

It should also be noted that multiple brokers can be run on the same host. To do so set alternate data directories for each qpidd instance.

1.1.3.3.  Using a configuration file

Each option that can be specified on the command line can also be specified in a configuration file. To see available options, use --help on the command line:

./qpidd --help

A configuration file uses name/value pairs, one on each line. To convert a command line option to a configuration file entry:

a.) remove the '--' from the beginning of the option. b.) place a '=' between the option and the value (use yes or true to enable options that take no value when specified on the command line). c.) place one option per line.

For instance, the --daemon option takes no value, the --log-to-syslog option takes the values yes or no. The following configuration file sets these two options:

daemon=yes
log-to-syslog=yes

1.1.3.4.  Can I use any Language client with the C++ Broker?

Yes, all the clients work with the C++ broker; it is written in C+, but uses the AMQP wire protocol. Any broker can be used with any client that uses the same AMQP version. When running the C+ broker, it is highly recommended to run AMQP 0-10.

Note that JMS also works with the C++ broker.

1.1.4.  Authentication

1.1.4.1.  Linux

The PLAIN authentication is done on a username+password, which is stored in the sasldb_path file. Usernames and passwords can be added to the file using the command:

saslpasswd2 -f /var/lib/qpidd/qpidd.sasldb -u <REALM> <USER>

The REALM is important and should be the same as the --auth-realm option to the broker. This lets the broker properly find the user in the sasldb file.

Existing user accounts may be listed with:

sasldblistusers2 -f /var/lib/qpidd/qpidd.sasldb

NOTE: The sasldb file must be readable by the user running the qpidd daemon, and should be readable only by that user.

1.1.4.2.  Windows

On Windows, the users are authenticated against the local machine. You should add the appropriate users using the standard Windows tools (Control Panel->User Accounts). To run many of the examples, you will need to create a user "guest" with password "guest".

If you cannot or do not want to create new users, you can run without authentication by specifying the no-auth option to the broker.

1.1.5.  Slightly more complex configuration

The easiest way to get a full listing of the broker's options are to use the --help command, run it locally for the latest set of options. These options can then be set in the conf file for convenience (see above)

./qpidd --help

Usage: qpidd OPTIONS
Options:
  -h [ --help ]                    Displays the help message
  -v [ --version ]                 Displays version information
  --config FILE (/etc/qpidd.conf)  Reads configuration from FILE

Module options:
  --module-dir DIR (/usr/lib/qpidd)  Load all .so modules in this directory
  --load-module FILE                 Specifies additional module(s) to be loaded
  --no-module-dir                    Don't load modules from module directory

Broker Options:
  --data-dir DIR (/var/lib/qpidd)   Directory to contain persistent data generated by the broker
  --no-data-dir                     Don't use a data directory.  No persistent
                                    configuration will be loaded or stored
  -p [ --port ] PORT (5672)         Tells the broker to listen on PORT
  --worker-threads N (3)            Sets the broker thread pool size
  --max-connections N (500)         Sets the maximum allowed connections
  --connection-backlog N (10)       Sets the connection backlog limit for the
                                    server socket
  --staging-threshold N (5000000)   Stages messages over N bytes to disk
  -m [ --mgmt-enable ] yes|no (1)   Enable Management
  --mgmt-pub-interval SECONDS (10)  Management Publish Interval
  --ack N (0)                       Send session.ack/solicit-ack at least every
                                    N frames. 0 disables voluntary ack/solitict
                                   -ack

Daemon options:
  -d [ --daemon ]             Run as a daemon.
  -w [ --wait ] SECONDS (10)  Sets the maximum wait time to initialize the
                              daemon. If the daemon fails to initialize, prints
                              an error and returns 1
  -c [ --check ]              Prints the daemon's process ID to stdout and
                              returns 0 if the daemon is running, otherwise
                              returns 1
  -q [ --quit ]               Tells the daemon to shut down
Logging options:
  --log-output FILE (stderr)  Send log output to FILE. FILE can be a file name
                              or one of the special values:
                              stderr, stdout, syslog
  -t [ --trace ]              Enables all logging
  --log-enable RULE (error+)  Enables logging for selected levels and component
                              s. RULE is in the form 'LEVEL+:PATTERN'
                              Levels are one of:
                              trace debug info notice warning error critical
                              For example:
                              '--log-enable warning+' logs all warning, error
                              and critical messages.
                              '--log-enable debug:framing' logs debug messages
                              from the framing namespace. This option can be
                              used multiple times
  --log-time yes|no (1)       Include time in log messages
  --log-level yes|no (1)      Include severity level in log messages
  --log-source yes|no (0)     Include source file:line in log messages
  --log-thread yes|no (0)     Include thread ID in log messages
  --log-function yes|no (0)   Include function signature in log messages

1.1.6.  Loading extra modules

By default the broker will load all the modules in the module directory, however it will NOT display options for modules that are not loaded. So to see the options for extra modules loaded you need to load the module and then add the help command like this:

./qpidd --load-module libbdbstore.so --help
Usage: qpidd OPTIONS
Options:
  -h [ --help ]                    Displays the help message
  -v [ --version ]                 Displays version information
  --config FILE (/etc/qpidd.conf)  Reads configuration from FILE


 / .... non module options would be here ... /


Store Options:
  --store-directory DIR     Store directory location for persistence (overrides
                            --data-dir)
  --store-async yes|no (1)  Use async persistence storage - if store supports
                            it, enables AIO O_DIRECT.
  --store-force yes|no (0)  Force changing modes of store, will delete all
                            existing data if mode is changed. Be SURE you want
                            to do this!
  --num-jfiles N (8)        Number of files in persistence journal
  --jfile-size-pgs N (24)   Size of each journal file in multiples of read
                            pages (1 read page = 64kiB)

1.1.7. Timestamping Received Messages

The AMQP 0-10 specification defines a timestamp message delivery property. The timestamp delivery property is a datetime value that is written to each message that arrives at the broker. See the description of "message.delivery-properties" in the "Command Classes" section of the AMQP 0-10 specification for more detail.

See the Programming in Apache Qpid documentation for information regarding how clients may access the timestamp value in received messages.

By default, this timestamping feature is disabled. To enable timestamping, use the enable-timestamp broker configuration option. Setting the enable-timestamp option to 'yes' will enable message timestamping:

./qpidd --enable-timestamp yes
  

Message timestamping can also be enabled (and disabled) without restarting the broker. The QMF Broker management object defines two methods for accessing the timestamp configuration:

Table 1.1. QMF Management - Broker Methods for Managing the Timestamp Configuration

MethodDescription
getTimestampConfigGet the message timestamping configuration. Returns True if received messages are timestamped.
setTimestampConfigSet the message timestamping configuration. Set True to enable timestamping received messages, False to disable timestamping.

Example 1.1. Enabling Message Timestamping via QMF - Python

The following code fragment uses these QMF method calls to enable message timestamping.

# get the state of the timestamp configuration
broker = self.qmf.getObjects(_class="broker")[0]
rc = broker.getTimestampConfig()
self.assertEqual(rc.status, 0)
self.assertEqual(rc.text, "OK")
print("The timestamp setting is %s" % str(rc.receive))

# try to enable it
rc = broker.setTimestampConfig(True)
self.assertEqual(rc.status, 0)
self.assertEqual(rc.text, "OK")