Menu Search

9.11. Memory

9.11.1. Introduction

Understanding how the Qpid broker uses memory is essential to running a high performing and reliable service. A wrongly configured broker can exhibit poor performance or even crash with an OutOfMemoryError. Unfortunately, memory usage is not a simple topic and thus requires some in depth explanations. This page should give the required background information to make informed decisions on how to configure your broker.

Section 9.11.2, “Types of Memory” explains the two different kinds of Java memory most relevant to the broker. Section 9.11.3, “Memory Usage in the Broker” goes on to explain which parts of the broker use what kind of memory. Section 9.11.4, “Low Memory Conditions” explains what happens when the system runs low on memory. Section 9.11.5, “Defaults” lays out the default settings of the Qpid broker. Finally, Section 9.11.6, “Memory Tuning the Broker” gives some advice on tuning your broker.

9.11.2. Types of Memory

While Java has a couple of different internal memory types we will focus on the two types that are relevant to the Qpid broker. Both of these memory types are taken from the same physical memory (RAM).

9.11.2.1. Heap

Normally, all objects are allocated from Java's heap memory. Once, nothing references an object it is cleaned up by the Java Garbage Collector and it's memory returned to the heap. This works fine for most use cases. However, when interacting with other parts of the operating system using Java's heap is not ideal. This is where the so called direct memory comes into play.

9.11.2.2. Direct

The world outside of the JVM, in particular the operating system (OS), does not know about Java heap memory and uses other structures like C arrays. In order to interact with these systems Java needs to copy data between its own heap memory and these native structures. This can become a bottle neck when there is a lot of exchange between Java and the OS like in I/O (both disk and network) heavy applications. Java's solution to this is to allow programmers to request ByteBuffers from so called direct memory. This is an opaque structure that might have an underlying implementation that makes it efficient to interact with the OS. Unfortunately, the GC is not good at tracking direct memory and in general it is inadvisable to use direct memory for regular objects.

9.11.3. Memory Usage in the Broker

This section lists some note worthy users of memory within the broker and where possible lists their usage of heap and direct memory. Note that to ensure smooth performance some heap memory should remain unused by the application and be reserved for the JVM to do house keeping and garbage collection. Some guides advise to reserve up to 30% of heap memory for the JVM.

9.11.3.1. Broker

The broker itself uses a moderate amount of heap memory (≈15 MB). However, each connection and session comes with a heap overhead of about 17 kB and 15 kB respectively. In addition, each connection reserves 512 kB direct memory for network I/O.

9.11.3.2. Virtual Hosts

The amount of memory a Virtual Host uses depends on its type. For a JSON Virtual Host Node with a BDB Virtual Host the heap memory usage is approximately 2 MB. However, each BDB Virtual Hosts has a mandatory cache in heap memory which has an impact on performance. See below for more information.

9.11.3.3. Messages

Messages and their headers are kept in direct memory and have an additional overhead of approximately 1 kB heap memory each. This means that most brokers will want to have more direct memory than heap memory. When many small messages accumulate on the broker the 1 kB heap memory overhead can become a limiting factor.

When the broker is running low on direct memory it will evict enqueued messages from memory and flow them to disk. For persistent messages this only means freeing the direct memory representation because they always have an on-disk representation to guard against unexpected failure (e.g., a power cut). For transient messages this implies additional disk I/O. After being flowed to disk messages need to be re-read from disk before delivery.

Please, note that messages from uncommitted transactions are not flowed to disk as part of running into low direct memory conditions, as they are not enqueued yet. The Connection has its own threshold for keeping messages from uncommitted transactions in memory. Only when Connection threshold is breached, the uncommitted messages on the connection are flowed to disk.

9.11.3.4. Message Store

Berkeley DB (BDB)

The broker can use Oracle's BDB JE (BDB) as a message store to persist messages by writing them to a database. BDB uses a mandatory cache for navigating and organising its database structure. Sizing and tuning this cache is a topic of its own and would go beyond the scope of this guide. Suffice to say that by default Qpid uses 5% of heap memory for BDB caches (each Virtual Host uses a separate cache) or 10 MB per BDB store, whichever is greater. See the official webpage especially this page for more information. For those interested, Qpid uses EVICT_LN as its default JE cacheMode.

Derby

TODO

9.11.3.5. HTTP Management

Qpid uses Jetty for the HTTP Management (both REST and Web Management Console). When the management plugin is loaded it will allocate the memory it needs and should not require more memory during operation and can thus be largely ignored.

9.11.4. Low Memory Conditions

9.11.4.1. Low on Heap Memory

When the broker runs low on heap memory performance will degrade because the JVM will trigger full garbage collection (GC) events in a struggle to free memory. These full GC events are also called stop-the-world events as they completely halt the execution of the Java application. Stop-the-world-events may take any where from a couple of milliseconds up to several minutes. Should the heap memory demands rise even further the JVM will eventually throw an OutOfMemoryError which will cause the broker to shut down.

9.11.4.2. Low on Direct Memory

When the broker detects that it uses 75% of available direct memory it will start flowing incoming transient messages to disk and reading them back before delivery. This will prevent the broker from running out of direct memory but may degrade performance by requiring disk I/O.

9.11.5. Defaults

By default Qpid uses these settings:

  • 0.5 GB heap memory
  • 1.5 GB direct memory
  • 5% of heap reserved for the BDB JE cache.
  • Start flow-to-disk at 75% direct memory utilisation.

As an example, this would accommodate a broker with 50 connections, each serving 5 sessions, and each session having 1000 messages of 1 kB on queues in the broker. This means a total of 250 concurrent sessions and a total of 250000 messages without flowing messages to disk.

9.11.6. Memory Tuning the Broker

9.11.6.1. Java Tuning

Most of these options are implementation specific. It is assumed you are using Oracle Java 11.

9.11.6.2. Qpid Tuning

  • The system property qpid.broker.bdbTotalCacheSize sets the total amount of heap memory (in bytes) allocated to BDB caches.
  • The system property broker.flowToDiskThreshold sets the threshold (in bytes) for flowing transient messages to disk. Should the broker use more than direct memory it will flow incoming messages to disk. Should utilisation fall beneath the threshold it will stop flowing messages to disk.
  • The system property connection.maxUncommittedInMemorySize sets the threshold (in bytes) for total messages sizes (in bytes) from connection uncommitted transactions when messages are hold in memory. If threshold is exceeded, all messages from connection in-flight transactions are flowed to disk including those arriving after breaching the threshold.

9.11.6.3. Formulae

We developed a simple formula which estimates the minimum memory usage of the broker under certain usage. These are rough estimate so we strongly recommend testing your configuration extensively. Also, if your machine has more memory available by all means use more memory as it can only improve the performance and stability of your broker. However, remember that both heap and direct memory are served from your computer's physical memory so their sum should never exceed the physically available RAM (minus what other processes use).

memoryheap = 15 MB + 20 kB * Nsessions + (1.7 kB + (120 + averageSizeheaderNameAndValue ) * averageNumberheaders)* Nmessages + 100 kB * Nconnections

memorydirect = 2 MB + (200 B + averageSizemsg *2)* Nmessages + 1MB * Nconnections

Where N denotes the total number of connections/sessions/messages on the broker. Furthermore, for direct memory only the messages that have not been flowed to disk are relevant.

Note

The formulae assume the worst case in terms of memory usage: persistent messages and TLS connections. Transient messages consume less heap memory than persistent and plain connections consume less direct memory than TLS connections.

9.11.6.4. Things to Consider

Performance

Choosing a smaller direct memory size will lower the threshold for flowing transient messages to disk when messages accumulate on a queue. This can have impact on performance in the transient case where otherwise no disk I/O would be involved.

Having too little heap memory will result in poor performance due to frequent garbage collection events. See Section 9.11.4, “Low Memory Conditions” for more details.

OutOfMemoryError

Choosing too low heap memory can cause an OutOfMemoryError which will force the broker to shut down. In this sense the available heap memory puts a hard limit on the number of messages you can have in the broker at the same time.

If the Java runs out of direct memory it also throws a OutOfMemoryError resulting the a broker shutdown. Under normal circumstances this should not happen but needs to be considered when deviating from the default configuration, especially when changing the flowToDiskThreshold.

If you are sending very large messages you should accommodate for this by making sure you have enough direct memory.