Menu Search
Apache License

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Qpid Interoperability Test Users Guide

1. Introduction

qpid-interop-test is an AMQP client interoperability test suite. It tests various aspects of the AMQP protocol and/or test client features against each other to ensure that they can interoperate.

The test suite consists of tests and shims. Each test has a set of test-cases which make up the test. Each test case will pass or fail a specific feature or piece of functionality under test.

Each test has a set of shims, which are small and specific clients which send and receive messages, and is written using one of the client libraries under test. For example, the amqp_types test has shims for the following clients: * AmqpNetLite * ProtonCpp * ProtonPython2 * ProtonPython3 * RheaJS

To obtain both self- and interoperability testing, each test program will run each shim against every other shim in the role of both sender and receiver. For the amqp-type-test example above, this will result in the following combinations of shims being used:

Sender shim

Receiver shim

1

AmqpNetLite

AmqpNetLite

2

AmqpNetLite

ProtonCpp

3

AmqpNetLite

ProtonPython2

4

AmqpNetLite

ProtonPython3

5

AmqpNetLite

RheaJS

6

ProtonCpp

AmqpNetLite

7

ProtonCpp

ProtonCpp

8

ProtonCpp

ProtonPython2

9

ProtonCpp

ProtonPython3

10

ProtonCpp

RheaJS

11

ProtonPython2

AmqpNetLite

12

ProtonPython2

ProtonCpp

13

ProtonPython2

ProtonPython2

14

ProtonPython2

ProtonPython3

15

ProtonPython2

RheaJS

16

ProtonPython3

AmqpNetLite

17

ProtonPython3

ProtonCpp

18

ProtonPython3

ProtonPython2

19

ProtonPython3

ProtonPython3

20

ProtonPython3

RheaJS

21

RheaJS

AmqpNetLite

22

RheaJS

ProtonCpp

23

RheaJS

ProtonPython2

24

RheaJS

ProtonPython3

25

RheaJS

RheaJS

so that for each test case, 25 individual tests are run. The test program will by default run all the available shims against each other in this way, but it is possible to control which shims are used using the --include-shim or --exclude-shim arguments (see below).

3. Building

a. Install dependencies:

  • Build tools: git, gcc-c++, cmake, maven, json-cpp

  • Qpid Proton: qpid-proton-cpp-devel, python2-qpid-proton, python3-qpid-proton

b. Decide on local vs system install

Local install:

Installs all of the Proton and QIT bits in a local directory. This is useful for limited testing where you don’t want to have these files in your system directories. Also, if you don’t have root privileges, then this is the only way to install. The drawback is that you may need to adjust some environment settings (PATH, PYTHONPATH, LD_LIBRARY_PATH) so that the test will run.

System install:

Installs the files into traditional system locations. This type of install requires root privileges. As the files are located in expected locations, no environment settings need be made.

c. Build qpid-interop-test

System install:

 $ cd qpid-interop-test
 $ mkdir build
 $ cd build
 $ cmake ..
 $ make
 $ sudo make install

Local install:

 $ cd qpid-interop-test
 $ mkdir build
 $ cd build
 $ cmake -DCMAKE_INSTALL_PREFIX=<path/to/local/install/dir> ..
 $ make install

4. Running

The tests by default assume a broker is available and running. The assumed default is at localhost:5672. For other broker location(s), use the --sender and --receiver arguments to specify where the clients should interact, see below.

The tests do not start or stop brokers.

There are several tests in the test suite:

  • amqp_types_test.py - Tests all of the AMQP primitive types. This primarily tests the encoding and decoding of AMQP types.

  • amqp_large_content_test.py - Tests large messages of various types. Messages sizes are 1MB, 10MB, 100MB. Compound types (lists, maps, etc) send elements of various sizes so that the total payload is the target size.

  • jms_messages_test.py - Tests JMS message types (as implemented by Qpid-jms over AMQP) from all the Qpid clients (including non-jms clients)

  • jms_hdrs_props_test.py - Tests various combinations of JMS headers and properties are correctly sent and received by the various clients.

Each test is executed directly.

Command-line arguments

Table 1. Common to all tests

--help

Print help. This is useful for seeing the available argument options and defaults for some arguments.

--sender

Node to which test suite will send messages. Format: ip-address:port Default: localhost:5672

--receiver

Node from which test suite will receive messages. Format: ip-address:port Default: localhost:5672

--no-skip

Do not skip tests that are excluded by default for reasons of a known bug. Warning: some tests may lock up of freeze rather than fail.

--broker-type

Specify the broker manually, which eliminates the test connection made to the broker to determine its identity through connection properties. If "None" is specified, then Artemis broker will be assumed, but this will change in the future when Artemis fixes the connection properties issue. Format: For our current brokers: one of: "ActiveMQ", "qpid-cpp", "qpid-dispatch-router". Artemis does not currently pass its name in connection properties, and is equivalent to "None".

‘--timeout’

Timeout for the test in seconds (default: 10 seconds, may vary with test). If the test does not complete within this time, it will be terminated and marked as a failure.

--include-shim

Name of shim to include. Cannot be used together with --exclude-shim. May be used multiple times to include more than one shim.

--exclude-shim

Name of shim to exclude. Cannot be used together with --include-shim. May be used multiple times to exclude more that one shim.

--xunit-log

Enable the generation of xUnit log files at the conclusion of the tests.

--xunit-log-dir

Path where xUnit log files are written.

--description

Description of test used in xUnit log file.

--broker-topology

Table 2. amqp-types-test

--include-type

Name of AMQP type to include. Cannot be used together with --exclude-type. May be used multiple times to include more than one type.

--exclude-type

Name of AMQP type to exclude. Cannot be used together with --include-type. May be used multiple times to exclude more than one type.

Table 3. amqp-complex-types-test

--include-type

Name of AMQP complex type to include. Cannot be used together with --exclude-type. May be used multiple times to include more than one type.

--exclude-type

Name of AMQP complex type to exclude. Cannot be used together with --include-type. May be used multiple times to exclude more than one type.

--include-subtype

Name of AMQP subtype to include. This is the AMQP type used within a primary complex type (for example, the types used within an AMQP list). Cannot be used together with --exclude-subtype. May be used multiple times to include more than one subtype. NOTE: There is a special * wildcard that can be used for this, and will allow all valid types to be used together in the same test.

--exclude-subtype

Name of AMQP subtype to exclude. Cannot be used together with --include-subtype. May be used multiple times to exclude more than one subtype.

Table 4. amqp-large-content-test

No other parameters. There is currently no way to select/limit the message size, but there an issue open to address this limitation.

Table 5. jms-messages-test

--include-type

Name of JMS message type to include. Cannot be used together with --exclude-type. May be used multiple times to include more than one type.

--exclude-type

Name of JMS message type to exclude. Cannot be used together with --include-type. May be used multiple times to exclude more than one type.

Table 6. jms-hdrs-props-test

--include-type

Name of Java property type to include. Cannot be used together with --exclude-type. May be used multiple times to include more than one type.

--exclude-type

Name of Java property type to exclude. Cannot be used together with --include-type. May be used multiple times to exclude more than one type.

There is currently no way to control/limit the JMS header types in this test, but there is an issue open to address this limitation.

Examples:

To limit amqp_types_test to boolean type only: $ amqp_types_test --include-type boolean

To limit amqp_types_test to Qpid-cpp and Proton-python shims only: $ amqp_types_test --include-shim ProtonCpp --include-shim ProtonPython

To test against a pair of Dispatch Routers and a broker running on the local machine as follows (first set up the brokers and routers):

+----------+     9001 +----------+    5672 +--------+
| sender   |--------->| dispatch |-------->|        |
|  shim    |          | router 1 |         |        |
+----------+          +----------+         |        |
                                           | broker |
+----------+     9002 +----------+    5672 |        |
| receiver |<---------| dispatch |<--------|        |
|  shim    |          | router 2 |         |        |
+----------+          +----------+         +--------+

$ amqp_types_test --sender localhost:9001 --receiver localhost:9002

5. Output

All the tests will list each test as it runs and whether it passes or fails. If a test fails, then the details of the failure will be printed at the end of the test.

 ======================================================================
 FAIL: test.B.MESSAGE.JMS_TYPE_HEADER:string+JMS_REPLYTO_HEADER:topic.QpidJms->ProtonCpp (__main__.PartB_JmsHeaderCombination_TestCase)
 ----------------------------------------------------------------------
 Traceback (most recent call last):
   File "./install/lib/python2.7/site-packages/qpid_interop_test/jms_hdrs_props_test.py", line 422, in inner_test_method
     receive_shim)
   File "./install/lib/python2.7/site-packages/qpid_interop_test/jms_hdrs_props_test.py", line 313, in run_test
     self.fail(str(receive_obj))
 AssertionError: JmsReceiver error: Unexpected JMS message header: JMS_PRIORITY: Expected default priority (4), found priority 0

Currently, the tests do not produce text log files. However, by using the --xunit-log command-line option, xunit log files will be generated at the conclusion of the test in the xunit_logs directory. The directory may be changed by using the --xunit-log-dir command-line option. Log file management is the responsibility of the user, and if xunit logging is used regularly, these files will accumulate, possibly exhausting the available disk space. NOTE that if a test is interrupted (by using ctl+c, for example), then the log files will not be generated.)