... | ... | @@ -2,6 +2,42 @@ |
|
|
|
|
|
Validate _multiple_ *publishers* (aka. producers) and _multiple_ *consumers* working on the same [queue](https://en.wikipedia.org/wiki/Message_queue), exchange messages via some [AMQP](https://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol) [message broker](https://en.wikipedia.org/wiki/Message_broker) without any message [corruption](https://en.wikipedia.org/wiki/Data_corruption). Messages consumed should only be consumed *once*; In other words, no consumer should receive a message that has already been seen by another consumer.
|
|
|
|
|
|
## Birds eye view
|
|
|
|
|
|
```mermaid
|
|
|
graph LR;
|
|
|
|
|
|
Producer1{Producer 1};
|
|
|
Producer2{Producer 2};
|
|
|
ProducerX[...];
|
|
|
ProducerN{Producer n};
|
|
|
|
|
|
AMQP((Message Queue));
|
|
|
|
|
|
Consumer1(Consumer 1);
|
|
|
Consumer2(Consumer 2);
|
|
|
ConsumerX(...);
|
|
|
ConsumerN(Consumer n);
|
|
|
|
|
|
Producer1 -- send --- AMQP;
|
|
|
Producer2 -- send --- AMQP;
|
|
|
ProducerX -. send .- AMQP;
|
|
|
ProducerN -- send --- AMQP;
|
|
|
|
|
|
AMQP -- receive --- Consumer1;
|
|
|
AMQP -- receive --- Consumer2;
|
|
|
AMQP -. receive .- ConsumerX;
|
|
|
AMQP -- receive --- ConsumerN;
|
|
|
```
|
|
|
|
|
|
Link to version with subgraphs [https://goo.gl/BQMKXf](https://goo.gl/BQMKXf)
|
|
|
|
|
|
### The following test cases shall be possible to test
|
|
|
|
|
|
* 1 producer, n subscribers (typical example: 'newsletter', etc.)
|
|
|
* n producers and 1 subscriber (typical example: [IoT](https://en.wikipedia.org/wiki/Internet_of_things))
|
|
|
* m producers and n subscribers ((typical?) example: price generation for products sold at whatever PoS)
|
|
|
|
|
|
## Scenario 1
|
|
|
|
|
|
* *Scope*: Check valid message exchange, [cryptographically](https://en.wikipedia.org/wiki/Cryptography) proven message exchange
|
... | ... | @@ -54,6 +90,52 @@ The generating process (aka [publisher](https://en.wikipedia.org/wiki/Publish%E2 |
|
|
|
|
|
Since the process sends non-[ASCII](https://en.wikipedia.org/wiki/ASCII) characters over the channel, we can see if the encoding/decoding works correct or not. In theory this will also test if [JSON](https://en.wikipedia.org/wiki/JSON) module from [Python](https://en.wikipedia.org/wiki/Python_(programming_language)) does it's job correctly, but given our experience, this is usually not the problem.
|
|
|
|
|
|
## Scenario 3 - Queue instance not available
|
|
|
|
|
|
Questions:
|
|
|
* What happens if the queue is not available?
|
|
|
* What are the concepts/available options in regards to business continuity and (high) availability?
|
|
|
|
|
|
### Script
|
|
|
|
|
|
TBD: Shall the script keep trying to send messages or not?
|
|
|
|
|
|
## Scenario 4 - Message producer/consumer not available
|
|
|
|
|
|
Questions:
|
|
|
* Which information is stored in the message queue if subscribers have not been able to pick up messages?
|
|
|
* Can we report such situation automatically in a dashboard or via e-mail? Especially in cases where a messages haven't been picked up within their [TTL](https://en.wikipedia.org/wiki/Time_to_live)
|
|
|
|
|
|
### Testing
|
|
|
|
|
|
Simplistic approach: Just run a producer, but no consumers.
|
|
|
|
|
|
## Questions
|
|
|
|
|
|
### User/Role and privilege concept
|
|
|
|
|
|
* General introduction to the role - and user concept, including introduction of standard privilege settings (i.e. subscriber, administrator, ...).
|
|
|
* Authentication/authorization sshall be performed via [claims based](https://en.wikipedia.org/wiki/Claims-based_identity) [SAML 2.0](https://en.wikipedia.org/wiki/SAML_2.0)
|
|
|
|
|
|
### Prioritization and validity of messages (TTL)
|
|
|
|
|
|
* Is it possible to send messages with higher priority in order to make sure it's delivered as fast as possible, in case there are already several messages in the queue?
|
|
|
* Is setting TTL on individual messages possible in order specify how long a specific message is valid? Is the message then automatically removed from the queue?
|
|
|
|
|
|
### Acknowledgement (fire and forget vs. guaranteed delivery/processing)
|
|
|
|
|
|
* Show how consumer can acknowledge or not acknowledge message delivery/processing.
|
|
|
* Is the message queue able to show which consumer has fetched a message and if it has been acknowledged by this specific consumer?
|
|
|
* Show different status messages can have.
|
|
|
|
|
|
#### Script
|
|
|
|
|
|
While basic_ack/_nack can easily shown scripted, the different status a message may eventually have, can only be shown in the dashboard.
|
|
|
|
|
|
### Dashboard
|
|
|
|
|
|
The solution should provide a web interface ([WUI](https://en.wikipedia.org/wiki/Web_application)) where all relevant information can be tracked, like messages received/sent per queue, per day/month/year, errors per queue and so on.
|
|
|
|
|
|
## Comparison
|
|
|
|
|
|
All these tests will be done with [CloudAMQP](https://cloudamqp.com) as well (separately).
|
... | ... | |