Changes between Version 80 and Version 81 of benchmarks

Timestamp:

11/23/11 16:55:52 (13 years ago)

Author:

bartek

Comment:

--

benchmarks

@@ -150 +150 @@
 The QCG-Notification performance was compared to the performance of two other services implementing brokered version of WS-Notification specification. These were Apache !ServiceMix (version 3.3.1) and IBM !WebSphere Application Server (version 7.0). 
 
-A reliable performance tests of the notification systems are difficult to realise. Note that the proceeding time of a single, small message is very short, difficult to correct measurement and highly depends on many aspects, such as underlying best effort transport protocols, network protocols or service's configuration. 
+A reliable performance tests of the notification systems are difficult to realise. Note that the proceeding time of a single, small message is highly short, difficult to correct measurement and highly depends on many aspects, such as machine configuration, underlying best effort transport protocols or network protocols. To overcome this problem, the time was measured not on the basis of a single send of a notification, but on the basis of multiple sends (called here bunch) assumed as an elementary event.
 
 Taking into account these objectives as well as the fact that the tested notification systems are different: each system provides a set of specific functions that may be unavilable in the another systems (for example, !ServiceMix doesn’t provide support for Full topic dialect), the decision was to test only the most common features and focus on key scenarios.
-The elementary tests were performed many times and the received results were averaged.
+ 
+The elementary events were performed many times and the received results were averaged.
 
 === Testbed ===
@@ -161 +162 @@
 
 === Test 1 - Notification Delivery without Active Subscriptions ===
-The first test concerned the measurement of the time of sending a bunch of notifications (hovewer each notification is sent separately) while there were no active subscriptions in the tested system. That means that the time was counted on the Publisher side and the notifications was not forwarded by the brokers as there were any Notification Consumers interested in receiving notifications. 
+The first test concerned the measurement of the time of sending a bunch of notifications (hovewer each notification is sent separately) while there were no active subscriptions in the tested system. That means that the time was counted on the Publisher side and the notifications were not forwarded by the brokers as there were any Notification Consumers interested in receiving notifications. 
 * filtering based on: topics;
 * notification dialect: Simple;
-* notification payload: 320 bytes;
+* notification payload: about 320 bytes;
 * number of notifications in one bunch: 1000.
 
@@ -170 +171 @@
 
 ''Pros:''
-* Possibility to compare :
- * approximately constant number of tasks,
- * "the task flow" (when one task is finished, another begins).
-* The program may be used to measure the overall capacity of the system.
+* Possibility to compare performance of the systems in the most basic, but typical scenario;
+* The delay imposed by the notification systems may be easily measured by comparison to the times of sending notifications between lightweight Publisher and Notification Consumer.
 
 ''Cons:''
-* The measured submitting time may be distorted (the response of the service on the submit request does not necessarily imply the submit to the queueing system).
+* The Publisher installed only on a single machine;
+* The services in a default configuration and only marginally loaded (no subscriptions, single Topic Namespace);
 
 === Test 2 - Notification Delivery with Active Subscriptions ===