Changes between Version 80 and Version 81 of benchmarks

Show
Ignore:
Timestamp:
11/23/11 16:55:52 (12 years ago)
Author:
bartek
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • benchmarks

    v80 v81  
    150150The QCG-Notification performance was compared to the performance of two other services implementing brokered version of WS-Notification specification. These were Apache !ServiceMix (version 3.3.1) and IBM !WebSphere Application Server (version 7.0).  
    151151 
    152 A reliable performance tests of the notification systems are difficult to realise. Note that the proceeding time of a single, small message is very short, difficult to correct measurement and highly depends on many aspects, such as underlying best effort transport protocols, network protocols or service's configuration.  
     152A reliable performance tests of the notification systems are difficult to realise. Note that the proceeding time of a single, small message is highly short, difficult to correct measurement and highly depends on many aspects, such as machine configuration, underlying best effort transport protocols or network protocols. To overcome this problem, the time was measured not on the basis of a single send of a notification, but on the basis of multiple sends (called here bunch) assumed as an elementary event. 
    153153 
    154154Taking into account these objectives as well as the fact that the tested notification systems are different: each system provides a set of specific functions that may be unavilable in the another systems (for example, !ServiceMix doesn’t provide support for Full topic dialect), the decision was to test only the most common features and focus on key scenarios. 
    155 The elementary tests were performed many times and the received results were averaged. 
     155  
     156The elementary events were performed many times and the received results were averaged. 
    156157 
    157158=== Testbed === 
     
    161162 
    162163=== Test 1 - Notification Delivery without Active Subscriptions === 
    163 The first test concerned the measurement of the time of sending a bunch of notifications (hovewer each notification is sent separately) while there were no active subscriptions in the tested system. That means that the time was counted on the Publisher side and the notifications was not forwarded by the brokers as there were any Notification Consumers interested in receiving notifications.  
     164The first test concerned the measurement of the time of sending a bunch of notifications (hovewer each notification is sent separately) while there were no active subscriptions in the tested system. That means that the time was counted on the Publisher side and the notifications were not forwarded by the brokers as there were any Notification Consumers interested in receiving notifications.  
    164165* filtering based on: topics; 
    165166* notification dialect: Simple; 
    166 * notification payload: 320 bytes; 
     167* notification payload: about 320 bytes; 
    167168* number of notifications in one bunch: 1000. 
    168169 
     
    170171 
    171172''Pros:'' 
    172 * Possibility to compare : 
    173  * approximately constant number of tasks, 
    174  * "the task flow" (when one task is finished, another begins). 
    175 * The program may be used to measure the overall capacity of the system. 
     173* Possibility to compare performance of the systems in the most basic, but typical scenario; 
     174* The delay imposed by the notification systems may be easily measured by comparison to the times of sending notifications between lightweight Publisher and Notification Consumer. 
    176175 
    177176''Cons:'' 
    178 * The measured submitting time may be distorted (the response of the service on the submit request does not necessarily imply the submit to the queueing system). 
     177* The Publisher installed only on a single machine; 
     178* The services in a default configuration and only marginally loaded (no subscriptions, single Topic Namespace); 
    179179 
    180180=== Test 2 - Notification Delivery with Active Subscriptions ===