Martin Spier's Blog

This came to my attention on the last few weeks since several projects came with this request and nobody had done this before.
First of all, I’d like to explain a little bit about how the Citrix protocol works on LoadRunner.
Once LoadRunner establishes a connection with the Citrix server, we have two kinds of transactions, Outbound and Inbound. The first comprehending keystrokes and mouse gestures going from the client to the Citrix server. The second comprehend window commands and screen refreshes going from the Citrix server to the client.
Differently from other protocols, were we send a package (HTTP post, WebService call, etc) and wait for the response to measure the response time, the Citrix protocol does not have this kind synchronization. Basically what happens is, we send a keystroke for example, the server responds with a acknowledgment, and that’s it, there is no way of knowing if the applications has processed the request and presented the results. Basically, there is no synchronization between Inbound and Outbound transactions.

In order to measure the response times for Citrix transactions, we need to find a way to “understand” what is coming from the Citrix server and manually synchronize it. There are two ways of doing that, the first is to monitor for window changes, like a window opening, etc. The second is to monitor a part of the screen for a refresh. The problem with both of them is to predict what is going to change, like the name of window that is going open or which part of the screen will refresh. This makes the scripting part really difficult and time consuming, because we cannot predict exactly what is going to be changed, like for example, we’re monitoring for a screen change, but an error message appears, the script will understand that there was a change on the screen and continue, consequently crashing.

Another point to notice is that Citrix is a proprietary solution, based on a widely know protocol (RDP). RDP’s performance is based solely on the bandwidth capacities and being a proprietary solution, we cannot change the protocol, consequently, we cannot increase the performance of the protocol, we can only work with the bandwidth between the Citrix server and the client. This bandwidth varies too much from time to time (peak hours, etc), so the results will vary a lot depending on the time we run the tests.
Based on this assumption, in order to have reliable results, we need to run several rounds of tests during different times, so we can have an idea about the overall performance.

Assuming these problems, I tried to find different ways of getting the response times from Citrix applications. Based on a few articles on the subject, I found that there are more reliable ways of doing that, other than using LoadRunner scripts to capture response times.
Going back to the requirements, we can separate the measure of “response time” to do certain action into three steps. The first is the time it takes from sending a command from the client to the Citrix server. The second is the time the application takes to process that request. The third is the time it takes for Citrix to send the screen refresh with the results to the client. So:

Send Keystroke + Time to Process + Send Screen Refresh = Overall Response Time to Perform an Action

On most cases it is easy and reliable to measure the “time to process” each action locally, without Citrix, so what we need to measure is the time it takes for sending a keystroke to Citrix and the time it takes for the client to receive the screen refresh. Basically is the delay of sending a keystroke and the delay of receiving the screen.

We know that this delays are affected mainly by three factors, the bandwidth limitations, the latency and the amount of data transferred. The first step is to find out how much data is transferred. We can do that by two ways, one is to use a sniffer on the network to capture exactly how much data is sent and received on each transaction. The second is to estimate this amount based on known factors.
For the outbound transactions, this is quite easy, since mouse gestures and keystrokes usually consume the same bandwidth. For the inbound transactions we know that the RDP protocol uses JPG compression to send the screen refreshes, and it only sends the updates for the parts that were changed. This way we can estimate the amount of data transferred by using a simple formula to calculate the JPG size using the image size, density and color content of the refreshed areas.

Knowing the amount of data to be transferred on inbound and outbound transactions we have two options to know the delay. The first is to measure the Round Trip Time (RTT) from the Citrix to the client and the client to the Citrix server, using the amount of data estimated before. The second is to estimate the RTT with known formulas, that use this three factors to estimate the probable delay on the RTT.

After all this technical explanations, I believe that the best solution is to:

1. Measure the time to process each action locally, without Citrix.
2. Estimate the average amount of data transferred from the client to the Citrix server, using the standard values for keystrokes and mouse gestures.
3. Estimate the average amount of data transferred from the Citrix server to the client on the screen refreshes, using captures JPG images of the updates screens.
4. Measure the RTT with the estimated data for the outbound transactions on different times. This can be easily done by a network engineer once we know the amount of data.
5. Measure the RTT with the estimated data for the inbound transactions on different times.
6. Sum all this measures and define our average response time to execute an action over Citrix.

This way we have more reliable results, already on a fine grain, making it easier to identify problems, since we will know if the response times are high on the processing part or the Citrix part.

Note that we’re not trying to measure the Citrix server performance from a load perspective. There are known metrics for Citrix servers that usually satisfy the sizing requirements.

Related posts:

1. Recording Citrix scripts on LoadRunner
2. How to Obtain an ICA File Through Citrix Web Interface 4.5 and 4.6
3. How to Change the TimeOut on LoadRunner
4. Performance Testing and its Benefits
5. Creating Oracle 12 Scripts on LoadRunner