Apple introduced a useful tool with Lion that allows you to test application performance under various network conditions. The Network Link Conditioner (NLC) is primarily designed for testing iOS apps, though it can also be used for testing websites via Safari in the iOS simulator.
Once you install XCode on Lion, you will find the NLC preference pane installer located within /Developer/Applications/Utilities/Network Link Conditioner/. A reboot is required before the NLC daemon will run.
Once you have installed the Network Link Conditioner you can find the NLC icon in the “other” section of the system preferences. Once activated the NLC tool applies to the network interface of your Mac so it will impact ALL traffic on your computer when you turn it on. The nice thing about using NLC is that you can change the bandwidth profile on the fly.
The NLC is a nice addition to OS X and might just come in handy when running performance profiling from Lion.
Performance testing is an expensive exercise. It takes specialist skills and a significant amount of planning and understanding of the application in order to replicate the hundreds or thousands of users you are expecting. Another option of course is to let your end users do the performance testing for you. Take the announcement from Blizzard below for Diablo III.
Beginning this Friday everyone is invited to log in and help us put the game and servers through their paces in this three day stress test as we march toward the game’s release.
It is really viable to get your end users to stress test your application? Continue Reading…
In most organisations, you will find that while they have a process, nobody seems to know it exactly, or even where to go to find it. The problem, it seems is with the way in which processes are documented. Process documents are usually lamented over at the time of their writing, then shelved without much thought at all. The reason for this I believe is that there is primarily only two times when a process document is actually referenced:
When a new employee joins the team, and is shown how things are done
When a higher manager asks “how does your team operate”
In my mind, I would much prefer a simpler process flow that is actually used by staff, even if it doesn’t cover every possible eventuality along the way. The visual process document provides the most effective way of presenting the flow of how we go about completing our tasks. Its typically printable on one page (though it might have to be A3), it’s pinnable to your office cubicle, and sometimes as importantly, can be pasted into powerpoint presentations for the business.
One of the most important parts of a successful Performance Test is getting the requirements right. No matter what tool you use or how well you script, your test won’t succeed if you are not accurately simulating end-user activity on the application. Traditionally, the user volume has been a mix of current load levels from log files and projected load from the business. More recently, analytics have been so easily implementable that every site these days, even small blogs, have analytics information.
So how can we use Analytics to help build a realistic load profile for performance testing? Continue Reading…
After a recent application update, I noticed that all of our scripts started to fail. It turns out that the AUT now included a piece of tracking code, and our load generators did not resolve the domain name. The error message that came up was:
Error -26627: HTTP Status-Code=404 (Not Found) for "http://tracking.headwired.com/track.asp" [MsgId: MERR-26627]
So why would a 404 cause an error and not just a warning? Turns out that iframes are treated as a critical resource. By default, images, scripts, css are all treated as non-critical resources resulting in a warning only, however iframes are considered to be critical and hence cause an error. It makes sense for iframes to be a critical resource… in 1999 when the virtual mall was a good idea, yet in 2011, iframes are used for tracking services only. Rants aside, I could find the option to treat images, scripts and css as critical (pictured below), however I could not find the option to treat iframes as non-critical.
By turning this option off, all 404’ed resources are treated as errors as well, as you can see in the following output log
Found resource "http://localhost/notfound.css" in HTML "http://localhost/iframe.html" [MsgId: MMSG-26659]
Found resource "http://localhost/notfound.js" in HTML "http://localhost/iframe.html" [MsgId: MMSG-26659]
Found resource "http://localhost/notfound.png" in HTML "http://localhost/iframe.html" [MsgId: MMSG-26659]
Detected non-resource "http://localhost/notfound-iframe.html" in "http://localhost/iframe.html" [MsgId: MMSG-26574]
Found resource "http://localhost/notfound-object.html" in HTML "http://localhost/iframe.html" [MsgId: MMSG-26659]
Error -26627: HTTP Status-Code=404 (Not Found) for "http://localhost/notfound.js" [MsgId: MERR-26627]
Error -26627: HTTP Status-Code=404 (Not Found) for "http://localhost/notfound.css" [MsgId: MERR-26627]
Error -26627: HTTP Status-Code=404 (Not Found) for "http://localhost/notfound.png" [MsgId: MERR-26627]
Error -26627: HTTP Status-Code=404 (Not Found) for "http://localhost/notfound-iframe.html" [MsgId: MERR-26627]
Error -26627: HTTP Status-Code=404 (Not Found) for "http://localhost/notfound-object.html" [MsgId: MERR-26627]
web_url("web_url") highest severity level was "ERROR", 1627 body bytes, 1438 header bytes [MsgId: MMSG-26388]
So back to the problem, a new domain that I needed to block had surfaced and if the domain name resolved, then it would have been included in the tests without my knowledge. While there are two schools of thought here, in the particular testing that I was doing, the customer did not want their tracking sites being hit from the performance tests, it was time to change approach and move to the include only list.
Why was this important? Well now I don’t have to catch every new tracking feature in order to block it, and my ever expanding list of blocked hostnames was now a thing of the past and everything was coming up Milhouse.
My test HTML page is as follows for anyone who is interested.
Joel Deutscher is an experienced performance test consultant, passionate about continuous improvement. Joel works with Planit's Technical Testing Services as a Principal Consultant in Sydney, Australia.
