Skip to main content

Posts

Showing posts from May, 2006

Kevin Lawrence is still right

Back in March 2005 Kevin Lawrence wrote this great article for Better Software called "Grow Your Test Harness Naturally"

He's got a link to the PDF on his blog, luckily.

It's one of those articles that keeps being cited by various people one runs into here and there. I read it over and over until I Got It.

The basic premise is: don't write the framework first. Write some tests, then extract the common code from those tests, write more tests, abstract common code, write more tests, refactor, repeat. The refactoring generates the framework.

The thing is, I had never had enough code complex enough, running against an app stable enough, to really and truly build a framework this way.

Until now.

I have a large number things to do with Watir.

First I automated each path through the code individually. I did about 10 of these.
Each of my paths had to set up a certain set of rules, so I abstracted a method "make_rules"
Each of my paths had to set some data befo…

Internet Explorer Basic Authorization vs iFrame is just wrong and stupid

UPDATE October 2009: it seems Angrez has added support for Basic Auth to a fork and it should hit the main Watir distro RSN. News will be here. Good things come...

So I wanted to bypass the 401 Authorization Required screen that IE kicks up without resorting to actually having to manipulate the popup. So I hacked Watir's goto() method, which works great:

def goto(url)
#@ie.navigate(url)
@ie.navigate(url,nil,nil,nil,"Authorization: Basic AbdXyz46bG1ubw==\n")
wait()
sleep 0.2
return @down_load_time
end

I know I can get away with this because the user/password on the test systems are never going to change, and even if they do, it's an easy script change.

The trouble is that the main page has an iFrame, and for some reason, when called from a script, stupid IE won't use Basic Authorization credentials to request the iFrame.

So now I'm getting the authorization popup when IE tries to load the iFrame contents. …

Two things about performance testing

A couple of things to stick in your back pocket:

1) Think geometrically, not arithmetically. Think about orders of magnitude.

10 is the same as 20 is the same as 50.
2000 is the same as 5000 is the same as 7000.
100,000 is the same as 200,000 is the same as 300,000.

The interesting information comes between the orders of magnitude: is 10,000 faster or slower than 10? is 1000 faster or slower than 100,000?

2) Think about precision and accuracy.

It is precise to say "10 records per second on this OS on this hardware on this network".
It is accurate to say "a rate of between 1 and 10 records per second on any OS on any reasonable hardware".
It is accurate to say "10 records per second running locally; 1 record per second over the network; 10 seconds per record over VPN".

Accuracy is critical in the general case; precision is critical in the specific case.