In our SOAP API, we have a function that I believe is intended to take care of small jobs, special cases, cleanup areas, things like that. Using this function bypasses a lot of the niftier features of our application. I have some functional tests for it, it works just fine.
Not long ago one of our customer support people asked me what kind of performance we could expect running almost a million transactions with this function in at least 30 threads. I pointed out that that was not at all what the function was designed to do. He made the excellent point that that was what the customer wanted to do, and we should be able to provide some information for them.
Well, crap. I am simply not set up to run that kind of a test. Neither Ruby nor Windows is very good at handling threads, and I don't have a *nix box I can appropriate immediately. (I'm working on that as we speak.) I did suggest a test that the customer could run on a test server with their own code, but I was disappointed not to be able to say something useful immediately.
But I had been working on this blog, and I re-read my post about soapui, and remembered that it had a load-test feature. Just like the functional-test interface, the load-test was really nicely designed, and I could create a 30-thread, 60-second load test for the function in question in just a few minutes, and get some very nice graphical output. My colleague was pleased, and I hope the customer was too.
I still believe that soapui is not scriptable enough to be my primary regression-test tool, but once again, for exploring-- this time for exploring performance-- you just can't beat it. Very very nicely done.
Again.
Not long ago one of our customer support people asked me what kind of performance we could expect running almost a million transactions with this function in at least 30 threads. I pointed out that that was not at all what the function was designed to do. He made the excellent point that that was what the customer wanted to do, and we should be able to provide some information for them.
Well, crap. I am simply not set up to run that kind of a test. Neither Ruby nor Windows is very good at handling threads, and I don't have a *nix box I can appropriate immediately. (I'm working on that as we speak.) I did suggest a test that the customer could run on a test server with their own code, but I was disappointed not to be able to say something useful immediately.
But I had been working on this blog, and I re-read my post about soapui, and remembered that it had a load-test feature. Just like the functional-test interface, the load-test was really nicely designed, and I could create a 30-thread, 60-second load test for the function in question in just a few minutes, and get some very nice graphical output. My colleague was pleased, and I hope the customer was too.
I still believe that soapui is not scriptable enough to be my primary regression-test tool, but once again, for exploring-- this time for exploring performance-- you just can't beat it. Very very nicely done.
Again.