I’m chairing next week’s Ark Group conference on SOA (Achieving Interoperability in Systems Architecture) at the Vibe Hotel in North Sydney, Australia.
You can download the conference brochure here. My ThoughtWorks colleague Halvard Skogsrud and I are presenting the workshop entitled “Bearing the standards of interoperability”.
Recently, Les Hatton wrote a compelling article entitled The Chimera of Software Quality. (Les Hatton, “The Chimera of Software Quality,” Computer, vol. 40, no. 8, pp. 104, 102-103, Aug., 2007)
A section of the article headed The cost of poor quality points out the economic impact of poor software quality, and the almost belligerently ignorant and/or apathetic approach to it by people and businesses in the technological nations.
He summarises this in what he postulates as Hatton’s Law:
The technological societies will collectively trash around [US]$250 per person per year on systems which will never see the light of day or, if they do, do not come close to what their users wanted, assuming they were asked in the first place. This they will ignore.
(emphasis mine.)
If you aren’t flabbergasted or at least peeved by that figure (except because you’ve already come to the sad realisation), stop reading now and go and defragment your hard disk, ignoramus.
As we say Downunder, “Goodonya, Les!”. Although he didn’t suggest where that money should go, he did indicate that it’s not just about the money, it’s about the engineering legacy we’re leaving. I have no idea if he shares my views, but other engineering disciplines have helped make the world better by avoiding the cost of poor quality. Pastoral tools, houses, clothes, toilets, aqueducts, fishing nets, boats, glass, the list goes on (and will probably not include the iPhone, people!).
Of course in software there are tactical actions taken, especially when shareholders start raising concerns over short-term cost-benefits of IT (or, more accurately, when executive bonuses are impacted by it by more than a few percentage points). But this is not enough.
Sometimes, I’m accused of being aggressive in the indignation I display when I see woeful technological systems. I suppose that’s because something inside of me has always been thinking that the money spent on building that pile of steaming crap could have been used for much more constructive purposes. In that sense, I’m not nearly aggressive enough. Perhaps I’ll stop calling it “technical debt” and start calling it “karmic debt”.
Stop writing shitty software and you could do something even better than increase shareholder value or boost your annual bonus. You could help take a stand against poverty - in your own country and around the world.
There are many corporations who divert some resources to social responsibility. Think how much more they could do if they did some of the simplest, mind-numbingly basic things to avoid spending massive sums on poor software. Those things aren’t just throwing money at a problem, but the money or more freedom from cost constraints allows corporations and their employees to do so much more.
Even corporations in industries you’d normally least expect to understand are being more socially responsible in a way that doesn’t raise the eyebrows of the cynics like me. I’ve personally witnessed the activities undertaken by employees at Westpac, a large Australian bank. I also know that avoiding the cost of poor software quality would allow them to do more. ThoughtWorks is not immune either. We use a couple of systems internally that are so fantastically crappy and impact both our effectiveness and efficiency that it beggars belief (I must point out this is relative to a benchmark one would expect of a company with our ideals, and would be otherwise more than reasonable almost anywhere else — we have the luxury of being very picky when critiquing internally). One of my clients laughed at me the other day when he found out what email client we use, because he made a lot of money (and stomach ulcers) supporting it in years past and he knew the impact that its poor quality has on its users (not that most of the alternatives are a benchmark for excellence).
ThoughtWorks is on an ambitious mission to change the nature of IT. In doing so, we hope to contribute to a better world. Avoiding the cost of poor software quality is one of the things we’re doing to help.
Most of my ThoughtWorks colleagues love tests. We love writing tests and watching them fail, and we love writing the code that then makes them pass. We love tests so much that most of the time taken in building the software is in the running of the tests to make sure we haven’t done something dotty to break the software.
We hate when the time taken to run all those tests takes more than a few minutes, and this causes a lot of discussion internally as well as on our blogs. My strategic preference is to make the application smaller and thus the builds quicker, by building the software as discrete components (built and deployed independently) that are composed into an application. But before that, there’s a long list of things to make the builds and tests quicker. The list is never long enough, though.
We regularly discuss/concoct innovations in testing at the Sydney TW office. A few weeks ago Stacy Curl started a discussion on an idea he’s had for a while around optimising acceptance tests. During this discussion, we talked about various techniques, including profiling and coverage, which then prompted thoughts around knowing which code each test covered.
This then blossomed into a requirement which was to instrument the code that is covered by each test such that when that code is changed, the tests that cover it are known. This then lets us run only the tests that “need” to be run when we change some code. I say “need” because there are always things below or outside the code that may alter behaviour, so a full test suite still has to be run at some point, just not in a way that delays feedback on the most likely problems.
In essence, we wanted a tool that defines the smoke tests (pre-commit build, initial CruiseControl pipeline build) dynamically. The “changed” code could be based on file system modified times, Subversion status, CruiseControl modificationset, etc.
Unfortunately, the context provided by tools like Clover, Emma and Cobertura isn’t sufficient to indicate which test is running (no stack is available in the metrics). The upshot was I went off and looked at using AspectJ aspects to trace the code during a test run. Gianny Damour helped with early optimisations of my advices and gave me the introduction to “cflow” which allowed me to remove my own stack-peeking logic. I was going to call it TestThis! (as in this.method()) but because of the annoyance in lowercase called it TestMe instead (no throwback to VB intended), with the almost obligatory “J” prefix for Java. Josh Price rightly pointed out that the resultant name “JTestMe” is lame - another reason to merge with ProTest (see below).
Only the source file name is tracked. While we could instrument to the method level, we felt that this was too granular and that any change to the source file should trigger all tests that touch any of that class’ logic. It also works for superclasses, inner classes and called classes - that is, the tests may not call the code directly, but because the code is within their control flow, they’re all tracked.
public aspect TestCoveringAClassObserver {
pointcut allTestMethods(TestCase test):
target(test) && // capture the TestCase
execution(public void TestCase+.test*()) // and advise execution of any JUnit test methods
;
before(TestCase test):
cflow(allTestMethods(test)) && // within the control flow of the allTestMethods advice
(execution(!private * *(*)) || execution(!private * *())) && // when executing any non-private method
(execution(* !TestCase+.*()) || execution(* !TestCase+.*(*))) && // that is not in the test case class
within (!com.thoughtworks.testme..*) // and not within any JTestMe class
{
recordLink(thisJoinPointStaticPart.getSourceLocation(), test);
}
;
}
Even with the extra guards to avoid recursive advice explosions and to ignore instrumenting the test code itself, you can see that the core piece of code is a ridiculously few lines of AspectJ. The “database” updated by the recordLink() method is a fat ugly (but very fast) object de/serializer that I haven’t bothered cleaning up much.
To allow you to easily run your code with or without instrumentation (you may be nervous that AspectJ has subtly altered behaviour), I’ve used the AspectJ Load Time Weaving (from the AspectWorkz merge). It also means you can continue to use your own custom JUnit test runners. Instrumented - use aj5’s java --javaagent). Normal - use java.
At CITCONF in Sydney a few weeks back, where I gave this a quick demo, the crowd were very encouraging and the Atlassian Clover (nee-Cenqua) guys, who demonstrated a tops new version, were academically interested in the neat and tidy use of aspects (they can’t use them for a reason I can’t remember?? Will find out next meetup).
There’s a long to-do list, the first of which are make it more “one-click” to use and explore a merge into ProTest (a test prioritising tool - and I figure the tests that JTestMe identifies are just another way to prioritise tests). Others are SVN integration and Ant task, and a filter for the excellent Continuous Testing tool (which also allows for prioritising tests).
It’s just been accepted into the Codehaus so have a peek and let me know. The usage is clunky (change the JTESTME_HOME values and the aj5 java agent path and test cases in the Windows cmd files) but it works for HelloWorld demonstration purposes (no automatic file modification detection yet).
Powered by WordPress