Sunday, January 9, 2011

Archive: Testing legacy web applications using a black-box method

Cross-posted/archive:Originally a comment on an InfoQ article, later became an article on my personal blog

Recently, a whole module of a legacy web application written in PHP4 around 10 years ago (and constantly "maintained" since) was needed to have new features.

We're talking about 1000s of lines of code within one function, or even case of a switch case.

Most of the time, people don't refactor maintained legacy applications, as somebody told me "the first rule of support development is: don't change anything other than requested, just add your stuff."

I haven't been able to track the application back to its beginnings but its imminent to me that if-else branches don't grow to 600 lines by themselves, without human intervention. Somebody has to mess these up, and someone has to have such kind of thinking. This is pretty general in enterprise programmng:

  • most of the tasks are about legacy applications

  • people fear to clean things up

  • it's not about development, but adding requested features and fixing bugs.


Also, PHP is a dynamic language, and therefore formal refactoring tools are usually unavailable. For example, PHP refactoring support in netbeans is basically non-existing.

So, what would you do here?

I decided that a system's answer is dependent only from its input and its context. This seems pretty straightforward:


System ( input, context ) -> output


OK, what is the input of a web application? Of course, its HTTP request! In PHP, it's hard to think about any other input.

What's the context? Context is given by two components basically: the underlying platform, whatever it is (no matter you have a framework or just common libraries, we call these platform together), and the persistent data layer. So:


Web app (request, persistent data) -> answer


(I know, I forgot platform to add, but in refactoring scenarios, platform should stay the same anyway. In case you change platforms too, there are other complications which we won't talk about this time.)

What's the answer? First, it's an HTML (or XML, JSON, etc) output. We didn't have to care about it in this particular case. The other output is: changes to the persistence layer. It's unusual for web applications to change anything other than their database and cache layers.So:


Web app(request, persistent data) -> (written-out response, persistent data' )


OK, what to do? We have an old system and we want to refactor it to a new system, and the question was: are they equal in functionality?

Question is: Web app == Web app' ?

Let's see what I did:

  • Ask a manual tester to go through every possible combination on the user interface
  • Recorded these into files (serialize($_REQUEST)), or, even better, (serialize($GLOBALS))
  • Ask the DB layer to NOT write anything to the DB (ugly global variable hack, if it is present, only select queries are executed), this way, we ensure that we keep a consistet state
  • Record every writing operation (so, instead of executing them, take note of them)
  • An algorhythm:

    1. load the serialized request,

    2. start recording db,

    3. run the original controller,

    4. collect db recordings,

    5. re-load request (in case it was modified by the original controller - we could never know)

    6. run the new controller

    7. collect db recordings

    8. see if the two are equal



This way, I could be sure, that in all of the scenarios a manual tester could come up with, both of the controllers behave the same way.

After the original recordings, I did a few additional points:

  1. re-load the request again
  2. enable db writing
  3. run the new controller
  4. display result.


This way I could create an - albeit slower - but seemingly normally functioning version of the software, which did everything it did previously, and it was verified that functionality haven't changed with the new controller.

I called this blackbox-harness test.

What do you think?

No comments:

Post a Comment