Oracle Scratchpad

September 24, 2009

Philosophy – 6

Filed under: Philosophy — Jonathan Lewis @ 8:15 pm BST Sep 24,2009

Testing for correctness:

  • Have you tested with good data and all the different possibilities of bad data
  • Does the code do the right thing … and nothing more.

[The Philosophy Series]


  1. This is one of the things I have the hardest times convincing people of in the RAD type of environment (and I’m sure the modern equivalents of RAD must have the same issues). I’ve changed one program 3 times in the last two days, for reasons I predicted in arguments over it years ago. You need to temper what the users want with foresight, they tend to be focused too narrowly on function and not focused enough on data. And vice-versa.

    As far as the right thing and nothing more, that could get into some deep philosophy. Yes, you can design provably correct software, but that’s really, really expensive at the enterprise level, even non-competitive when selling large applications. Look what’s happened to the ultimate software, the US space program software. Who can maintain it now? People start making silly mistakes like US/metric conversions. Hardware still winds up in out-of-bound conditions. New stuff just isn’t done right.

    Comment by joel garry — September 24, 2009 @ 9:00 pm BST Sep 24,2009 | Reply

  2. I think the key statement here is “… and nothing more”

    While it needs to be said, I think most people do test for positive outcomes, and sometimes basic negative ones. However side effects are oft forgotten. And Oracle provides us with more and more functionality/ways to create side effects. Sometimes these are difficult to track, usually from negative responses – but how much effort due you spend chasing potential problems that may not exist?

    I think I/we just answered myself.

    Comment by Scott Wesley — September 25, 2009 @ 4:46 am BST Sep 25,2009 | Reply

    • Scott,
      I think you’ve highlighted the biggest problem – how do you know when you’ve done enough … “you can’t prove a negative”.

      One of the big gaps in testing, though – and one that’s quite hard to think through – is the idea of concurrency. We can’t imagine all the things that could go wrong, but we can ask the question “what might other people be doing in the same place at the same time?”. That one question would probably pre-empt quite a lot of the problems I see with Oracle systems.

      Comment by Jonathan Lewis — September 25, 2009 @ 5:03 pm BST Sep 25,2009 | Reply

  3. Some guys put testing on top of the development process, i.e. they write tests before they write single line of code. TDD is all about it.

    Comment by Timur Akhmadeev — September 25, 2009 @ 6:24 am BST Sep 25,2009 | Reply

    • Yes indeed, I think it’s very easy to fall into the trap of defining tests based on what you know the code that you’ve written is intended to do, rather than on what it is supposed to do. A very good step when software development is to consider first how you will test that the code is correct _before_ writing it. Preferably you’d write the test code first in fact.

      Comment by David Aldridge — September 25, 2009 @ 6:45 am BST Sep 25,2009 | Reply

      • Testing what you know is supposed to happen — that’s why it’s good to give the program spec to someone else to create the test data. They don’t know what your code is going to do, so they’re more likely to create data that finds your errors.

        Comment by Jonathan Lewis — September 25, 2009 @ 4:59 pm BST Sep 25,2009 | Reply

    • There’s a package called dbms_space_admin (or something like that) that includes procedures for patching up space management bitmaps.

      I was once told, though I can’t tell how accurate the tale was, that the code to fix the bitmaps was one of the first things written when the entire bitmap management stuff was created.

      I think that captures the idea of how to think about testing.

      Comment by Jonathan Lewis — September 25, 2009 @ 4:57 pm BST Sep 25,2009 | Reply

  4. “Tests are done to find bugs and not to show their absence”.

    The more we find bugs the less we have risk to meet them in Production.

    The best way I have found to find more bugs is to run my code against a large volume of data; is to run my code in an overnight batch job with data coming from several sources.
    Code is run in a for loop and all errors are logged into a dedicated table without exiting the loop in case of error.

    Comment by Mohamed Houri — September 25, 2009 @ 7:51 am BST Sep 25,2009 | Reply

  5. Null data testing. What happens when you run you program with no data to process? It’s caught me out a couple of times.

    I agree with Timur and David,it’s hard to beat the process of writing you test cases (and developing your test data) before you start writing your code. In fact, it’s nice just to see some properly structured test data once in a while… *sigh*

    Comment by mwidlake — September 25, 2009 @ 1:48 pm BST Sep 25,2009 | Reply

RSS feed for comments on this post.

Comments and related questions are welcome.

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Powered by