Oracle Scratchpad

June 8, 2008

Scientific Method

Filed under: Philosophy — Jonathan Lewis @ 9:53 pm BST Jun 8,2008

I’ve finally found out why I seem to disagree with Don Burleson more frequently that I do with other people on the internet.

From a recent OTN thread:

Me: “you’re supposed to design a theory to match the facts, not select the facts to match the theory.”

Burleson: “I think it’s the other way around, Jonathan, the scientific method requires that you start with a hypothesis.”

So that’s my problem – I let the facts stand in the way of a perfectly nice theory.



  1. Actually, both are valid scientific methods – known as deductive and inductive.

    I can start by noticing a fact at a test environment (say, I added a /*+run_faster*/ hint and the query now runs faster) and I form a theory (The hint optimized the execution plan for the query), and now I have to run more tests to verify my theory (look at traces, run the query more times, change to other hints, etc) and then use the additional facts that my tests generated to confirm or reject my theory.

    Or, I can start with a theory (The second time I run a query it will be faster regardless of hints due to having needed blocks in cache) and then run few tests (run queries and look at trace, try to use very small cache, clean cache between runs, etc) and use those facts to verify or modify my theory as needed.

    It doesn’t really matter if you start with theories or facts as long as you actually use the facts you (and others!) find to improve, refine, validate or invalidate your theories. The only important thing is that you have to keep consistency – make sure your theories and facts are not contradictory.

    (Note that I did not follow the original debate and have little opinion about it. Just wanted to clarify a point regarding the scientific method.)

    Comment by prodlife — June 9, 2008 @ 3:00 am BST Jun 9,2008 | Reply

  2. @Prodfile : I enjoyed your reminder about deductive and inductive reasoning. However, I believe Jonathan’s point was that you can’t just select the facts you like and ignore the others. Our theories are ours, and we can change them as needed, but the facts are not “ours” to do with as we wish.

    From what I have seen on the Web, Don Burleson uses the “post hoc, ergo propter hoc” method: I did something, things ran faster, therefore what I did made things run faster. Dilbert’s boss once fixed the Internet this way ;)

    Comment by Stew Ashton — June 9, 2008 @ 6:23 am BST Jun 9,2008 | Reply

  3. As always:

    If you know what the result should look like…you can develop a better theory.

    Have a nice day! :lol:

    Comment by lascoltodelvenerdi — June 9, 2008 @ 6:26 am BST Jun 9,2008 | Reply

  4. Actually I believe it is not really worthwhile to waste more bandwidth on Don. I once tried to extract precise information from him but he simply went mute. If someone does not want to have a discussion founded by facts and proper arguments then be it. It’s sad if he leads others astray but eventually everybody can and has to think for himself.

    Thomas S. Kuhn’s theory about scientific revolutions provides some comfort for us even in this case: he observed that theories are replaced through extinction of their supporters. Don will at some point in time go out of business and so will other supporters of the common myths leaving the field for Jonathan, Richard etc. :-)

    Further reading:

    Comment by Robert Klemme — June 9, 2008 @ 2:58 pm BST Jun 9,2008 | Reply

  5. My issues with DKB aren’t so noble. I just find this guy offensive, I have yet to see him post any evidence to back up any of his claims. Anytime I read something by Jonathan,Tom or Richard Foote there is evidence behind it…trace files, execution plans, statistics, logical test results. Don just talks, always stating that he has witnessed the opposite of what someone is saying, but never providing proof. Maybe it is because I have both an earring and a tattoo that I am so put off.

    Comment by Tony — June 9, 2008 @ 7:08 pm BST Jun 9,2008 | Reply

  6. This made me think of two things:-

    1. Trying to establish a very ropey correlation by making a line fit the data points on a graph for an experiment that is strewn with inconsistancies.

    2. The spats that Joe Celko used to have with Chris Date ?, anyone remember those ?.

    Comment by Chris Adkin — June 9, 2008 @ 7:19 pm BST Jun 9,2008 | Reply

  7. It looks like Mr. Burleson is conflating “theory” and “hypothesis”.* If I recall correctly, the scientific method begins with a hypothesis and concludes with a theory.

    * (Which is also what advocates of “Intelligent Design” do when they say things like, “Evolution is only a theory.” )

    Comment by Dave Edwards — June 9, 2008 @ 8:10 pm BST Jun 9,2008 | Reply

  8. Inductive and deductive, aka a priori and a posteriori respectively, and both perfectly valid.

    The difficulty here arises (as I see it), however, where, say, a Statspack output has been doctored to fit a predefined and predicted pattern; though unfortunately for the good doctorer, 10g Statspack output has been interpolated with the 9i Statspack output (on a 9i database), thereby sort of invalidating the whole exercise, and driving a coach and horses through any credibility that this individual might have hitherto secured/retained.

    That rather sums up neatly the issue here I’d say: one (Jonathan) is properly fastidious about the evidence, supported by analysis, and the other is obsessed with the appearance of almost supernatural predictive powers, and will go to no little lengths to buttress those a priori claims with extremely dubious a posteriori evidence.

    Comment by SeánMacGC — June 10, 2008 @ 8:54 am BST Jun 10,2008 | Reply

  9. The other thing about the scientific method is it must be open to peer review. If you have a test that supports your hypothesis others must be able to reproduce it to check your results. Saying I have a really big complicated database rebuilding it in a blocksize of X gives a performance increase/decrease of Y is not proof unless you can show me the data.

    If you can’t show me the data then build a test case to demonstrate it, if you can’t build a test case you haven’t understood what happened or your lying to me. the scientific method is robust because you have to publish your data and others can question it and re run your experiments this leads to a process that looks illogical and haphazard from the outside but produces logical and consistant results.

    Comment by Chris — June 10, 2008 @ 10:45 am BST Jun 10,2008 | Reply

  10. Prodlife,

    I see that Stew Ashton has addressed your first point. It looks as if you were misdirected by Burleson’s attempt to hide the real issue behind an irrelevant observation about the scientific method. You might note, in passing, that his claim that “the theory must precede the facts” happens to be one that you have argued against.

    I don’t think it’s fair to be critical in normal conversation about the precise use of the terms ‘theory’ and ‘hypothesis. To most people they mean the same thing, except one of them is easier to spell.

    Of course, the proponents of ID who deliberately try to associate the scientific use of the word “theory” with the colloquial use of the word should be viewed as liars and charlatans.

    It would be helpful if you supplied some evidence in support of your comments – but I assume you may have had the following in mind:
    where a statspack report can’t make up it’s mind whether it’s a 10g AWR report or a 9i statspack report.

    This was the really disappointing thing about sp009’s contribution to the thread. I thought we were moving towards a point where he would be able to show some ‘real world’ results, and we would be able to say: “yes, you’re getting a benefit, and it’s because …” giving other people the opportunity to look for cases where they might get the same benefit.

    Comment by Jonathan Lewis — June 10, 2008 @ 12:17 pm BST Jun 10,2008 | Reply

  11. My apologies Jonathan, absolutely correct, and yes, that’s the one.

    Comment by SeánMacGC — June 10, 2008 @ 12:23 pm BST Jun 10,2008 | Reply

  12. “Actually, both are valid scientific methods – known as deductive and inductive.”

    Only deductive is scientific. Why? Because science is the study of empirical evidence. Deduction is a form of study, induction is not.

    Jonathan’s comment “you’re supposed to design a theory to match the facts, not select the facts to match the theory.” is incorrect. Theory should not be designed to match the facts, as it limits the theory to that which is observed. Don Burleson is correct in that “I think it’s the other way around, Jonathan, the scientific method requires that you start with a hypothesis.” Exactly. Start with a hypothesis (induction) and use the scientific method to test it. In Jungian terminology (as explained by van der Hoop) the creativity and induction is intuition (N), factual study is Thinking (T).

    Jonathan touched on this in another comment in that thread “the importance of constructuing experiments to test a hypothesis, and given other people the chance to see how careful you have to be to design the test properly”. And he really said it by finishing his comment with “You’re supposed to start with observations (facts), then construct a theory, then make predictions based on the theory, then test the theory to see if the predictions are correct.”

    IOW, hypotheses have no limits. Jonathan advocates the scientific method by using use the practical idea of having facts to limit the hypothesis before testing it. In a sense, Jonathan does do what he quoted from Burleson, but it is directed. Greg Rahn was more to the point with “Isolated and controlled experiments are very meaningful if constructed correctly”…”taking a complex problem and simplifying it so that it can be understood, and then confirming that the observations made in isolation are also pertinent in the original situation.” Exactly.

    The problem here is that Burelson said “Oracle officially recommends hypothesis testing with real-world workloads, not contrived artificial test cases.” and he wanted to use that to come up with some general theory, and does not provide any backup.

    I think Jonathan used the wrong statements. In the exchange itself, Burleson is correct. However, the way they applied them shows quite clearly the practicality of Jonathan approach, and the absurdity of Burleson’s.

    Comment by Brian Tkatch — June 11, 2008 @ 7:30 pm BST Jun 11,2008 | Reply

  13. Brian,

    I think you’ve followed the misdirection that I pointed out in commit #10; and you’ve made the same mistake as Prodlife – as pointed out by Stew Ashton in comment #2.

    The first part of my comment is: “you’re supposed to design a theory to match the facts” which you object to because it limits the theory. But the negation of my comment would be “you’re allowed to design a theory that doesn’t match the facts”..

    Requiring a theory to match the facts doesn’t limit the theory to match nothing but the currently known facts. Once a theory matches all the known facts it allows you to make predictions (of new facts) that may be tested.

    The second part of my comment is: “you can’t select the facts to match the theory” which, when negated, would be “you can ignore the facts that don’t match the theory”.

    I don’t think you could possibly state that my comment was wrong. The fact that Burleson wishes to contradict the comment is surprising – the fact that he introduces an irrelevant and distracting claim about the scientific method is not.

    Comment by Jonathan Lewis — June 11, 2008 @ 9:26 pm BST Jun 11,2008 | Reply

  14. Jonathan,

    Interesting points by using the opposites. The statements were not meant to be boolean, thus can be proven by the negation of the opposite) but the point is taken. I’m pretty sure we are saying almost the same thing. But we choose to use different words.

    My comment was more directed at the first comment stating expressing the thought that induction is equal to deduction in the scientific method. That it surely is not, because it precedes it, but the method itself is *entirely* deduction. And that is what Burleson got right. Not that it mattered here in Oracle, as you pointed out the misdirection.

    Anyway, i just bought Cost-Based Oracle Fundamentals (last night). Its completely unrelated to the discussion here, other than that i’m willing to pay to see what you have to write. :)

    Comment by Brian Tkatch — June 12, 2008 @ 12:21 pm BST Jun 12,2008 | Reply

  15. I have followed the post with utter interest.
    Maybe Mr Buerleson was simply not understood.
    Perhaps, just as Wittgenstein, he claims the scientific method shall not be expected to
    give complete explanations of the world, nor be applied where it does not belong.

    To be consistent, I believe the propossition “What we cannot speak about we must pass over in silence” must also be applied.

    Comment by Alexis Gil Gonzales — June 17, 2008 @ 10:09 pm BST Jun 17,2008 | Reply

RSS feed for comments on this post. TrackBack URI

Comments and related questions are welcome.

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Powered by