Many-to-many join on non-unique datasets… I wasn’t visualizing that quite right. I think that merge sort and merge join algorithms got a little jumbled in my head. It seemed “artificial” to me because I thought you unnecessarily invented the third pointer. The third pointer is needed for a merge join (though not for a merge sort).

My apologies for accusing you of being artificial. :) Thanks for clearing up my misunderstanding.

In what way is the example artificial ? I have rows which are non-unique in the first data set and some of those rows correspond to multiple rows in the second data set. The data is NOT artificial.

Your point about “two pointers” and the sequential nature of walking the data is valid – and it’s the reason why we usually go to the expense of sorting and buffering the two sets of data.

But to do the merge join we need a third pointer that starts from the position dictated by the second pointer (when it is in position) and walks the data set until it comes to a row which no longer matches current item in the first data set. At that point, the first pointer will move forward, the second point may, or may not, move forward, and the third pointer has to be set to the latest value of the second pointer.

Irrespective of any thoughts about optimising the efficiency of the mechanism, though, the merge join at this point is still following the pattern: for each row in the first rowsource locate the matches in the second source.

Your example at the top of the post (scanning the second set) is artificial. A properly written merge join algorithm keeps a single pointer into both datasets and advances one or the other until either terminates. Certainly no looping is needed on the second dataset! The whole algorithm is a single loop – there shouldn’t be any nesting. Certainly when I designed and wrote merge joins for algorithms classes there weren’t nested loops.

The caveat of course would be that both datasets are presorted. And of course SORTING requires some for of nesting and can’t beat O(n lg n). But if the input sets are truly presorted then the join itself should not require a loop and should execute in O(n) time, right?

It’s certainly correct to say that there are cases where, for the same volume of data and same amount of memory used, the merge join CAN outperform the hash join for exactly these reasons.

Nice article..

“If the sorted data sets for a merge join are too large to fit into memory Oracle dumps them to disc, but only reads through them (at most) once each”

And in the last part of your article you show that if there is a suitable index exist on join column(s) of table(s) involve in join
optimizer tend to avoid sorting which eventually lead to save on memory and CPU cycles

Is it correct to say that in such scenario merge join outperform compare to hash join?

Thanks

We (the designers/DBAs/programmers) do not have to create a hash table to get a hash join. I introduced the real single table hash cluster in a nested loop as a way of showing the EFFECT of what Oracle does internally when it chooses to do a hash join.

The choice of hash join, merge join, or nested loop join is determined entirely by the optimizer based on its assessment of data volume, data scatter, and available machine resources.

I do not know if Cost Based Oracle Fundamentals is easily available in India.

Thanks for this Article.

Just one doubt , the execution plan will decide to go for Merge Join depending upon the sql we write ? is this correct , unlike Hash join we need not create seperate Hash Tables i guess .. so please confirm if this determination is entirely based on the select statement plus the objects accessed?

Secondly the book you have mentioned , is it available in India? if yes then kindly let me know the publisher details as I am not getting in book stores here….

Thanks,
Kartik