<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:georss="http://www.georss.org/georss" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:media="http://search.yahoo.com/mrss/"
		>
<channel>
	<title>Comments on: Partitioned Bitmaps</title>
	<atom:link href="http://jonathanlewis.wordpress.com/2011/07/01/partitioned-bitmaps/feed/" rel="self" type="application/rss+xml" />
	<link>http://jonathanlewis.wordpress.com/2011/07/01/partitioned-bitmaps/</link>
	<description>Just another Oracle weblog</description>
	<lastBuildDate>Wed, 19 Jun 2013 22:21:35 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.com/</generator>
	<item>
		<title>By: Jonathan Lewis</title>
		<link>http://jonathanlewis.wordpress.com/2011/07/01/partitioned-bitmaps/#comment-41063</link>
		<dc:creator><![CDATA[Jonathan Lewis]]></dc:creator>
		<pubDate>Mon, 25 Jul 2011 07:40:59 +0000</pubDate>
		<guid isPermaLink="false">http://jonathanlewis.wordpress.com/?p=6605#comment-41063</guid>
		<description><![CDATA[1. Your plan is not visiting the table - if it were you would (typically) see a &quot;table access by rowid&quot; operation being fed by the index operation.
2. bitmap index join is not restricted to single-column bitmap indexes - your problem here is that the optimizer is losing sight of the &quot;partition-wise&quot; type of strategy that could be used.
.]]></description>
		<content:encoded><![CDATA[<p>1. Your plan is not visiting the table &#8211; if it were you would (typically) see a &#8220;table access by rowid&#8221; operation being fed by the index operation.<br />
2. bitmap index join is not restricted to single-column bitmap indexes &#8211; your problem here is that the optimizer is losing sight of the &#8220;partition-wise&#8221; type of strategy that could be used.<br />
.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: goiyala3</title>
		<link>http://jonathanlewis.wordpress.com/2011/07/01/partitioned-bitmaps/#comment-40953</link>
		<dc:creator><![CDATA[goiyala3]]></dc:creator>
		<pubDate>Fri, 08 Jul 2011 02:21:11 +0000</pubDate>
		<guid isPermaLink="false">http://jonathanlewis.wordpress.com/?p=6605#comment-40953</guid>
		<description><![CDATA[thanks for reply jonathan. I still confused about my actual plan. 
 1.  Is my query plan visited the actual table block ?
 2.  if Bitmap index join can take place only when it is single column bitmap index, then all the bitmap join operations will suffer with this problem
     of building huge hash table?]]></description>
		<content:encoded><![CDATA[<p>thanks for reply jonathan. I still confused about my actual plan.<br />
 1.  Is my query plan visited the actual table block ?<br />
 2.  if Bitmap index join can take place only when it is single column bitmap index, then all the bitmap join operations will suffer with this problem<br />
     of building huge hash table?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Jonathan Lewis</title>
		<link>http://jonathanlewis.wordpress.com/2011/07/01/partitioned-bitmaps/#comment-40947</link>
		<dc:creator><![CDATA[Jonathan Lewis]]></dc:creator>
		<pubDate>Thu, 07 Jul 2011 10:47:32 +0000</pubDate>
		<guid isPermaLink="false">http://jonathanlewis.wordpress.com/?p=6605#comment-40947</guid>
		<description><![CDATA[Goiyala3,

In general, bitmap indexes are very useful in data warehouse systems, and the most dramatic benefit you get from bitmap indexes appears when Oracle combines multiple bitmap indexes (which are often just single column indexes) to reduce the visits to table blocks to the smallest necessary set.

In this case, though, it&#039;s unfortunate that there is an &quot;obvious&quot; optimisation that should apply when you do a bitmap index join that simply hasn&#039;t been allowed for in the optimizer code. 

Bear in mind, though, that the query you&#039;re looking at would have to visit every row (hence block) in the table if the bitmap indexes weren&#039;t there - so you&#039;re not using bitmap indexes in the &quot;standard&quot; way of avoid unnecessary visits to table blocks, so you&#039;ve just been a bit unlucky at the edges of the code when you combine bitmap indexes, partitioned tables, and count(*) optimisation.]]></description>
		<content:encoded><![CDATA[<p>Goiyala3,</p>
<p>In general, bitmap indexes are very useful in data warehouse systems, and the most dramatic benefit you get from bitmap indexes appears when Oracle combines multiple bitmap indexes (which are often just single column indexes) to reduce the visits to table blocks to the smallest necessary set.</p>
<p>In this case, though, it&#8217;s unfortunate that there is an &#8220;obvious&#8221; optimisation that should apply when you do a bitmap index join that simply hasn&#8217;t been allowed for in the optimizer code. </p>
<p>Bear in mind, though, that the query you&#8217;re looking at would have to visit every row (hence block) in the table if the bitmap indexes weren&#8217;t there &#8211; so you&#8217;re not using bitmap indexes in the &#8220;standard&#8221; way of avoid unnecessary visits to table blocks, so you&#8217;ve just been a bit unlucky at the edges of the code when you combine bitmap indexes, partitioned tables, and count(*) optimisation.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: goiyala3</title>
		<link>http://jonathanlewis.wordpress.com/2011/07/01/partitioned-bitmaps/#comment-40939</link>
		<dc:creator><![CDATA[goiyala3]]></dc:creator>
		<pubDate>Wed, 06 Jul 2011 08:27:32 +0000</pubDate>
		<guid isPermaLink="false">http://jonathanlewis.wordpress.com/?p=6605#comment-40939</guid>
		<description><![CDATA[Jonathan

I only asked this question in previous thread. Table size 35gb and bitmap index size on SRC_SYS,PROD_KEY are below 40mb. If i created btree indexes then it would be around 1gb size. while reading documents  about tuning DWH systems, it said having individual bitmap indexes on columns, those indexes  can participate in  Bitmap AND/OR operation and will do much better than the btree indexes for adhoc queries which has multiple column on the predicates.

I thought of instead of scanning 35 gb table / scanning btree indexes of 1GB on this column, create bitmap indexes and joining of 35 mb bitmap  index with 40 mb bitmap index would be better than the above.

But running this query, with Full table scan plan, finishes faster than the above plan. FTS plan, did not take this much of sort area.
Bitmap join plan, converting into rowids took around 3Gig sort and took a long time.

Is have bitmap indexes on DWH fact tables will not be benefitted on bitmap AND/OR operation?]]></description>
		<content:encoded><![CDATA[<p>Jonathan</p>
<p>I only asked this question in previous thread. Table size 35gb and bitmap index size on SRC_SYS,PROD_KEY are below 40mb. If i created btree indexes then it would be around 1gb size. while reading documents  about tuning DWH systems, it said having individual bitmap indexes on columns, those indexes  can participate in  Bitmap AND/OR operation and will do much better than the btree indexes for adhoc queries which has multiple column on the predicates.</p>
<p>I thought of instead of scanning 35 gb table / scanning btree indexes of 1GB on this column, create bitmap indexes and joining of 35 mb bitmap  index with 40 mb bitmap index would be better than the above.</p>
<p>But running this query, with Full table scan plan, finishes faster than the above plan. FTS plan, did not take this much of sort area.<br />
Bitmap join plan, converting into rowids took around 3Gig sort and took a long time.</p>
<p>Is have bitmap indexes on DWH fact tables will not be benefitted on bitmap AND/OR operation?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Randolf Geist</title>
		<link>http://jonathanlewis.wordpress.com/2011/07/01/partitioned-bitmaps/#comment-40915</link>
		<dc:creator><![CDATA[Randolf Geist]]></dc:creator>
		<pubDate>Sat, 02 Jul 2011 14:38:40 +0000</pubDate>
		<guid isPermaLink="false">http://jonathanlewis.wordpress.com/?p=6605#comment-40915</guid>
		<description><![CDATA[On second thoughts I&#039;m not entirely convinced that the problem has anything to do with the HASH UNIQUE operation. It seems to me more like the general issue that the index join is simply not capable of taking advantage of partition-wise joins, probably because the partition key is not part of the join condition (although it could be implied).

Of course your plan proposed would be ideal since it could run the main HASH UNIQUE operation as part of the partition-wise operation, but since according to the original trace the main problem is the HASH JOIN, not the HASH UNIQUE, I think turning the HASH JOIN into a partition-wise join would eliminate most of the excess work.

I&#039;ve did a quick test with a manual re-write of an index join as you&#039;ve proposed in your series about index joins and managed to arrive at a plan that does a partition-wise index join based on the two bitmap indexes and a single HASH UNIQUE operation on top of that. It&#039;s not the plan you&#039;ve proposed (since it does not a splitted HASH UNIQUE with one as part of the partition-wise operation) but I think solves the main problem described here.

Simple index join without any other operation does not make use of partition-wise operations:

[sourcecode]
---------------------------------------------------------------------------
&#124; Id  &#124; Operation                      &#124; Name             &#124; Pstart&#124; Pstop &#124;
---------------------------------------------------------------------------
&#124;   0 &#124; SELECT STATEMENT               &#124;                  &#124;       &#124;       &#124;
&#124;   1 &#124;  VIEW                          &#124; index$_join$_001 &#124;       &#124;       &#124;
&#124;*  2 &#124;   HASH JOIN                    &#124;                  &#124;       &#124;       &#124;
&#124;   3 &#124;    PARTITION LIST ALL          &#124;                  &#124;     1 &#124;     4 &#124;
&#124;   4 &#124;     BITMAP CONVERSION TO ROWIDS&#124;                  &#124;       &#124;       &#124;
&#124;   5 &#124;      BITMAP INDEX FULL SCAN    &#124; T_BIDX1          &#124;     1 &#124;     4 &#124;
&#124;   6 &#124;    PARTITION LIST ALL          &#124;                  &#124;     1 &#124;     4 &#124;
&#124;   7 &#124;     BITMAP CONVERSION TO ROWIDS&#124;                  &#124;       &#124;       &#124;
&#124;   8 &#124;      BITMAP INDEX FULL SCAN    &#124; T_BIDX2          &#124;     1 &#124;     4 &#124;
---------------------------------------------------------------------------
[/sourcecode]

The plan I&#039;ve arrived at:

[sourcecode]
---------------------------------------------------------------------
&#124; Id  &#124; Operation                         &#124; Name    &#124; Pstart&#124; Pstop &#124;
---------------------------------------------------------------------
&#124;   0 &#124; SELECT STATEMENT                  &#124;         &#124;       &#124;       &#124;
&#124;   1 &#124;  SORT AGGREGATE                   &#124;         &#124;       &#124;       &#124;
&#124;   2 &#124;   VIEW                            &#124;         &#124;       &#124;       &#124;
&#124;   3 &#124;    HASH UNIQUE                    &#124;         &#124;       &#124;       &#124;
&#124;   4 &#124;     PARTITION LIST ALL            &#124;         &#124;     1 &#124;     4 &#124;
&#124;*  5 &#124;      HASH JOIN                    &#124;         &#124;       &#124;       &#124;
&#124;   6 &#124;       BITMAP CONVERSION TO ROWIDS &#124;         &#124;       &#124;       &#124;
&#124;   7 &#124;        BITMAP INDEX FAST FULL SCAN&#124; T_BIDX4 &#124;     1 &#124;     4 &#124;
&#124;   8 &#124;       BITMAP CONVERSION TO ROWIDS &#124;         &#124;       &#124;       &#124;
&#124;   9 &#124;        BITMAP INDEX FAST FULL SCAN&#124; T_BIDX3 &#124;     1 &#124;     4 &#124;
---------------------------------------------------------------------
[/sourcecode]

It required however the bitmap indexes to include the partition key (and use it as part of the join condition in addition to the ROWID) which is certainly a nuisance.

Randolf]]></description>
		<content:encoded><![CDATA[<p>On second thoughts I&#8217;m not entirely convinced that the problem has anything to do with the HASH UNIQUE operation. It seems to me more like the general issue that the index join is simply not capable of taking advantage of partition-wise joins, probably because the partition key is not part of the join condition (although it could be implied).</p>
<p>Of course your plan proposed would be ideal since it could run the main HASH UNIQUE operation as part of the partition-wise operation, but since according to the original trace the main problem is the HASH JOIN, not the HASH UNIQUE, I think turning the HASH JOIN into a partition-wise join would eliminate most of the excess work.</p>
<p>I&#8217;ve did a quick test with a manual re-write of an index join as you&#8217;ve proposed in your series about index joins and managed to arrive at a plan that does a partition-wise index join based on the two bitmap indexes and a single HASH UNIQUE operation on top of that. It&#8217;s not the plan you&#8217;ve proposed (since it does not a splitted HASH UNIQUE with one as part of the partition-wise operation) but I think solves the main problem described here.</p>
<p>Simple index join without any other operation does not make use of partition-wise operations:</p>
<pre class="brush: plain; title: ; notranslate">
---------------------------------------------------------------------------
| Id  | Operation                      | Name             | Pstart| Pstop |
---------------------------------------------------------------------------
|   0 | SELECT STATEMENT               |                  |       |       |
|   1 |  VIEW                          | index$_join$_001 |       |       |
|*  2 |   HASH JOIN                    |                  |       |       |
|   3 |    PARTITION LIST ALL          |                  |     1 |     4 |
|   4 |     BITMAP CONVERSION TO ROWIDS|                  |       |       |
|   5 |      BITMAP INDEX FULL SCAN    | T_BIDX1          |     1 |     4 |
|   6 |    PARTITION LIST ALL          |                  |     1 |     4 |
|   7 |     BITMAP CONVERSION TO ROWIDS|                  |       |       |
|   8 |      BITMAP INDEX FULL SCAN    | T_BIDX2          |     1 |     4 |
---------------------------------------------------------------------------
</pre>
<p>The plan I&#8217;ve arrived at:</p>
<pre class="brush: plain; title: ; notranslate">
---------------------------------------------------------------------
| Id  | Operation                         | Name    | Pstart| Pstop |
---------------------------------------------------------------------
|   0 | SELECT STATEMENT                  |         |       |       |
|   1 |  SORT AGGREGATE                   |         |       |       |
|   2 |   VIEW                            |         |       |       |
|   3 |    HASH UNIQUE                    |         |       |       |
|   4 |     PARTITION LIST ALL            |         |     1 |     4 |
|*  5 |      HASH JOIN                    |         |       |       |
|   6 |       BITMAP CONVERSION TO ROWIDS |         |       |       |
|   7 |        BITMAP INDEX FAST FULL SCAN| T_BIDX4 |     1 |     4 |
|   8 |       BITMAP CONVERSION TO ROWIDS |         |       |       |
|   9 |        BITMAP INDEX FAST FULL SCAN| T_BIDX3 |     1 |     4 |
---------------------------------------------------------------------
</pre>
<p>It required however the bitmap indexes to include the partition key (and use it as part of the join condition in addition to the ROWID) which is certainly a nuisance.</p>
<p>Randolf</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Log Buffer #227, A Carnival of the Vanities for DBAs &#124; The Pythian Blog</title>
		<link>http://jonathanlewis.wordpress.com/2011/07/01/partitioned-bitmaps/#comment-40913</link>
		<dc:creator><![CDATA[Log Buffer #227, A Carnival of the Vanities for DBAs &#124; The Pythian Blog]]></dc:creator>
		<pubDate>Sat, 02 Jul 2011 06:25:02 +0000</pubDate>
		<guid isPermaLink="false">http://jonathanlewis.wordpress.com/?p=6605#comment-40913</guid>
		<description><![CDATA[[...] Jonathan Lewis answers another burning question on on multi-column bitmap indexes and the inability of Oracle to create a bitmap index join when (to the human eye) the strategy was an obvious choice. [...]]]></description>
		<content:encoded><![CDATA[<p>[...] Jonathan Lewis answers another burning question on on multi-column bitmap indexes and the inability of Oracle to create a bitmap index join when (to the human eye) the strategy was an obvious choice. [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Randolf Geist</title>
		<link>http://jonathanlewis.wordpress.com/2011/07/01/partitioned-bitmaps/#comment-40909</link>
		<dc:creator><![CDATA[Randolf Geist]]></dc:creator>
		<pubDate>Fri, 01 Jul 2011 21:03:21 +0000</pubDate>
		<guid isPermaLink="false">http://jonathanlewis.wordpress.com/?p=6605#comment-40909</guid>
		<description><![CDATA[Well, the strategy of generating sub-results that are then propagated to a parent operation for doing the final result across all sub-results can already been seen when performing aggregates with parallel execution, for example a MAX() will be applied by each parallel slave on its own set, and a second, final MAX() will be done by the query coordinator across all sub-aggregates provided by the parallel slaves.

It is a pity that Oracle doesn&#039;t support this (yet) for above partition-wise join case.

Although not the same as what you&#039;ve proposed and not with the same efficiency, it would be interesting to see the plan when requesting parallel execution, something like 

select count(1) from (select /*+ parallel_index(REV_F) index_join(REV_F REV_F_IDX1 REV_F_IDX5) */ distinct SRC_SYS,PROD_KEY from dds.REV_F);

It will probably still suffer from the same problem, but at least it will very likely show two levels of HASH UNIQUE operations (although not at the coordinator level, so I&#039;m not entirely sure if it is the same as described above for the MAX() case), so it comes at least closer to your proposal.

Randolf]]></description>
		<content:encoded><![CDATA[<p>Well, the strategy of generating sub-results that are then propagated to a parent operation for doing the final result across all sub-results can already been seen when performing aggregates with parallel execution, for example a MAX() will be applied by each parallel slave on its own set, and a second, final MAX() will be done by the query coordinator across all sub-aggregates provided by the parallel slaves.</p>
<p>It is a pity that Oracle doesn&#8217;t support this (yet) for above partition-wise join case.</p>
<p>Although not the same as what you&#8217;ve proposed and not with the same efficiency, it would be interesting to see the plan when requesting parallel execution, something like </p>
<p>select count(1) from (select /*+ parallel_index(REV_F) index_join(REV_F REV_F_IDX1 REV_F_IDX5) */ distinct SRC_SYS,PROD_KEY from dds.REV_F);</p>
<p>It will probably still suffer from the same problem, but at least it will very likely show two levels of HASH UNIQUE operations (although not at the coordinator level, so I&#8217;m not entirely sure if it is the same as described above for the MAX() case), so it comes at least closer to your proposal.</p>
<p>Randolf</p>
]]></content:encoded>
	</item>
</channel>
</rss>
