You now have an index with lots of empty space. It’s probably about 2,000 blocks long, every other block is completely empty but still in the structure, every other block has one entry in it. (This is an approximation, I haven’t created the data to check accuracy but the image is about right).

If you run the query:

select * from table where id_col is not null and some_flag = ‘X’

Oracle will (probably) use the index, and has to start at the left hand edge of the index (where the first available entry has id_col = 1000) and use the forward pointers to move from block to block in the index.

This is a fairly common coding method for queue driven tasks. If the task can’t process that small number of rows (those old multiples of 1,000) you see a lot of wasted work happening before the tasks gets to the usable data.

If your code allows you to ignore the “bad” table rows before you go to look at them, you reduce the waste. If you coalesce the index (possibly on a regular basis) you reduce the waste even more.

Important point: An index block does not go on to the free list until it is completely empty. Then, when it is on the free list, it is still linked into the index structure in its original position until it finally gets re-used. In my example I might have 1,000 empty blocks on the free lists, but I’m not going to take any of them out of the structure until I insert more rows into the table and need more space at the top, or right hand edge, of the index. The coalesce command cleans things up, and removes empty blocks from the structure.

Sorry, I don’t understand why the index would even bother walking the blocks where the table rows don’t exist (the As an added penalty . . .).

Workflow tends to add rows to the right hand end of a “queueing” index, and delete (most) rows from the left hand end. Such indexes are perfect candidates for something I call the FIFO problem (and Tom Kyte refers to as ‘sweepers’).

The index may be nearly empty over a great long section of the left hand side, and well packed at the right-hand end. So your 50% space used may really be half the index with 0%, and half the index with 100% packing.

If you have to walk a long way through the empty part of the index before getting any useful table data, you are wasting resources in the index. As an added penalty, if you have to walk through a lot of the index and then go to table rows that you can’t process (which is probably why they’re still there in an ‘old’ part of the index) then you’re wasting even more resources.

This looks like an example of an index where you need to do a detailed analysis (there are a couple of scripts on my website that may help: Index Efficiency, and Index Efficiency pt. 2). It is possible, of course, that if you just think about the way the data is used and the impact this has on the index then you won’t even need to investigate the index.

Once you understand the usage, you can decide if you need to do a rebuilds from time to time, or whether you can simply get by with a regular coalesce.

Swathe is indeed the correct spelling based on the etymological lineage given in the 2nd edition of ODE which ties this word back to the Old High German word schwabe which is what you did in the mid-sixteenth century Germany when you had to clear a field of grass or wheat as there weren’t a lot of database indices to clear in those days…

Cheers,
Jeroen

                                     OPTIMAL  COMPR
    BTREE           USED      %SPACE  COMPR   SPACE
   SPACE(k)       SPACE(k)     USED   COUNT   SAVE
-------------- -------------- ------ ------- -------
     2,979,069      2,662,420     90       2       0


For those without immediate dictionary access, my original image was of a (traditional) farmer sweeping across a field with a scythe.

The first thing to remember, though, is that the clustering_factor is Oracle’s count of the number of table block jumps you take as you walk the index in order [See Chapter 5 of the book for a complete description]. But if you have (for example) multiple freelists, or have used ASSM (automatic segment space management) in the tablespace, then the number of jumps could be very large compared to the number of blocks visited, as you could keep revisiting the same blocks.

The second thing to consider is that the number of rowids you use from the index can be much less than the number of index row entries that you examine – and I suspect that this has happened here.

I would guess that the activity is similar to the following. Imagine an ‘order lines’ table with an index (status, product_id, order_date). Imagine a query ‘select … where status = ‘A’ and order_date = trunc(sysdate)’.

All the rows for today will be packed into just a few blocks in the table, but if you use this index to drive the query, you could have thousands of index entries to scan in the index for (say) ProductX where ths status is ‘A’ to find the few rows for today. Do the same for ProductY and so on, and you find that you scan a HUGE swathe of the index, reading hundred of blocks, but just load and re-read a few dozen table blocks.

This type of accident can happen quite easily if you set the optimizer_index_cost_adj to a low value (anything less than about 20 starts to look a little dangerous) and the optimizer_index_caching to a high number (anything above about 80 starts to look a little dangerous).

I see that your input has been mess up by the way that white space is eliminated. Check the notes about this on the opening page of the blog. If you want to do a global edit on the tkprof file to change all spaces to the “nbsp” tag and re-post just that piece as a new comment, I’ll paste it back into the original post.

Elapsed times include waiting on following events:
  Event waited on                             Times   Max. Wait  Total Waited
  ----------------------------------------   Waited  ----------  ------------
  SQL*Net message to client                    2006        0.00          0.00
  db file sequential read                    232869        0.34        599.23
  latch free                                      8        0.02          0.09
  SQL*Net message from client                  2006        0.03          6.37
********************************************************************************

As it can be seen that there were 232,869 single block reads done during the course of the query. Per the raw trace file, 227861 of those reads were done against the index where as 5008 were against the table which adds up to 232,869. This translate to a ratio of almost 45:1 of index blocks read to the table blocks read. I was expecting larger number of table blocks read than the index blocks due to the high CF?