Oracle Scratchpad

March 16, 2015

Tablespace HWM

Filed under: Infrastructure,Oracle — Jonathan Lewis @ 10:22 am BST Mar 16,2015

The following question appeared the Oracle-L list-server recently:

In order to resize a datafile to release space at the end, we need to find whatever the last block_id that is at the start of that free contiguous space.
Problem is that we have a very large database such that querying dba_extents to find the last block is probably not an option. The standard query(ies) that make use of dba_extents runs for hours at stretch and also  sometimes fails with a ‘snapshot too old’ (just gives up).
Is there an alternative to using dba_extents?

I was surprised to hear that a suitable query against dba_extents could last for hours, although for locally managed tablespaces Oracle does have to read the segment header block for every single segment in the tablespace to get the segment extent map and that might make things a little slow. (A follow-up post explained that part of the problem was that the tablespaces were locally managed, so maybe it wasn’t just a case of an unlucky execution plan.)

If you look hard enough there’s probably an alternative strategy for dealing with any problem – and it might even be a good one. In the case of tablespace highwater marks, how about looking at dba_free_space instead of dba_extents ? If there’s space that can be released from a file it starts at the block after the last used block, e.g.:

        tablespace_name, file_id, block_id, blocks, block_id + blocks - 1 last_block  
        tablespace_name = 'TEST_8K_ASSM_AUTO' 
order by 
        file_id, block_id

------------------------------ ---------- ---------- ---------- ----------
TEST_8K_ASSM_AUTO                       6        128        256        383
TEST_8K_ASSM_AUTO                       6       8576      12200      20775

2 rows selected.

alter database datafile '{file name}' resize {block size * 8,575};

Database altered.

If you do try this then one of two things happen – either you manage to resize the file to the current minimum it can be, or you fail with Oracle error ORA-03297: file contains used data beyond requested RESIZE value and the file can’t be resized until you move some objects which are above the highest chunk of free space, so you’re back to dba_extents to find out which segment is causing the problem.

If you want to try using optimistic approach but don’t want to run some SQL that might cause an Oracle error you could always compare the details from dba_free_space with the details from dba_data_files to see if any space has been used AFTER the last free chunk – but there’s a little trap to making that check. You’ll notice that the last block of the free space is 20,775; but look what dba_data_files says about the last block in the data file(s):

SQL> select file_id, blocks, blocks - 1 last_block, user_blocks, file_name from dba_data_files order by file_id;

---------- ---------- ---------- ----------- ------------------------------------------------------------------
         1     129280     129279      129152 /u01/app/oracle/oradata/TEST/datafile/o1_mf_system_938s4mr3_.dbf
         2     267520     267519      267392 /u01/app/oracle/oradata/TEST/datafile/o1_mf_sysaux_938s551h_.dbf
         3     131200     131199      131072 /u01/app/oracle/oradata/TEST/datafile/o1_mf_undotbs1_938s5n46_.dbf
         4      25600      25599       25472 /u01/app/oracle/oradata/TEST/datafile/o1_mf_users_938s6bhn_.dbf
         5     131200     131199      131072 /u01/app/oracle/oradata/TEST/datafile/o1_mf_test_8k_bcdy0y3h_.dbf
         6      20782      20781       20648 /u01/app/oracle/oradata/TEST/datafile/o1_mf_test_8k__bfqsmt60_.dbf

6 rows selected.

There are 20,782 blocks in the data file (though the numbering starts at zero, so the last block is 20,781) so there seem to be blocks in the data file that are beyond the last blocks of free space. You’ll have to trust me when I say that there’s no data beyond the free space, I’ve dropped all the (other) segments in this tablespace and purged the recyclebin: the last free space chunks stops short of the end of the file by 6 blocks. The presence of the user_blocks column in dba_data_files helps to explain what’s going on. You can consider a datafile to be made of three components: the space management part, the part that can hold legally sized extents, and a part at the end of file which is too small to hold the smallest extent that can legally be created in the tablespace.

The details depends on the version of Oracle, the definition of the tablespace, initial size of the file, and how the file has grown. In recent versions of Oracle, and assuming you haven’t done something silly with a very small starting size and massive growth, the space management part is likely to be a chunk of 1MB at the start of the file (64KB for older versions). For a locally managed tablespace the chunk at the end of the file could be anything up to one block less than the defined size for “uniform” extent allocation, or one block short of 64KB for system allocated extents.

In my example I have blocks = 20,782, and user_blocks = 20648: that’s because the tablespace was created in a recent version of Oracle with system allocated extents and 8KB blocks: 20,782 = 20648 + 128 (space management header) + 6 (dead space at end of file); the value of user_blocks allows for 2,581 extents of 64KB, and the last six blocks of the file are (currently) unusable. (I have a more extreme example of wasted space in an example I published a couple of years ago.)


When the question first came up my first thought was simply to dump the tablespace space management block but realised just a bit too late that dba_free_space was a much easier option. If anyone does care to pursue the bitmap dump you’ll have to work out all the details because there are variations on the theme that are probably only going to appear with very large datafiles or if you’ve converted from dictionary managed to locally managed. The method starts with the dbms_space_admin package which allows you to dump a tablespace bitmap into the session’s trace file:

execute dbms_space_admin.tablespace_dump_bitmaps('TEST_8K')

Header Control:
RelFno: 5, Unit: 128, Size: 294400, Flag: 1
AutoExtend: NO, Increment: 0, MaxSize: 0
Initial Area: 126, Tail: 294399, First: 8, Free: 2283
Deallocation scn: 148317558.2950
Header Opcode:
Save: No Pending Op
File Space Bitmap Block:
BitMap Control:
RelFno: 5, BeginBlock: 128, Flag: 0, First: 8, Free: 63472
FF00FF0000000000 0000000000000000 0000000000000000 0000000000000000
0000000000000000 0000000000000000 0000000000000000 0000000000000000

This tablespace was locally managed with a block size of 8KB and uniform extents of 1MB (which equates to 128 blocks), so we’re looking at a bitmap where one bit represents 128 blocks. Since the Oracle version is 11gR2, and the file doesn’t fall into the special “tiny” category the header section is 1MB / 128 blocks; the bitmap starts in block 2 (the third block of the file) which is why the size of the “Initial Area” is 126 blocks rather than 128.  The first free extent is number 8 (counting from zero) and there are 2,283 free extents in the file.

If I use my space-reporting script to report the details of the free and used extents in the tablespace I can start to align the bitmap with the extents and work out how to interpret the ones and zeros. This is what I’ve got at present:

------- ----------- ----------- ---------- ---------- ---------------------------- ------------------
      5         128         255        128 TEST_USER  T1                           TABLE
                256         383        128 TEST_USER  T1                           TABLE
                384         511        128 TEST_USER  T1                           TABLE
                512         639        128 TEST_USER  T1                           TABLE
                640         767        128 TEST_USER  T1                           TABLE
                768         895        128 TEST_USER  T1                           TABLE
                896       1,023        128 TEST_USER  T1                           TABLE
              1,024       1,151        128 TEST_USER  T1                           TABLE
              1,152       2,175       1024 free       free
              2,176       2,303        128 TEST_USER  T3                           TABLE
              2,304       2,431        128 TEST_USER  T3                           TABLE
              2,432       2,559        128 TEST_USER  T3                           TABLE
              2,560       2,687        128 TEST_USER  T3                           TABLE
              2,688       2,815        128 TEST_USER  T3                           TABLE
              2,816       2,943        128 TEST_USER  T3                           TABLE
              2,944       3,071        128 TEST_USER  T3                           TABLE
              3,072       3,199        128 TEST_USER  T3                           TABLE
              3,200     294,399     291200 free       free

As you can see, the 8 x 1-bit (starting FF) aligns with the first 8 allocated extents of 128 block each, then the 8 x 0-bit with the 1,024 free blocks, followed by a further 8 x 1-bit and 8 x 128 block extents.  Furher investigations are left as an exercise to the interested reader.



  1. “Quck and dirty” datafile resize script:

      for i in 1..99 loop
        for r in 
          select 'alter database datafile '''||file_name
                 ||''' resize '||trunc(bytes*i/100) as cmd 
            from dba_data_files 
             execute immediate r.cmd;
           exception when others 
             then null;
         end loop;              
     end loop;  

    Unscientific, however usefull when space/time/boss pressure is high but dba_extents is slow.

    Comment by Leonid Mikhailov — March 16, 2015 @ 11:12 am BST Mar 16,2015 | Reply

    • define “dirty”
      I find only one improvement in your script

        e exception;
        pragma exception_init(e, -03297);
        for i in 1..99 loop
          for r in
            select 'alter database datafile '''||file_name
                   ||''' resize '||trunc(bytes*i/100) as cmd 
              from dba_data_files 
               execute immediate r.cmd;
             exception when e 
               then null;
           end loop;              
       end loop;  

      Besides that, it is quite perfect ?

      Comment by Matthias Rogel — March 17, 2015 @ 7:04 pm BST Mar 17,2015 | Reply

      • If you’re going to do that then I think you should use a binary chop method to maximise the reduction with the minimum number of failures. You probably take out some fairly aggressive locks on each attempt to resize (I haven’t actually checked) so it’s probably a good idea to minimise the number of times you do it.

        Comment by Jonathan Lewis — March 17, 2015 @ 7:14 pm BST Mar 17,2015 | Reply

        • sounds quite reasonable.
          And when counting simply up in one-percent-steps one should probably break after the first success …

          Comment by Matthias Rogel — March 17, 2015 @ 8:23 pm BST Mar 17,2015

  2. Hi,
    I’ve a query on SYS.X$KTFBUE which I’ve made long time ago when LMT came (with very inefficient DBA_EXTENTS) and I was working on database with thousands of files. I still use it and find it fast to get the statements to resize datafiles to minimal size. For security, it generates it only when datafile is autoextensible and can extend back to initial size. In case it helps anybody, here it is:

    Comment by Franck Pachot (@FranckPachot) — March 17, 2015 @ 8:22 pm BST Mar 17,2015 | Reply

  3. Take care, dba_free_space is an alternative, except on Query Against DBA_FREE_SPACE is Slow After Applying (Doc ID 1904677.1)

    Comment by Geert De Paep — March 23, 2015 @ 3:31 pm BST Mar 23,2015 | Reply

    • Geert,

      Thanks for that warning and the reference.

      Interestingly I don’t have a problem – but that’s likely to be a luck/costing thing based on the number of rows the optimizer thinks I have in ts$, file$, and recyclebinbin$. (The problem is the optimizer finding a bad plan for the bit of the query that examines the recyclebin for dropped tables that still use space but need to be reported as free space.)

      Comment by Jonathan Lewis — March 24, 2015 @ 7:42 am BST Mar 24,2015 | Reply

      • Yes indeed. In some databases it is slow, in others it is fast. And indeed, when you have a lot of objects in the recycle bin, this seems to be more problematic. Anyway, if you need to access dba_free_space only once to shrink datafiles, this should be ok. But if you have monitoring scripts in the database for monitoring tablespace free space, this can be a problem. Then it is recommended to use the alternative view from the Oracle Support note.

        Comment by Geert De Paep — March 24, 2015 @ 7:57 am BST Mar 24,2015 | Reply

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at