Comments on: Compression Units – 4

By: Greg Rahn

Greg Rahn — Thu, 09 Aug 2012 21:20:55 +0000

@Jonathan – HCC was built with scans in mind as it is more costly to get the CU, uncompress it, then grab a single row. Not every query needs to be a FTS, but if the majority are not, then HCC probably isn’t the right compression choice.

@pieboy13 – Since the column level compression techniques are not publicly documented by Oracle, I can not comment specifically. What I will say is that there are a number of techniques are used by column major databases to encode/compress data and there is a good chance that HCC uses those as well.

By: Jonathan Lewis

Jonathan Lewis — Thu, 09 Aug 2012 16:56:26 +0000

pieboy13,
I’ve just generated a data set at “archive low” – here’s a sample from the rowid check code (note – around 11 blocks, and 7,800 rows per CU):

       FNO        BNO      L_FNO      L_BNO     F_DIFF     B_DIFF ROWS_PER_CU
---------- ---------- ---------- ---------- ---------- ---------- -----------
         5    3602372          5    3602383          0         11        7877
         5    3602383          5    3602394          0         11        7880
         5    3602394          5    3602405          0         11        7768
         5    3602405          5    3602416          0         11        7722
         5    3602416          5    3602428          0         12        7723

And here’s s chunk from the block dump of block 3602383:

tab 0, row 1, @0x32
tl: 5693 fb: --H-F--N lb: 0x0  cc: 1
nrid:  0x0176f7d0.0
col  0: [5681]
Compression level: 03 (Archive Low)
 Length of CU row: 5681
kdzhrh: ------PC CBLK: 11 Start Slot: 00
 NUMP: 11
 PNUM: 00 POFF: 5561 PRID: 0x0176f7d0.0
 PNUM: 01 POFF: 13577 PRID: 0x0176f7d1.0
 PNUM: 02 POFF: 21593 PRID: 0x0176f7d2.0
 PNUM: 03 POFF: 29609 PRID: 0x0176f7d3.0
 PNUM: 04 POFF: 37625 PRID: 0x0176f7d4.0
 PNUM: 05 POFF: 45641 PRID: 0x0176f7d5.0
 PNUM: 06 POFF: 53657 PRID: 0x0176f7d6.0
 PNUM: 07 POFF: 61673 PRID: 0x0176f7d7.0
 PNUM: 08 POFF: 69689 PRID: 0x0176f7d8.0
 PNUM: 09 POFF: 77705 PRID: 0x0176f7d9.0
 PNUM: 10 POFF: 85721 PRID: 0x0176f7da.0
CU header:
CU version: 0   CU magic number: 0x4b445a30
CU checksum: 0xb561315
CU total length: 90376
CU flags: NC-U-CRD-OP
ncols: 8
nrows: 7880
algo: 0
CU decomp length: 89337   len/value length: 778484

(This happens to be a case where one CU ends and another begins in the same block – so this CU is row 1 in the block rather than row 0.)
I mentioned in my previous reply to Greg that I thought I’d seen 11 blocks for a “query” CU – but looking at this result I think I probably was remembering an experiment with “archive low”.

By: pieboy13

pieboy13 — Thu, 09 Aug 2012 16:00:22 +0000

Thanks for the details guys.

I went back and had a look at a few tables I had access to at the moment and it seems like the three lower level compression settings (Query Low through Archive Low) all have around 4 blocks per CU (32K). While Archive High has a significantly higher number generally around 32 (256K). I haven’t really seen anything in between although it’s an admittedly haphazard look at a very small data set.

Greg, can you expand any on the column level (I’ll call it pre-processing) compression?

By: Jonathan Lewis

Jonathan Lewis — Thu, 09 Aug 2012 12:44:55 +0000

Greg,

I think we’re probably making the same point while talking in different ways about different aspects of the same feature.

I appreciate that it would take less CPU (though more space) to handle all your data if it were stored using LZO rather than bzip2 even if the CU was a uniform 256KB in both cases; but the aspect I was thinking of was that it takes less CPU to unpack a single row from a 32KB LZO CU than it would to unpack the same row from a 256KB LZU CU.

If your strategic choice is “compress for query” then you are presumably expecting to query the data, and probably not expecting every query to be a tablescan – so the CPU spent selecting single rows at random is likely to be a sufficiently important factor that a 32KB limit on CUs becomes a much better strategy than a 256KB limit.

By: Greg Rahn

Greg Rahn — Wed, 08 Aug 2012 15:44:39 +0000

I agree that the CPU cost (and elapsed time) of compression and decompression are important considerations along with the compression ratio because there are trade offs being made, however, I would place the emphasis on not how large the CU is, but what the compression algorithm and objective are because that is really the driving factor for the CU size. The top level compression algos include LZO, gzip (medium), gzip (high) and bzip2 [1]. The emphasis with the “archive” level compression is really reduced size, but the CU size and algo both contribute to CPU costs. Even if the CU size was constant across the four HCC levels, as the compression level increases, more CPU is required. e.g. LZO requires less CPU than bzip2 on an identical data set.

[1] http://bit.ly/My60wt

By: Jonathan Lewis

Jonathan Lewis — Wed, 08 Aug 2012 09:00:14 +0000

Greg

Thanks for the comments and links.

I should have pointed out that compression for query seemed to use much smaller compression units. (I have a memory that I saw query high produce CUs of 11 blocks on one occasion – but possibly it was actually archive low and my memory is at fault.)

To my mind, the different limits on the sizes of query HCCs and archive HCCs is probably the most significant difference in strategy – the CPU cost of decompressing a column from an HCC that’s allowed to extend to 256KB is likely to be much greater than the CPU cost of decompressing a column from a 32KB HCC. That (I assume) is why my example from the previous post (http://jonathanlewis.wordpress.com/2012/07/27/compression-units-3/ ) showed the CPU usage going up by a factor of 40 even though the size of the table dropped by only 25% (49,000 blocks down to 36,600 blocks) when I switched from query high to archive high compression.

By: Greg Rahn

Greg Rahn — Wed, 08 Aug 2012 07:26:05 +0000

Just clarifying a bit of what was demonstrated:
When a segment uses HCC, dbms_rowid.rowid_block_number(rowid) represents the CU (the logical rowid), not the database block that one is generally familiar with. This also means that any block that has non-head CU pieces in it will have no logical rowids associated with it, which explains the 32 block “gap”.

IIRC for HCC “query” compression, CUs are typically 32K, while “archive” compression CUs are typically no larger than 256K as mentioned.

When an HCC segment is loaded, compression analysis chooses the compression algo per column (of which there are several) which is why there is a enqueue wait for “enq: ZH – compression analysis” in 11gR2.

Another good reference is:
http://canali.web.cern.ch/canali/docs/Compressing_VLDS_Oracle_UKOUG09_LC_CERN.ppt