Comments on: Thinking Big

By: Jonathan Lewis

Jonathan Lewis — Tue, 11 Mar 2008 20:33:14 +0000

I’ve just referenced this note from an item on the Oracle forums, and noticed that it’s not obvious how I got from 16 byte keys to 200 entries per block when running at 70% usage.

You have to know that a standard b-tree index entry is actually going to be: key size + 2 bytes row overhead + 2 bytes row pointer + 6 bytes rowid + a possible one byte for rowid length for a total of 27 bytes. (And if you’ve forgotten to include length bytes in your key length, it may be a little more).

By: Hit Ratios (4) « Oracle Scratchpad

Hit Ratios (4) « Oracle Scratchpad — Wed, 26 Sep 2007 21:23:56 +0000

[...] key index has a height of four but still fits comfortably in 18G. Based on the example shown in this entry, 6G of memory would be adequate for such a index if you had a 16 byte key and the index was [...]

By: Jonathan Lewis

Jonathan Lewis — Sun, 08 Apr 2007 16:00:26 +0000

Dave,

Whatever else this does, it will probably add a significant overhead to data maintenance. The “skinny table” approach is one that should be used as carefully as the “add another index” approach. i.e. weighed carefully in terms of resource costs and benefits.

If you really have 84 “atomic” data items per employee, then you ought to have 84 data columns in a table. You might then use row-level triggers to populate a few skinny tables with the subset of columns used by a few very popular queries.

I certainly wouldn’t consisder creating 84 tables with one data item per table – the join costs of putting together the data for even a simple query would be prohibitive.

By: dave

dave — Fri, 06 Apr 2007 12:07:56 +0000

What advantges do skinny tables provide? I currently have a database consisting of 84 data elements per employee. I am considering creating a skinny table for each and every data element( index field, data value). My database has approximately 150 million records whcih spans a 20 year period of time.

I believe the skinny table design will significantly improve the performance of all my applications (more responsive to queries) at a major sacrifice of utilizing more disk storage just for indices. My estimate is 90% of the available storage ( 800 Gigabytes) will be used for indices alone and 6% will be the actual data values

Appreciate any additional comments of pros and cons for use of skinny tables.

By: SAN Array Cache and Filers Hate Sequential Writes « Kevin Closson’s Oracle Blog: Platform, Storage & Clustering Topics Related to Oracle Databases

Tue, 20 Mar 2007 17:07:51 +0000

[...] Lewis has taken on a recent Oracle-l thread about Thinking Big. It’s a good blog entry and I recommend giving it a read. The original Oracle-l post read like [...]

By: Jonathan Lewis

Jonathan Lewis — Mon, 19 Mar 2007 22:14:13 +0000

Cos, yes – IOTs for “thin” tables are an option – but the trade-offs are the extra I/O caused by the randomness of inserts, the smaller number of rows per block, and the larger size of secondary indexes. Getting the cost/benefit balance right can be tricky.

DBA, thanks for the feedback – the IOT always add an interesting element to the sizing / resourcing problem.

By: Jonathan Lewis

Jonathan Lewis — Mon, 19 Mar 2007 21:39:00 +0000

Amit, probably not. The figures are showing 64GB just to buffer one index completely. How much SSD would you have to buy to buffer all the indexes you want, just to eliminate one I/O per probe from the cost ? How much SSD would you need to buffer the whole (row-size unspecified) table in order to eliminate the significant volume of I/Os ? The whole SSD argument is a little suspect when you start talking about big databases. In fact, you should take a look at Kevin Closson's blog for some interesting comments on CPU stalling, and memory access speeds to see just how misleading the SSD argument can be.

By: Amit Kulkarni

Amit Kulkarni — Mon, 19 Mar 2007 13:01:41 +0000

Would using a SSD like BitMicro or RamSan help?

By: Peter Scott

Peter Scott — Mon, 19 Mar 2007 12:30:20 +0000

What the SAN can give you is a write-cache benefit for the log writer, database writer, and any direct path writes and re-reads from sorting and hashing But only within the limits of the cache size – stray into the massive sorts and hashes of the data warehouse world and things can slow down, even more so for the RAID 5 configured SAN user

By: dba

dba — Mon, 19 Mar 2007 09:18:38 +0000

your ball-park figures are pretty good.
I have a real-world system like that here, and the three billion-row index is taking little more than 75GB (key compressed, partitioned IOT)

response times are fairly consistent with what you predicted.