ZFS — on which DxFS is based — would indeed break up the 1MB read in your example into smaller reads, but that doesn’t mean that each of those smaller chunks would become its own I/O operation. ZFS aggregates adjacent operations to minimize the number of I/Os it issues.

You’re right that for COW filesystems like ZFS, WAFL, and others a 1MB contiguous read might require many random reads depending on how the file was written. At Delphix we address that with a combination of prefetching and caching. Users of Delphix often see a storage reduction of 10:1. They see a similar increase of efficiency in DRAM caching. Several customers use Delphix with large memory configurations — not that expensive today — so, say, 1TB of memory can effectively keep all virtual databases cached.

Yes there are a number of ways to thin provision a clone database. One of the most interesting is clonedb because it runs with Oracle 11.2.2+ with default install, no extra license or specialized storage. With clonedb you are stuck with the initial copy of the database. If you want a clone from a different time you have to create a new copy of production which sort of defeats the purpose. There is some flexibility in that one could apply redo to the base copy to get clones at different points in time but of course the farther away one is from the base copy the longer and harder it is going to be to stand up the clone.

On the other hand there are limited methods that allow one to roll the base copy forward and free one from dragging around the original copy and/or creating full new copies for new clones. All these methods require specialized hardware. There is of course Netapp, there is EMC with SRDF , there is Oracle ZFS storage appliance. All require export storage skills including knowledge of how to mount file systems to target machines, how to snapshot, what files to track and how to track them such as datafiles, controlfiles, redo logs, archive logs, how to layout files across LUNs and stapshot the correct LUNs. In order to automate this so an end users can provision their own clone is a tremendous amount of work. CERN gave a presentation in January at a UKOUG conference and said they had coded over 25,000 lines of code to try and make a Netapp solution capable of providing end users provisioning of clones (ie taking out the storage admin, sysadmins, dbas)

Unlike all of these Delphix is a software stack that installs on commodity x86 hardware and can use any storage you give it. It’s fully automated. It takes a one time full copy of the source database and then does incremental forever collection from the source databases. From their end users can provision clones onto any registered host with the correct Oracle executable choosing down to the SCN the point in time to recover to. All recovery, mount of file systems, setting up init.ora etc is done automatically. The new virtual database is also tracked by Delphix and a clone of the virtual database can be forked off from this virtual database. Forking can be done as many times and as deep as one wants. This is great for passing copies of developer databases to QA groups for example.

As I like to say, why if thin provisioning is so great saving massive amounts of storage and enabling quick provisioning of database copies is it not seeing more adoptions? The reason is the barrier to entry with specialized storage requirements, expert storage admin skills and major amounts of scripting to make it turn key keep most of us out of the technology. Delphix breaks down all those barriers by making it turn key simple, automated, running on any x86 hardware and using any storage. Now we are seeing huge adoption rates.

It’s like the internet was there before browsers. Tools like bulletin boards, telnet and ftp were there but it wasn’t until web browsers came out that the internet usage exploded.

FlexClone is the product, it just needed wrapping in some simple scripts to make it play nicely with Oracle, allocate an IP address for a listener, and so on. The question “How do you supply a terabyte sized database to five different development teams without using up 5TB of disc space ?” was solved a very long time ago. Nowadays there are loads of ways to do it, VMware and LVM being two others that spring immediately to mind.

I’ve got access to some documents on “use cases” to get me started with some of the answer, but I’ll wait until I’ve had hands on before I start writing replies. Apart from the obvious restrictions imposed by EHCC, a similar type of problem that crossed my mind was the need for a development “gold copy” that didn’t contain sensitive data such as credit card numbers; I think there’s a use case on that one in the library as well.

Was this a “productised” approach, or was this like me doing rman standby databases on Oracle 6 with a few cunning shell scripts ?