we hit this unpleasant situation several times in 9iR2. We had few huge tables (several 100s of GB), which were filled by direct path reads every day. There were some weekly reports which used only part of those tables, but still selected large volume of data. The only sensible execution plan was using indexes and nested loops joins. There was a lot of work done by reports to do the delayed block cleanout work. But from time to time, it was really expensive because of finding commit time (when few commit intensive jobs started in parallel with our reports)
The nasty thing is oracle does not provide simple solution to this issue. Oracle introduced (I think it was 9iR2) optimization for FTS of data imported by direct path read operation. Oracle is able to find chunks of data filled by one direct path job and will perform delayed block cleanout only for the very first data block ( this is one of the reasons why blocks formatted by direct path have initially 3 ITL slots). Mentioned optimization is not possible for INDEX RANGE SCAN, causing several headaches to me and my colleagues, because we were unable to clean all “dirty” blocks in efficient way (just like with single session direct path reads in 11g).

The issue could be even worse for data stored in read only tablespaces.

Regards
Pavol Babel

Feel free to bring it up. In fact I’ll probably be in the audience anyway.

I’m not sure that doing periodic scans would help because the problem relates to changes made after the report starts running.

In the old days (pre-automatic undo management, undo autotuning etc.) the problem would probably have shown up as an ORA-01555 error. The fact that there’s lots of undo available to protect you from the crash is the source of the extra work you do trying to complete the report.

I think the minscn is supposed to help when it’s possible to determine logically a minimum SCN by which the commit must have happened – for example the commit on a block in a read-only tablespace must have happened before the tablespace went read-only.

In a busy system, though, if you are constantly changing (and committing) data and I start to run a long query, there have to be cases where I can know that you’ve committed a transaction, but need to know exactly when you committed that transaction.

Thanks, wseful follow-up (even though the scale of the issue isn’t in the same order of magnitutude).

I’ve always assumed that this could happen on a RAC system that tried to do OLTP on one node and reporting on the other, but I’ve not actually seen it on a production system. If you have access to metalink there are a couple of bugs/notes addressing exactly this type of symptom. In part it relates to the effects of undo autotuning and the way that you can end up with very large undo segments.

I guess there really is no such thing as a free lunch. The scenario you describe would be pretty common for people trying to leverage their standby as reporting environment. I guess one way to mitigate this type of behavior would be to periodic full scans of the most used reporting tables but even that may not help if the tables are read direct: http://antognini.ch/2009/07/impact-of-direct-reads-on-delayed-block-cleanouts/, This is a really good caveat for people to be aware of when considering Active Standby Database. I’m actually presenting on how to leverage some Oracle features to clone databases at this year’s RMOUG event. Would you mind if I brought this up (giving you your due credit of course)?

John

NAME                                                                                VALUE
---------------------------------------------------------------- ------------------------
session logical reads                                                     347,206,851,483
consistent gets                                                           314,535,839,356
consistent gets - examination                                              46,844,294,185
transaction tables consistent reads - undo records applied                      9,809,799
transaction tables consistent read rollbacks                                       15,213

I think I’d have to agree with your assessment. There are a couple of anomalies that I want to look into in case they are contributing (or exacerbating) factors, but I think the effect appears because

a) the standby can’t cleanout blocks when it visits them – it’s read-only
b) the primary isn’t going to clean every block out very promptly because the nature of the application means that some blocks that get changed will only be revisited (at least in the near future) by the reports.
c) lots of blocks will be changed, committed, but not cleaned out in the primary as some of the biggest reports are running in the standby

If I am understanding this correctly, this is not a bug but rather just a normal consequence of the active standby continuing to recover as it is begin queried. Is my understanding correct?

Thanks,

John