Oracle Scratchpad

June 12, 2013

Not In Nasty

Filed under: Bugs,CBO,Oracle — Jonathan Lewis @ 5:31 pm UTC Jun 12,2013

Actually it’s probably not the NOT IN that’s nasty, it’s the thing you get if you don’t use NOT IN that’s more likely to be nasty. Just another odd little quirk of the optimizer, which I’ll demonstrate with a simple example (running under 11.2.0.3 in this case):

create table t1
as
select	*
from	all_objects
where	rownum <= 10000
;

update t1 set
	object_type = null
where
	object_type = 'TABLE'
;

commit;

begin
	dbms_stats.gather_table_stats(
		ownname		 => user,
		tabname		 =>'T1',
		method_opt	 => 'for all columns size 1'
	);
end;
;

select
	sample_size, num_nulls, num_distinct, histogram
from
	user_tab_columns
where
	table_name = 'T1'
and	column_name = 'OBJECT_TYPE'
;

This code is going to give you 10,000 rows but the number of distinct values for object_type and the number of nulls will depend on the version and options installed; however you should get about 15 distinct values for object_type and about 1,000 nulls in that column.

If you want to run a query to count the rows where the object_type is not one of: ‘INDEX’,'SYNONYM’, or ‘VIEW’, there are two obvious ways of writing it, so let’s put them into a single query with a UNION ALL, and see what Oracle predicts as the cardinality (cut-n-pasted from an SQL*Plus session):

SQL> set autotrace traceonly explain
SQL> select
  2     count(*)
  3  from       t1
  4  where      object_type != 'INDEX'
  5  and        object_type != 'SYNONYM'
  6  and        object_type != 'VIEW'
  7  union all
  8  select
  9     count(*)
 10  from       t1
 11  where      object_type not in ('INDEX', 'SYNONYM', 'VIEW')
 12  ;

Execution Plan
----------------------------------------------------------
Plan hash value: 575959041

----------------------------------------------------------------------------
| Id  | Operation           | Name | Rows  | Bytes | Cost (%CPU)| Time     |
----------------------------------------------------------------------------
|   0 | SELECT STATEMENT    |      |     2 |    12 |    38  (53)| 00:00:01 |
|   1 |  UNION-ALL          |      |       |       |            |          |
|   2 |   SORT AGGREGATE    |      |     1 |     6 |            |          |
|*  3 |    TABLE ACCESS FULL| T1   |  5903 | 35418 |    19   (6)| 00:00:01 |
|   4 |   SORT AGGREGATE    |      |     1 |     6 |            |          |
|*  5 |    TABLE ACCESS FULL| T1   |  7308 | 43848 |    19   (6)| 00:00:01 |
----------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

3 - filter("OBJECT_TYPE"<>'INDEX' AND "OBJECT_TYPE"<>'SYNONYM' AND "OBJECT_TYPE"<>'VIEW')
5 - filter("OBJECT_TYPE"<>'INDEX' AND "OBJECT_TYPE"<>'SYNONYM' AND "OBJECT_TYPE"<>'VIEW')

Look at operations 3 and 5: the way you write the query makes a significant difference to the cardinality predicted by the optimizer – despite the fact that the NOT IN predicate is transformed internally so that the predicate sections of the two parts of the UNION ALL are exactly the same.

If you’re wondering, the NOT IN option is the one that does the rational arithmetic – it has calculated the number of rows where object_type is not null, then allowed for filtering out 3/15ths of those rows because the query rejects 3 rows out of the 15 distinct values reported by the statistics. The list of inequalities has applied some arithmetic that tries to allow for the nulls three times – the more values you reject the worse the estimate gets.

It’s quite likely that you won’t notice the effect in many cases – but if you run queries against a column with a large percentage of nulls, then the differences can be very large and knock-on effects of this error could be dramatic.

June 10, 2013

Metadata bug

Filed under: Bugs,Oracle — Jonathan Lewis @ 6:42 pm UTC Jun 10,2013

Here’s a funny little bug – which probably won’t cause any damage – that may remind you that (most of) the people who work for Oracle are just ordinary people like you and me who make ordinary little mistakes in their programming. It’s a bug I discovered by accident because I just wanted to check something about how a particular undo tablespace had been defined, and I called dbms_metadata instead of querying dba_tablespaces. Here’s the cut-n-paste from an SQL*Plus session on 11.2.0.2:


SQL> select dbms_metadata.get_ddl('TABLESPACE','UNDOTBS1') text from dual;

TEXT
----------------------------------------------------------------------------------------------------

  CREATE UNDO TABLESPACE "UNDOTBS1" DATAFILE
  SIZE 209715200
  AUTOEXTEND ON NEXT 5242880 MAXSIZE 32767M,
  '/u01/app/oracle/oradata/TestDB11/redo03.log' SIZE 209715200
  AUTOEXTEND ON NEXT 5242880 MAXSIZE 32767M,
  '/u01/app/oracle/oradata/TestDB11/undotbs01.dbf' SIZE 209715200
  AUTOEXTEND ON NEXT 5242880 MAXSIZE 32767M
  BLOCKSIZE 8192
  EXTENT MANAGEMENT LOCAL AUTOALLOCATE
   ALTER DATABASE DATAFILE
  '/u01/app/oracle/fast_recovery_area/TESTDB11/flashback/o1_mf_8v4qrxy1_.flb' RESIZE 959447040
   ALTER DATABASE DATAFILE
  '/u01/app/oracle/oradata/TestDB11/redo03.log' RESIZE 959447040
   ALTER DATABASE DATAFILE
  '/u01/app/oracle/oradata/TestDB11/undotbs01.dbf' RESIZE 959447040

SQL> select file_name from dba_data_files where tablespace_name = 'UNDOTBS1';

FILE_NAME
----------------------------------------------------------------------------------------------------
/u01/app/oracle/oradata/TestDB11/undotbs01.dbf

SQL> select member from v$logfile;

MEMBER
----------------------------------------------------------------------------------------------------
/u01/app/oracle/oradata/TestDB11/redo01.log
/u01/app/oracle/oradata/TestDB11/redo02.log
/u01/app/oracle/oradata/TestDB11/redo03.log

As you can see, a call to dbms_metadata.get_ddl() is telling me that my undo tablespace is made up of two files (and then wants to alter three (!) of them). One of which looks suspiciously like it might be one of my redo log files, and the third file is one of the flashback logs in my fast recovery area! So I queried dba_tablespaces to find that (as expected) I had only one file in my undo tablespace, and then I queried v$logfile to check whether I really did have a log file called redo03.log and I checked the fast recovery area to see when that last file had appeared.

I don’t suppose this error in dbms_metadata will have any serious side effects – although I did wonder if there might be some ramifications for a “full transportable export” (an 11.2.0.3 option) from datapump.

A quick check on MOS revealed this to be a known bug fixed in 12.1: “Bug 10177856 : DBMS_METADATA(‘TABLESPACE’, …) INCLUDES WRONGLY TEMPFILES AND ONLINE REDO-LOGF” The specific bug (and there were three reported by my search) described the SYSTEM tablespace report also reporting the temporary tablespace(s) – so I did a quick check on my SYSTEM and SYSAUX to see what would happen, and the results made it easy to see how the problem had happened – when it was collecting filenames the code in dbms_metadata had failed to check file types properly.

My query for SYSTEM reported system.dbf along with temp01.dbf and redo01.log and an OMF-named flashback log.
My query for SYSAUX report sysaux.dbf along with temp_special.dbf and redo02.log and an OMF-named flashback log.

Here’s a query (runnable only by SYS) showing why those particular extra files appeared on my system:

SQL> select fnfno, fntyp,fnnam from x$kccfn where fnfno <= 3 order by fnfno, fntyp;

     FNFNO      FNTYP FNNAM
---------- ---------- ---------------------------------------------------------------------------
         0        200 /u01/app/oracle/oradata/TestDB11/bct.dbf

         1          3 /u01/app/oracle/oradata/TestDB11/redo01.log
                    4 /u01/app/oracle/oradata/TestDB11/system01.dbf
                    7 /u01/app/oracle/oradata/TestDB11/temp01.dbf
                   24 /u01/app/oracle/fast_recovery_area/TESTDB11/flashback/o1_mf_8v439xsc_.flb

         2          3 /u01/app/oracle/oradata/TestDB11/redo02.log
                    4 /u01/app/oracle/oradata/TestDB11/sysaux01.dbf
                    7 /u01/app/oracle/oradata/TestDB11/temp_special.dbf
                   24 /u01/app/oracle/fast_recovery_area/TESTDB11/flashback/o1_mf_8v43b0hp_.flb

         3          3 /u01/app/oracle/oradata/TestDB11/redo03.log
                    4 /u01/app/oracle/oradata/TestDB11/undotbs01.dbf
                   24 /u01/app/oracle/fast_recovery_area/TESTDB11/flashback/o1_mf_8v4qrxy1_.flb
 

The code is reporting the names of the corresponding files of types 3, 7 and 24 whenever it reports a file of type 4 (which is the data files). There are other possible file types, of course, and if I had other features enabled perhaps I would see some of them. Running dbms_metadata.get_ddl with tracing enabled showed me that it queries view ku$_tablespace_view, which includes a join to view ku$_file_view, which includes a three-part union all of x$kccfn including subqueries for files of type 4 and 7 specifically, and a subquery with no restriction on file type but a join on file number to the flashback file list x$ktfthc.

Although the bug is reported as fixed in 12.1, it seems to be fixed in 11.2.0.3 (so that deals with my worry about datapump) – it was just unlucky (or lucky, depending on your viewpoint) that I happened to run my first query on version 11.2.0.2.

June 4, 2013

ROWID

Filed under: ANSI Standard,Bugs,lateral view,Oracle — Jonathan Lewis @ 9:09 am UTC Jun 4,2013

Here’s a suggestion to help you avoid wasting time. If you ever include the rowid in a query – not that that should happen very commonly – make sure you give it an alias, especially if you’re using ANSI SQL. If you don’t, you may find yourself struggling to work out why you’re getting an irrational error message. Here’s an example that appeared recently on the OTN forum, with the output cut-n-pasted from a system running 11.1.0.7:

select 
	'1' 
from 
	dual a
left join 
	(
	select	c.dummy, b.rowid
	from	dual b
	join	dual c
	on b.dummy = c.dummy
	) d
on 	a.dummy = d.dummy
;

select
*
ERROR at line 1:
ORA-01445: cannot select ROWID from, or sample, a join view without a key-preserved table

The error doesn’t really seem to fit the query, does it?
If you want to bypass the problem all you have to do is give b.rowid (line 7) an alias like rid.

As far as I can tell, the problem arises from a defect in the code that Oracle uses to transform the query before optimising it. Given the nature of the transformation it’s possible that you’re only going to see the problem manifest like this if the SQL uses ANSI forms; in this particular case changing from the ANSI left outer join syntax to Oracle’s standard (+) syntax also bypasses the problem. To narrow down the cause, I simply enabled event 10053 (the optimizer trace) and got lucky. The trace file showed an interesting “unparsed SQL” statement which, stripped of double-quote marks and re-formatted, was as follows (in 11.1.0.7 – this may vary with version of Oracle):

SELECT 
	A.DUMMY QCSJ_C000000000600000,
	from$_subquery$_007.DUMMY_0 QCSJ_C000000000600001,
	from$_subquery$_007.ROWID_1 ROWID 
FROM 
	SYS.DUAL A, 
	LATERAL( 
		(
		SELECT 
			D.DUMMY DUMMY_0,
			D.ROWID ROWID_1 
		FROM	(
			SELECT 
				from$_subquery$_005.QCSJ_C000000000500001_1 DUMMY,
				from$_subquery$_005.QCSJ_C000000000500003_2 ROWID 
			FROM	(
				SELECT 
					B.DUMMY QCSJ_C000000000500000,
					C.DUMMY QCSJ_C000000000500001_1,
					B.ROWID QCSJ_C000000000500003_2 
				FROM 
					SYS.DUAL B,
					SYS.DUAL C 
				WHERE 
					B.DUMMY=C.DUMMY
				) from$_subquery$_005
			) D 
		WHERE
			A.DUMMY=D.DUMMY
		)
	)(+) from$_subquery$_007

Note the alias of ROWID appearing in lines 4 and 15. If you tried to run this SQL from the command line (after fiddling the event to enable lateral() views), or even just the simple select running from lines 13 to 26, you would get error ORA-00923: FROM keyword not found where expected. My guess is that the context in which the optimisation takes place means that this error is re-raised as the error ORA-01445 that we ultimately see.

April 16, 2013

systimestamp

Filed under: Bugs,Oracle,Troubleshooting — Jonathan Lewis @ 12:46 am UTC Apr 16,2013

For your entertainment – there’s nothing up my sleeves, this was a simple cut-n-paste after real-time typing with no tricks:

20:39:51 SQL> create table t1 (t1 timestamp);

Table created.

20:39:55 SQL> insert into t1 values(systimestamp);

1 row created.

20:39:59 SQL> select t1 - systimestamp  from t1;

T1-SYSTIMESTAMP
---------------------------------------------------------------------------
+000000000 04:59:50.680620

1 row selected.

20:40:08 SQL>

My laptop runs Oracle so quickly that it took only 4 seconds for 5 hours to elapse !
11.2.0.3 on 64-bit Linux – the client is running with TZ=EST5EDT, while the server is running UK Time (currently BST (GMT+1))

Comments available on MOS: 340512.1 Timestamps & time zones – Frequently Asked Questions
Another MOS note, thanks to Jure Bratina in the comments: 227334.1 – “Dates & Calendars – Frequently Asked Questions” in the question

Update:

As Niall quotes in the comments: “times are difficult”.
Oracle supplies three timestamps: systimestamp, localtimestamp, and current_timestamp. (For reasons of consistency, only one of uses an underscore ;) )
Oracle also supplies three timestamp types: timestamp, timestamp with time zone, and timestamp with local time zone.
Oracle also supplies two timezone calls: dbtimezone, and sessiontimezone

If you need to figure out all the details of how these things hang together, I think you need to set your machine timezone to something that isn’t UTC (or GMT as I still tend to call it), then use two separate machines as clients, with their timezones set to two other timezones (again avoiding UTC).

I’ve done a few experiments but without being so rigorous in my settings – my machine was running on GMT, but I opened a (UNIX) session and set the session time zone to EST5EDT to start the database, while running other (UNIX) session with different TZ settings. The reason I should have restarted the machine in a different timezone is that Oracle “normalises” some timestamps to UTC – which means there are cases when I can’t be certain whether the stored value is in UTC because it has been normalised or because it simply was the actual machine time.

So here’s a little experiment (11.2.0.2, instance started in EST5EDT, unix session running in UTC, connecting across the network to the server).

select
        current_timestamp,
        localtimestamp,
        systimestamp
from
        dual
;

CURRENT_TIMESTAMP
---------------------------------------------------------------------------
LOCALTIMESTAMP
---------------------------------------------------------------------------
SYSTIMESTAMP
---------------------------------------------------------------------------
17-APR-13 11.37.10.870658 AM +01:00
17-APR-13 11.37.10.870658 AM
17-APR-13 06.37.10.870554 AM -04:00

Notes:
systimestamp reflects the instance timestamp – which is 5 hours earlier than the session timestamp.
systimestamp returns a timestamp with time zone, not just a timestamp
localtimestamp and current_timestamp show the client time, but localtimestamp doesn’t show the timezone, current_timestamp does (the +1:00 appears because Daylight Saving Time (British Summer Time) is active so my session is one hour ahead of UTC, while the database is 4 hours behind.)

Another quick test:

create table t1 (
        t0              timestamp,
        tz              timestamp with time zone,
        tl              timestamp with local time zone,
        ts_type         varchar2(20)
)
;

insert into t1 values(
        systimestamp, systimestamp, systimestamp,
        'sys Timestamp'
);

commit;

select * from t1;

T0
---------------------------------------------------------------------------
TZ
---------------------------------------------------------------------------
TL                                                                          TS_TYPE
--------------------------------------------------------------------------- --------------------
17-APR-13 06.44.04.353489 AM
17-APR-13 06.44.04.353489 AM -04:00
17-APR-13 11.44.04.353489 AM                                                sys Timestamp

select
        dump(t0,16),
        dump(tz,16),
        dump(tl,16),
        ts_type
from
        t1
;
DUMP(T0,16)
------------------------------------------------------------------------------------------------------------------------
DUMP(TZ,16)
------------------------------------------------------------------------------------------------------------------------
DUMP(TL,16)
------------------------------------------------------------------------------------------------------------------------
TS_TYPE
--------------------
Typ=180 Len=11: 78,71,4,11,7,2d,5,15,11,d0,68
Typ=181 Len=13: 78,71,4,11,b,2d,5,15,11,d0,68,10,3c
Typ=231 Len=11: 78,71,4,11,b,2d,5,15,11,d0,68
sys Timestamp

Notes:
T0 – the timestamp column, has the instance timestamp in it – but doesn’t have any timezone information stored; the raw dump show the value 6:44:04 (7, 2d, 5 – convert from hex and substract one). Anyone on ANY timezone will see their output showing 6:44:04 if they select this column.

TZ – the timestamp with time zone column, has the instance timestamp, but has stored it as (b, 2d,5 – 11:44:04) with time zone information (10,3c) that allows the session to know what “global” moment the information really represents and the location (or, rather, time zone) where is was entered.

TL – the timestamp with local time zone, has the instance timestamp, but has stored it as (b, 2d, 5 – 11:44:04) with NO timezone information. So the output when you query this column is adjusted to suit the local timestamp. It’s the right “global” moment, and it displays as the relevant local time. But, as a penalty, it’s lost the information about where (in which time zone) it was entered.

I think that examination of the content of the raw dumps of the three different types may help you understand why you need to store timestamps in a column type that includes a time zone – if you don’t then you lose some information, and time-based arithmetic will give you some surprises if your application crosses timezones.

Next Issue:

Indexing time (though the link in the comments below to Tony Hasler’s blog probably gives you all the answers you need), and an Oracle design error that I’ve visited before.

March 20, 2013

Lock Bug

Filed under: Bugs,deadlocks,Locks,Oracle — Jonathan Lewis @ 12:08 pm UTC Mar 20,2013

Here’s an oddity that I ran into a little while ago while trying to prepare a sample trace file showing a particular locking pattern; it was something that I’d done before, but trace files can change with different versions of Oracle so I decided to use a copy of 11.2.0.2 that happened to be handy at the time to check if anything had changed since the previous (11gR1) release. I never managed to finish the test; here are the steps I got through:


-- created a table t1 to work with, no data needed, column definitions don't matter.

Session 1:
lock table t1 in row exclusive mode;        -- mode 3, lock acquired

Session 2:
lock table t1 in row exclusive mode;        -- mode 3, lock acquired

Session 3:
lock table t1 in exclusive mode;            -- mode 6, goes into wait for session 1

Session 1 (again):
lock table t1 in share row exclusive mode;  -- mode 5, should move to converter queue, wait for session 2

The mode 3 to mode 5 conversion is what happens if you have a foreign key referential integrity constraint without a covering index and run code that does something like “delete child rows for parent X, delete parent X” (which also happens under the covers if your foreign key is declared as “on delete cascade”).

Notice the “should” on the lock conversion line – this is exactly what happens in 10.2.0.3 and 11.2.0.3; but on the system I was using session 1 got an immediate deadlock (ORA-00060) error – in the absence of a deadlock scenario !

My problem is this – the instance that gives the deadlock error is 64-bit 11.2.0.2 running RAC on Linux (OEL 5); the instances that don’t give the error are 32-bit, non-RAC, running on Windows XP Pro. SO is the anomaly due to some difference in:

  • Windows vs. Linux
  • 32 bit vs. 64 bit
  • RAC vs. non-RAC
  • Specific version of Oracle

I’m inclined to think it’s a version dependent bug, but it’s possible that it’s a necessary side effect of RAC. So if you’ve got 11.2.0.3 RAC, or 11.2.0.2 non-RAC, or the terminal releases of 10g and 11.1 whether RAC or non-RAC, I’d like to hear from you which versions – if any – produce the same deadlock and which don’t. (And the scientific method being what it is, anyone with 11.2.0.2 on RAC on Linux might like to confirm – or contradict – my result.)

Update 8:30 p.m. GMT

The results in so far tend to support the idea that this may be a RAC-related issue. Justin Mitchell’s result is particularly interesting, and I confirmed on my 11.2.0.2 RAC system that if the two starting sessions are on different nodes then I don’t get the deadlock behaviour.

Users of RAC will probably be aware of the fact that v$lock.block commonly sets itself to 2 (potential blocker) on RAC the moment you acquire the lock; and this is what happens to both the “row exclusive” locks. When we request the exclusive lock, both TM locks change from block=2 to block=1 if they are on the same node; if they are on different nodes then only one of them changes.

Unfortunately Oleksandr didn’t see a deadlock when he tried RAC 11.2.0.3 – and since he didn’t make any comment about using multiple nodes we will have to wait to see if he can repeat the test and tell us whether his results are consistent with Justin’s, or whether they suggest that there are further considerations in effect.

January 17, 2013

dbms_xplan bug

Filed under: Bugs,dbms_xplan,Oracle — Jonathan Lewis @ 6:22 pm UTC Jan 17,2013

Here’s a very long post (which is mainly an example) demonstrating a little bug in the “explain plan” functionality. It’s a variation of a bug which I thought had been fixed in 11g, but it still appears in some cases. Take a look at this execution plan, which comes from explaining “select * from dba_tab_cols” – the bit I want to emphasise is in lines 1 to 10:

------------------------------------------------------------------------------------------------------
| Id  | Operation                       | Name       | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                |            |     8 | 21704 |       |  2387   (1)| 00:00:29 |
|   1 |  TABLE ACCESS BY INDEX ROWID    | OBJ$       |   480 | 14400 |       |     3   (0)| 00:00:01 |
|*  2 |   INDEX RANGE SCAN              | I_OBJ1     |   192 |       |       |     2   (0)| 00:00:01 |
|   3 |   TABLE ACCESS BY INDEX ROWID   | OBJ$       |   480 | 14400 |       |     3   (0)| 00:00:01 |
|*  4 |    INDEX RANGE SCAN             | I_OBJ1     |   192 |       |       |     2   (0)| 00:00:01 |
|   5 |    TABLE ACCESS BY INDEX ROWID  | OBJ$       |   480 | 14400 |       |     3   (0)| 00:00:01 |
|*  6 |     INDEX RANGE SCAN            | I_OBJ1     |   192 |       |       |     2   (0)| 00:00:01 |
|   7 |     TABLE ACCESS BY INDEX ROWID | OBJ$       |   480 | 14400 |       |     3   (0)| 00:00:01 |
|*  8 |      INDEX RANGE SCAN           | I_OBJ1     |   192 |       |       |     2   (0)| 00:00:01 |
|   9 |      TABLE ACCESS BY INDEX ROWID| OBJ$       |   480 | 14400 |       |     3   (0)| 00:00:01 |
|* 10 |       INDEX RANGE SCAN          | I_OBJ1     |   192 |       |       |     2   (0)| 00:00:01 |
|* 11 |  HASH JOIN                      |            |   480 |  1929K|       |     5  (20)| 00:00:01 |
|* 12 |   INDEX RANGE SCAN              | I_OBJ1     |   480 | 12480 |       |     2   (0)| 00:00:01 |
|  13 |   TABLE ACCESS FULL             | USER$      |    59 |   117K|       |     2   (0)| 00:00:01 |
|  14 |  NESTED LOOPS OUTER             |            |     1 |  2084 |       |     3   (0)| 00:00:01 |
|  15 |   TABLE ACCESS BY INDEX ROWID   | COL$       |     1 |    56 |       |     2   (0)| 00:00:01 |
|* 16 |    INDEX UNIQUE SCAN            | I_COL3     |     1 |       |       |     1   (0)| 00:00:01 |
|  17 |   TABLE ACCESS BY INDEX ROWID   | ATTRCOL$   |     1 |  2028 |       |     1   (0)| 00:00:01 |
|* 18 |    INDEX UNIQUE SCAN            | I_ATTRCOL1 |     1 |       |       |     0   (0)| 00:00:01 |
|  19 |    TABLE ACCESS BY INDEX ROWID  | ATTRCOL$   |     1 |  2028 |       |     2   (0)| 00:00:01 |
|* 20 |     INDEX UNIQUE SCAN           | I_ATTRCOL1 |     1 |       |       |     1   (0)| 00:00:01 |
|* 21 |  FILTER                         |            |       |       |       |            |          |
|* 22 |   HASH JOIN RIGHT OUTER         |            | 72962 |   188M|       |  2387   (1)| 00:00:29 |
|  23 |    TABLE ACCESS FULL            | USER$      |    59 |   118K|       |     3   (0)| 00:00:01 |
|* 24 |    HASH JOIN RIGHT OUTER        |            | 72962 |    45M|       |  2384   (1)| 00:00:29 |
|* 25 |     TABLE ACCESS FULL           | OBJ$       |   583 | 30899 |       |   205   (0)| 00:00:03 |
|* 26 |     HASH JOIN RIGHT OUTER       |            | 72962 |    41M|       |  2178   (1)| 00:00:27 |
|  27 |      TABLE ACCESS FULL          | COLTYPE$   |  2886 |   174K|       |   275   (0)| 00:00:04 |
|* 28 |      HASH JOIN                  |            | 72962 |    37M|       |  1902   (1)| 00:00:23 |
|  29 |       TABLE ACCESS FULL         | USER$      |    59 |  1770 |       |     3   (0)| 00:00:01 |
|* 30 |       HASH JOIN                 |            | 72962 |    35M|       |  1899   (1)| 00:00:23 |
|  31 |        INDEX FAST FULL SCAN     | I_USER2    |    59 |  3068 |       |     2   (0)| 00:00:01 |
|* 32 |        HASH JOIN RIGHT OUTER    |            | 72962 |    31M|  2976K|  1896   (1)| 00:00:23 |
|  33 |         TABLE ACCESS FULL       | HIST_HEAD$ | 16280 |  2782K|       |    70   (0)| 00:00:01 |
|  34 |         NESTED LOOPS            |            | 72962 |    19M|       |   660   (1)| 00:00:08 |
|  35 |          TABLE ACCESS FULL      | OBJ$       | 47960 |  3840K|       |   205   (0)| 00:00:03 |
|  36 |          TABLE ACCESS CLUSTER   | COL$       |     2 |   404 |       |     1   (0)| 00:00:01 |
|* 37 |           INDEX UNIQUE SCAN     | I_OBJ#     |     1 |       |       |     0   (0)| 00:00:01 |
|* 38 |   TABLE ACCESS CLUSTER          | TAB$       |     1 |    26 |       |     2   (0)| 00:00:01 |
|* 39 |    INDEX UNIQUE SCAN            | I_OBJ#     |     1 |       |       |     1   (0)| 00:00:01 |
|  40 |   NESTED LOOPS                  |            |     1 |    78 |       |     3   (0)| 00:00:01 |
|* 41 |    INDEX RANGE SCAN             | I_OBJ4     |     1 |    39 |       |     2   (0)| 00:00:01 |
|* 42 |    INDEX RANGE SCAN             | I_USER2    |     1 |    39 |       |     1   (0)| 00:00:01 |
------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   2 - access("O"."OBJ#"=:B1)
   4 - access("O"."OBJ#"=:B1)
   6 - access("O"."OBJ#"=:B1)
   8 - access("O"."OBJ#"=:B1)
  10 - access("O"."OBJ#"=:B1)
  11 - access("O"."OWNER#"="USER#")
  12 - access("O"."OBJ#"=:B1)
  16 - access("CL"."OBJ#"=:B1 AND "CL"."INTCOL#"=:B2-1)
  18 - access("RC"."OBJ#"(+)=:B1 AND "RC"."INTCOL#"(+)=:B2-1)
       filter("CL"."INTCOL#"="RC"."INTCOL#"(+))
  20 - access("TC"."OBJ#"=:B1 AND "TC"."INTCOL#"=:B2)
  21 - filter((("O"."TYPE#"=3 OR "O"."TYPE#"=4) OR "O"."TYPE#"=2 AND  NOT EXISTS (SELECT 0
              FROM "SYS"."TAB$" "T" WHERE "T"."OBJ#"=:B1 AND (BITAND("T"."PROPERTY",512)=512 OR
              BITAND("T"."PROPERTY",8192)=8192))) AND ("O"."TYPE#"<>4 AND "O"."TYPE#"<>5 AND "O"."TYPE#"<>7
              AND "O"."TYPE#"<>8 AND "O"."TYPE#"<>9 AND "O"."TYPE#"<>10 AND "O"."TYPE#"<>11 AND
              "O"."TYPE#"<>12 AND "O"."TYPE#"<>13 AND "O"."TYPE#"<>14 AND "O"."TYPE#"<>22 AND
              "O"."TYPE#"<>87 AND "O"."TYPE#"<>88 OR BITAND("U"."SPARE1",16)=0 OR ("O"."TYPE#"=4 OR
              "O"."TYPE#"=5 OR "O"."TYPE#"=7 OR "O"."TYPE#"=8 OR "O"."TYPE#"=9 OR "O"."TYPE#"=10 OR
              "O"."TYPE#"=11 OR "O"."TYPE#"=12 OR "O"."TYPE#"=13 OR "O"."TYPE#"=14 OR "O"."TYPE#"=22 OR
              "O"."TYPE#"=87) AND (SYS_CONTEXT('userenv','current_edition_name')='ORA$BASE' AND
              "U"."TYPE#"<>2 OR "U"."TYPE#"=2 AND "U"."SPARE2"=TO_NUMBER(SYS_CONTEXT('userenv','current_edit
              ion_id')) OR  EXISTS (SELECT 0 FROM SYS."USER$" "U2",SYS."OBJ$" "O2" WHERE "O2"."TYPE#"=88
              AND "O2"."DATAOBJ#"=:B2 AND "U2"."TYPE#"=2 AND "O2"."OWNER#"="U2"."USER#" AND
              "U2"."SPARE2"=TO_NUMBER(SYS_CONTEXT('userenv','current_edition_id'))))))
  22 - access("OT"."OWNER#"="USER#"(+))
  24 - access("AC"."TOID"="OT"."OID$"(+))
  25 - filter("OT"."TYPE#"(+)=13)
  26 - access("C"."OBJ#"="AC"."OBJ#"(+) AND "C"."INTCOL#"="AC"."INTCOL#"(+))
  28 - access("O"."SPARE3"="U"."USER#")
  30 - access("O"."OWNER#"="U"."USER#")
  32 - access("C"."OBJ#"="H"."OBJ#"(+) AND "C"."INTCOL#"="H"."INTCOL#"(+))
  37 - access("O"."OBJ#"="C"."OBJ#")
  38 - filter(BITAND("T"."PROPERTY",512)=512 OR BITAND("T"."PROPERTY",8192)=8192)
  39 - access("T"."OBJ#"=:B1)
  41 - access("O2"."DATAOBJ#"=:B1 AND "O2"."TYPE#"=88)
  42 - access("O2"."OWNER#"="U2"."USER#" AND "U2"."TYPE#"=2 AND
              "U2"."SPARE2"=TO_NUMBER(SYS_CONTEXT('userenv','current_edition_id')))
       filter("U2"."SPARE2"=TO_NUMBER(SYS_CONTEXT('userenv','current_edition_id')))

If you try reading this execution plan using the rule of thumb (ROT) “the first thing that happens is near the top and over to the right” you’ll run into problems because (a) that’s not a good guideline and (b) there are a number of scalar subqueries in the select list and some filter subqueries in the where clause confusing the issue. Here’s the definition of the view dba_tab_cols:

select u.name, o.name,
       c.name,
       decode(c.type#, 1, decode(c.charsetform, 2, 'NVARCHAR2', 'VARCHAR2'),
                       2, decode(c.scale, null,
                                 decode(c.precision#, null, 'NUMBER', 'FLOAT'),
                                 'NUMBER'),
                       8, 'LONG',
                       9, decode(c.charsetform, 2, 'NCHAR VARYING', 'VARCHAR'),
                       12, 'DATE',
                       23, 'RAW', 24, 'LONG RAW',
                       58, nvl2(ac.synobj#, (select o.name from obj$ o
                                where o.obj#=ac.synobj#), ot.name),
                       69, 'ROWID',
                       96, decode(c.charsetform, 2, 'NCHAR', 'CHAR'),
                       100, 'BINARY_FLOAT',
                       101, 'BINARY_DOUBLE',
                       105, 'MLSLABEL',
                       106, 'MLSLABEL',
                       111, nvl2(ac.synobj#, (select o.name from obj$ o
                                 where o.obj#=ac.synobj#), ot.name),
                       112, decode(c.charsetform, 2, 'NCLOB', 'CLOB'),
                       113, 'BLOB', 114, 'BFILE', 115, 'CFILE',
                       121, nvl2(ac.synobj#, (select o.name from obj$ o
                                 where o.obj#=ac.synobj#), ot.name),
                       122, nvl2(ac.synobj#, (select o.name from obj$ o
                                 where o.obj#=ac.synobj#), ot.name),
                       123, nvl2(ac.synobj#, (select o.name from obj$ o
                                 where o.obj#=ac.synobj#), ot.name),
                       178, 'TIME(' ||c.scale|| ')',
                       179, 'TIME(' ||c.scale|| ')' || ' WITH TIME ZONE',
                       180, 'TIMESTAMP(' ||c.scale|| ')',
                       181, 'TIMESTAMP(' ||c.scale|| ')' || ' WITH TIME ZONE',
                       231, 'TIMESTAMP(' ||c.scale|| ')' || ' WITH LOCAL TIME ZONE',
                       182, 'INTERVAL YEAR(' ||c.precision#||') TO MONTH',
                       183, 'INTERVAL DAY(' ||c.precision#||') TO SECOND(' ||
                             c.scale || ')',
                       208, 'UROWID',
                       'UNDEFINED'),
       decode(c.type#, 111, 'REF'),
       nvl2(ac.synobj#, (select u.name from "_BASE_USER" u, obj$ o
                         where o.owner#=u.user# and o.obj#=ac.synobj#),
            ut.name),
       c.length, c.precision#, c.scale,
       decode(sign(c.null$),-1,'D', 0, 'Y', 'N'),
       decode(c.col#, 0, to_number(null), c.col#), c.deflength,
       c.default$, h.distcnt,
       case when SYS_OP_DV_CHECK(o.name, o.owner#) = 1
            then h.lowval
            else null
       end,
       case when SYS_OP_DV_CHECK(o.name, o.owner#) = 1
            then h.hival
            else null
       end,
       h.density, h.null_cnt,
       case when nvl(h.distcnt,0) = 0 then h.distcnt
            when h.row_cnt = 0 then 1
	    when (h.bucket_cnt > 255
                  or
                  (h.bucket_cnt > h.distcnt
                   and h.row_cnt = h.distcnt
                   and h.density*h.bucket_cnt < 1))
                then h.row_cnt
            else h.bucket_cnt
       end,
       h.timestamp#, h.sample_size,
       decode(c.charsetform, 1, 'CHAR_CS',
                             2, 'NCHAR_CS',
                             3, NLS_CHARSET_NAME(c.charsetid),
                             4, 'ARG:'||c.charsetid),
       decode(c.charsetid, 0, to_number(NULL),
                           nls_charset_decl_len(c.length, c.charsetid)),
       decode(bitand(h.spare2, 2), 2, 'YES', 'NO'),
       decode(bitand(h.spare2, 1), 1, 'YES', 'NO'),
       h.avgcln,
       c.spare3,
       decode(c.type#, 1, decode(bitand(c.property, 8388608), 0, 'B', 'C'),
                      96, decode(bitand(c.property, 8388608), 0, 'B', 'C'),
                      null),
       decode(bitand(ac.flags, 128), 128, 'YES', 'NO'),
       decode(o.status, 1, decode(bitand(ac.flags, 256), 256, 'NO', 'YES'),
                        decode(bitand(ac.flags, 2), 2, 'NO',
                               decode(bitand(ac.flags, 4), 4, 'NO',
                                      decode(bitand(ac.flags, 8), 8, 'NO',
                                             'N/A')))),
       decode(c.property, 0, 'NO', decode(bitand(c.property, 32), 32, 'YES',
                                          'NO')),
       decode(c.property, 0, 'NO', decode(bitand(c.property, 8), 8, 'YES',
                                          'NO')),
       decode(c.segcol#, 0, to_number(null), c.segcol#), c.intcol#,
       case when nvl(h.row_cnt,0) = 0 then 'NONE'
            when (h.bucket_cnt > 255
                  or
                  (h.bucket_cnt > h.distcnt and h.row_cnt = h.distcnt
                   and h.density*h.bucket_cnt < 1))
                then 'FREQUENCY'
            else 'HEIGHT BALANCED'
       end,
       decode(bitand(c.property, 1024), 1024,
              (select decode(bitand(cl.property, 1), 1, rc.name, cl.name)
               from sys.col$ cl, attrcol$ rc where cl.intcol# = c.intcol#-1
               and cl.obj# = c.obj# and c.obj# = rc.obj#(+) and
               cl.intcol# = rc.intcol#(+)),
              decode(bitand(c.property, 1), 0, c.name,
                     (select tc.name from sys.attrcol$ tc
                      where c.obj# = tc.obj# and c.intcol# = tc.intcol#)))
from sys.col$ c, sys."_CURRENT_EDITION_OBJ" o, sys.hist_head$ h, sys.user$ u,
     sys.coltype$ ac, sys.obj$ ot, sys."_BASE_USER" ut
where o.obj# = c.obj#
  and o.owner# = u.user#
  and c.obj# = h.obj#(+) and c.intcol# = h.intcol#(+)
  and c.obj# = ac.obj#(+) and c.intcol# = ac.intcol#(+)
  and ac.toid = ot.oid$(+)
  and ot.type#(+) = 13
  and ot.owner# = ut.user#(+)
  and (o.type# in (3, 4)                                     /* cluster, view */
       or
       (o.type# = 2     /* tables, excluding iot - overflow and nested tables */
        and
        not exists (select null
                      from sys.tab$ t
                     where t.obj# = o.obj#
                       and (bitand(t.property, 512) = 512 or
                            bitand(t.property, 8192) = 8192))))

I’ve left the line numbers in for this one to make it easier to refer to bits of the code. Notice that there are several nvl2() calls in the select list that include the option to query table obj$ – these are at lines 11, 19, 23, 25 and 27; these are the five scalar subqueries represented by lines 1 to 10 of the execution plan, and given that they are all within the same decode() statement you can appreciate that the corresponding pairs of lines in the execution plan should all be at the same indentation rather than forming the steadily increasing cascade that they do. This is an error in the calculated depth column of the plan table – an error which I first mentioned a few years ago with an example from 10g adding a throwaway comment that I had found a similar bug in 11g – but I’d not got around to showing an example until today and this example from 11.2.0.3

Footnote:

As a little instruction in reading execution plans – the main section of the query starts at line 21, with lines 22 – 37 representing the list of tables in the “from” list – including two tables representing the view _CURRENT_EDITION_OBJ.
Lines 38 and 39 represent the “not exists” subquery against table tab$.
Lines 40 – 42 represent an existence subquery embedded in the definition of view _CURRENT_EDITION_OBJ.
Lines 11 – 13 represent the inline scalar subquery in the nvl2() call at line 40 of the view definition.
Lines 14 – 20 represent the scalar subqueries in the decode() call at line 99 of the view definition, and again we have a depth problem, because lines 19 and 20 represent the second scalar subquery in that decode and line 19 ought to be indented to the same level as line 14.

If you write some old-style code to query the plan table, using a connect by between the id and parent_id columns, you’ll find that the indentation is correct – here’s the output from the plan table when reported by the old utlxpls.sql script from Oracle 8i:

Plan Table
--------------------------------------------------------------------------------
| Operation                 |  Name    |  Rows | Bytes|  Cost  | Pstart| Pstop |
--------------------------------------------------------------------------------
| SELECT STATEMENT          |          |     8 |   21K|   2387 |       |       |
|  TABLE ACCESS BY INDEX ROW|OBJ$      |   480 |   14K|      3 |       |       |
|   INDEX RANGE SCAN        |I_OBJ1    |   192 |      |      2 |       |       |
|  TABLE ACCESS BY INDEX ROW|OBJ$      |   480 |   14K|      3 |       |       |
|   INDEX RANGE SCAN        |I_OBJ1    |   192 |      |      2 |       |       |
|  TABLE ACCESS BY INDEX ROW|OBJ$      |   480 |   14K|      3 |       |       |
|   INDEX RANGE SCAN        |I_OBJ1    |   192 |      |      2 |       |       |
|  TABLE ACCESS BY INDEX ROW|OBJ$      |   480 |   14K|      3 |       |       |
|   INDEX RANGE SCAN        |I_OBJ1    |   192 |      |      2 |       |       |
|  TABLE ACCESS BY INDEX ROW|OBJ$      |   480 |   14K|      3 |       |       |
|   INDEX RANGE SCAN        |I_OBJ1    |   192 |      |      2 |       |       |
|  HASH JOIN                |          |   480 |    1M|      5 |       |       |
|   INDEX RANGE SCAN        |I_OBJ1    |   480 |   12K|      2 |       |       |
|   TABLE ACCESS FULL       |USER$     |    59 |  117K|      2 |       |       |
|  NESTED LOOPS OUTER       |          |     1 |    2K|      3 |       |       |
|   TABLE ACCESS BY INDEX RO|COL$      |     1 |   56 |      2 |       |       |
|    INDEX UNIQUE SCAN      |I_COL3    |     1 |      |      1 |       |       |
|   TABLE ACCESS BY INDEX RO|ATTRCOL$  |     1 |    1K|      1 |       |       |
|    INDEX UNIQUE SCAN      |I_ATTRCOL |     1 |      |      0 |       |       |
|  TABLE ACCESS BY INDEX ROW|ATTRCOL$  |     1 |    1K|      2 |       |       |
|   INDEX UNIQUE SCAN       |I_ATTRCOL |     1 |      |      1 |       |       |
|  FILTER                   |          |       |      |        |       |       |
|   HASH JOIN RIGHT OUTER   |          |    72K|  188M|   2387 |       |       |
|    TABLE ACCESS FULL      |USER$     |    59 |  118K|      3 |       |       |
|    HASH JOIN RIGHT OUTER  |          |    72K|   45M|   2384 |       |       |
|     TABLE ACCESS FULL     |OBJ$      |   583 |   30K|    205 |       |       |
|     HASH JOIN RIGHT OUTER |          |    72K|   41M|   2178 |       |       |
|      TABLE ACCESS FULL    |COLTYPE$  |     2K|  174K|    275 |       |       |
|      HASH JOIN            |          |    72K|   37M|   1902 |       |       |
|       TABLE ACCESS FULL   |USER$     |    59 |    1K|      3 |       |       |
|       HASH JOIN           |          |    72K|   35M|   1899 |       |       |
|        INDEX FAST FULL SCA|I_USER2   |    59 |    2K|      2 |       |       |
|        HASH JOIN RIGHT OUT|          |    72K|   31M|   1896 |       |       |
|         TABLE ACCESS FULL |HIST_HEAD |    16K|    2M|     70 |       |       |
|         NESTED LOOPS      |          |    72K|   19M|    660 |       |       |
|          TABLE ACCESS FULL|OBJ$      |    47K|    3M|    205 |       |       |
|          TABLE ACCESS CLUS|COL$      |     2 |  404 |      1 |       |       |
|           INDEX UNIQUE SCA|I_OBJ#    |     1 |      |      0 |       |       |
|   TABLE ACCESS CLUSTER    |TAB$      |     1 |   26 |      2 |       |       |
|    INDEX UNIQUE SCAN      |I_OBJ#    |     1 |      |      1 |       |       |
|   NESTED LOOPS            |          |     1 |   78 |      3 |       |       |
|    INDEX RANGE SCAN       |I_OBJ4    |     1 |   39 |      2 |       |       |
|    INDEX RANGE SCAN       |I_USER2   |     1 |   39 |      1 |       |       |
--------------------------------------------------------------------------------

November 30, 2012

v$sqlstats

Filed under: AWR,Bugs,Oracle,Troubleshooting — Jonathan Lewis @ 5:45 pm UTC Nov 30,2012

Sometimes you find bugs on MOS (Metalink, OCIS, whatever) that make you feel positively ill. I’ve just been on a customer site where (in passing) they mentioned that one of their historic queries avainst v$sqlstats now tool just over one second (CPU) in 11.2.0.3 when it had previously taken about 200ms on 10.2.0.4***. After a little checking it seemed likely that the change was possibly related to the fact that they had increased the size of the SGA significantly, allowing for a much larger shared pool and library cache; however there have been numerous code changes in the shared pool area on the route from 10g to 11g, so I decided to check MOS to see if anyone else had seen a similar problem. I found this:

Bug 13550185 : QUERIES ON V$SQLSTATS MAY CAUSE CRASH OR HANG

Version 11.2.0.3

Fixed in 12.1

Last updated: 29th Nov 2012

There is a documented workaround in the base bug - stop collecting SQL execution statistics, but this rather defeats the point of having AWR snapshot capturing all that “SQL ordered by … “ stuff. On the other hand, if some queries against v$sqlstats can cause sessions to crash or the database to hang then every AWR snapshot might be the one that hangs your database !

The bug in question references Solaris 10 on SPARC, but a quick search for the bug number finds 66 other entries in the bug database and the first one I looked at was on Redhat Linux 5; and the bug also describes itself as “platform generic”. On a positive note, the status is currently “80 – Development to QA/Fix Delivered Internal”, and when I raised the reported the threat to the client, they contacted their Oracle Support contact and discovered that there was a patch available already (number 15033625), and that they had already installed it as a side effect of installing a merge patch to fix a problem with corruption in the shared pool. If you search the patch lists on MOS you’ll find that there are patches for many different versions in the 11.2.0.2 and 11.2.0.3 range, for several different platforms.

*** An increase from 200 ms to 1 second may not seem something worth worrying about, but the client ran the query every 10 seconds because (as it said on the packet in 10g) v$sqlstats is latch free and very cheap to run and (b) they have SLAs which mean they basically need to know if any of their critical, high-frequency, queries are going bad within one minute of the problem starting to occur.

September 11, 2012

FBI Delete

Filed under: Bugs,Function based indexes,Indexing,Oracle — Jonathan Lewis @ 5:56 pm UTC Sep 11,2012

A recent post on Oracle-l complained about an oddity when deleting through a function-based index.

I have a function based index but the CBO is not using it. The DML that I expect to have a plan with index range scan is doing a FTS. Its a simple DML that deletes 1000 rows at a time in a loop and is based on the column on which the FBI is created.

Although execution plans are mentioned, we don’t get to see the statement or the plan – and it’s always possible that there will be some clue in the (full) plan that tells us something about the code that the OP has forgotten to mention. However, function-based indexes have a little history of not doing quite what you expect, so I thought I’d take a quick look at the problem, starting with the simplest possible step – do function-based indexes and “normal” b-tree indexes behave differently on a delete. Here’s the data set I created for my test:
(more…)

September 2, 2012

Bugs

Filed under: Bugs,Oracle — Jonathan Lewis @ 8:06 pm UTC Sep 2,2012

Last week I published some code that demonstrated how a fast refresh of an aggregate materialized view could fail because of the internal use of the sys_op_map_nonnull() function. In some ways it was an easy problem to explain because all you had to do was trace the call to refresh and see what was going on. Today’s bug is completely different – it’s virtually impossible to see the connection between the failure and its cause. Here (as a cut-n-paste) is an example of what happens when the bug appears:

execute dbms_stats.gather_schema_stats(user)
*
ERROR at line 1:
ORA-01760: illegal argument for function
ORA-06512: at "SYS.DBMS_STATS", line 13336
ORA-06512: at "SYS.DBMS_STATS", line 13682
ORA-06512: at "SYS.DBMS_STATS", line 13760
ORA-06512: at "SYS.DBMS_STATS", line 13719
ORA-06512: at line 1

(more…)

August 30, 2012

Surprises

Filed under: Bugs,Oracle,trace files,Troubleshooting — Jonathan Lewis @ 5:55 pm UTC Aug 30,2012

I’ve given examples in the past of how you can be suprised by Oracle when a mechanism that has “always worked” in the past suddenly just doesn’t work because some unexpected internal coding mechanism hits a boundary condition. One example I gave of this was rebuilding indexes online – where a limitation on the key size of index organized tables made it impossible to do an online rebuild of an index on a heap table because of an ORA-01450 (maximum key length) error that was raised against the (index-organized) journal table that  Oracle creates internally to support the rebuild.

Slightly more recently the topic of the sys_op_map_nonnull() function came up in some comments about the ora_hash function – and I made a passing comment about the function appearing in materialized view refreshes and then never followed it up. Fortunately I got an email from Jared Still and Don Seiler a few days ago which contained a nice short demonstration of the problem so, with their permission, I’ve published it below.
(more…)

August 9, 2012

Debugging

Filed under: Bugs,Execution plans,Oracle,subqueries — Jonathan Lewis @ 7:04 pm UTC Aug 9,2012

One of my recent assignments involved a company that had run into some performance problems after upgrading from 10.2.0.3 to 11.2.0.2. We had spent half an hour on the phone discussing the system before I had arrived, and I’d made a couple of suggestions that had solved most of their problems before I got on site – but they still wanted me to come in and give them some specific ideas about why the critical part of the solution had helped.

The most critical piece of advice I had given them (after listening very carefully to their description of the system) was to get rid of ALL the histograms they had on their system, and then watch very carefully for any signs that they might need to re-introduce a handful of histograms over the next few weeks.

One of their critical queries completed in less that 2 seconds when histograms were removed, but took 33 seconds to complete when histograms were in place. With their permission, the following notes record my investigation of this puzzle, the underlying Oracle bug (possibly not yet documented) that caused it, and the optimum workaround that was available to them.
(more…)

June 14, 2012

eBS

Filed under: Bugs,Oracle — Jonathan Lewis @ 6:23 am UTC Jun 14,2012

Just in from Mark Bobak on the Oracle-L list server: a warning for the people who have to deal with the client environment for their eBS systems. (It’s likely that the relevant person in your organization has already received this message direct from Oracle Corp., but there’s no harm in double-checking.)

June 8, 2012

Unique Fail

Filed under: Bugs,CBO,Execution plans,Oracle,Statistics — Jonathan Lewis @ 5:54 pm UTC Jun 8,2012

As in – how come a unique (or primary key) index is predicted to return more than one row using a unique scan. Here’s and example (running on 10.2.0.3 – but the same type of thing happens on newer versions):
(more…)

March 21, 2012

ACS

Filed under: Bugs,Infrastructure,Oracle,Upgrades — Jonathan Lewis @ 6:15 pm UTC Mar 21,2012

You’ve probably heard about adaptive cursor sharing, and possibly you’ve wondered why you haven’t seen it happening very often on your production systems (assuming you’re running a modern version of Oracle). Here’s a little bug/fix that may explain the non-appearance.

MOS Doc ID 9532657.8 Adaptive cursor sharing ignores SELECTs which are not fully fetched.

This bug is confirmed in 11.2.0.1, and fixed in 11.2.0.3. The problem is that the ACS code doesn’t process the statistical information from the cursor unless the cursor reaches “end of fetch” – i.e. if you don’t select all the data in your query, Oracle doesn’t consider the statistics of that execution when deciding whether or not to re-optimise a statement.

It’s quite possible, of course, for an OLTP system, and particularly a web-based system, to execute a number of that allow the user to fetch data one “page” at a time, and stop before fetching all the data – so this bug (or limitation, perhaps) means that some critical statements in your application may never be re-optimized. If this is the case, and you know that you have some such statements that should generate multiple plans, then you could add the hint /*+ bind_aware */ to the SQL.

Upgrade woes: as ever, when a bug is fixed, it’s possible that a few people will suffer from unfortunate side-effects. In the case of this bug, Oracle may start to re-optimize and generated multiple child cursors for SQL statements that (from your perspective) didn’t need the extra work. If you’re very unlucky this may have an undesirable impact on execution performance, and library cache activity.

Thanks to Leonid Roodnitsky for sending me a note about this bug after attending one of my tutorial days last month.

March 12, 2012

First_rows hash

Filed under: Bugs,CBO,Execution plans,Oracle — Jonathan Lewis @ 1:49 am UTC Mar 12,2012

Just like my posting on an index hash, this posting is about a problem as well as being about a hash join. The article has its roots in a question posted on the OTN database forum, where a user has shown us the following execution plan:

---------------------------------------------------------------------------------------------------------
| Id  | Operation                     | Name                    | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT              |                         |    14 |   896 |    63   (7)| 00:00:01 |
|   1 |  SORT GROUP BY                |                         |    14 |   896 |    63   (7)| 00:00:01 |
|   2 |   NESTED LOOPS                |                         |       |       |            |          |
|   3 |    NESTED LOOPS               |                         |    14 |   896 |    62   (5)| 00:00:01 |
|*  4 |     HASH JOIN                 |                         |    14 |   280 |    48   (7)| 00:00:01 |
|   5 |      VIEW                     | V_SALES_ALL             |   200 |  1800 |     4   (0)| 00:00:01 |
|   6 |       UNION-ALL               |                         |       |       |            |          |
|   7 |        INDEX FAST FULL SCAN   | PRODUCTS_DATES_IDX      |   100 |   900 |     2   (0)| 00:00:01 |
|   8 |        INDEX FAST FULL SCAN   | PRODUCTS_DATES_IDX_HARD |   100 |   900 |     2   (0)| 00:00:01 |
|*  9 |      VIEW                     | index$_join$_003        |  2238 | 24618 |    44   (7)| 00:00:01 |
|* 10 |       HASH JOIN               |                         |       |       |            |          |
|* 11 |        INDEX RANGE SCAN       | PRODUCTS_GF_INDEX2      |  2238 | 24618 |     6   (0)| 00:00:01 |
|  12 |        INDEX FAST FULL SCAN   | PRODUCTS_GF_PK          |  2238 | 24618 |    45   (3)| 00:00:01 |
|* 13 |     INDEX UNIQUE SCAN         | DATES_PK                |     1 |       |     0   (0)| 00:00:01 |
|  14 |    TABLE ACCESS BY INDEX ROWID| DATES                   |     1 |    44 |     1   (0)| 00:00:01 |
---------------------------------------------------------------------------------------------------------

(more…)

Next Page »

Theme: Rubric. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 1,437 other followers