Just after I finished reading chapter 7 I faced an ORA-04020 on a client’s production system. The theory I learnt in chapter 7 helped me a lot to understand the context of the problem. However, when I look at the tracefile that was produced, I still have a question (although the problem at the client’s site has been resolved). I include an excerpt of the tracefile below:

[...]
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 – 64bit Production
With the Partitioning option
[...]
System name: AIX
Node name: [...]
Release: 1
Version: 7
[...]
Redo thread mounted by this instance: 1
Oracle process number: 70
Unix process pid: 11927750, image: oracle@[...] (J003)

*** 2013-04-25 19:39:54.145
*** SESSION ID:(471.65) 2013-04-25 19:39:54.145
*** CLIENT ID:() 2013-04-25 19:39:54.145
*** SERVICE NAME:(SYS$USERS) 2013-04-25 19:39:54.145
*** MODULE NAME:(DBMS_SCHEDULER) 2013-04-25 19:39:54.145
*** ACTION NAME:(AVQ$ISLRM0_42100) 2013-04-25 19:39:54.145

A deadlock among DDL and parse locks is detected.
This deadlock is usually due to user errors in
the design of an application or from issuing a set
of concurrent statements which can cause a deadlock.
[...]
ORA-04020: deadlock detected while trying to lock object K.DEF_SEC_ENV
——————————————————–
object waiting waiting blocking blocking
handle session lock mode session lock mode
——– ——– ——– —- ——– ——– —-
700000297c255b0 7000002a889eb68 700000285b96050 X 7000002a889eb68 700000285c0a3f0 S
700000298c42b90 7000002aaa29498 7000002826d79d0 X 7000002a889eb68 700000285c0b700 S
——————————————————–
[...]

From how I read the deadlock graph, the session identified by 7000002a889eb68 was locking itself as it was holding a shared lock on library cache object 700000297c255b0 and wanted to get an exclusive lock on that object. Moreover this session was holding a shared lock on library cache object 700000298c42b90, which was requested to be locked in exclusive mode by a session identified by 7000002aaa29498.

The two library cache objects were PL/SQL packages. The scenario was that the client was installing a software upgrade, and forgot to stop all application processes connecting to the database (as would be required), so the solution was to repeat the installation after properly terminating all processes connected to the database.

The surviving session (7000002a889eb68) was the application process the client forgot to stop, while the session that terminated with an ORA-04020 was one of the processes doing the actual installation.

The point I don’t understand, is why the above scenario is considered a deadlock. Couldn’t session 7000002a889eb68 just convert it’s shared lock to an exclusive lock (if there were other sessions holding a shared lock on the object it obviously couldn’t, but then I’d expect to see those sessions in the deadlock graph and to be listed as the session blocking session 7000002a889eb68). Also, terminating session 7000002aaa29498 as Oracle did doesn’t resolve the locking graph – and in fact ORA-04020 occurred two more times throughout the faulty installation, each time with session 7000002a889eb68 in the same place in the corresponding deadlock graph, locked on the same object (700000297c255b0) and with two other installation processes being blocked by this session and terminating with an ORA-04020.

I just add the remainder of the tracefile below in case it contains any information that is relevant to understand the problem.

Thank you
kind regards
Martin

[...]
———- DUMP OF WAITING AND BLOCKING LOCKS ———-
——————————————————–
————- WAITING LOCK ————-
—————————————-
SO: 0x700000285b96050, type: 78, owner: 0x7000002a3ea7fb0, flag: INIT/-/-/0×00 if: 0×3 c: 0×3
proc=0x7000002aa7250e8, name=LIBRARY OBJECT LOCK, file=kgl.h LINE:8548 ID:, pg=0

LibraryObjectLock: Address=700000285b96050 Handle=700000297c255b0 RequestMode=X CanBeBrokenCount=1 Incarnation=2 ExecutionCount=0

User=7000002a889eb68 Session=7000002ac94e810 ReferenceCount=0 Flags=[0000] SavepointNum=152b0
LibraryHandle: Address=700000297c255b0 Hash=14f3dcb0 LockMode=S PinMode=S LoadLockMode=0 Status=INVL
ObjectName: Name=K.BASE_PAR#
FullHashValue=a198494a548895777a38651614f3dcb0 Namespace=TABLE/PROCEDURE(01) Type=PACKAGE(09) Identifier=274315 OwnerIdn=29
Statistics: InvalidationCount=1 ExecutionCount=4 LoadCount=6 ActiveLocks=1 TotalLockCount=128 TotalPinCount=145
Counters: BrokenCount=1 RevocablePointer=2 KeepDependency=0 BucketInUse=306 HandleInUse=306 HandleReferenceCount=0
Concurrency: DependencyMutex=700000297c25660(0, 600, 0, 0) Mutex=700000297c256e0(0, 1961, 0, 0)
Flags=PIN/TIM/[00002801]
WaitersLists:
Lock=700000297c25640[700000285b960c0,700000285b960c0]
Pin=700000297c25620[700000297c25620,700000297c25620]
LoadLock=700000297c25698[700000297c25698,700000297c25698]
Timestamp: Current=10-13-2012 01:55:44
HandleReference: Address=700000297c25758 Handle=700000299dc22a8 Flags=OWN[200]
LibraryObject: Address=70000028f5c0ca8 HeapMask=0000-0015-0015-0000 Flags=EXS/LOC[0004] Flags2=[0000] PublicFlags=/SWR[0008]
DataBlocks:
Block: #=’0′ name=KGLH0^14f3dcb0 pins=0 Change=NONE
Heap=70000028f5c1c60 Pointer=70000028f5c0d48 Extent=70000028f5c0c28 Flags=I/-/P/A/-/-
FreedLocation=0 Alloc=1.773438 Size=3.976562 LoadTime=59819567764
Block: #=’2′ name=PLDIA^14f3dcb0 pins=1 Change=NONE
Heap=70000028f5c1110 Pointer=70000028f5bfca8 Extent=70000028f5bfc28 Flags=I/-/P/A/-/-
FreedLocation=0 Alloc=54.085938 Size=56.000000 LoadTime=59819567764
Block: #=’4′ name=PLMCD^14f3dcb0 pins=0 Change=NONE
Heap=70000028f5c11e8 Pointer=70000028f583278 Extent=70000028f5831f8 Flags=I/-/-/A/-/-
FreedLocation=0 Alloc=6.960938 Size=8.031250 LoadTime=59819568229 ————- BLOCKING LOCK ————
—————————————-
SO: 0x700000285c0a3f0, type: 78, owner: 0x7000002a3f836e0, flag: INIT/-/-/0×00 if: 0×3 c: 0×3
proc=0x7000002aa7250e8, name=LIBRARY OBJECT LOCK, file=kgl.h LINE:8548 ID:, pg=0

LibraryObjectLock: Address=700000285c0a3f0 Handle=700000297c255b0 Mode=S CanBeBrokenCount=1 Incarnation=1 ExecutionCount=0

User=7000002a889eb68 Session=7000002ac94e810 ReferenceCount=1 Flags=CNB/[0001] SavepointNum=3254
LibraryHandle: Address=700000297c255b0 Hash=14f3dcb0 LockMode=S PinMode=S LoadLockMode=0 Status=INVL
ObjectName: Name=K.BASE_PAR#
FullHashValue=a198494a548895777a38651614f3dcb0 Namespace=TABLE/PROCEDURE(01) Type=PACKAGE(09) Identifier=274315 OwnerIdn=29
Statistics: InvalidationCount=1 ExecutionCount=4 LoadCount=6 ActiveLocks=1 TotalLockCount=128 TotalPinCount=145
Counters: BrokenCount=1 RevocablePointer=2 KeepDependency=0 BucketInUse=306 HandleInUse=306 HandleReferenceCount=0
Concurrency: DependencyMutex=700000297c25660(0, 600, 0, 0) Mutex=700000297c256e0(0, 1961, 0, 0)
Flags=PIN/TIM/[00002801]
WaitersLists:
Lock=700000297c25640[700000285b960c0,700000285b960c0]
Pin=700000297c25620[700000297c25620,700000297c25620]
LoadLock=700000297c25698[700000297c25698,700000297c25698]
Timestamp: Current=10-13-2012 01:55:44
HandleReference: Address=700000297c25758 Handle=700000299dc22a8 Flags=OWN[200]
LibraryObject: Address=70000028f5c0ca8 HeapMask=0000-0015-0015-0000 Flags=EXS/LOC[0004] Flags2=[0000] PublicFlags=/SWR[0008]
DataBlocks:
Block: #=’0′ name=KGLH0^14f3dcb0 pins=0 Change=NONE
Heap=70000028f5c1c60 Pointer=70000028f5c0d48 Extent=70000028f5c0c28 Flags=I/-/P/A/-/-
FreedLocation=0 Alloc=1.773438 Size=3.976562 LoadTime=59819567764
Block: #=’2′ name=PLDIA^14f3dcb0 pins=1 Change=NONE
Heap=70000028f5c1110 Pointer=70000028f5bfca8 Extent=70000028f5bfc28 Flags=I/-/P/A/-/-
FreedLocation=0 Alloc=54.085938 Size=56.000000 LoadTime=59819567764
Block: #=’4′ name=PLMCD^14f3dcb0 pins=0 Change=NONE
Heap=70000028f5c11e8 Pointer=70000028f583278 Extent=70000028f5831f8 Flags=I/-/-/A/-/-
FreedLocation=0 Alloc=6.960938 Size=8.031250 LoadTime=59819568229 ————- WAITING LOCK ————-
—————————————-
SO: 0x7000002826d79d0, type: 78, owner: 0x7000002a3f13810, flag: INIT/-/-/0×00 if: 0×3 c: 0×3
proc=0x7000002aa71bb48, name=LIBRARY OBJECT LOCK, file=kgl.h LINE:8548 ID:, pg=0

LibraryObjectLock: Address=7000002826d79d0 Handle=700000298c42b90 RequestMode=X CanBeBrokenCount=1 Incarnation=1 ExecutionCount=0

User=7000002aaa29498 Session=7000002aaa29498 ReferenceCount=0 Flags=[0100] SavepointNum=126f08
LibraryHandle: Address=700000298c42b90 Hash=be6d0403 LockMode=S PinMode=S LoadLockMode=0 Status=VALD
ObjectName: Name=K.DEF_SEC_ENV
FullHashValue=71ccc043dbfdfb6ee394b13dbe6d0403 Namespace=TABLE/PROCEDURE(01) Type=PACKAGE(09) Identifier=58571 OwnerIdn=29
Statistics: InvalidationCount=0 ExecutionCount=0 LoadCount=1 ActiveLocks=1 TotalLockCount=1 TotalPinCount=1
Counters: BrokenCount=1 RevocablePointer=1 KeepDependency=0 BucketInUse=4 HandleInUse=4 HandleReferenceCount=0
Concurrency: DependencyMutex=700000298c42c40(0, 3, 0, 0) Mutex=700000298c42cc0(0, 175, 0, 0)
Flags=PIN/TIM/[00002801]
WaitersLists:
Lock=700000298c42c20[7000002826d7a40,7000002826d7a40]
Pin=700000298c42c00[700000298c42c00,700000298c42c00]
LoadLock=700000298c42c78[700000298c42c78,700000298c42c78]
Timestamp: Current=11-16-2012 19:10:57
HandleReference: Address=700000298c42d38 Handle=700000299dc22a8 Flags=OWN[200]
LibraryObject: Address=700000282c65820 HeapMask=0000-0005-0005-0000 Flags=EXS/LOC[0004] Flags2=[0000] PublicFlags=NST[0001]
DataBlocks:
Block: #=’0′ name=KGLH0^be6d0403 pins=0 Change=NONE
Heap=7000002984ce330 Pointer=700000282c65910 Extent=700000282c657a0 Flags=I/-/P/A/-/-
FreedLocation=0 Alloc=0.882812 Size=4.000000 LoadTime=59819579077
Block: #=’2′ name=PLDIA^be6d0403 pins=1 Change=NONE
Heap=700000282c65a68 Pointer=700000282c64820 Extent=700000282c647a0 Flags=I/-/P/A/-/-
FreedLocation=0 Alloc=29.679688 Size=32.000000 LoadTime=59819579077 ————- BLOCKING LOCK ————
—————————————-
SO: 0x700000285c0b700, type: 78, owner: 0x7000002a3f836e0, flag: INIT/-/-/0×00 if: 0×3 c: 0×3
proc=0x7000002aa7250e8, name=LIBRARY OBJECT LOCK, file=kgl.h LINE:8548 ID:, pg=0

LibraryObjectLock: Address=700000285c0b700 Handle=700000298c42b90 Mode=S CanBeBrokenCount=1 Incarnation=1 ExecutionCount=0

User=7000002a889eb68 Session=7000002ac94e810 ReferenceCount=1 Flags=CNB/[0001] SavepointNum=375b
LibraryHandle: Address=700000298c42b90 Hash=be6d0403 LockMode=S PinMode=S LoadLockMode=0 Status=VALD
ObjectName: Name=K.DEF_SEC_ENV
FullHashValue=71ccc043dbfdfb6ee394b13dbe6d0403 Namespace=TABLE/PROCEDURE(01) Type=PACKAGE(09) Identifier=58571 OwnerIdn=29
Statistics: InvalidationCount=0 ExecutionCount=0 LoadCount=1 ActiveLocks=1 TotalLockCount=1 TotalPinCount=1
Counters: BrokenCount=1 RevocablePointer=1 KeepDependency=0 BucketInUse=4 HandleInUse=4 HandleReferenceCount=0
Concurrency: DependencyMutex=700000298c42c40(0, 3, 0, 0) Mutex=700000298c42cc0(0, 175, 0, 0)
Flags=PIN/TIM/[00002801]
WaitersLists:
Lock=700000298c42c20[7000002826d7a40,7000002826d7a40]
Pin=700000298c42c00[700000298c42c00,700000298c42c00]
LoadLock=700000298c42c78[700000298c42c78,700000298c42c78]
Timestamp: Current=11-16-2012 19:10:57
HandleReference: Address=700000298c42d38 Handle=700000299dc22a8 Flags=OWN[200]
LibraryObject: Address=700000282c65820 HeapMask=0000-0005-0005-0000 Flags=EXS/LOC[0004] Flags2=[0000] PublicFlags=NST[0001]
DataBlocks:
Block: #=’0′ name=KGLH0^be6d0403 pins=0 Change=NONE
Heap=7000002984ce330 Pointer=700000282c65910 Extent=700000282c657a0 Flags=I/-/P/A/-/-
FreedLocation=0 Alloc=0.882812 Size=4.000000 LoadTime=59819579077
Block: #=’2′ name=PLDIA^be6d0403 pins=1 Change=NONE
Heap=700000282c65a68 Pointer=700000282c64820 Extent=700000282c647a0 Flags=I/-/P/A/-/-
FreedLocation=0 Alloc=29.679688 Size=32.000000 LoadTime=59819579077 ——————————————————–
This lock request was aborted.

Is my understanding correct that the strategy to hold cursors open which you describe starting on page 176 is the same that is done (automatically) by the PL/SQL interpreter when the PL/SQL cursor cache comes in action? Is that what you are saying by “the Oracle pre-compiler allows you to generate code that holds cursors without having to do any special coding”?
Or phrased differently is my understanding correct that the code on page 177 would behave identically (performance-wise) if you moved the calls to dbms_sql.open_cursor, dbms_sql.parse and dbms_sql.close_cursor inside the loop, as the PL/SQL cursor cache would still keep the cursor open?
If so, what would be the benefit to implement this in PL/SQL code (as the optimization is done automatically by PL/SQL anyway) – or did you just write the code to demonstrate how the PL/SQL cursor cache works?

thank you
kind regards
Martin

On page 183 you write that the sub-pool concept used to cause ORA-04031 in earlier releases. Is this no longer the case – and if so since which release and how did Oracle fix the problem (the most obvious approach would be to have a session try each of the sub-pools in turn in case it’s allocation request to the first sub-pool fails). On page 191 (second paragraph) however you mention that the scenario of no sufficiently contiguous memory in one sub-pool can cause an ORA-04031, without referring to old versions.

Also on page 183 you mention that one of the reasons to introduce sub-pools was to combat ORA-04031. In what way do the sub-pools address the ORA-04031? Starting on that page 183 you list three damage limitation strategies, none of which seems to be directly related to the concept of sub-pools (the third is related to the sub-sub pools, i.e. durations, which could however be implemented independently of the sub-pools as well)

thank you
kind regards
Martin

I have a question regarding the free-lists which you describe starting on page 186. Is there one such list per sub-sub pool (which I don’t think is written explicitly in the book, but I might also have skipped it when reading through the chapter). If so, are the different free lists just shown in order in the heapdump and is there any labelling telling what sub-sub pool the list belongs to? If this is not the case (i.e. one single free-list for the entire shared pool), how can Oracle determine to which sub-sub pool a chunk belongs that is found in the free-list and therefore decide if the chunk may be used to serve the corresponding memory request?

thank you for clarification
kind regards
Martin

On page 197 you mention “vanish” as one of the possibilities what a parse call can do. Does this refer to other possibilities than the PL/SQL cursor cache (where cursors are held open imlicitly). This might be inferred from the phrasing “particlularly from pl/sql code” (which seems to suggest that there are other possibilities in addition to the PL/SQL cursor cache). If so, could you please tell me what possibilities you were thinking of? Otherwise this first point would be the same as the fourth point where you mention the PL/SQL cache explicitly?

thank you
kind regards
Martin

I have managed to repeat this behaviour only in EE 9.2.0.8 while on EE 10.2.0.4 and 11.2.0.3 was working as expected ie to show one executions after flushing shared pool.

Regards,
Sasa

I can’t reproduce your results. You had “alter session flush shared_pool”, by the way, but I assume this was a typo, so I’ve corrected it.

After the flush shared pool, I found that v$sql and v$sqlarea had been flushed, but v$sqlstats still showed the statement with 100 executions. After running the statement one more time, all three of them showed one execution. (This was 11.2.0.3 on Windows XP Pro 32-bit.)

Looks like in 11.2.0.3 , session_cached_cursors behave differently than in the earlier patchsets/versions.
Here is a small test case.

###

Create table test (a number);
Insert into test values(1);
commit;

-- Session Cached Cursor is set to 100.

-- Run this SELECT 100 times.
SELECT * FROM TEST WHERE A=1 ;

SELECT EXECUTIONS FROM V$SQLAREA WHERE SQL_TEXT='SELECT * FROM TEST WHERE A=1';

Output
=======
100 

(This is expected)

Hold on , don’t exit this session. Now flush the shared pool

ALTER SYSTEM FLUSH SHARED_POOL;

SELECT EXECUTIONS FROM V$SQLAREA WHERE SQL_TEXT='SELECT * FROM TEST WHERE A=1';

Output
=======

No rows selected 

(This is expected)

Now , run the same select statement just once.

SELECT * FROM TEST WHERE A=1 ;

SELECT EXECUTIONS FROM V$SQLAREA WHERE SQL_TEXT='SELECT * FROM TEST WHERE A=1';

Output
=======
101 

V$SQLAREA now shows 101 executions. But the fact of the matter is that the previous 100 executions
have been flushed out of the shared pool. So , this count should be 1 .
I notice this behavior only in 11.2.0.3 and not in the earlier patchsets/versions.

Further , if I set session_cached_cursor=0 ,
I don’t get the same issue. I correctly see the execution count as 1

Best Regards,
Vishal

Thank you for providing the output summary for your core_dc_activity_01.sql script. I examined the script while reading the book and noticed that there was a single result (likely one column selected, one predicate) for 10.2.0.3 and 11g included in the script file. Prior to posting this potential errata item, I had not created your snap_latch_child and snap_rowcache packages in a test database, so I did not execute the test script to confirm that the number of dc_histogram_defs gets was dependent on the number of columns in the SELECT clause. Logic reasoning suggested that there should be some form of increase in the dictionary cache gets to obtain the column definition and average row lengths of the selected columns, but it did not seem reasonable that dc_histogram_defs would show an increase in the number of gets.

After seeing your comment, I created the snap_latch_child and snap_rowcache packages in a test database and experimented a bit with the core_dc_activity_01.sql script. I am able to reproduce the 2,000, 3,000, 3,000, and 4,000 results that you provided, so the statement in the book is correct.

I made several slight adjustments to your script. Below is output (modified to show just the snap_rowcache.end_snap output) when selecting all four columns with either two columns listed in the WHERE clause or a single column in the WHERE clause. In both cases there were 4,000 gets of dc_histogram_defs (only when hard parses were required):

SQL> prompt	select four columns with two predicates
SQL> declare
  2  	m_n	number;
  3  	m_v	varchar2(10);
  4          m_m2    varchar2(100);
  5          m_id    number;
  6  begin
  7  	for i in 1..1000 loop
  8  		execute immediate
  9  			'select n1, v1, padding, id from t1 where padding = rpad(''x'',100) and id = ' || i
 10  			into m_n, m_v, m_m2, m_id;
 11  	end loop;
 12  end;
 13  /
 
PL/SQL procedure successfully completed.
 
SQL> execute snap_rowcache.end_snap
---------------------------------
Dictionary Cache - 28-Dec 22:39:52
Interval:-      0 seconds
---------------------------------
Parameter                 Usage Fixed    Gets  Misses   Scans  Misses    Comp    Mods Flushes
---------                 ----- -----    ----  ------   -----  --------------    ---- -------
dc_segments                   0     0   4,000       0       0       0       0       0       0
dc_users                      0     0   3,000       0       0       0       0       0       0
dc_objects                    0     0   4,013       0       0       0       0       0       0
dc_global_oids                0     0      12       0       0       0       0       0       0
dc_histogram_defs             0     0   4,000       0       0       0       0       0       0
dc_object_grants              0     0      20       0       0       0       0       0       0
 
PL/SQL procedure successfully completed.

SQL> prompt	select four columns with one predicate
SQL> declare
  2  	m_n	number;
  3  	m_v	varchar2(10);
  4          m_m2    varchar2(100);
  5          m_id    number;
  6  begin
  7  	for i in 1..1000 loop
  8  		execute immediate
  9  			'select n1, v1, padding, id from t1 where id = ' || i
 10  			into m_n, m_v, m_m2, m_id;
 11  	end loop;
 12  end;
 13  /
 
PL/SQL procedure successfully completed.
 
SQL> execute snap_rowcache.end_snap
---------------------------------
Dictionary Cache - 28-Dec 22:39:53
Interval:-      0 seconds
---------------------------------
Parameter                 Usage Fixed    Gets  Misses   Scans  Misses    Comp    Mods Flushes
---------                 ----- -----    ----  ------   -----  --------------    ---- -------
dc_segments                   0     0   4,000       0       0       0       0       0       0
dc_users                      0     0   3,000       0       0       0       0       0       0
dc_objects                    0     0   4,012       0       0       0       0       0       0
dc_global_oids                0     0      12       0       0       0       0       0       0
dc_histogram_defs             0     0   4,000       0       0       0       0       0       0
global database name          0     0       2       0       0       0       0       0       0
 
PL/SQL procedure successfully completed.

The script (core_dc_activity_01.sql) goes through four variations of query, ending with:

select n1, v1
from t1
where padding = rpad(‘x’,100)
and id = {constant}

Note that the columns in the select list are not the ones used in the where clause.

The four variants tested by the script were:

one column selected, one predicate
one column selected, two predicates
two columns selected, one predicate,
two columns selected, two predicates

The number of gets on the dictionary cache entry dc_histogram_defs (in 10.2.0.3) were

2,000
3,000
3,000
4,000

suggesting that every column referenced in the query – whether in the select list or the where clause – requires dictionary access.

Perhaps if I had included multiple sets of the results my claim would have been clearer, but I was concerned that this would result in a fairly large section of repetitious text for a small increase in information content.