Dom – I didn’t post all three enqueue dumps to this comment, but the other two looked exactly like the one I that I did post. They also showed enqueues held in KJUSEREX. I was hoping you’d just take my word for it since copying the other enqueue dumps seemed a bit redundant!

Even in the scenario you worked out, you showed the transaction which I posted with an X lock — which is inconsistent with the Wait-For-Graph. Any ideas about the inconsistency?

I read this deadlock scenario like this:

Transaction 7AA000-0001-00000A15 holds a mode 6 / X / KJUSEREX (5) lock on 0xb60011.0x1a642  
Transaction 7AA000-0001-00000A15 wants a mode 4 / S / KJUSERPR (3) lock on 0x7370009.0x190f7  
but is blocked by  
Transaction 412000-0001-00002C54 holds a mode 4 / S / KJUSERPR (3) lock on 0x7370009.0x190f7  
Transaction 412000-0001-00002C54 wants a mode 4 / S / KJUSERPR (3) lock on 0x749000e.0x1b0d4  
but who itself is blocked by  
Transaction 7F2000-0001-000008C2 holds a mode 4 / S / KJUSERPR (3) lock on 0x749000e.0x1b0d4  
Transaction 7F2000-0001-000008C2 wants a mode 4 / S / KJUSERPR (3) lock on 0xb60011.0x1a642  
who is blocked by the mode 6 of Transaction 7AA000-0001-00000A15 initially listed. 

Because ASH is captured every 1 second and deadlock detection is typically 3 seconds, V$ACTIVE_SESSION_HISTORY can often provide useful additional information.

Quoting Jonathan from this OTN thread (which was about similar lock modes but across rac nodes)
https://forums.oracle.com/forums/thread.jspa?messageID=10659671
“Generally tx/4 is something to do with indexes, although there are a couple of exotic causes that relate to internal activity”

Global Wait-For-Graph(WFG) at ddTS[0.86ce] :
BLOCKED 0x47c45f1a08 3 wq 2 cvtops x1 TX 0x749000e.0x1b0d4(ext 0x2,0x0)[412000-0001-00002C54] inst 1 
BLOCKER 0x47c3c30990 3 wq 1 cvtops x28 TX 0x749000e.0x1b0d4(ext 0x2,0x0)[7F2000-0001-000008C2] inst 1 
BLOCKED 0x47c4fb3760 3 wq 2 cvtops x1 TX 0xb60011.0x1a642(ext 0x2,0x0)[7F2000-0001-000008C2] inst 1 
BLOCKER 0x47c45f1838 3 wq 1 cvtops x28 TX 0xb60011.0x1a642(ext 0x2,0x0)[7AA000-0001-00000A15] inst 1 
BLOCKED 0x47c3fbbca8 3 wq 2 cvtops x1 TX 0x7370009.0x190f7(ext 0x2,0x0)[7AA000-0001-00000A15] inst 1 
BLOCKER 0x47c3589590 3 wq 1 cvtops x28 TX 0x7370009.0x190f7(ext 0x2,0x0)[412000-0001-00002C54] inst 1 

Just 3s across the board. But up above, in the enqueue dumps, it is clear that all three of these enqueues are being held in KJUSEREX mode (5) and all three deadlocked transactions (holding locks in EX mode) are requesting the locks in PR mode (3). I’m not sure why the graph is displaying only 3s – maybe a bug in 11.2.0.3.0 or maybe it is only showing the modes of the requestors and not the holders?

----------resource 0x4fc68109c0----------------------
resname       : [0x749000e][0x1b0d4],[TX][ext 0x2,0x0]
grant_bits    : KJUSERNL KJUSEREX 
grant mode    : KJUSERNL  KJUSERCR  KJUSERCW  KJUSERPR  KJUSERPW  KJUSEREX
count         : 2         0         0         0         0         1
Granted_locks : 1 
Cvting_locks  : 2 

**I'm guessing gl=granted_lock, rl=requested_lock, rp=resource_pointer?

GRANTED_Q :
lp 0x47c45f1838 gl KJUSEREX rp 0x500558c850 [0xb60011][0x1a642],[TX][ext 0x2,0x0]
  master 1 gl owner 0x47c0c31fe8 possible pid 94619 xid 7AA000-0001-00000A15 bast 0 rseq 103 mseq 0 history 0x14951495
  open opt KJUSERDEADLOCK  
CONVERT_Q: 
lp 0x47c3c30b60 gl KJUSERNL rl KJUSERPR rp 0x500558c850 [0xb60011][0x1a642],[TX][ext 0x2,0x0]
  master 1 gl owner 0x47c1271750 possible pid 94617 xid 79A000-0001-00000892 bast 0 rseq 103 mseq 0 history 0x49ab549a
  convert opt KJUSERGETVALUE  
lp 0x47c3589760 gl KJUSERNL rl KJUSERPR rp 0x500558c850 [0xb60011][0x1a642],[TX][ext 0x2,0x0]
  master 1 gl owner 0x47c2005628 possible pid 94637 xid 81A000-0001-000007F2 bast 0 rseq 103 mseq 0 history 0x49ab549a
  convert opt KJUSERGETVALUE  

----------enqueue 0x47c45f1838------------------------
grant_level      : KJUSEREX
req_level        : KJUSEREX
bast_level       : KJUSERNL
pid              : 94572
possible pid     : 94619
xid              : 7AA000-0001-00000A15
lock_state       : GRANTED
Open Options     : KJUSERDEADLOCK 
Convert options  : KJUSERNOQUEUE KJUSERNODEADLOCKWAIT 
valblk           : 0xf0140000000000007083b5a4ff7f0000 .p

sorry for the data dump… but I’m wondering if anyone else has seen inconsistency like this?

also, wondering if anyone has ideas about figuring out what this enqueue is protecting? can you make any sense out of valblk perhaps?

Here is the way I remember them (the ‘Sub’ is explained in dbmslock.sql comments):

S and X are Shared and eXclusive locks concerning the whole table (if talking about TM locks)
SS/RS and SX/RX are Shared and eXclusive locks concerning only a Subset of the table (just some Rows)
SSX/SRX is a Shared lock for whole table + eXclusive lock concerning only a Subset of the table (just some Rows)

Then I can remember the compatibility matrix knowing that eXclusive prevents Shared or eXclusive except when this incompatibility is only at Subset level (as they can be a different subset – concerning different rows).

Regards,
Franck.

I cannot recall at the moment, but have you written an article that explores the interaction of a holder’s LMODE and a requester’s REQUEST mode – and the possible causes that lead to the combinations? Such an exploration might help answer the question of “What happened in the program that lead to session 1 blocking session 2?” or “Why did session 1 and session 2 deadlock?”. The exploration would be very helpful when a DBA performs a self-join on V$LOCK.

For example:
* TYPE=TX, LMODE=6, REQUEST=4 = session 2 is attempting to insert a row with the same primary key value or unique index value as a row just inserted by session 1
* TYPE=TX, LMODE=6, REQUEST=4 = ITL Problem or bitmap index fragment problem
* TYPE=TX, LMODE=6, REQUEST=6 = session 2 is attempting to update a row that is locked by session 1′s transaction (likely session 1 already updated the row)
* TYPE=TM, LMODE=3, REQUEST=5 = possible missing index on a foreign key column