Comments on: No change

By: Jonathan Lewis

Jonathan Lewis — Wed, 25 Nov 2009 13:22:51 +0000

Networks can so easily be forgotten. I’ve been to several sites that had to be told why a report that ran in seconds in London was taking minutes in New York. (After all, if people keep telling you that “SQL*Net message from client” is an idle wait … sometimes it’s the most important thing going on.)

By: Michael Fontana

Michael Fontana — Mon, 23 Nov 2009 21:45:26 +0000

I had a recent situation where network latency was affected for a given server doing inserts of data collected from external devices elsewhere in the network. Everything was the same, locally, and it took a time to track down the fact that network latency was causing the problem. Another problem of this nature was network throughput being affected on this same system by a load – in this case from network backups.

Things can get really interesting when queries are done across a network, and folks are not that forthcoming about looking into these problems first before complaining about database performance.

By: PdV

PdV — Sat, 21 Nov 2009 13:55:28 +0000

Good Topic. This can turn into a reference-list of “things to look into”.

Did anyone mention the “front-end” yet ?
Is it possible that the calling-process (java, sqlplus, dotnet, whatever) may simply take its time peeling off the data record by record whereby network-latency and front-end-sluggishness kick in ?

By: Jonathan Lewis

Jonathan Lewis — Thu, 19 Nov 2009 18:15:57 +0000

Wayne,

There’s one particularly interesting point in your list that may not be commonly known:

“The physical location of the data you need is changing, (data block, index block, undo segment, control file. disk read or memory read.)”

If you’re running on WAFL (write anywhere file layout) which, I believe, is the technology used by ZFS and Netapp and possibly a couple of other platforms then when you update an Oracle block it won’t be written back to the same location on the disk.

In unlucky circumstances this could introduce a surprising difference to the performance of the same query visiting the same basic data set if that data was “slowly changing”.

By: mdavis

mdavis — Mon, 16 Nov 2009 05:58:50 +0000

I’m surprised nobody has suggested this one yet:

* Oracle bug

By: Wayne Lang

Wayne Lang — Sun, 15 Nov 2009 17:20:56 +0000

No change is a relative term. In this case, the question you asked is not changed (The query is not changed) and how the way oracle to find the answer is not changed. (The execution plan is not changed.) Everything else is changing. The demand for resource is changing because your session is complete with others for CPU, memory, IO, network, latches and locks. The data need for the answer is changing with the DML from other sessions. The physical location of the data you need is changing, (data block, index block, undo segment, control file. disk read or memory read.) Even if your session is the only one in database, background processes are still running. Maybe your server is shared. Most likely your SAN is shared. Even if you have a dedicated system and network just to run this query, your server backup is running, Memory management is going (including memory paging) CPU performance is not the same because of Multiple CPUs on system each with millions and millions transistors, The power supply voltage is up or down, the data center air temperature is not a constant, the nearby microwave is leak radiation… Well you got the picture.

By: Hans Forbrich

Hans Forbrich — Sun, 15 Nov 2009 16:29:44 +0000

My vote is for ‘obscure interactions’.

User defined functions (typically in the SELECT list expressions or in the WHERE clause) introduce a whole separate level of interaction. If they happen to involve queries of their own and are run for each row, the issues previously mentioned may be magnified.

VPD, query rewrite, other ‘policy’ or ‘trigger’ interactions may have a hidden impact that again exacerbated the standard issues.

(And, with RAC, one question becomes ‘are you on the node that has the data’?)

By: mwidlake

mwidlake — Sun, 15 Nov 2009 12:17:06 +0000

(late to the party, as always)
1) RAC – swapping from being allocated to the node hosting and processing a large percentage of the data your query uses to another (or the otheer processing code swapping to a new node).
or – your RAC nodes are not equal in power (bad situation to have but it happens)
2) Some other major non-oracle user of resouce is swapping on and off your server/your network/
your storage. (You mention this right at the start Jonathan, but I think it’s worth repeating as the other resource user may be nothing to do with your database or even Oracle and it so easy to miss that).

By: Jonathan Lewis

Jonathan Lewis — Sat, 14 Nov 2009 20:48:43 +0000

Mohamed, It sounds like the best thing you can do to investigate is to adopt the strategy I used for the case which showed that a large part of the variation was due to caching effects, and recent (or concurrent) activity, viz: wrap the query in a script that checks v$mystat and v$session_event for the work done and time lost. In your case you might also want to include a call to "select * from table(dbms_xplan.display_cursor);" immediately after you run the query so that you can capture the actual plan used in the execution, and a query against v$pq_tqstat to check what went on between the parallel query slaves.

By: Jonathan Lewis

Jonathan Lewis — Sat, 14 Nov 2009 15:36:08 +0000

Timur,

It’s unfortunate that the documentation (and Metalink, and even some of the white papers) often have little errors of this type in them. In many cases it comes from talking about the basic cases, and overlooking the more subtle cases.

There are several cases where object != segment, but to many people they are synonymous and it’s easy to forget things like clusters, partitioned tables, IOTs and so on.

Technically the linked list is based on the data_object_id – following which your comment is a natural consequence.