Sometimes people ask questions like this and happen to get lucky that I recognise the problem, or have some very specific suggestion that may help – but I’m not trying to be a replacement for Oracle Support. In your case, all I can say is that it sounds like a slow memory leak (on x64, the 32-bit “big jump” bit sounded like a runaway pl/sql process or a violent explosion of logons); the only suggestion I have to make to track it down is to run some code in a fairly tight loop that keeps calculating and reporting changes in v$sysstat in case that shows you any sudden strange changes. Apart from that, there’s the VMMAP utility that MOS references – could that give you some clue that (for example) connects the constant growth with one or two specific processes such as m000 or DBWR ?

Sorry I didn’t notice this one when it arrived.
You don’t say which version of Oracle – not that that would actually have made any difference in this case, but it’s generally a useful detail to include.
Given that the MAX is less than the current value I would be inclined to assume it’s wrong. As a general guideline, when the SYSTEM figures don’t seem to make sense I look at the session figures to see if there are any clues there.

I am facing the same situation and the same problem.
On one windows 2003 server (32bits), an oracle database (10.2.0.4 32bits) used to crash with ORA-4030 error. First, I have increased the sga_max and sga_target, giving the database more memory after I noticed that the virtual memory was continuously increasing. So I used the /3GD parameter on the server and allowed the database to use about 2,8 Gb of memory. It was not enough… I have a small program to monitor the virtual memory (using pslist, using windows registry …) and I saw that from time to time in a month, the virtual memory was increasing suddenly (about 300 M° in two or three minutes) but it was never released. Lately, I have upgraded the database because this was the only thing I could do to solve the problem (the 11g upgrade is not my customer’s priority …). So I moved the database to a 64bits Windows 2003 server, EE R2 with about 45 Gb of RAM . I upgraded the database to a Oracle 10.2.0.4 64bits and continued monitoring the virtual memory. It is still increasing … Now the memory is about 3,8 Gb and it’s growing .. slowly but surely…
I also tried to add the “secret parameter” to the init.ora file, but it did nothing. I tried to use dead connection detection but it was useless either …
I would be pleased to paste a small snapshot of the monitoring but I don’t know how to do it here … If you want it, mail me and I’ll send it back…

Any idea would be much appreciated ;-)

Regards,

Joel

Statistic	                                  Begin Value	 End Value
session pga memory max	709,926,624	764,105,568
session cursor cache count	11,892	                14,633
session uga memory	433,981,789,904	562,878,155,584
opened cursors current	208	                 251
logons current	                100	                104
session uga memory max	1,726,565,024	2,172,072,408
session pga memory	518,379,264	570,019,360

session uga memory seems to be very high . I have following questions :

1. what could be the reason of so high session uga memory , what does it really mean ?
2. session uga memory max is smaller then session uga memory – does this indicate any thing which we should fix or analyze.

Thanks !
Ajeet

Thanks for the follow-up information.

Since you were kind enough to offer some suggestions on resolving this, I thought I’d follow up with the solution I eventually tracked down

It turns out there is a known bug in Oracle 10gR2 (and we have now confirmed in in 10gR1 as well) on x64 chips whereby threads are not cleaned up correctly in the Oracle JVM after the call completes leading to “zombie” threads which take up memory but never get cleaned out. This means that on an x64 windows platform the Virtual Memory of the Oracle.exe process increases continuously
if you use the JVM.

What do you need to hit this bug
——————————–
1. 64 bit chip (note: not 64 bit OS – we hit this on 32 bit)
2. 10gR1 or 10gR2 (note: supposed to be fixed in 11gr1)
3. Windows OS (2003 in our case)
4. Run Java in the database

Symptoms
——–
1. Virtual memory of Oracle.exe increases continuously
2. If you run VMMap (sysinternals) you will see a massive allocation for Thread Stack space (we had 600Mb in thread stack with 20 active sessions!)
3. The thread stack breakdown will show lots of threads without an associated thread id

Solution
——–
1. Go onto metalink and read 1062406.1

The solution is an undocumented parameter and I am therefore not about to publish it in case somebody decides to set it without being advised to by Oracle Support.

I’m guessing a simple test case would be easy to do – x64 box, set up some java stored procedures, run them and keep watch on the virtual memory!

Thanks so much for responding. I’ve checked through metalink and there are a fair few memory leak bugs in our version, so an upgrade is definitely on the cards in the future!

In the mean time, I think I may have found a lead to what could be causing this. I happened to take a statspack this afternoon as part of debugging this and found…

session pga memory 361,307,828
session pga memory max 491,724,468
session uga memory 764,521,322,824
session uga memory max 393,031,272

Now that 700Mb looks suspiciously like the difference between the expected size and the actual VM size windows shows. What is odd is that a query against v$sesstat shows 39Mb being used for UGA.

Somewhere to start at least.

Thanks for your insights

Since Oracle operates on Windows as a single process with multiple threads it is perhaps a little more likely that a memory leak from an Oracle “process” might not be released.

Since you’ve got a fairly elderly copy of Oracle there, you may have a bug that has been fixed in a later release so it’s worth checking metalink and patch release “bugs fixed” listing to see if you can find a match.

The only little oddity that stands out is the gap in process IDs (particulary the jump from 60 to 88) – it’s not terribly important, but might be a hint that something odd is happening with process (thread) re-use.

The only thing I can think of is to run up a little task that takes a snapshot of the differing view points every minute or two (don’t bother capturing the SGA, do check the different sources for PGA memory) to see if you spot any oddity (e.g. a timed pattern of growth, or sudden jumps).

This looks a little like something we experience, but the symptoms are slightly different. Have you any advice as to what we could do to try to debug.

Symptoms: over time the virtual memory used by the oracle process (10.1.0.5 on Windows 2003R2) increases until it hits the 3Gb limit. SGA and PGA info show massively less memory in use than the VM shown by sysinternals.

Base Data:

From sysinternals using pslist -m oracle

Name                Pid      VM      WS    Priv Priv Pk   Faults   NonP Page
oracle             6548 1996236 1210708 1203804 1596496 18076609    103  324

From Oracle using data dictionary queries

SGA

Total System Global Area 1048576000 bytes
Fixed Size                   792728 bytes
Variable Size             568584040 bytes
Database Buffers          478150656 bytes
Redo Buffers                1048576 bytes

Process Memory

  PID SPID  USED_MB ALLOC_MB FREEABLE_MB MAX_MB      
----- ----- ------- -------- ----------- ------      
    1             0        0           0      0      
    2 4704        0        1           0      1      
    4 1356        0        1           0      1      
    6 2784        0        1           0      4      
    8 9104        0        1           0      1      
   10 4364        4       12           0     12      
   12 6340        0        1           0      4      
   14 5428        0        1           0      3      
   16 7800        0        1           0      1      
   18 1664        1        1           0      2      
   20 3776        2        4           0      7      
   22 10096       4        9           0      9      
   24 3616        4        9           0      9      
   26 2036        0        1           0      1      
   28 8168        1        2           0      2      
   30 3724        0        1           0      1      
   32 6972        3        3           0      8      
   34 2628        2        3           0      9      
   36 4528        2        3           0     13      
   38 8236        2        3           0      7      
   42 5628        1        3           0      3      
   44 9796        3        5           0      7      
   46 1704        0        1           0      1      
   48 3984        2        2           0      2      
   52 6660        2        3           0      5      
   54 8600        2        4           0      4      
   56 5872        2        2           0      7      
   58 1776        1        2           0      2      
   60 3368        2        2           0      6      
   88 4324        3        3           0      6      

So I’m using 2Gb of virtual memory for a 1Gb SGA and maximum 150Mb or so of PGA memory and this keeps increasing. Is it possible for a PGA memory leak not to be cleared when the process exits?

Any suggestions for queries / dumps to run that might point me in the right direction?