Oracle Scratchpad

Direct SGA Access – pt 2

Attaching to the shared memory segment(s)

A 64-bit machine is theoretically capable of addressing 16 exabytes of memory – roughly 16 million terabytes. Obviously no-one is likely (ever?) to own a machine with that much memory – but it is extremely convenient that you can tell your computer to set up a “physical to logical” mapping that associates a portion of your physical memory to some arbitarily chosen range in that enormous address space.

This is particularly important when you have several programs (such as the many background and shadow processes of an Oracle instance) that want to share and manipulate large chunks of memory. If all the programs can agree (for example) that some structure like x$ksuse should start at logical address 0x8E000000 then none of the programs needs to worry about the actual physical location of that memory – it’s up to the operating system to deal with the “translation”.

When an Oracle instances starts up, the initiating process will call the operating system to allocate several large chunks of memory as “Shared Memory Segments” and associate them with specific logical addresses. Every other Oracle process will then be able to reference those memory areas using the logical addresses and not have to worry about finding out the physical memory addresses.

With minor variations for platform the following (operating system) command will list all the current shared memory segments, allowing you to identify the ones that were allocated by the oracle userid:

[oracle@linux183 ~]$ ipcs -m

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status      
0x00000000 294912     oracle     777        16384      1          dest         
0x00000000 327681     oracle     777        7372800    2          dest         
0x00000000 557058     oracle     777        7372800    2          dest         
0x00000000 524291     oracle     600        524288     2          dest         
0x00000000 655364     oracle     600        524288     2          dest         
0x00000000 753669     oracle     600        16777216   2          dest         
0x00000000 42139654   oracle     600        524288     2          dest         
0x00000000 29294599   oracle     600        10485760   62                      
0x00000000 851976     oracle     600        524288     2          dest         
0x00000000 29327369   oracle     600        1879048192 62                      
0x00000000 29360138   oracle     600        8388608    62                      
0x445feb18 29392907   oracle     600        20480      62                      
0x00000000 40239116   oracle     777        4145152    2          dest         
0x00000000 5439502    oracle     777        16384      2          dest         
0x00000000 12910607   oracle     600        524288     2          dest         
0x00000000 40435728   oracle     777        4718592    2          dest         
0x00000000 44171281   oracle     777        4259840    2          dest         
0x00000000 41156626   oracle     777        3932160    2          dest         

[oracle@linux183 ~]$ 

As you can see there are 18 segments allocated on my system at present, all of them allocated by O/S user oracle although, for reasons I don’t know, 14 of them have been flagged for destruction (status = dest). [It’s possible with the experiments I was doing, and frequent bounces of the database (startup force) I manged to introduce a few orphan segments]. You’ll notice that some of them have a relatively large value for nattch, the number of attached processes: it’s a safe bet that those are the shared memory segments for the Oracle instance I’ve got running.

There are only 4 segments still active, and a key feature of those segments is that their access permissions (perms) are “600“, which is the numeric representation of “owner only can read and write”. (The three digits are, in order, for “owner”, “group”, “all” and each digit is a bitmap (4,2,1) for “read”, “write”,”execute”. The fact that the segments can only be read by their owner is an important detail when you want to attach to them.

One of the most important details about each segment is its id (shmid). When you want to attach to a segment you have to specify two things – the shmid of the segment, and the address in your program’s virtual memory space where the segment should reside..

You’ll notice that there’s nothing in the ipcs output that says anything about a shared segment’s address and that’s because it is up to each process that attaches to choose the virtual address it wants to use for the memory. The oracle executables have a few hard-coded addresses for attaching the various memory segments (there is some variation for a couple of rare platforms, and there is a mechanism for modifying the embedded values that I’ve seen used a couple of times on 32-bit systems) and the first thing you have to be able to do in order to program direct SGA access is to make sure you know what those addresses are.

If you want to guarantee that you use the correct addresses in your “direct attach” program (rather than assuming that your instance is using the standard addresses) you can ask the Oracle instance what addresses it’s using by connecting to a suitably privileged schema and issuing the oradebug ipc command:

SQL> oradebug setmypid
Statement processed.
SQL> oradebug ipc
IPC information written to the trace file
SQL> select  tracefile
  2  from    v$process
  3  where   addr = (
  4                  select  paddr
  5                  from    v$session
  6                  where   sid = sys_context('userenv','sid')
  7          )
  8  /

TRACEFILE
--------------------------------------------------------------------------------
/u01/app/oracle/diag/rdbms/or19/or19/trace/or19_ora_21982.trc

SQL> ed /u01/app/oracle/diag/rdbms/or19/or19/trace/or19_ora_21982.trc

The volume of information that gets dumped by this call has grown over the last few years but the key section will look similar to the following:

  Area #0 `Fixed Size' containing Subareas 2-2
  Total size 00000000008b6830 Minimum Subarea size 00000000
   Area  Subarea    Shmid    Segment Addr    Stable Addr    Actual Addr
      0        2 29294599 0x00000060000000 0x00000060000000 0x00000060000000
               Subarea size     Segment size   Req_Protect  Cur_protect
                          00000000008b7000 0000000000a00000 default       readwrite
 Area #1 `Variable Size' containing Subareas 0-0
  Total size 0000000070000000 Minimum Subarea size 01000000
   Area  Subarea    Shmid    Segment Addr    Stable Addr    Actual Addr
      1        0 29327369 0x00000061000000 0x00000061000000 0x00000061000000
               Subarea size     Segment size   Req_Protect  Cur_protect
                          0000000070000000 0000000070000000 default       readwrite
 Area #2 `Redo Buffers' containing Subareas 1-1
  Total size 0000000000749000 Minimum Subarea size 00001000
   Area  Subarea    Shmid    Segment Addr    Stable Addr    Actual Addr
      2        1 29360138 0x000000d1000000 0x000000d1000000 0x000000d1000000
               Subarea size     Segment size   Req_Protect  Cur_protect
                          0000000000749000 0000000000800000 default       readwrite
 Area #3 `skgm overhead' containing Subareas 3-3
  Total size 0000000000005000 Minimum Subarea size 00000000
   Area  Subarea    Shmid    Segment Addr    Stable Addr    Actual Addr
      3        3 29392907 0x000000d2000000 0x000000d2000000 0x000000d2000000
               Subarea size     Segment size   Req_Protect  Cur_protect
                          0000000000005000 0000000000005000 default       readwrite

My instance reports 4 areas (corresponding to the 4 active areas that were not marked for destruction in the ipcs output) and, stripping this output to the bare minimum, we can see the shmids and addresses we have to use to attach to the segments.

 Area #0 `Fixed Size' Total size 00000000008b6830
                    Shmid    Segment Addr
                 29294599 0x00000060000000

 Area #1 `Variable Size' Total size 0000000070000000
                    Shmid    Segment Addr
                 29327369 0x00000061000000 

 Area #2 `Redo Buffers' Total size 0000000000749000
                    Shmid    Segment Addr
                 29360138 0x000000d1000000 

 Area #3 `skgm overhead' Total size 0000000000005000
                    Shmid    Segment Addr
                 29392907 0x000000d2000000 

You’ll notice that areas 0 to 2 have familiar sounding names, so if you want to go back to more familiar ground for a moment, here’s some matching information accessible through SQL:

SQL> select name, value, to_char(value,'xxxxxxxx') hex_value from v$sga;

NAME                          VALUE HEX_VALUE
------------------------ ---------- ---------
Fixed Size                  9136176    8b6830
Variable Size             788529152  2f000000
Database Buffers         1090519040  41000000
Redo Buffers                7639040    749000

 

You’ll notice that v$sga and “oradebug ipc” aren’t completely consistent in what they call “Variable Size” but in this case it’s easy to work out that 0x2f000000 + 0x41000000 = 0x7000000

Better still, if you’re using one of the more recent vesions of Oracle (18c and above) you can get a direct report of the shared segment information (with some highly undesirable column names that I’ve had to quote because of the embedded spaces). You’ll notice from this query that Oracle hasn’t fully utilized every allocation:

SQL> select shmid, "SEG_START ADDR", "SIZE", "SEGMENT SIZE", "AREA NAME" from x$ksmssinfo order by shmid;

     SHMID SEG_START ADDR         SIZE SEGMENT SIZE AREA NAME
---------- ---------------- ---------- ------------ --------------------------------
  29294599 0000000060000000    9138176     10485760 Fixed Size
  29327369 0000000061000000 1879048192   1879048192 Variable Size
  29360138 00000000D1000000    7639040      8388608 Redo Buffers
  29392907 00000000D2000000      20480        20480 skgm overhead


4 rows selected.

You need to get the shared memory segment information into your program somehow, and this last query looks like the easiest option to access the information – provided you can access the database in the standard way.

Of course, you could just use ipcs -m and make a good guess about which segment should use which address – but if you’re running more than one instance on the machine you’ll have to be very careful about grouping the segments correctly (nattch will probably be a helpful hint).

In fact, for the purposes of finding and reading things like the x$ksuse structure we’re really only interested in two of the areas – the “Fixed Size” and the “Variable Size” – so here’s a very simple piece of C code that will attach just those two segments, sleep for a while, then detach, sleep and exit. (The sleep() is there so I have a few moments to call ipcs -m to see if nattch on 2 of the 4 segment has increased by 1; then to check in a similar way that it was the explicit detach rather than program termination that released the segments.

I’ve chosen to #define the target addresses, but accept the shmids from the command line and my error trapping is virtually non-existent. I’ve included a hat-tip to Kyle Hailey – it’s about 30 years since I did any serious C programming (Windows 3.0, analysing real-time data streams from a stock ticker) and I needed to start with some sample code to get the little grey cells going. You’re welcome to point out any errors or oddities in the code.

/*

        File:           attach_shm.c
        Dated:          March 2022
        Inspired by:    Kyle Hailey

        cc -o attach_shm  attach_shm.c

        select 
                shmid, "SEG_START ADDR", "SIZE", "SEGMENT SIZE", "AREA NAME" 
        from 
                x$ksmssinfo 
        order by 
                shmid
        ;

             SHMID SEG_START ADDR         SIZE SEGMENT SIZE AREA NAME
        ---------- ---------------- ---------- ------------ --------------------------------
          29294599 0000000060000000    9138176     10485760 Fixed Size
          29327369 0000000061000000 1879048192   1879048192 Variable Size
          29360138 00000000D1000000    7639040      8388608 Redo Buffers
          29392907 00000000D2000000      20480        20480 skgm overhead

        Usage:  ./attach_shm {Fixed Size shmid} {Variable Size shmid}
                ./attach_shm 29294599 29327369

*/

#include <stdio.h>
#include <stdlib.h>     
#include <sys/shm.h>    

#define FIXED     0x0000000060000000
#define VARIABLE  0x0000000061000000

main(int argc, char **argv) {

        int     shmid[2];
        void    *addr_fix;
        void    *addr_var;

        if (argc != 3) {
                fprintf(stderr, "Usage: oracle_attach {fixed shmid} {Variable shmid}\n");
                exit(EXIT_FAILURE);
        }

        shmid[0] = atoi(argv[1]);
        shmid[1] = atoi(argv[2]);

        fprintf(stderr,"Fixed shmid:    %i\n",shmid[0]);
        fprintf(stderr,"Variable shmid: %i\n",shmid[1]);

        addr_fix = shmat(shmid[0], (void *)FIXED, SHM_RDONLY );
        if (addr_fix != (void *)-1) {
                fprintf(stderr, "Fixed address:    %p %d \n", addr_fix, addr_fix);
        }
        else {
                fprintf(stderr, "Failed to attach Fixed size \n");
                exit(EXIT_FAILURE);
        }

        addr_var = shmat(shmid[1], (void *)VARIABLE, SHM_RDONLY );
        if (addr_var != (void *)-1) {
                fprintf(stderr, "Variable address: %p %d \n", addr_var, addr_var);
        }
        else {
                fprintf(stderr, "Failed to attach Variable size \n");
                exit(EXIT_FAILURE);
        }
        
        fprintf(stderr, "Sleeping for 5 seconds\n");
        sleep(5);

        fprintf(stderr, "Detached Fixed size:    %i\n", shmdt(addr_fix));
        fprintf(stderr, "Detached Variable size: %i\n", shmdt(addr_var));

        fprintf(stderr, "Sleeping for 5 seconds\n");
        sleep(5);

        exit(EXIT_SUCCESS);
}

Although I’ve only had to attach two shared memory segments I have a vague memory that I have seen some very large systems where the “Variable Size” is made up of several segments. If this is correct it shouldn’t be a problem, you just have to add a few extra calls to shmat().

Once you’ve got the shared memory attached, you can start adding code to crawl all over it, looking up constants and flags and following endless chains of pointers; but you need to learn a little more about the dynamic performance views and underlying memory structures before you can start bypassing them, and that’s the topic of part 3.

Summary

You need to have a highly privileged O/S account to attach to the oracle owner’s shared memory segments as the segments are declared as “owner only can read and write”. The ipcs -m command will list all the shared memory segments on the server, but you have to pick the right one(s) to attach.

To attach to a shared memory segment (and use it sensibly) you need to know its shmid, and the logical address where it should live. To find the correct logical addres the only way (that I know of) is to ask Oracle. There are many ways to do this – including guessing or assuming that it’s the same for the current system as it is for other systems you can access.

Leave a Comment »

No comments yet.

RSS feed for comments on this post.

Comments and related questions are welcome.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Website Powered by WordPress.com.

%d bloggers like this: