Quickly find foreign-keys depended tables

Posted by David Yahalom under DB2 LUW

Today I needed a quick way to find out all of the tables foreign keys are referencing for a given table. That is, if I have a table named TABLE1 which has a few foreign keys defined on it, I want a list of all the tables these foreign keys are referencing.

The easier way to do this is by querying the the database catalog tables (’data dictionary’). It took me some googleing to find out exactly what DB2 system catalog contains the relevant foreign key information.

SYSCAT.SYSRELS

You can easily use the following query to get a list of all tables referenced via the foreign keys of a given table(s):

select distinct reftbname
from sysibm.sysrels
where tbname in ('TABLE1', TABLE2', 'TABLE3')

Truncate in DB2, DB2 CLP tricks, simple DB2 tracing

Posted by David Yahalom under DB2 LUW

Truncate table in DB2

One of the things I really missed when going DB2 from Oracle is the truncate table command.

In Oracle you have the option to quickly truncate a table using the truncate table command. This is a really good feature as the difference between doing truncate or a DML delete is huge in terms of performance.

You have to remember that when doing a truncate you are actually doing a non-logged delete of the table which is much less I/O intensive then a regular DML delete. This is most useful if you want to truncate a table during work hours without increasing server load.

However, apparently, DB2 does have a truncate feature, the command is just not as straightforward as writing truncate table in Oracle.

To truncate a table in DB2, simply write:

alter table schema.table_name activate not logged initially with empty table

From what I was able to read, this will delete the table content without doing any kind of logging which will go much easier on your server’s I/O.

Change the DB2 CLP prompt

Working with multiple servers or databases in DB2 can be dangerous if you are using the command line interface and forget which session is connected to which server.

So, in order to save yourself allot of future headaches and lost sleep hours while trying to mount restore tapes you can issue the following command which will change the DB2 CLP prompt to a meaningful (user@database):

db2set DB2_CLPPROMPT=(%da@%d):

You can get creative and also use the following environment variables in building your custom DB2 prompt:

%d = Currently connected database name
%i = Attached instance name
%ia = Authorization ID of the current instance
%n = Newline (for a two line prompt)
%da = Authorization ID of the current database

Nifty DB2 CLP commands you usually forget:

history

Lists the last 20 commands entered and prefixes each with a number. The maximum number of commands kept in memory can be customized with the DB2 registry variable DB2_CLP_HISTSIZE.

runcmd command_number_n
[/sourcode]

Re executes command number n from the list given by the history command. If n is not specified (or n = -1), the previous command is invoked.

[sourcecode language="sql"]
edit command_number_n
[/sourcode]

Edits the command number n using an editor defined by the DB2 registry variable DB2_CLP_EDITOR. If not set, this uses the vi editor on Linux/UNIX and Notepad on Windows.

[sourcecode language="sql"]
Exclamation mark (!)
[/sourcode]

This is the escape character that lets you issue operating system commands from within the CLP interactive mode

How to monitor locks:

[sourcecode language="sql"]
1) db2 update monitor switches using LOCK on
2) db2 get snapshot for locks on opal_d > /tmp/locks.txt
3) db2 reset monitor all

Do a simple DB2 trace

DB2 has several ways to do an SQL query trace. This method is using the explain plan with snapshotsql prefix:

1) Create a text file with the query you want to trace. For example, create a file named c:\query1.sql and paste your query inside.

2) Add the following text before the SELECT operator:

explain plan with snapshot for

So that your query will look something like this:

explain plan with snapshot for
SELECT count(*) from syscat.tables;

3) Run the following command:

db2 -tvf c:query1.sql

This will prepare the execution plan and save it in the explain tables within the system catalog.

4) Run the following command which will output the execution plan to a txt file:

db2exfmt -d database_name -u username password -1 -o c:explain_plan.txt

Solve DB2 “stuck” shared quiesce problems

Posted by David Yahalom under DB2 LUW

I knew this day was coming and I was just too afraid to admit it. After years of experiencing all kinds of strange Oracle problems (ORA-600 this, Ora-600 that) I thought moving on to a simpler architecture (DB2) would prove more weirdness-resistant. Actually, I knew all long this is not true but I liked the false sense of hope it gave me. -sigh- Anyway, I’m giving DB2 an official “pain in the ass” medal for the problems it caused me today. Oracle earned its fair share of these medals in the past too, so DB2 should feel just at home.

Today, one of our developers came to me saying that a DML operating she wrote is “hanging” and the table on which this operation was issued was non-responsive. Just as she walked into my room the “hang” released and we were able to query the problematic table again. We wrote some quick queries to verify all is well. Ten seconds after that I received a phone call from another developer saying that he is received a “SQL0290N Table space access is not allowed. SQLSTATE=55039″.

Okay, so something went bad with the Quiesce process in DB2?

Relaxed, I thought the problem is probably caused because the table is in load pending state or a quiesce state. So I quickly did a list tablespaces command and saw the tablespace #3 is indeed in quiesced shared mode.

The handbook procedure in such cases is to do a quiesce reset on any table from the problematic tablespace. Well, I tried that and guess what? Nada. While the command itself executed okay, the result was more of the same. Table space access is not allowed errors.

This seems like a phantom quiesce mode in DB2 (a wonderful architectural “feature” DB2 probably learned from Satan himself, nonetheless). In such a case, there are a few more trick to try like to try and reacquire the quiesce by the same user that caused it and then to reset it or to try a load terminate operation. Neither of these worked. Tablespace is still in quiesce mode. Maybe putting the tablespace in backup pending mode should do the trick? No such luck. You simply can’t because the table is already Quiesced shared.

The solution that worked for me, after doing a lot of guesswork (and I’ll put money that it will probably work for you in such situations as well) is:

1) Do a list tablespaces show detail command to find out the tablespace that is in QUIESCE SHARED mode.

Tablespace ID = 3
Name = PROD_TBS1
Type = Database managed space
Contents = All permanent data. Large table space.
State = 0x0001
Detailed explanation:
Quiesced: SHARE
Total pages = 2723840
Useable pages = 2723808
Used pages = 2181056
Free pages = 542752
High water mark (pages) = 2708288
Page size (bytes) = 4096
Extent size (pages) = 32
Prefetch size (pages) = 32
Number of containers = 1
Number of quiescers = 1
Quiescer 1:
Tablespace ID = 3 <<--- This is the tablespace id that is in Quiece mode
Object ID = 12 <<--- This is the table inside the tablespace that is causing the QUIESCED SHARED

2) Run this query to find out which object is loking the tablespace in QUIECE SHARED mode:

SELECT SUBSTR(TAB1.DEFINER,1,15) AS DEFINER, SUBSTR(TAB2.TABSCHEMA,1,15) AS TABSCHEMA,SUBSTR(TAB1.TABNAME,1,15) AS TABNAME, SUBSTR(TAB2.TBSPACE,1,15) AS TBSPACE_NAME FROM SYSCAT.TABLES TAB1, SYSCAT.TABLESPACES TAB2 WHERE TABLEID=OBJECT_ID AND TAB2.TBSPACEID=TABLESPACE_ID AND TAB1.TBSPACEID=TAB2.TBSPACEID;

This is the result you will receive:

DEFINER TABSCHEMA TABNAME TBSPACE_NAME
--------------- --------------- --------------- ---------------
DB2INST9 TABSCM TABLE1 PROD_TBS1

There we have it. The problematic table name.

3) Now, lets try what we tried before, but this time, instead of just using a “quiesce table” command on a random table from the problematic tablespace, lets use it on the same table that is causing the problems.

db2 quiesce tablespaces for table TABSCM.TABLE1 share

Now we have re-quiesced the table back with the same locking level as it was originally quiesced with (shared).

3) And in order to get rid of the annoying quiesce do a:

db2 quiesce tablespaces for table TABSCM.TABLE1 reset

After this all is back to normal.

The reason I’m annoyed is because there is nothing online that suggests doing this. There are bits and bytes of info here and there but nothing concrete. Even IBMs solution for this is to simply to Re-Quiesce the tablespace (and not the table itself):

To remove a phantom quiesce:

1. 1 Connect to the database with the same user ID used when the quiesce mode was set.
2. Use the LIST TABLESPACES command to determine which table space is quiesced.
3. Re-quiesce the table space using the current quiesce

http://publib.boulder.ibm.com/infocenter/db2luw/v8/index.jsp?topic=/com.ibm.db2.udb.doc/core/r0002003.htm

Monitor locks in DB2

Posted by David Yahalom under DB2 LUW

Since locking in DB2 can be a real problem if not managed correctly, it is very important to keep a close eye on row and table locks.

I know of two easy ways to monitor locks.

The easiest one is simply to execute the following query from the DB2 CLI:

SELECT SUBSTR(appl_name, 1,15) as APPLICATION_WAITING,
SMALLINT(agent_id) as USER_WAITING_ID,
SUBSTR(authid,1,15) AS USER_WAITING,
SMALLINT(agent_id_holding_lk) AS USER_HOLDING_ID,
lock_mode AS HOLDING,
lock_object_type AS TYPE,
lock_mode_requested AS REQUESTED_LOCK_MODE
FROM SYSIBMADM.LOCKWAITS

Then use this to drill down to a specific lock for more details:

SELECT appl_status,
SUBSTR(PRIMARY_AUTH_ID,1,10) AS USER_HOLDING_LOCK,
SUBSTR(appl_name, 1,15) AS APP_HOLDING_LOCK,
SUBSTR(client_nname,1,20) AS CLIENT_HOLDING_LOCK
FROM SYSIBMADM.SNAPAPPL_INFO
WHERE AGENT_ID = <>>

Another way to monitor locks which will result in more detail displayed is:

1) Allow DB2 to monitor locks in the database:

db2 update monitor switches using LOCK on

2) Run the following command which will output the monitoring statistics to a text file of your choice:

db2 get snapshot for locks on database_name > /tmp/locks.txt

3) Keep in mind that there is a certain amount of overhead involved while using the monitor locks option, this is why I always turn the locking monitor off when it is not needed.

db2 reset monitor all

IBM migration toolkit

Posted by David Yahalom under DB2 LUW

I want to recommend a great free tool from IBM called the “IBM migration toolkit” (now avail. in V2 beta).

This GUI tool allows you to migrate data from Oracle, SQL and a varaity of other databases to IBM DB2 (both LUW and Z/OS). I found this very useful with my DB2 deployment and if you are working in a place where you need to transfer data from Oracle/SQL server to DB2, try this tool fist before going to higher-end (pricey) solutions as it might fit your needs.

“The IBM Migration Toolkit is an easy-to-use tool that allows you to migrate your data from a wide variety of source databases to either DB2 or Informix Dynamic Server, regardless of platform.”

Is my DB2 optimizer a tad drunk?

Posted by David Yahalom under DB2 LUW

Today I have noticed a strange phenomena in one of our DB2 databases. When doing a low zero-selectivity query (that is, query without a WHERE clause), the DB2 optimizer preferred to use an index scan rather than do a table scan.

That seemed a bit strange so I tried running the same query twice. Once without any indexes on the underlying table (to force a table scan) and then again with (to allow the optimizer use an index).

The results really surprised me.

According to the optimizer, the table scan query will perform slower than the index scan query.

I know that with Oracle systems, the rule of the thumb is that using a full table scan would perform faster than an index scan when the total number of rows retrieved from a table is between 1% and 15% of table data.

It is very logical. If the database has to read the entire table wouldn’t it be faster just to read the blocks directly from the table container rather than doing an INDEX READ + ROWID lookup for the entire table data?

The way I see it:

SELECT * FROM TABLE

Performing a TABLE SCAN to get the data: X I/O blocks read from the table.
Performing an INDEX SCAN to get the data: X I/O blocks read from the table + Y I/O blocks read from the index.

Or am I wrong?

Any of you can shed some light on why the DB2 optimizer reports the table scan as slower than an index scan in this case?

Here is the real world example of what I mean:

SELECT COUNT *
FROM TABLE1;

Using an INDEX:

Access Plan:
-----------
Total Cost: 72284.4
Query Degree: 1
Rows
RETURN
( 1)
Cost
I/O
|
95047
FETCH
( 2)
72284.4
26250.8
/---+---
95047 95047
RIDSCN TABLE: USER
( 3) TABLE1
556.668
275.886
|
95047
SORT
( 4)
556.667
275.886
|
95047
IXSCAN
( 5)
427.788
275.886
|
95047
INDEX: USER
TABLE1_IDX1

Without an INDEX:

Access Plan:
-----------
Total Cost: 73526
Query Degree: 1
Rows
RETURN
( 1)
Cost
I/O
|
95047
TBSCAN
( 2)
73526
27725
|
95047
TABLE: USER
TABLE1

Leverage your Oracle 10g skills to learn DB2 9.1 for Linux, UNIX and Windows

Posted by David Yahalom under DB2 LUW, Oracle

Leverage your Oracle10g skills to learn DB2 9.1 for Linux, UNIX and Windows.

This article makes direct comparison between the Oracle architecture (instance, databases, physical files, network, configuration files, etc) and the DB2 architecture thus allowing Oracle DBAs to understand the similarities and differences between Oracle and DB2.

Because of its comparative nature (showing you how the same things work in DB2 compared to Oracle) I think its a great first read if you are an Oracle DBA new to DB2.

I do have some comments I would like to add:

1) IBM says:

“DB2 does contain a binary file known as the system database directory that contains entries of all the databases you can connect from your DB2 machine.”

This compares best to Oracle tnsnames.ora file. I don’t understand why IBM didn’t draw this comparison.

2) IBM says:

“Every Oracle database contains a table space named SYSTEM, which Oracle creates automatically when the database is created. Other table spaces for user, temporary and index data need to be created after the database has been created, and a user needs to be assigned to these table spaces before they can be used.”

While I get IBMs point (that you have to create tablespaces manually after the create database command) This is not technically true. Frist, when you issue a create database command in Oracle you explicitly specify the creation of a temp tablesapce. Second, after the database creation all other tablespaces you have created can be used without being “assigned to users” simply by using them in the storage clause of the object creation statement.

3) IBM Says:

“As indicated earlier, Oracle’s data buffer concept is equivalent to DB2’s bufferpool; however, DB2 allows for multiple bufferpools to exist. There is no predefined number of bufferpools that you can create, and they can have any name.”

Again, not entirely true. While less flexible, Oracle 9i/10g allows for multiple db buffers with different block size. The limitation is one buffer per one block size while in DB2 it is one buffer per tablespace.

Great, simple and free db2 monitor software

Posted by David Yahalom under DB2 LUW

Today I have stumbled upon this great little free tool called “DB2 Monitor”.

This is a fast windows GUI application for performance monitoring. Shows active connections with lock information, SQLs in db cache, database snapshots, database hit ratios, database objects information, list history. Can help resolve locking problems and identify worst performing queries.

Its fast, has a very simple user interface and just works every time! (something I can’t really say about most of IBMs JAVA monitoring applications for DB2).

Recommended.

You can get DB2 Monitor from here.

Update all database statistics with a single command

Posted by David Yahalom under DB2 LUW

As with all RDBMS implementation, DB2 UDB uses the statistics information in the catalog table to derive the best execution plan. We, as DBAs, should regularly run the RUNSTATS command to keep our database statistics updated for optimal query performance.

RUNSTATS ON TABLE SCHEMA_NAME.TABLE_NAME

Runstats works as advertised but what happen when you need to quickly update statistics for a large group of tables at once? Allot of DBAs (including myself) would write a script that creates another script, a “RUNSTATS ON TABLE_NAME” script, with the table name concated. This will, of course, work, but there is a simpler and better way to do this in DB2 using the reorgchk command.

To update stats for the entire database use:

REORGCHK UPDATE STATISTICS on TABLE ALL

To update all the tables of a particular schema use:

REORGCHK UPDATE STATISTICS on SCHEMA schema_name

You should note, however, that the RUNSTATS command generated by the REORGCHK UPDATES STATISTICS command collects statistic on the table only without distribution. To have distributed statistics on your tables you should use the RUNSTATS command instead.

Understanding DB2 backup filenames

Posted by David Yahalom under DB2 LUW

When backing up a db2 database the output filename might be confusing at first.
The filename generated by the backup utility might seem long an confusing but it is actually pretty straight forward and very informative.

For example, if I issued the following command:

db2 backup database dbone to /storage/backup_path/

I might get a file named:

DBONE.0.DB2INST1.NODE0000.CATN0000.20070101140000.001

So, what does all the numbers mean?

On UNIX/LINUX:

DBONE.0.DB2INST1.NODE0000.CATN0000.20070101140000.001
(1)-----(2)---(3)---------(4)---------(5)-------------(6)-----------(7)

On Windows:

DBONE.0DB2INST1NODE0000CATN000020070101140000.01
(1)-----(2)----(3)---------(4)----------(5)---------------(6)--------(7)

1) Database alias
2) Backup type.
3) Instance name
5) Database Partition (always NODE0000 for a non-partitioned database).
5) Catalog partition number (always CATN0000 for a non-partitioned database).
6) Date&time of when the backup file was taken.
7) Time image sequence number.