Migrating from SPARC to x86

Posted by David Yahalom under RAC, Hardware, Solaris, Unix, Oracle

Remember the SPARC Vs. x86 post I made some time ago? I’ve talked about sizing old SPARC hardware compared to new x86 based servers.

Well, I have just finished a rather big project migrating several single-instance Oracle 8i databases running on old(ish) SPARC machines to a state of the art Oracle 10g RAC on commodity x86 Linux servers.

I have migrated my client from this:

Two: 4 X UltraSparc III+ 1GHZ CPUs, 8GB RAM.
Two: 3X UltraSparc II 300Mhz CPUs, 4GB RAM.

To this:

Two HP DL580G5: 2X Intel QuadCore Xeon X7350@2.93GHz, 16GB RAM (list price: $15,000).
One HP DL380G5: 2X Intel QuadCore Xeon E5345@2.33GHz, 8GB RAM (list price: $8,000).

The workload remained the same. The old config had several different unique schemas on each physical server. Each applications running connected to a different schema on a different physical server. No shared data between servers. This was done to spread the workload across several machines. Five years ago they (client) considered it a poor man’s cluster.

When installing and configuration the RAC cluster, I have combines all of the different schemas under a single database.

I’ll try and published more detailed performance benchmarks later on, but now all I can say is this: in terms of raw CPU power (taking storage I/O and memory restrictions out of the function) the RAC cluster smokes, obliterates and completely destroys all of the older SPARC servers combined.

The RAC cluster consist a total of 6 CPUs, each with 4 cores (a total of 24 cores).  The average load on each node is around the 2.0-2.2 mark. Meaning a combined load of ~6/24 or roughly 25%.  Given the same workload,  the old SPARC machines reached well above 100%. For example, one of the “more powerful” Ultrasparc III+ 1Ghz machines (total of 4 CPUs) reached load ranges of above 20!

The workload is a mixed multi-million user, multi-session web application with short, cached queries and transactions plus several single-threaded very cpu-intensive financial applications that perform thousands of logical DML operations per second. One of these DML intensive processes caused a 4XUltraSprac III+ 1Ghz machine to reach average loads of 15-20. The same process running on the same data barely tickles the QuadCore Xeon.

Now, please keep in mind two things:
1) This isn’t scientific, far from it. At least not yet. I have also performed more concrete benchmarks, but until I’ll have time to arrange all the data into user-readable format you’ll just have to take my word on it.

2) You might say that comparing 4 year old SPARC hardware to brand new x86 machines might not be fair. And I’ll agree. It isn’t. But what’s important is to take the general sizing figures from this post and try to adapt them to occasions where you have a site “suck” with old SPARC hardware and you want sizing information to decide how exactly SPARC Mhz translates to x86 Mhz.

The bottom line is this: don’t be afraid to migrate old SPARC machines to modern Linux x86 hardware. It will usually be allot faster.
And…. you can achieve great performance when using cheap, x86 servers in a RAC cluster and very high ROI.

Show full process name / path / string in Solaris using ps

Posted by David Yahalom under Solaris

Let’s say you have a JAVA process running on an old Sun Solaris machine. You want to see the full path of the process running including any run-time variables that were passed to it.

Using ps -eaf | grep -i java gives us this:

[root@hostname ~]# which ps
/bin/ps

[root@hostname ~]# ps -eaf | grep -i java

nobody  4589  4588   0 14:26:35 ?           0:21 /usr/java1.4/bin/java -Djava.awt.headless=true -Xms200m -Xmx300m org.apache.jse

As you can see the process name is trimmed and you can’t see all parameters passed to Java. No matter what parameters or scripting you’ll try and do, using /bin/ps (default) will crop your process name.

There is, however, an entriely different ps program we can use.

[root@hostname ~]#/usr/ucb/ps -auxwww | grep -i java

nobody    4589  0.1  4.228480842528 ?        S 14:26:35  0:21 /usr/java1.4/bin/java -Djava.awt.headless=true -Xms200m -Xmx300m org.apache.jserv.JServ /jserv/etc/jserv.properties

Fantastic!

Limelight

Posted by David Yahalom under Security, Oracle

I would like to thank the several people that have emailed me about this. Pete Finnegan, who is one of the most prominent figures in the world of Oracle databases and Oracle Security has mentioned AuthorityBase on his web site!

He has a writeup mentioning the Oracle Security presentation I’ve created for my company (XpertOne1) and he seemed to liked it.

Check it out at Pete’s site!

Intermittent ORA-12545 When Trying To Connect To RAC Database

Posted by David Yahalom under RAC, Oracle

Well, got another RAC tidbit for you today. I’ve been doing some very interesting RAC installations in the past few weeks and came across several “bugs” (or “random features” in Oracle-tongue ) I felt like sharing.

So, you have a brand new 10g RAC cluster installed and when you try to connect using OCI (probably JDBC thin as well) only to recieve intermittent ORA-12545 errors. The client will connect fine every other attempt or so.

This error happens when you enable server side load balancing but the client does not have domain address / DNS search function setup (or valid /etc/hosts file) so it cannot translate a hostname to a proper IP address.

The solution is simple, make sure that the client from which you are trying to connect is able to resolve all the hostnames in your RAC installation. Either via registering the nodes in a DNS server or by updateding the /etc/hosts file (remember, Windows also has an /etc/hosts file under <WINDOWS>\system32\drivers\etc\hosts).

I’ve seen many DBAs forget that for server-side LOB you need to be able to resolve each cluster node hostname from the client EVEN when you only use IPs in your TNSNAMES.ORA entries. Even without server-side LOB, it’s good practice to be able to resolve RAC hostnames from the client.

This isn’t actually a bug, it’s by-design and if you understand they way server-side LOB works in RAC you’ll see why the client has to be able to resolve RAC hostnames to IP addresses.

Ora-12520 When listeners on VIP in 10g RAC Setup

Posted by David Yahalom under RAC, Oracle

A few days ago I’ve stumbled across an annoying problem with Oracle 10.2.0.1 (yep, need to patch it) RAC causing my OCI client fail to connect with an ORA-12520 error (ORA-12520: TNS:listener could not find available handler for requested type of server ).

Everything was configured correctly on both the server side and the client.

1. Remote_Listener parameter is set to an alias which is defined in the Tnsnames.ora file on both the nodes.
2. The hostname is setup correctly in the tnsnames.ora file.
3. The IP address, VIP and hostname are all properly configured in /etc/hosts on all nodes.

This is a documented Oracle bug with a complete note on this in Metalink (342419.1). 10g RAC instances have a problem where the instances are not registering themselves correctly with the virtual IP address. They get registered to the real ip address where the listener is not listening.

The workaround is very simple:

Define a LOCAL_LISTENER entry for each node in the node’s local TNSNAMES.ORA file:

LISTENER_NODE_XXX =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = <node1-vip>)(PORT =1521))
)

Then issue the following command in the sqlplus prompt:

sql>Alter system set LOCAL_LISTENER= 'LISTENER_XXXX' scope=both sid='SID1'.

Remember to do the same for all other nodes in the cluster (each node should have an entry in its TNSNAMES.ORA file with the LISTENER_NODE_XXX definition + setting the LOCAL_LISTENER spfile parameter to the same TNSNAMES.ORA entry).

Simulate load on your linux server using a one-liner

Posted by David Yahalom under Unix, Linux, General IT

Yesterday I needed a quick way to simulate load on a Linux sever. I wanted to find out that my monitoring system is working and sending alerts.

While there are many many tools that can do this, all I needed was a simple bash one-liner shell script that can make the CPUs beg for mercy.

What I ended up using is this:

dd if=/dev/zero bs=100M | gzip | gzip -d | gzip | gzip -d | gzip | gzip -d > /dev/null &

Send a few of these babies to the background and you’ll start seeing smoke coming from your server soon enough.

Start your servers: Intel’s next-gen CPU smokes!

Posted by David Yahalom under Hardware, Oracle

UPDATE: Whoops! Had bad URL in link to Nehalem Benchmarks. Fixed!

AnandTech, one of the best sources of hardware news & reviews,  have just published preliminary benchmarks of Intel’s next-gen CPU - the Nehalem.

Even when the CPU is not using final silicon and being coupled with a very early and unstable motherboard, it smoked the benchmarks suppressing a Penryn-based Core2 quad by as much as %50! In fact, a 2.6Ghz Nehalem is faster than a 3.2Ghz Penryn! Amazing.

This got me thinking about the x86 server market, especially when compared to non-x86 servers such as SPARC or PowerPC machines.
Even now we see that dollar-to-dollar, x86 servers can sometime outperform non-x86 hardware.

Nowadays when Oracle RAC clusters based on cheap $7000 x86 servers outperform single-instance super expensive high-end Sun/IBM servers, Intel’s new Nehalem could very well be the final proof many DBAs need in order to ditch their old SPARC/PowerPC servers and transition to Linux x86 RAC installations.

Anyway, AMD is going to launch a new chip design in early 2009 and I still see great promise in IBMs Cell CPU deign.

Interesting times ahead.

Read more about Nehalem’s architecture and on-die memory controller here.

Oracle Virtual Directory

Posted by David Yahalom under ETL, General IT, Oracle

Another presentation I’ve made not too long ago at an Israel Oracle User Group meeting (sponsored in part by Xpert-One1, the consulting and solution provider company I work for) was about a relatively little known product (at least by Oracle terms) called Oracle Virtual Directory.

OVD allows for Enterprise Level LDAP without synchronization.

Oracle Virtual Directory provides LDAP and XML views of existing enterprise identity information without synchronizing or moving data from its native locations.

OVD can connect to pretty much anything JAVA can connect to and expose several different LDAP directories and RDMBS data repositories as a single LDAP tree.

Think about it. You can virtually “unify” all the different directories in your organizations (be it RDBMS servers or LDAP directories) as a single directory service - which is so much easier to work with. And without any sort of synchronization.

Very very cool.

You can get my presentation here.

OVD is availiable for download, free, from OTN. 

The secure Oracle database - howto

Posted by David Yahalom under Security, Oracle

Last week I’ve held a 45 minute presentation about Oracle-made DB security solutions at a customer convention held by Xpert-One1, the company I work for (Xpert-one1).

It was a pretty neat presentation focusing on how to achieve 360 degrees of protection for your database, how most security exploits originate from within the organization (an IDC study shows as much as %80) and how while network security is well understood (firewalls, VPNs, etc) database security is almost always forsaken (not many businesses expect their DBAs to be security focused).

My presentation covers the following Oracle products and how the fit in creating a secure database:

Oracle Database Vault
Oracle Advanced Security
Oracle Transparent Table Encryption
Oracle Label Security & Oracle Virtual Private Database
Oracle Secure Backup
Oracle Grid Control

You can download my presentation here.

It’s high level, since the audience we were targeting were CTOs, IT managers, security guys and not actual DBAs.

Fell free to leave comments or contact me if you have questions or want more information.

The magic that is DUAL!

Posted by David Yahalom under Oracle

I’ve stumbled across a very interesting write-up at AskTom regarding Oracle’s DUAL table. Some very interesting discussion going there including the origin of the name “Dual” for the obviously singular dual table. Recommended read.

And for some quick trivia. What’s the command masked below responsible for such output?

SVRMGR> select * from dual;
D
-
X
1 row selected.

SVRMGR> XXXXXXXXXXXXXXXXXXXX;
Statement processed.

SVRMGR> select * from dual;
ADDR     INDX       INST_ID    D
-------- ---------- ---------- -
01680288          0          1 X
1 row selected.

To find the answer browse the post at AskTom.