Don’t be Fooled by Oracle’s Smoke and Mirrors

Conor O’Mahony, an IBM employee, closely watches Oracle. In this post, Conor examines Oracle’s recent announcements related to its SPARC line of products. You can read more from Conor on his Database Diary blog.

This week, Oracle hosted a video and issued a press release with a series of announcements related to its SPARC line of products. I am going to take a few moments to examine those announcements.

Interestingly, during the course of their announcement Webcast, Oracle frequently used the purported success of their Exadata and Exalogic products to back up many of their assertions about the new SPARC offerings, even though neither Exadata nor Exalogic use their SPARC systems (at least not yet anyway). Also interesting was that they spent so much time talking about the SPARC SuperCluster and their latest SPECjEnterprise2010 benchmark result, yet it turns out that Oracle did not actually use their SPARC SuperCluster for either that benchmark result or their TPC-H benchmark result. They assembled a system that is similar to the SuperCluster, but it is not the SuperCluster. (Instead of using the ZFS storage appliance, this benchmark uses F5100 flash arrays to store the database completely on solid state drives.)

These tactics are symptomatic of the smoke and mirrors that plague these announcements. Let’s see what they actually announced:

• They announced a SPECjEnterprise2010 benchmark result⁽¹⁾ that beat IBM’s best performance result in the same benchmark by over 2.4 times, delivered 20 percent higher performance per processor than IBM POWER7, and supposedly delivered 6.7 times better price/performance. Let’s have a look at these claims. Yes, Oracle do have a bigger benchmark result, but they also used a lot more hardware. Here are the details:

  Item	Oracle	IBM	Comments
  App Servers	4 x 32 core SPARC T4-4	1 x 64 core Power 780	Oracle had 2x the number of cores
  Database Servers	2 x 32 core SPARC T4-4	1 x 32 core Power 750 Express	Oracle had 2x the number of cores
  Database Server Memory	2TB (1TB per server)	512GB	Oracle had 4x the amount of memory
  Storage Tier	8 Sun Fire 4270 M2 servers (48 cores)	DS5300 Disk System	Oracle had 48 additional cores for database I/O
  Storage Tier Memory	8 x 8GB of memory for added caching	8GB of cache	Oracle had 8x the amount of disk cache
  Total HDD	72 x 2TB drives = 144TB	128 x 146GB drives = 18TB	Oracle had 8x more database storage
  Total SSD	16TB	None	Oracle used 16TB of SSD for database; IBM used none.
  Database Integrity Checking	Turned off	Left on

  You can see that Oracle didn’t just use twice as much processor cores for this benchmark run. They also used a lot more server memory, additional storage processors, more storage memory, more hard disk drives, and solid state drives. You certainly need to take all of this into account when considering the relative performance of the systems. As well, of course, as the fact that they disabled integrity checking to boost performance, which is something that I doubt anyone would do in a production system.

  The other thing to be aware of is that Oracle very cleverly derived their own independent pricing analysis. And that pricing analysis did not compare the total costs for these systems; instead it compared only the application server portion of the costs. In other words, they chose to not consider the cost of the database software (which includes licensing both Oracle Database and Oracle RAC on 64 processor cores). If they did add that cost, the conclusions would have been very different. You can be sure that they took these steps for a very good reason

• They announced a TPC-H 1000GB benchmark result⁽²⁾ that is currently in 10th place for performance. They then asserted that they “delivered over 2.4 times better performance per processor at 1/3 the price/performance than an eight processor IBM Power 780.” However, you should note that the IBM Power System’s result was not running IBM DB2, which of course is optimized with deep exploitation of the Power Systems platform to drive faster performance. The Power Systems result they refer to uses Sybase IQ.

  Also, if you look at the ratio of storage to database size, you will notice that the Oracle system needed more than 10 times more storage than the database size. Whereas the IBM result needed less than 4 times more storage than the database size.

  I also believe there is an error in their claim; the price/performance for the Oracle system is $4.20 ^{(see UPDATE below)}, the price/performance for the IBM system is $6.85 USD, which is actually 2/3 the price/performance and not the 1/3 Oracle are claiming. Their cost comparisons are even more troublesome when you consider: Sun and Oracle TPC Price/Performance Tactics Revealed.

• They announced a number of benchmarks for their own applications software. These are not industry standard benchmarks, but Oracle benchmarks based on internal testing.

Make no mistake, Oracle are aggressively trying to stem the tide of customers moving off of SPARC. To understand their desperation just look at the following chart showing the Unix market share numbers over the past decade:

In 2010, IBM Power Systems successfully displaced 712 Oracle/Sun installations. In the first half of this year, IBM have done this 362 times. I would love to hear from Oracle regarding how many customers moved in the opposite direction. For more details of this momentum, make sure to visit the Power Systems Web page.

Oracle knows that IBM Power Systems have superior performance, reliability, availability, security, and virtualization. Oracle knows that IBM Power Systems currently lead in a number of other industry benchmarks that they failed to mention. They know they have a fight on their hands to play catch up when it comes to the SPARC processors, and much of what they are doing right now is simply noise to cover up the real story.

UPDATE: For their TPC-H benchmark, Oracle used a 3-year term license, rather than a standard software license. They also used a special incident support offering, rather than their standard software support. If Oracle used their standard software licenses and their standard software maintenance, the price/performance for their TPC-H benchmark result would have been $7.91 per QphH (rather than the $4.20 per QphH they reported).