Comments on: What makes open source databases so scalable?

By: Jeff

Jeff — Mon, 27 Mar 2006 01:17:43 +0000

Dennis,
You should read this post:
http://www.salesforcewatch.com/2006/03/recap_philadelp.html
“Benioff: Hi Larry, your Oracle software is causing big problems for my company.”

We don’t really know what the config of the SF Oracle db, but one could presume that they weren’t trying to cut corners here so if the fix is to install “more oracle” then I’m still left with the question of how scalable oracle is in this environment. Basically, anything can scale given enough hardware and software, I’m wondering what gives you the best MPG (or in your case, KPG)?

By: Dennis Howlett

Dennis Howlett — Mon, 27 Mar 2006 00:41:11 +0000

Jeff – SF.com has made it clear their issues were much more to do with operational problems than their use of Oracle DB. That of course may be ‘pressured doubletalk.’ What I do know is that scaling is a function of many factors – not just the DB. You pretty much say so yourself (it seems to me). Paul Strassman (to whom you pointed some weeks back) talks pretty specifically about how Google scales. The DB seems a relatively minor part of the overall architectural equation.

By: Dan Creswell

Dan Creswell — Fri, 24 Mar 2006 09:43:05 +0000

I’m not sure whether opensource databases are any more/less scalable than Oracle, I think it’s all in perception.

(1) I suspect a lot of the new generation of web-based services are in many cases not yet serving big numbers but we have the opposite impression. For example, there was a big noise recently about an RoR site serving 2.5 million hits in a day which everyone perceived as a big deal – but get the calculator out and thats about 30 requests a second which is far from mega demanding.

(2) In the case of Amazon and Google and possibly EBay, they have a different approach to scaling. They do it through compartmentalization of data (partitioning) and careful use of data staleness constraints amongst other things. The norm is to cram everything into one centralized, clustered database whilst Amazon and co create lots of smaller databases and scatter data amongst them thus preventing any single database from taking too much load. They then apply common sense tech solutions like caching. i.e. These guys are using a different _architectural approach_ from the norm whilst using the same _technology_ as everyone else. This difference is reflected in the recent introduction of partitioning to WebSphere indicating that the smart people have had this figured out for a while and now it’s being brought to the average J2EE joe in a nice package.

By: Jeff

Jeff — Fri, 24 Mar 2006 06:10:18 +0000

2 hours a year? Yeah I could live with that. I bet a lot of SFdC customers are wishing they could be so lucky as to have only 2 hours of unplanned downtime.

Regarding the SLA, that’s exactly what FreeCRM.com did, 99.9% uptime or your money back (although for a free service money back isn’t very exciting, but FreeCRM does have a licensed product and they do private label implementations).
http://www.marketwire.com/mw/release_html_b1?release_id=73652

I couldn’t get that survey link to load, but I did find a reference to it in an eweek story. Interesting data.

By: Nitin

Nitin — Fri, 24 Mar 2006 05:08:19 +0000

1. 99.98% uptime translates of 2 hours of downtime a year. I wonder anyone running salesforce.com or rightnow can live with that kind of unplanned downtime. I agree that SFDC has had larger downtimes, but I wish to point out that even 99.99% uptime is not something a SaaS vendor would be too excited about in providing in an SLA.

2. If open source database were indeed scalable, why would they be not present in any top 10 metrics for the largest databases:
http://www.wintercorp.com/VLDB/2003_TopTen_Survey/TopTenWinners.asp

By: Jeff

Jeff — Thu, 23 Mar 2006 20:20:11 +0000

Bill,
First a correction: I am not certain that ebay is running on an open source database. I do know they are a big Microsoft shop, but cannot say with any certainty what dbms they are running.

You point echoes one that I made, which is that Oracle is much more than a database. However, I would disagree with the notion that open source databases are being deployed in environments that are less demanding that transaction processing imposes. As an example, we have about 1,500 implementations of SAP on MySQL and we’re probably one of the more demanding environments for transaction databases. Also, Streambase has been kicking ass in fin services and trading applications (highly demanding environments) and that is built on Sleepycat.

The thing about SAP that is revealing in this debate is that we’re not even using the advanced features in Oracle because we have our own transaction processing, security, management, etc. features as part of the application stack. You could run SAP on Oracle 7 just as easily at 10g.

Finally, Salesforce.com is running on Oracle and has experienced significant outages in the last year, RightNow is running on MySQL and has published uptime of something like 99.98%. It would be hard to argue that Rightnow is less intensive of an app than SFdC.

By: Bill Burnham

Bill Burnham — Thu, 23 Mar 2006 19:30:16 +0000

I like bashing Oracle as much as the next guy, but the reality is that the reason OpenSource DBs may *seem* more scalable is that people generally use them for the easy stuff. The front ends of big web sites tend to scale nicely thanks to the fact that they are often heavily replicated, read-only databases with huge caches. The queries they do handle tend to be garden variety “fire and forget” stuff. That doesn’t compare to a DB in a highly transactive environment that is processing complex read/writes for multiple applications across large numbers of multi-deminisional tables. I’ll bet Amazon, Yahoo, et. al. aren’t running their finance DBs on top of an open source engine. That’s not to say that open source DBs can’t eventually compete quite well in this arena (and a few already do), it’s just to say that the perception that opensource DBs scale better than traditional DBs is a flawed because open source DBs tend to be deployed today in far les complex environments. Send them into the core enterprise and they will have to do the same work as the traditional DBs and with either scale about the same, or more likely, much worse, because they haven’t been optimized to do work in that kind of an environment…yet.