Columnar Data Storage

Hasso Plattner, SAP
Image by dfarber via Flickr

At the SAP academic research conference yesterday Hasso Plattner spent a lot of time talking about database design and why it’s still important. More significantly, he drilled into why re-architecting applications to take advantage of a fundamentally differently database than what we are used to with relational databases is critical if we are to simplify code bases and develop new generations of applications that take off where the current state of the art ends.

This area has a pretty steep learning curve so to get started here’s a good explanation of the distinction between columnar vs. row database architecture.

Reblog this post [with Zemanta]

Microsoft launches its alternative to Amazon’s SimpleDB

It was long thought that open source would represent the biggest challenge to traditional database vendors, but fast forward to today and the law of orthogonal technology innovation kicks into high gear as hosted databases go from a “wouldn’t that be pretty cool” to very real offerings.

Microsoft has begun signing up testers for SQL Server Data Services (SSDS), a forthcoming service that will allow customers and developers to host their data in a Microsoft-hosted database. So what is (and isn’t) SSDS, exactly? [From Microsoft launches its alternative to Amazon’s SimpleDB]

While the idea of taking down Oracle’s database business is indeed appealing to me, I doubt that will be the outcome of both SimpleDB and SDSS (seriously, why the hell can’t Microsoft acronyms as product names habit?) because for the time being the core enterprise software market is still oriented around on-premise offerings.

However, two interesting things have happened in recent years… I can’t think of a single company that has built a new product or company on a BEA stack or with an Oracle database or any other proprietary software stack. Open source technologies get their fair share of attention but if you were to poll 100 startups that have formed in recent years you would find a significant number of them are built on Amazon Web Services.

Secondly, non-relational database technologies are making inroads into the traditional enterprise market. StreamBase is one example, founded by one of the fathers of relational database technology, Mike Stonebraker, the company has been focusing on complex event processing in financial services. SimpleDB itself is another example, representing a dramatic departure from relational database systems.

These services are starting to make their way on to enterprise desktops and just like was the case with open source we can expect that enterprise osmosis will bring them into IT. Salesforce.com is also playing a role in this as well with their Force.com offering, which combines data storage with application development tools.

I haven’t seen any acceleration of Force.com in the marketplace, but this is probably less about the technology and more about the peripatetic approach to marketing it, which certainly hasn’t been helped by their identity crisis that drives the confusion around what name is it being called this month.

Give it a couple of years and I strongly believe that big enterprise IT shops and systems integrators will be enthusiastic users of these new hosted infrastructure technologies if for no other reason than the cost of building with them and maintaining them as needs scale is a fraction of on premise infrastructure.