In the last couple of months I’ve come to realize that Object Storage is the worst thing you can recommend to an average IT organization (or even to a reseller).
And yes, I know, I’ve been trying to convince all my readers and clients how cool it is in all its aspects and potential… but when come to reality it’s not true!
Don’t get me wrong, I’m still a strong supporter of it, but It’s usually very hard to understand and sell this technology and it needs a different, smarter, approach… just only from the marketing point of view!
Users want solutions, not backends
Object storage does not make sense for most organizations out there. Accessing storage via APIs is out of reach for most traditional IT shops and, even when you have developers, they want to access data via NFS/SMB (or a local file system) because they are used to doing it in that way. Inertia is so strong in status quo changes that it will take a long time before we can actually see a real evolution on how we access our storage infrastructures (I mean access storage via APIs here).
On the other hand, IT organizations have issues with storage and they constantly looking for solutions. Object storage is not a solution per se, but when coupled with the right tools (gateways, interfaces or specific software) it becomes a really compelling solution for a number of reasons!
It’s all about simplification (of the message and the infrastructure)
If you have had a conversation with a CIO or an IT manager lately, one of the most recurrent words in their talks is “simplification”! And, even more so, if you talk to someone who manages storage, Simplification (with a capital S) is above everything.
Of course, complexity is always to be avoided but now, with this explosive data growth that most organizations are experiencing, infrastructure TCO can explode in your hands too.
Simplification is one of the advantages of Object Storage. A few years back (in 2010) I wrote an article where I stated that object storage would eventually become the next unified storage: traditional protocols (e.g. NFS, SMB or iSCSI) alongside new services and access protocols. I’m still very confident about the idea of a horizontal platform being useful to implement many different vertical services. You can call it an ecosystem or a set of end-to-end solutions.
Please talk about solution-based storage (or cloud storage)
From this point of view, talking about solution-based storage is not that weird. Companies like HDS started years ago building a smart ecosystem which can now bring a set of ready-to-deploy enterprise solutions where cloud NAS and sync&share are only the most visible of a broader catalog.
Others like EMC, started later but are now doing a pretty god job of buying up startups like Cloud scaling and Maginatics (just to mention the last one) and building a similar ecosystem.
Many startups are building their sets of native and third party solutions to compete at best too, but it looks like some of them are doing better than others (Scality is a vivid example here, but others like Caringo or DDN are adding similar interfaces as well).
In any case with all this interest in private cloud, you can’t avoid thinking about storage. Implementing enterprise cloud storage can be very easy at this point, since it helps to simplify the storage infrastructure while guaranteeing an impressive ROI.
Solution-based Storage is much more a SME thing than you think
Solution-based storage is not only for the larger enterprise. Companies like Cloudian, for example, starts with 10TB licenses and others, like DDN, now have solutions that start at about 200TB. Fortunately, it’s no longer an at-least-1-PB-to-start-talking story… it just doesn’t make any sense! (never did! IMHO)
Some of the vendors now are also packaging their software in appliances which are affordable and easy to deploy by relatively unskilled sysadmins/resellers in minutes.
Furthermore, vendors like HDS and Cloudian have began to show cloud tiering functionalities (leveraging on-premises installation and public cloud at the same time). In this case a small enterprise can contain the cost by deploying a small on-premises infrastructure for critical data and grow on public cloud for other necessities.
SMB market is one of the most interesting because the number of opportunities. If simplification is the goal in larger organizations, it’s even more important for the SMB where resources are fewer and usually less skilled. At the same time, for this kind of enterprises, adopting new technologies is historically easier than for others. If I were an object storage vendor I would sell my solution to thousands medium sized organizations instead of a dozen of larger ones. All of the smaller ones will incessantly grow in the future, and if they adopt my object storage backend now, sooner or later they will buy more!
But Petabyte-scale storage is the future
Even though I’ve just said that SMB is a good market for Object Storage (or better, solution-based storage), we can’t ignore the potential of object storage in huge multi-Petabyte installations. Companies like Scality, DDN and Cleversafe can all claim huge installations (and some of them are really impressive like the story recently presented by Scality with the 500PB at Los Alamos National Lab). They are very few now but the future is there… the only problem is to understand how distant this future will be. In any case this will remain a very high end market for few vendors with very high scalable, feature rich and rock solid products. Take it with a grain of salt, but you could consider them the VMAX-like class of object storage!
Everybody loves Ceph and Openstack
Ceph, which is now backed by RedHat through the InkTank acquisition, is becoming more and more relevant in many talks with vendors and end users. I’m not saying that a lot of users are really implementing it in production (well, some of them do actually, but it’s not the majority at the moment). The interesting thing is that some people thinks about Ceph as the right storage platform for Openstack (indeed, so do I) and as a viable option for object storage implementations.
Even though, at the moment, it is relatively limited in performance scalability (due to a poorly implemented gateway to serve S3 and Swift APIs) the development of the product is going in fast forward mode and it would be very interesting to see the next stage.
For example, vendors like Fujitsu, who don’t have their own object storage solution, are now starting to propose Ceph-based products to address Multi-Petabyte and/or Solution-based storage needs. I don’t know wether they’ll be able to succeed or not but there is some potential here, especially if community grows and RedHat will be able do its part. (regarding the particular case of Fujitsu, I have my concerns about this specific solution, which are somehow very similar to what Storagebod wrote in his blog, but that’s another story)
Ceph is quickly becoming the biggest Swift (and SwiftStack?) competitor but, at the same time Swift APIs are also getting some attention. In fact, if you look around, many object-storage vendors are now working on Swift compatibility too. This doesn’t automatically translates in real world business today, but it will certainly help to be more integrated with Openstack tomorrow.
SwiftStack (the startup that is proposing an enterprise version of Swift and founded by the inventors of Swift APIs), is not ready to compete neither in the general purpose Multi-Petabyte nor in the Solution-based storage market but it has recently raised another $16M. This, in conjunction with the interest on Swift APIs, will be sure of great help for building a more mature and relevant product with gateways and interfaces built from third parties.
Private cloud (storage) and final notes
Object storage is one of the most interesting pieces of private (and hybrid) cloud implementations. It’s totally undervalued but it always has the best ROI and TCO when it comes to big multi-purpose storage implementations. In some cases, not now but probably in the near future, it might be considered also for VM-storage when coupled with the right server-side caching mechanisms. (Scality has recently demonstrated this capability and I would like to see it seriously implemented with a solution like Pernix!)
Object-storage isn’t selling? It’s only a marketing problem. As I mentioned at the beginning of this post, end users don’t buy back-end technologies but solutions to their problems. When you talk about multi-Petabyte or Solution-based storage the end user changes his mind and start to see object storage in a totally different light. For example, the success of some startups like CTERA have been made possible because they propose a Solution-based storage but if you look into their architecture it wouldn’t have been possibile to do it without leveraging object storage at the back-end.
Disclaimer: I’m working, or have recently worked, with some of the companies cited in this post
Fun fact, Ceph is still not a 1.0 product. Let the ramifications of that sink in.
Well, Ceph’s commercial sponsor InkTank cashed a $175M check from Red Hat at the end of April, so maybe not having that 1.0 release didn’t cost them much. SwiftStack is actually funded to the tune of $24M. I don’t think they were particularly looking for the most recent $16M in funding, but I guess it is hard to say no and better to take it when you don’t need it…yet.
I agree that you don’t sell “object storage” per se. You sell what you can do with it to solve data storage and management problems. Solution selling is a practice that is well understood by vendors and their channel partners. I like the message of “storage simplification” as it addresses the fact that NAS and SAN environments are complex and need skilled management. They are also expensive and must be replaced every 3 to 5 years as the vendors bring out the new and “end of life” their old storage systems. They also were never designed to cope with the explosive growth of “unstructured data” being generated by users and machines.
I’ve noticed that some object storage software vendors have carved out particular niche markets or verticals. Cloudian, which has received about $45M in funding, has set itself apart from these niche players in several important ways, which I think will eventually make them a very successful player in the object storage market. First, they support both storage providers and enterprise customers. Part of this comes from their heritage working on mobile data projects with carrier class providers. Second, Cloudian clusters can start on the very small scale…tens of TBs and scale to hundreds of PBs. This allows Cloudian to reach down to the smallest SMB and up the largest enterprise customers. Third, Cloudian offers an appliance based cluster building block which is built for them and supported by them. This is the classic “turnkey” approach to making it easy to deploy and support a storage cluster. Fourth, Cloudian is laser-focused on AWS S3 compatibility. AWS has the largest third party ecosystem for cloud related solutions. Anything that works with AWS S3 can almost be guaranteed to work with Cloudian. The network effects of AWS are huge and Cloudian customers can derive direct benefits by being able to tier data to AWS S3 or Glacier from their Cloudian cluster.
Ceph/InkTank/Red Hat and Swift/SwiftStack are not as market-ready and do not have a broad range of solutions because they only tepidly support AWS S3 in addition to their native APIs, which have not seen large adoption yet in the object storage market.
“In some cases, not now but probably in the near future, it might be
considered also for VM-storage when coupled with the right server-side
caching mechanisms. ”
That is exactly what the open-source project Open vStorage (http://openvstorage.com/) does. It turns object storage into block storage. It uses SSDs & PCIe flash inside the host as Tier 1 storage and accumulates multiple writes (log based approach) into a larger object and pushes that towards the object store (Swift or Ceph) as Tier 2 storage. The near future is already here 😉