Well, the title is a somewhat strong (and full of buzzwords) maybe, but it’s just a way to get your attention… and I have to admit that finding a title for this post wasn’t easy.
Lately I’ve been working with various Object Storage vendors, and I’ve collected some interesting information about what you can expect from object storage solutions in the near future (some of which are already available) and what it will mean for your enterprise hybrid cloud storage infrastructure.
Two trends (preface)
There are two visibly opposite trends in modern IT: moving data closer to the CPU or moving data closer to the network. They are not exclusive to each other and, on the contrary, they can be considered two faces of the same coin. (I dealt with it here in this blog).
In the first case Flash memory is the enabling technology and a lot of effort has been made to:
- shrink latencies and increase IOPS,
- implement modern architecture with strong caching capabilities,
- mange data locality (VM data is organized to stay close to its CPU, like some VSAs for example).
The final goal, as always in this case, is to perform a greater number of operations/transactions as quickly as possible. Hence this part of the infrastructure is optimized for $/IOPS.
In the other case, the needs are totally different and are related to file and content management of all sorts. Access is wherever and on whatever device, something traditional storage is not able to cope with… and Objects (files+metadata) are the key. Due to the nature and the quantity of data to manage, the first unit of measure here is $/GB.
In my opinion, in the near future we will be seeing more and more enterprises adopting this dual Flash+Objects strategy.
Why not Objects
There are two main problems with Object Storage today (perceptions, I might say):
- suspicion: it’s a totally different approach when compared to traditional storage.
- misconception: it is considered a very high end storage solution for ISPs/CSPs, web 2.0 companies and larger enterprises.
They are both totally off track, but sometimes not even the vendors can contribute in helping to change the end user’s mind/perception.
In any case, things are changing and bject Storage platforms are becoming more interesting for mid sized customers as we’ll.
Cheap becomes cheaper
Object storage is cheap but, sometimes, its not cheap enough to justify the migration from NAS boxes.
In fact, Private Object Storage is not cheap per se. The cost of the infrastructure (Object Storage+Gateways+Software+…) is high at the start and most of the savings, both in terms of TCA and TCO, kick in when the infrastructure grows beyond a certain number (1PB could probably be considered as the break-even point, and this is why most OS vendors primarily look for customers with 1+ PB of data 😉 ).
On the other hand, services like Amazon S3 are cheaper (and they will get even cheaper in the future) but they also come with all the issues of the public cloud.
Things are changing though.
Best of two worlds
Some Object Storage vendors are working on new features that will bring the pros of the two worlds with the promise of mitigating the cons. Long story short: a hybrid approach that can be translated with:
- maintaining a complete control of the infrastructure and stored data;
- having access to the right resource (and the right price) in function of the needs;
In practice we are talking about integrating object storage systems installed on-premises with public cloud services.
Just a brilliant idea
The idea is quite simple and can easily be summed up with the ability to maintain control over data and, above all, metadata while the physical store location can be managed through specific data-placement policies.
In this case, (please, let me use the buzz-word) we could talk about “Software-Defined Object Storage 2.0″ where the control plane (data+metadata management layer) and data plane (the physical storage) are separate.
It simply means that you can build your (small) private object store and you can use the public cloud(s) if and when needed.
Compatibility with protocols like S3 becomes fundamental to communicate (on both sides!)
Two possible scenarios (just the beginning)
The first two different scenarios that come to my mind are auto/manual tiering and DR.
In the first case we’re talking about the expansion of a local object storage system to the public cloud and moving less accessed data there with the intent of taking advantage of the low cost (thanks to the cloud war between Google, Microsoft and Amazon).
A second opportunity comes from the DR (that’s more complicated indeed, but it’s theoretically possible). A copy of all objects could be maintained on the cloud and it would be a great saving when compared to an on-premises full redundant solution.
Why it matters
All this kinds of solutions will lead to a certain democratization of Object Storage and more enterprises will be able to adopt it.
You can start small and stay small by growing to the cloud while maintaing full control over your data! Your most accessed data will remain local while cold data will automagically migrate to cheaper locations.
At the same time, you can take advantage of the pay-as-you-go model of the cloud while mixing CAPEX and OPEX in function of budget, business and technical needs.
Last but not least, your storage infrastructure will be more agile and flexible than ever, it will be transparent for your users and maybe for you too! 😉
Interesting! Thanks for sharing.
How is data security / theft addressed in this ‘tiering’ approach? Is it Cloud Provider dependent (Encryption, controlled physical access to DC etc.) or does the Software-defined object storage model govern that?
best,
Shaloo
Shaloo,
as I told you on the other thread, implementations are at the beginning and there are some tradeoffs (it any case implementation of tiering varies from vendor to vendor). 😉
Hi
I completely understand the topic. Did you already have look at the open-source project called http://openvstorage.com/ . It allows you run VMs on top of an object store and uses local flash inside the host as caching. Sounds very much inline with your thoughts.
Kind regards,
Wim Provoost
Product manager Open vStorage
No, but sounds interesting and I’ll check it out ASAP.