Cloud Computing: What should I do?

single white cloud on blue sky

“I know Cloud Computing is the ‘in-thing’, so what should I be doing in Cloud Computing?” This is a question that people often ask me when I tell them that I started a company aimed at Cloud education. The answer to this question is again a question, “Who are you?”  Lest you think I am being rude there, let me say that what I want to know from the person is the role that he / she is currently playing. The answer I give will depend on it.

Let me try and answer this for various roles in an Enterprise starting from the very top one. This is ofcourse a 10,000ft view and there will be lot of finer detail that I may not have covered. Yet I think this will give you a good idea of the direction you must take in case you want to work in the Cloud.

1. CIO:  It should be quite obvious that lot of CIOs would be under pressure to either move to the Cloud or atleast explore the possibility of moving to the Cloud. The CIO ofcourse has to look at the long term pan of the company and work out the economics. So if you are CIO you should be know about the various deployment models of the Cloud (Public, Private, Hybrid) and the various service models (IaaS, PaaS and SaaS). You should be able to understand the economics thoroughly (calculating the spend on Cloud is not as easy as you think.) Also the SLAs offered by each provider, their reputation for high uptime and security of your data need to taken into account. The call which the CIOs will generally need to make is whether they want a Cloud model or if they don’t want to manage infrastructure, should they go for Managed Services. To make this decision, they need to understand the difference between a Private Cloud and Managed Data Center. From what I hear and read both have their own advantages and disadvantages. A lot will depend on the applications that are being used by the Enterprise. Needless to say a long term vision about the Infrastructure and getting the best value for money would be the CIOs aim.

Architect : ‘Cloud Architect’ has different connotations depending on where you are working or would like to work. If you want to be a ‘Cloud Architect’ with a Cloud Provider you need to understand the nuts and bolts of how the Cloud is formed. A Cloud Architect in a Cloud Provider space sees less of the Cloud and more of the infrastructure. So while you need to have a clear idea of what Cloud means to the consumer, you must be well versed with Data Center technologies. You should be an expert in either Server Virtualization, Storage or Networking and should be able to understand how Software-Defined-Anything (Storage, Server, Networking) works. Additionally understand the newer technologies like Containers (Docker) which is being used in the Cloud context. Your job will be to Architect the infrastructure to ensure optimal and efficient use. So try and be a domain expert in one of the areas I listed above : Server, Storage or Networking. Depending on the Cloud Provider you may want to become an expert in OpenStack in case of IaaS providers and Cloud Foundry in case of PaaS providers.

If you are a ‘Cloud Architect’ in a company which wants to consume the Cloud, the expectations from you are different. You need to understand the infrastructure services provided by the Cloud Providers and plan for migrating your applications to the Cloud. If will need a good understanding of the services provided to perform efficient migration. You can also plan on developing some of your applications on the Cloud itself and for this too you need to have a clear idea of the services offered by the provider. For example, if you want to migrate your applications to Amazon AWS you should probably think of getting yourself Amazon certified, which will force you to read and understand all the offerings of Amazon.

Similarly if you are more interested in PaaS for your development team, you must understand the offerings from various PaaS vendors whether it Google App Eng, Microsoft Azure, Amazon Elastic Beanstalk, IBM BlueMix or anyone else. You need to understand the IDEs they provide, the language support they provide and how easy it is to deploy your application.

Project Managers / Technical Leads:  Understanding the deployment scenarios and service offerings of the Cloud Vendor is key. Understanding the economics and keeping a good grip on the money spent will be a major task. For like virtualization, here too we can have a sprawl due to the easy nature of provisioning a VM and the ease at which you can consume any service. So understanding how each service of the provide will put a drain on your exchequer is important. For the final control on the developers lies with the Project Managers and Technical Leads. Understanding the infrastructure and your own application well will let you realistically estimate how easy or difficult it is to migrate to the Cloud

Developers: Even before you understand the Cloud, ensure you can code well in one of the languages used for web services: Ruby, PHP, Python or Java. (You have Node.Js and GO as well but one of the four should suffice for now). Once you have mastered the language it will become easy for you to use any API and interact with the Cloud. You will be seeing all the services offered by the Cloud Provider from a programming perspective and once you understand the APIs you can develop lot of programs based on the Cloud. Overall understanding of Cloud Computing is required also with an inquisitive mind and a good grip on a programming language. Given that providers like Amazon AWS give you free tier for a year and the APIs are readily available, an enterprising programmer can develop her own application on the web in a very short time. If you are going to develop Enterprise class applications you need to understand the three-tier application model and some of the frameworks depending on the language you chose.

Administrators: Amongst different categories the Administrators have the most to learn. Again we have two types of Administrators here: One at the Cloud Providers premises and one at the Consumer premises. If you are the Consumer, then you will need to understand how to use the Management Console or CLI of the Cloud Provider. For example, if you apps are hosted on Amazon you should know how to use the Amazon Console and Amazon CLI to manage your apps in AWS. If it is a different provider then you must understand their UI and CLI. System Admin certification of Amazon AWS and CloudStack certification may be helpful

If you are an Administrator at a Cloud Provider, then as with the Architect, you need to be a specialist in Storage Administration, Network Administration or Server Administration. You must thoroughly understand the concept of Virtualization and how it is applied to Storage, Server and Networking.

Read about various server virtualization technologies like VMware, Xen, KVM, OVM, Virtual Box etc. if you are a Server Admin. Also understand the provisioning tools like Chef. Puppet, Ansible. Also understand container technology like Dockers and tools like Vagrant which allow you to launch VMs.

If you are a storage administrator understand how storage virtualization helps, understand replication and backup technologies. Storage is a very very important part of the Cloud and maintaining the SLAs wrt Storage is a major challenge. Understand what Scale Out Filesystem is and why they are useful in the Cloud Data Center.

My knowledge of networking is not great so I will refrain from giving lot of advice but surely do try and understand the concept of Software Defined Networking.

Hope this gives you an idea of what you must be concentrating on. Hopefully your journey to the Cloud will be smooth.



Recent interesting acquisitions in Storage Space

When there is great growth in an industry, you would expect the demand would to spur competition and we would expect the customer to have more choices and more vendors to procure from. I guess this works only upto a certain scale. Beyond which the opposite, consolidation of vendors,  happens. That is what I see happening in the Storage Industry  now. The demand for Storage is on the rise. Every company is showing wonderful results. Demands for newer technologies is also on the rise. In such a scenario, we are seeing lot of consolidation happening. So market growth leads to shrinking vendor base? I am sure there is some management theory explaining this phenomena. As to when consolidation happens in an industry etc.

These thoughts came to me when I look at the recent happenings in the Storage space. We saw Data Domain being bought by EMC last year. This year there two very major acquisitions. One was HP fighting off Dell in order to acquire 3Par Technologies. HP wanted an array like that of 3Par in their portfolio and went for it aggressively against Dell. It was a $2b + acquisition. 3Par has some nice technology and were quite well known for techniques like Thin Provisioning, Micro RAID, Wide Striping etc. There were getting noticed in the market and had a decent customer base. Everyone feels that this acquisition will help HP immensely in the Storage market.

The second acquisition which has a lot of people talking is that of EMC planning to acquire the Scale Out NAS vendor, Islion. This will also be a $2b + deal. From the comments I see, like HP with 3Par, this is also a buy to fill in a gap in EMC’s portfolio. The general opinion is that the current NAS product of EMC, Celerra, doesn’t scale up well and hence the need to buy a scale out NAS product. EMC was lacking a scale out NAS while the competition had their products. HP has both PolyServe and IBRIX. (Polyserve, btw, had lot of people from the erstwhile Sequent Computers and is based at Beaverton, Portland, Oregon. Some of whom I know), IBM has its Scale Out NAS (SONAS), NetApp has its own scale out product. So this product ensures EMC is also playing in this space.

The other interesting acquisition was IBM acquiring Storwize, a company involved in Primary Data Compression. Storwize had a compression appliance for NAS. This appliance would compress data before it was stored on disk. IBM after acquiring Storwize released a product called IBM Storwize v7000 Storage Array. The funny part was that this array had no Storwize technology in it!! It seems that IBM wants to brand its arrays as Storwize arrays and so only the name was used.

Other interesting acquisitions happened in the Database area. EMC acquired the company Greenplum, which is “massively parallel processing database platform” and IBM acquired the database company Netezza. Both these database companies were involved in building databases for high performance business analytics.

Most of these acquisitions happened keeping cloud in mind. Also on the back the mind of all traditional Storage companies is Oracle. Oracle now has Sun, Sun StorageTek, Virtual Iron and Exadata. And of course, they have their database. They do pose a serious threat in the Storage space. There was. for a brief, while a talk on whether they would acquire someone like NetApp to grow in the Storage space. You never know what will happen!!!

As I said in the beginning, while the Storage market is expanding, the vendor base is getting consolidated. Innovative startups and small companies with good track record are being gobbled up by the big players. So you eventually will end up with only the big guys in the fray.

Why ‘Inception’ is a no-brainer for Storage folks


If you haven’t heard about Christopher Nolan’s ‘Inception’, you must be living in your own dream land. This was probably the most hyped up movie after ‘Avatar’ and definitely much more discussed than ‘Avatar’. Just in case you are one of those who hasn’t seen or heard about ‘Inception’, here is a gist of that movie. The movie plays as a dream in a dream in a dream format. There is a debate about whether the dream level is 5 or 6 but we will not go there. The premise of the movie is that there are certain ‘dream thieves’ who can penetrate anyone’s dream and can plant an idea. On waking up the person thinks it is his own idea. This is ‘Inception’.

Fans of Christopher Nolan make take issues with the title of this blog post. How is it that I am so casually saying that the ideas presented in ‘Inception’ are no brainer for Storage folks (and Virtualization folks) as well? Wasn’t Nolan’s movie one of the more complicated ones in recent times? Can I offer some proof to support my assertion? Well, here you go guys and pardon any bad puns along the way.

First, lets take the Virtualization angle. And let us take a simple case. We have an hypervisor on which an operating system runs. This is like the first level dream state. As I was trying to practice something on VMware ESX but did not have the hardware, I ran across an article which says that ESX can be loaded in a VMware Workstation!!! Now check this out: You first load the operating system, say some Windows OS, on top of it you load VMware Workstation. In this workstation, you load VMware ESX. On top of this hypervisor you can load your operating system. This could again be Windows!!! Unlike Nolan’s linear dream, we are in a circular dream four levels deep!! And if you want to add to the complexity, you can run a virtual appliance inside this Windows OS, which runs on the hypervisor, which is loaded on top of VMware workstation, which is loaded on Windows OS!!!! There you go. And according to the article I read, this is supposed to be a valid use case for testing and training!!! (Of course no one would want to use it in production.)

Let’s now shift to the Storage world and this dream in a dream becomes very commonplace. First, lets take the most basic unit, the LUN. This itself is a virtual entity which is carved out of multiple disks (configured as some RAID). Sometimes you may combine these LUNs to form a Meta LUN or you can present multiple LUNs to the server, which can combine them using Volume Manager. These combined LUNs can then then be split up again and assigned as single disks!! If you have a virtualization equipment in the network, that adds to the fun. You can have HDS USP or IBM SVC or NetApp N Series, for example, and these will take LUNs from heterogeneous storage, combine them, split them and present the modified LUNs to the server. Again a case of dream within a dream within a dream. These are not some theoretical use cases but rather daily use cases.

When you combine Virtualization with Storage, the whole thing goes haywire. You can have your basic LUNs on your arrays, these can be presented to a virtualization like USP, which can combine multiple LUNs and present it to the hypervisor for its datastore. The Hypervisor in turn uses this Datastore LUNs, carves up smaller disks from it and then presents it to the operating systems running on the hypervisor. Assume one of these Virtual Machines is running a virtual appliance, then the disks assigned to the operating system is further divided and a small portion is presented as a disk to the virtual appliance!! If you try explaining this to Nolan I am sure he will stop making movies with flashbacks or dreambacks and stick to linear storytelling!!

You can also argue that when the dream is on, the underlying reality may change and you may wake up at a different than when the dream started. That can happen here as well. Many of the virtualizing equipments like USP or EMC’s Rainfinity will move your LUN from one physical storage to another one without the user realizing that the underlying reality has changed. In short, Virtualization and Storage work most efficiently when the user is in a dream state and has no clue about the actual reality!!!

But… you say, Nolan’s movie was about sowing an idea. What idea is being sowed by Server and Storage Virtualization. According to me, it is the idea called ‘Cloud’. As in ‘Inception’, someone has managed to sow the idea so deep that every vendor thinks it is his / her own idea !!! Will ‘Cloud’ work like a dream or is it just a dream. I cannot answer yet as I am not sure if I am in a dream or not!! The protagonist Cobb explains in the movie, “You never know the beginning of a dream”. Since I am yet to find out the beginning of the ‘Cloud’ trend, I am unsure of myself!!! So, as far as the ‘Cloud’ is concerned, the totem is still spinning. In a few years we will know if the spinning stops. Till then….

The shape of the cloud : Public or Private

Cloud over Kinner Kailash

” That wall has to be bigger if we have to use this pattern”, said my painter, “it will not be very effective on your wall”. This is something which happens to many of us all the time. With our painters, our carpenters, our architects and various other service providers. You want to do something but the person providing you the service has a comfort zone in which he / she wants to work. So rarely do you get exactly what you want. It will always be a compromise between your ideas and the ideas of the vendor. This is something which happens not only for individuals like us but also for large enterprises. You cannot execute everything on our own. You need to depend on the vendors. What vendors will say depends on their comfort zone, what products they have, what they want to seel and how much commission the sales guy is getting for selling a particular product. I guess anyone who has been in the industry long enough knows this. And these are exactly the factors which will influence the way the cloud would shape up eventually.

Last year there was a lot and lot of talk about the cloud, which is still continuing. Initially the talk seemed to be about one type of cloud, owned by a service providers like Amazon, Google etc, whose services all enterprises would avail of. Slowly it emerged that there two types of clouds. One, the public one. Which you see when you step out on the road. And the private one, which you see on your own ceiling. In other words, Public Cloud is the one which is run by a service provider and Private Cloud is the one which is run by your own IT department.

People may ask as to what Private Cloud means. After all has the IT department not been running the data center all along. How does the Private Cloud change things? I am not sure if the definition of Private Cloud has taken a concrete shape yet but here is my take. Private Cloud represents a huge change in thinking on the part of the IT department. You no longer need to buy things for each department separately or you don’t need to reserve resources for any one department / division or whatever is the unit of classification. The whole idea is to give resources when necessary, provision just the right amount of resources and give more resources when asked for. In essence the IT department owns all the resources as a huge pool, or cloud, and provisions as required internally. No more buying or provisioning resources for a particular division, which will be locked down for a long time. This would apply to all kinds of resources, the key among them being compute power and storage. Given the ‘give only as much needed’ philosophy and sharing of resources across the organization will definitely bring in optimal usage and definite cost savings.

Changing from one model to another one is not an easy task. So what are the likely challenges for such a movement? Since I haven’t handled big data centers, I cannot possibly give you all the scenarios, but these are most likely challenges that an organization would face when trying to build its Private Cloud:

– Change in the thinking pattern of the IT staff. They must forget how they procured and provisioned earlier. They need to think in terms of how to provision using the cloud paradigm

– Training the IT staff in newer technology areas like Virtualization and newer ways of provisioning which we will important in order to build a Private Cloud. They will need to design newer chargeback mechanisms as well

– The bigger challenge I see is how to move the current infrastructure into the Private Cloud. Big companies have tons of equipment and they are already provisioned and being used. How will you get all these into a common pool? No organization would want to build a Private Cloud by purely buying new equipment

– Another important aspect which will dictate on whether Private Cloud will be accepted within an organization or not will depend on the how the power structure would change is Private Cloud is implemented. We have seen more than once that many good ideas have been compromised due to this issue.

How will this Public / Private Cloud help the vendors. As I said earlier, what eventually will come up depends a lot on your vendor. As of now many vendors have started talking Private Clouds and this is understandable. Look at it this way. You spend a lot of time, energy and money building up relationships with your client. You are now in a position that the client trusts you and you know the client well enough even to influence their buying decisions. At this juncture, if you were to propose the Public Cloud idea, you are shooting yourself. The decision on which equipment to buy will then pass on to the service provider, with whom you may or may not have a great equation. If it is a Private Cloud, you can always show the benefit of the cloud to the customer without losing your influence or your orders!!

The way the Public / Private Cloud is being proposed is: Big Enterprises need a Private Cloud, Smaller Enterprises can use the Public Cloud. This again makes sense from the vendors point of view. Especially the big ones. Because running after small orders or small players is never something which big vendors want to do. In such cases, if the buying decision shifts away from the small players to a service provider, who will buy in large quantities, then it is easy for the big guys to target this service provider.

As I had said in the beginning of this post, what color I paint my walls depends a lot on my painters aesthetics as well. In the same way, the way cloud will evolve will depend on what the vendors feel would be beneficial to them in the long run. Based on this, we will be seeing a lot more talk on Private Clouds. When cloud becomes a reality, Public and Private Clouds will coexist.

You can read EMC’s Chuck Hollis take on private clouds here.  Here is a different take on the cloud and what it should or should not mean. As usual, Steve Duplessie does not mince words in this article on why the cloud will vapourise