The arrival of the cloud into data centers has meant major changes in the IT world, but you ain't seen nothing yet: The Internet of Things (IoT) will bring an even greater change. And no change driven by the IoT will be more drastic than the birth of unikernel-enabled, transient microservices. These services will bring a new level of responsiveness and efficiency to the cloud.
In the world of IoT, where everything from door handles to electric toasters will be chatting on your network, we need clouds that can scale to incredible heights, react with amazing speed, and reach new levels of functional economy. That will require tightly focused microservices that can be born the instant they are needed and perish the moment they're finished. These will be built on ultra-efficient unikernels and controlled by an enhanced theory of cloud orchestration that will allow tasks to appear and disappear almost instantaneously, without constantly troubling a centralized cloud logic controller.
Whether you are a Fortune 500 CIO or just someone who wants to talk to your smart devices around your home, you need to understand how all this works. Let's start with the cloud layer.
The cloud so far
For over a decade, the IT industry has been reaching for the clouds. With the aid of OpenStack and other open-source cloud orchestration projects, data centers worldwide have been enjoying the flexibility of cloud computing.
The components of these orchestration systems follow a typical pattern. The cloud orchestrator rules over all of the virtual machines in the cloud. VM instances for each necessary service are created ahead of time and then wait for incoming requests to process. When these existing service instances become too busy, the orchestrator adds capacity by starting up additional VMs. Then, when incoming requests fall off, the orchestrator reduces the number of service instances, freeing up resources for other needs. This system works fine, as long as the volumes aren't too extreme.
How the IoT messes things up
Then came the IoT. It introduced a level of complexity that begs for a slightly different solution. Imagine a world where every button you press, every switch you flip, demands a response of some sort. The number of potential events becomes astronomical. And the risk of huge, unforeseen spikes in activity also soars. (Just think about what might occur in the seconds following power restoration after a regional power blackout. How many events could that region generate in those few seconds?)
Also, the scope of the IoT is worldwide. We are not just talking about tracking events generated in some data center, but events potentially generated in every town, every home, every room, and from every appliance. Suddenly, the task is enormous. If every button and switch everywhere is a potential event, then you'll need a massive number of services to answer the call. These services need to be spread out everywhere, not just located in a handful of regional data centers. The services should be in proximity to the events themselves, so that a timely response can be registered.
But how can you accomplish this? You can't have large data centers on every street corner. Who could manage all these remote data centers to make sure that there is enough service capacity for every potential event? And the closer you get to the events themselves, the more likely you are to have expensive resources underutilized during certain times of day. How can we afford to do this?
The new cloud model with microservices
The current cloud model needs to expand to cover the potential realities of the IoT. We need something that can move fluidly as events occur, with no wasted resource overhead waiting for events to occur, and with no manual intervention required to manage service capacity. We need a model that implements an architecture based on transient microservices.
An architecture that uses transient microservices needs to support the following concepts:
- The microservice should be able to be born on demand
- It should be spry, with the ability to start up and shut down quickly
- It should fulfill its purpose and then immediately die
- The birth and death processes for each microservice should require no action from the cloud orchestration layer
- The architecture should be highly distributed, with lightweight processing hubs
Let's look at these points a bit more closely.
The microservice should be able to be born on demand.
If you are looking at the possibility of thousands, or millions, of local processing hubs that register every event that comes over the network, it makes no sense to have services sitting idle, waiting to process incoming events. That's incredibly wasteful. It makes far more sense to birth a microservice when the event appears on the network, so you attain maximum utilization of your resources.
The microservice should be spry, with the ability to start up and shut down quickly.
If the microservice needs to be born on demand, it will need to be fast enough to start up almost instantaneously. Traditional services can take several seconds, or even several minutes, to start up. Transient microservices need startup times and shutdown times measured in milliseconds.
The microservice should fulfill its purpose and die.
Since you need to attain the most efficient use of limited resources, a transient microservice should shut itself down as soon as it has completed its task. In this way, all resources allocated to the microservice can be released to the local processing hub, making those resources available to handle the next task.
A microservice's birth and death processes should require no action from the cloud orchestration layer.
Since speed is of the essence, there is no time for an orchestration layer to decide that the service needs to be born or needs to shut down. In the human body, reflex actions do not need to be processed by the brain. Instead, reflexes are handled by interneurons in the spinal column that assure maximum response time and keep the brain from being overwhelmed as it attempts to respond to a deluge of minor signals.
Likewise, with the IoT, we need a reflex-style processing model in which the orchestration layer (the “brain” of the cloud) does not need to respond to a torrent of event requests. This ensures the fastest possible response time, as well as keeping the orchestration layer from being devastated while trying to deal with every trivial event that happens under its purview.
The microservice architecture should be highly distributed, with lightweight processing hubs.
Following the illustration of human reflex actions, you can construct a highly distributed architecture that relies on very lightweight processing hubs located near where events are generated. Every home or office could have a tiny processing hub—possibly the size of a Roku or smaller—that uses its small footprint efficiently to process local events quickly and then sends resulting data upstream to heavier compute centers as needed.
At this point you may be thinking, “This theory is interesting, but how can I possibly achieve this? What technology is so small and quick that this concept will work?”
The key is unikernels: simple, small, fast machine images that can come and go quickly and on demand. Unikernels are single-purpose virtual machines composed of an application and the minimal set of operating system functions needed to make that application function. Applying concepts well known in the embedded programming space, unikernels remove the unneeded bulk of general-purpose operating systems, creating tiny images that you can start and stop quickly. Examples of this technology include a functional DNS server in an image that clocks in at under 200 KB (yes, kilobytes) and network devices that start in 30 milliseconds and run using 6 MB of memory.
With technology such as this, you could conceivably create inexpensive, tiny hubs to act as the spinal interneurons of the IoT nervous system. They would provide the fast reflex needed for each button press, while transmitting any essential messages upstream as needed by the IoT brain.
Embrace the IoT through unikernels and microservices
By deploying these unikernel-enabled transient microservices, you can effectively handle the workload expected from a fully populated IoT. Are there still a number of details to work out? Absolutely. But by embracing appropriate new technologies such as unikernels and putting them in the proper context (in this case, transient microservices), you can make the IoT work at scale.
Keep learning
Choose the right ESM tool for your needs. Get up to speed with the our Buyer's Guide to Enterprise Service Management Tools
What will the next generation of enterprise service management tools look like? TechBeacon's Guide to Optimizing Enterprise Service Management offers the insights.
Discover more about IT Operations Monitoring with TechBeacon's Guide.
What's the best way to get your robotic process automation project off the ground? Find out how to choose the right tools—and the right project.
Ready to advance up the IT career ladder? TechBeacon's Careers Topic Center provides expert advice you need to prepare for your next move.
