Across several days back in March, users were unable to login to the popular stock-trading app Robinhood. The application experienced multiple outages, including one that lasted for hours on the same day the Dow experienced its biggest point gain in history.
Users were locked out of Robinhood when they most wanted to use it, and this outage, made even worse by its terrible timing, made headlines everywhere. But Robinhood was just one of many applications that experienced trouble because of increased demand and volatility due to the COVID-19 pandemic.
“Trouble” under these circumstances, even if only over a few hours, can cause millions in losses as well as the swift disintegration of brand loyalty — which may be impossible to bounce back from.
When “shelter in place” orders were issued in cities around the world, it completely changed the way we live, work, eat, play and shop – particularly the way we eat and shop! And many businesses have been forced to lay off staff, rethink their business models on the fly, and accelerate their digital transformation.
Whether you are in a business, a government, or a nonprofit, there’s a good chance you’re facing unprecedented and highly volatile demand for your applications. This means you are also (hopefully) reimagining and future-proofing one particular part of your IT landscape that plays a mission-critical but unheralded role.
The part I’m talking about isn’t a particularly sexy element of your IT landscape. In fact, it is so unsexy that many in IT refer to it as “plumbing.” Digital plumbing, of course, but still plumbing. They call it plumbing because it is how business events flow, like water, across your organization.
So what are business events? They are the things that happen across your organization that people, machines, devices or applications care about.
- A customer places an order = a business event
- A package gets loaded onto a truck = a business event
- A credit card gets approved = a business event
For example, when you use your credit card to pay for a purchase in an e-commerce application, an event gets fired from that application across the network and out to all the applications that need to receive it, such as inventory, credit checking, shipping, etc.
And in many organizations, business events need to move across and through the organization and beyond – to the cloud, sometimes multiple clouds and, depending on the type of event, to devices or machines in the Internet of Things (IoT). But like with plumbing, if one of the applications is getting over-run by demand (clogged) and can’t receive a business event, you don’t want that failure to make all the rest of your plumbing fail.
This critical layer of your company's IT infrastructure can make the difference between fast or terrible application performance, happy or cranky customers, efficient and inefficient processes, outages or not. Yet many business leaders don’t know much about it.
Let’s explore the problems it can address, what the modern version of this layer is called, and how having it will help support your organization through the pandemic and beyond.
By changing the way we live, work, play, and shop, the drastic measures being implemented to slow the spread of COVID-19 have affected all business to some degree, some severely.
Companies are fighting to stay on top of new customer preferences and behaviours, new delivery and business models, new and constantly-shifting government regulations, and more. In the face of these unpredictable changes, government apps have crashed and left constituents without crucial guidance. Online shoppers have been left staring at loading screens as they struggle to complete online transactions. And a variety of Robinhood-esque outages have occurred, leaving customers completely in the dark.
Nearly every industry has seen their data movement strategies take a hit due to these spikes in demand.
Elsewhere amid COVID-19 chaos, a rush of online donations to charities like local food banks could be vulnerable to these kinds of overwhelmed systems. If users make a donation during a sudden spike in demand, the app could crash and leave the individual uncertain as to whether the transfer took place, meaning they have to follow up manually with their bank and/or with the already overwhelmed food bank to confirm. In this example, the data ends up in a black hole because the back-end system of the application can’t handle the stream of data coming in. It times out – effectively giving up and leaving the customer in the dark.
The move to a digitized business model and microservices
At the same time, the measures required to fight COVID-19 are pushing most businesses to leverage technology to become more agile and efficient. This trend, commonly called digital transformation, was already underway, but it’s now far more urgent for far more companies — likely in yours.
Many organizations are trying to achieve in mere months the kind of digital transformation they’d normally spend years on. They are building dozens and even hundreds of new web-based applications that all have to be connected to their back-end systems and to various cloud-native services for analytics and AI.
As part of this, they are replacing large monolithic enterprise applications with “microservices” that can be quickly combined and repurposed to meet a wide range of new business needs.
Digital transformation is about more than enterprise applications, though. It frequently involves the so-called “Internet of Things,” whether that be appliances in a consumer’s home, elevators in a skyscraper, sensors on an oil pipeline, or industrial control systems in a factory. In any case, to incorporate connected devices into your business processes means you need a way to efficiently collect information from, and send instructions to, a wide range of devices in a way that’s real-time, reliable and secure.
The rise of the modern event streaming and management layer
For decades, companies have relied on what IT calls “message-oriented middleware,” from companies such as IBM and TIBCO, as their digital plumbing. It worked for years as companies were operating data centers on their premises. Today, companies need to operate in real-time, migrate to the cloud and incorporate IoT into their business — and their existing digital plumbing is no longer cutting it.
The modern version of digital plumbing is called an event streaming and management platform, and it is what is needed to help your organization accelerate your digital transformation and deal with the increased volume and volatility brought on by COVID-19.
For example, we work with a multinational financial services company with over 17 million clients. They experienced huge increases in trading volume across their trading platform after the pandemic lock-down started, going from an average of 65 billion events a day passing through their platform to a high of 95 billion events. This could have caused the type of challenges we’ve mentioned here, but because they had invested in our event streaming and management platform, their applications continued to perform and their customers remained happy.
Most IT departments recognize the need to upgrade from antiquated plumbing to event streaming and management platforms, but they run into resistance from those who don’t understand how these deeply embedded legacy technologies will hamper their efforts to adapt in an age where agility is everything. With COVID-19 changing the game, event streaming and management platforms will be necessary to operate efficiently and satisfy customers in the “digitize or die” arena of the future.
The author is Mychelle Mollot, CMO of Solace. Mychelle is an experienced CMO and software executive with over 25 years experience leading marketing, product marketing, product, market strategy and go-to-market initiatives and teams.