Milliseconds Matter for Instant Digital Experiences | Digital CxO Summit

Good morning or afternoon, depending on where you're joining us from. My name is Henry Tan and I'm the product marketing team here at Redis. Today, I want to share why we think milliseconds matter for realtime digital experiences.

We believe real-time data is the emerging new category and opportunity for companies to embrace a real-time world that we live in today. Every business across all industries is becoming digital. Digital business are differentiating around customer experience, processing, analyzing, and making predictions with data in real time, enables better decision making and real business use cases.

We'd like to say latency is the new outage, and really what it's doing is that it's actually inhibiting your growth as a business. These trends and challenges are specifically impactful industries like financial services and retail. Think about, um, Uber Rideshare or your DoorDash food delivery.

Um, these services are where customers are demanding real time instant responses, right? You don't wanna wait too long for the Uber to tell you when the driver is coming. I've had that happen a number of times, especially when I was going to the airport, and it was definitely not fun.

Um, and so, you know, you want things like instant, um, you know, when you're using your mobile banking application, you wanna be able to get your balance information in a timely manner. So there are a number of different applications where realtime makes a huge difference, especially in terms of the customer experience. But what does realtime really mean?

Uh, well, let's try to put this, uh, definition that hopefully is a, is a little bit more clearer. Um, there's been a lot of research that eventually boils down to the fact that a hundred milliseconds of end-to-end response time is considered a real-time experience. That's the time when you have little to no delay and get to this phase, what I consider as the flow between you and the application that you're interacting with.

When you think about what means at the data layer level, um, it means you want a hundred milliseconds end to end latency, which includes the network traffic between the end user and the application server, and the time it takes for the application to fetch that data. Wherever the database is located. With average network times really right around a hundred milliseconds nowadays, that means modern databases need to respond in less than a millisecond For us to meet that a hundred overall, a hundred millisecond end-to-end latency.

If an application doesn't provide that millisecond or sub millisecond response time, you're gonna end up with distracted customers, right? They're gonna switch to a different website. If they were shopping for a particular item, they may decide to go to a competitor.

So let your risk of that transaction or that experience being at risk is much higher if you can then reduce that latency. So let's take a look at some examples of real-time, um, applications and use cases you may be familiar with. Okay.

Tell me if this has happened to you. I'm traveling to work for a conference, um, later this afternoon. I realize as I'm packing, I lost my charger.

This is kind of critical. Um, so I, I shop online, I go to my favorite electronics retailer, and I find the product that I need. It's the brand that I like, it's the specific type that I need.

So great. I also look up the, where's the nearest location so I can pick it up like immediately. I drive to the, I drive to the, to the location.

And guess what? The product actually wasn't there wasn't in stock, even though their website told me it was so clearly I'm not a happy customer. Now, there are a whole host of other examples of where real-time data is, is really needed for applications and for customer experience today.

Um, a few more examples are in the financial payment space. You can u you would use Zelle or Venmo, which I just did the other day to make a payment. Obviously you expect that experience to be seamless, um, and always on because who knows, maybe you're paying back a friend, um, but it is in the middle of the night, but you wanna make sure he gets it.

Um, you book travel tickets, um, and hotel rules pay with a credit card. You're playing an online game, which I don't do, but obviously many people do and they really expect the, the critical, um, response times because, hey, when you're playing your friends or you're playing in a group or competitively, you really need to make sure you have the biggest competitive advantage that you can get. Um, maybe you're checking a fee on any of your social media platforms.

Uh, you're researching stocks and want to do some sort of online trading and all of these daily activities, you expect realtime responses and realtime experiences. Now, many organizations rely on traditional databases that are unable to deliver on these kind of, uh, realtime experiences. So what is going on with organizations today?

New cloud native and digital only companies were founded in the past decade to provide these kind of real service, real time services, but, uh, they are hindered by these legacy databases. Um, these in some institutions have been around for 50 or even a hundred years, and they've built up a lot of tech debt and have many legacy systems. So it's not easy for them to transition, uh, off these legacy systems and to start fresh or to use, um, cloud native services.

Demo monetization and migration path needs to be more fully thought out for them to be able to transform their systems, especially the legacy databases to offer real-time services. They simply can't rip out and replace their legacy systems in a day or even weeks or months. But why are legacy systems such an obstacle?

And really the simple answer is they were not built for the digital modern age. If you look at evolutionary database lineage depicted on this diagram, relational databases and mainframe originate during a period of time around 50 to 60 years ago, which not only predated the cloud era, but when storage particularly Ram was very expensive when storage costs are the primary constraint at the time. Relational databases were designed primarily with this space storage as a first principle, not performance or not data access.

The consequences of this design was, is what is known as an impedance mismatch between the way we store data within relational databases as columns and of data and the way we access and visualize it as objects or documents or as a time series. So not only was the performance limited by the physical constraints of a spinning disc, but the extra transformation civilization and complexity to overcome the impedance mismatch limited the latency and throughput performance which digital applications desperately need and consumers expect. So what are companies to do?

Well, that's where no SQL databases entered the picture. No SQL databases were built using 180 degree different mental model than relational databases. While relational databases for storage first, uh, were optimized for storage first, no SQL databases were optimized for, uh, data access for which avoids that impedance mismatch.

This was possible since they originated in the two thousands with storage becoming much more affordable and cost effective, including, uh, ram, allowing for the viability of distributed data platforms, including in-memory data platforms like Redis, which was the precursor to database managed services in the cloud. And then of course, as you can see on the right hand side, we then move to the next wave of innovation, uh, which is then Gen F ai, which I will talk about, um, in a little bit later. Now, as part of modernization, many companies have turned to microservice architectures.

It's been around a while now, and companies have migrated to this architecture over monolithic architectures for developing new applications or refactoring existing ones. The importance of a modern high performance daily is still critical, I would argue even more important for distributed microservices. While we all know the benefits of microservices, which includes agility and faster time to market, there are definitely challenges with moving to a microservices architecture.

It's more complex because of the different services involved. You may have an application that literally has hundreds, if not maybe thousands of individual services that comprise that application. Um, so each one must be isolated, but yet be able to communicate and share data when, when needed.

Why isolate is because we don't wanna couple each of the different services so that each team that's responsible for each domain and each service can have their own release cycle. By coupling the services together, you are in effect going back to monolithic style, where you have one huge coase and one release cycle. So microservices are still bound by the performance limits of the data tier.

Maintaining data consistency across domains and across legacy databases, um, is a challenge. And so a modern data layer is that is needed for microservices, is really gonna be critical. Let's take a look at a specific example.

Next. C o s is a microservice design pattern that stands for command query responsibility segregation. Yes, it is a mouthful I know, but this design pattern illustrates really clearly, I believe how a low latency in memory data platform can address the challenges of microservices.

This pattern essentially separates the reads and fund the rights to optimize the read performance while maintaining the isolation or decoupling. If you look at this example here on a very simplistic financial services application, one service, which is connected to the system of record database is, is writing things like payment approval. So all those transactions needs to have all the durability and asset compliance related to a SQL or relational database.

However, we have a separate service called the Payment history microservice that really only needs re-access to that system of record that's hosted on a legacy SQL database. So how do we get the microservice payment history service to be able to re and get access to that data on the other domain or the other microservice without coupling the two together? The key is to have a separate reoptimize cache, um, together with a change data mechanism to prefetch the data that's in the system of record into the cache that is optimized for read.

And what you're doing then is unburdening the system of record, um, enabling you to get the latest updates from the transaction database, but yet not coupling the two together. So basically the data becomes available, um, as soon as possible to the change data mechanism. This way we adjust the four key challenges, legacy database performance, the complexity of having multiple services, uh, needing to access the same data, data consistency, and of course the speed and performance of an in-memory cache like Redis.

Let's take a look at another example. At a large bank that is implementing a new mobile um, banking application, they also implemented a similar pattern for their architecture. This particular bank offers multiple services including checking, savings, mortgages, credit cards, um, investment trading, but they're all using different databases, um, to store the user and account information.

It was taking way too long to get the data from the legacy systems by implementing and microservices design and using C Q R Ss, they're able to use an in-memory database to prefetch the user profiles and account information from each one of these different legacy sources, um, and basically speed up the login process and present the account dashboard of their balance information, whether it's their checking account or their mortgage balance. Uh, maybe they ones look at some of their most recent credit card transactions. Um, all these things were available to the user instantly by leveraging an in-memory data cache.

This architecture essentially then provided faster logins for the authentication, maintain performance during peak demand. Imagine that there are millions of your customers trying to access their accounts. Um, at the same time, maybe there's some sort of financial crisis and people wanna check immediately, you know, protect against single point of failure because of the ability of the data layer to achieve five lines of availability.

You know, take advantage of the multi-model capabilities of Redis as a data layer because we support multiple, uh, data models, including J SS O, which is the format this particular bank chose to store the user profile information as well as the account information. Another example that I think is really illustrative of the digital transformation move to a more customer service centric, um, platform and really enhance, uh, in this case the omnichannel customer experience. So Ultimate Beauty is a, um, large, um, US beauty retailer.

Um, they are the destination for things like cosmetics, fragrance, skincare products, haircare products and salons. Today, Ulta Beauty operates more than 1,450 retail stores across the country, and they maintain a very popular e-commerce platform, which includes, you know, collection of beauty tips, tutorials, and it can even help, recommend and guide you toward specific products like makeup, uh, based on your preferences and, um, feedback. So what were some of the challenges that elta, uh, encounter?

Well, number one, during covid they experienced rapid online growth as the physical stores were shut down for a period of time and they needed support a new, uh, model called Opus, which is buy online and pick up in store. So in order to implement that, they had to look at their inventory management system, which they were experiencing a lot of latency issues related to the backend cost to the inventory management system that they were using. They also wanted to prepare for the future and adopt a modern microservice, cloud native architecture for their new e-commerce platform to enable them to get to what they call the digital store of the future.

So with using a fast in-memory data layer, um, they were able to reduce their application ency, um, improve their inventory accuracy, which helped them to increase their conversion rate by using it in memory daily. That was, um, built, um, on, on cloud and are able to be accessed to the different cloud providers they were able to deliver, um, pickup built buy online and pick up in store. Um, in a matter of weeks during the, the haze and the craziness of covid, they're able to now develop a new digital website and platform, e-commerce platform that on the cloud that enables them to power cohesive, um, online experience as well as transition to their in-store, uh, physical presence.

So basically a true omnichannel customer journey. I love this quote from Omar, who is the IT director at ELTA Bey on this project. Today's shopper expects the great digital experiences.

We want to bring the physical and digital stores together to make shopping even more seamless for our guests. Red technology powers many of our features and we leverage it heavily in our omni-channel strategy as well in our new e-commerce platform we're building. I think this is just a great testimony to why you need a fast, high performant, highly available in memory data platform.

Now, I'm kind of switching tags on you. Um, I showed generative AI a little bit earlier in kind of my history, you know, lesson of the different, uh, database technologies. And you may be wondering, well, how does this relate to talking about customer experience and, um, fast data layers?

Well, you may have heard a thing called chat G P T, which unleash a tidal wave of innovation around large language models and enterprises are embracing it fully. More companies than ever are bringing the power of natural language processing to their products, and more importantly, to their customer service operations and interactions. So I just listed just but a few of the different areas that has a impact on customer experience through using generative AI technology.

You know, question and answer chat bots. They get them as your virtual assistant. Um, they can do things like make recommendations, um, think of customer support, where the agent can then leverage the Genive AI application to help them find answers to customer questions in a much more quicker and efficient way.

Um, being able to then focus on helping with, um, other types of questions that may, uh, take longer or be more complex. Uh, being able to search for previous histories and tell the application, you know, to take into account the prior interactions and what would be a good recommendation for a new product. All of these applications have a direct impact on the customer experience.

And where does a fast data layer fit in? Let's talk about any specific use case, you know, here at Redis, um, and at many other places, um, vector databases is becoming the norm to help augment what large language models do as it relates to these generative AI applications, especially in the areas of chatbots and virtual existence. Um, so similar to what, uh, M M E Cache does with other applications around caching.

There's a specific use case called large language models, semantic caching that also enhances the user experience, but is very dependent on having a fast data layer. And let me just kind of explain it pretty simply. So we all know that these new, uh, large language miles understands, um, na natural language, whether you're typing in or speaking it through, uh, some sort of audio connection.

But the way people, um, ask questions to commonly, you know, commonly, you know, topics, um, and how they do it the next time or with a different user are different, right? And so while you can cache similar, um, um, similar, um, questions and have the, the, uh, response already, which will speed up your response times, the key is that with semantic cache, you have to be able to search and understand that if I say code alarm versus condition in this particular example, uh, it means the same thing. And therefore you should retrieve the, the same cache on answer to what a C 3 84 code means and provide the same response to that question.

And the key here is not just fast caching, but to semantically understand that the difference between those, that there is no difference between those two queries. And so you are able to provide that intelligent caching provide the response in a much quicker way than what the large language models can do, uh, because they are inherently slow and obviously reduce the cost because each time that you make a call to a large language model, um, you are actually charged, um, especially for those commercial SaaS type models, providers. So that's a way we're again, a fast data layer that understand vectors, which is the language that large language models understands.

Uh, 'cause you have to convert that text to those numerical representations so that the algorithms in large language models can understand what they mean. So your database and your data layer should be able to handle multiple data types, including vectors, to be able to provide those seamless customer experiences, even in a generative AI application. So to conclude, given that data is key to being competitive, and we just discussed why Legacy Systems's work designed for the digital age, a realtime in-memory data platform like Redis is critical for seeing this customer experience.

A realtime data that handles realtime data at any scale, is able to run, um, anywhere is needed in the cloud, on-prem, um, in, you know, hybrid or be and be deployed automatically. Not only should the data layer provide real-time experiences to your customers, it also needs to be structured in a way that helps you stay competitive by building and delivering your apps faster to market in order for a real-time data layer or data platform. Do both of these provide real-time experiences and enable rapid app development through architectures like microservices?

It needs to provide you a choice of data models, be flexible, be simple to, and easy to deploy and use and needs to be resilient and be highly available available. So your developers and your DevOps people are not working to keep the infrastructure running, but to provide realtime digital experiences for your customers. So to answer the original title of this, uh, discussion, milliseconds do matter.

Here are some assets that, um, that you can learn more, uh, about the topics I cover today. Thank you for your time. com to learn more about Redis as a realtime data platform and what we can do to help you thrive in a real time digital world.

Thank you.