Experience the Era of High-Performance Applications | DevOps Experience 2023

Hi everyone. Thanks so much for your time today. My name is John Newnan.

I'm a product marketing manager here at Redis. Um, and today I'm really excited to discuss some exciting new functionality that we've recently introduced, um, to help developers, architects, and DevOps teams build high performance applications, while also reducing, um, some of the added complexity that often comes with an in-memory data layer. Um, so today we're gonna be primarily focusing on two tools that we've recently developed.

Um, one called Reddis Data Integration, and another called Redis Smart. And we're gonna discuss how they can make your lives easier. Um, so just what we're gonna be covering today, we're gonna start by quickly, um, discussing why realtime applications are so critical, um, followed by an introduction to Redis.

Next, we'll get into one of the key challenges that realtime applications introduce, which is complexity. Um, and then finally we'll get to the fun stuff, which is how you can use Redis to build real-time apps that reduce complexity. And we'll focus on two great, um, new tools developed by Redis that you can use.

One is Redis data integration to make it much easier to mirror data between your cache and your database. And the second is Redis Smart Cache to alleviate a lot of the challenges associated with implementing the most common caching pattern, um, which is cash aside. And then, uh, just as a quick programming note, um, please feel free to ask any questions via the chat.

We have, uh, Redis experts live that are working the chat, and we'll be happy to answer, uh, any questions that arise during today's session. So let's just very quickly start with some context to set the scene. Um, so I don't think it's a surprise to anyone here today, but, uh, we're all living in the age of real-time applications.

Uh, as we all know, over the past decade plus, um, so much of our lives has shifted from the physical to the digital world. Um, I mean, just think about the changes in your own life to retail, to shopping, to supply chains, to, to healthcare and how you speak to a doctor. Um, just given all these changes, it's probably not breaking news that, um, most businesses now primarily conducted digitally.

Um, and because so much business is now conducted digitally, performance of business, applications matter more than ever. Um, and there are real, um, measured business impacts to increasing speed. One 10th of a second decrease in latency, um, was measured by Deloitte to improve average order values, uh, of retail sites by nearly 10%.

Um, so businesses really, it's in their best interest to have, uh, highly performant applications. Um, but to meet today's user expectations, businesses need to get modern performance from their data layer, but they're also, they're often, um, hampered by legacy infrastructure or slow databases that just weren't built with today's modern performance needs in mind. Uh, so that's where Redis comes in.

Redis was introduced in 2009, uh, and it was really built to provide the speed needed by today's applications. It's commonly used, um, as a cache that can be used alongside existing databases, um, and, and current infrastructure to greatly speed up application performance. Um, and because of its speed, its flexibility and its overall capabilities, Redis has become one of the most widely used and most loved databases available anywhere today.

Okay, so we're all in this age of real time. Um, and Redis is this awesome tool. I know I'm biased, um, that is widely used when building real-time apps, but even despite Reds' greatness, um, there's still gonna be challenges.

And we're gonna focus on one of the key challenges of real-time applications, um, which is they can, they can add complexity. So developers and operators are already dealing with multiple databases for different use cases and data types they need to build with, and they need to maintain relational databases, key value databases in memory wide column streaming, uh, documents, search time series. The list just goes on and on, and it seems to always be growing.

Um, and they're deploying all these different types of databases in really complex hybrid and multi-cloud environments. So they may be using R D Ss and a w s Cosmos DB and Azure, Oracle and Kafka on premises. So now there are multiple database types for a whole variety of use cases.

Um, but each is deployed and managed in its own deployment environment. Well, the need for real-time applications can even further complicate this already quite complex picture. Um, so now you add an in-memory data store to be used as a cache to speed up data response times for many of these different databases.

Uh, so now developers need to build with an operators need to scale, monitor, and manage an in-memory caching layer as well. Uh, so just, it's an incredibly complex reality that presents a a number of real challenges. Um, so let's kind of get into how that complexity impacts the developers and the operators that face it.

Um, developers now have more technologies that they need to learn to code with. Ultimately, this slows down the development velocity and forces them to focus on learning new technologies, um, instead of building new value and added functionality into their applications that drive business value. Um, and operators are faced with slower and more challenging operations.

They need to operate a multitude of databases and their caches, uh, across clouds and on-premises environments. And this leads to wasted time and resources. They also need to deal with more frequent issues arising and those issues being slower to troubleshoot and resolve just because of all the complexity.

Um, and to top it all off, they also have to worry about data integrity and data consistency. Um, and this really complicated and, uh, complex data layer. Uh, so yeah, complexity is a problem, but, uh, we're also here to talk today about how Redis can help.

So, uh, let's dive into that. There are many ways that Redis helps with this problem around complexity. Um, first Redis is itself a very flexible technology.

So Redis works with many data types, and this is a benefit because it allows developers, uh, to consolidate a little bit around Redis. They can use one data platform instead of many individual databases. Uh, and it's also really easy to code with, uh, developers absolutely love it.

Um, then there's Redis Enterprise, which is our commercial offering, and it really greatly simplifies operations. It's reliable, it's highly resilient, resilient. It's very scalable, and it's available, fully managed in any cloud.

Um, and to top it off, we've also introduced new features to further reduce complexity. Um, the first is a new tool called Redis Data Integration that's used for integrating data between caches and their databases. And the second is a tool called Redis Smart Cache, which we've built to simplify, uh, the cache Aside pattern, which is the most common caching pattern to make, uh, query caching easier for our customers.

So let's dive in a little bit into these. We'll start with, uh, Redis Data integration, or r d i for short. Um, when building realtime applications, uh, data typically resides in both the cache for realtime access as well as a primary database.

And businesses have to ensure that they're able to integrate data between these two systems, um, without hampering performance or introducing issues around data consistency. Um, and that means that data must accurately and quickly get from the database to the cache and from the cache back to the database. Um, so at Redis, we created, uh, Redis data integration as a tool to make this process easier.

Um, r d I can be used to easily mirror data between a database and Redis Enterprise to ensure that, uh, data is consistent between both systems. It performs both data ingestion and transformation between Redis and a database, and won't get into that, uh, a little bit in the coming slides. Um, and we're also currently building right behind functionality into R D I.

Um, it's currently in preview, and it'll allow real-time rights to Redis with any changes updated asynchronously to an underlying database to help speed up that right, uh, performance. So how does Redis estate integration work? There are really two stages.

There's data ingest and data transformation. Um, data ingest is basically the process of identifying change data in a database and getting that data, um, from the source database into Redis Enterprise. Um, but this data is often in a data structure that's used by the primary database, and it needs to be transformed into a data type that can be stored in Redis.

Um, and that's where the transformation comes in. Uh, so let's walk through how R d I does both of these things. Um, when data is changed, an open source change data capture platform called dium captures changes to data and the source database, and it streams it into r D I.

Um, and this is the process that's depicted, uh, in the diagram to the left. Um, and then within Redis, the data may be further filtered, transformed, and mapped to one or more keys. Um, and RDIs currently supports transformation into a number of data types.

So hash, j ss o n set and Streams. Um, and this data transformation process is shown in the image to the right. So once data is sufficiently ingested and transformed, r d I writes that data to the, uh, destination Redis cache where it can be accessed by the application in real time.

Um, so that's great, that's how it works. But, you know, what are some of the benefits? How does r d I make your life easier?

Um, first and foremost, it makes it much easier to make your databases faster. It simplifies data integration between your caches and their databases. Um, it eliminates the need to refactor applications to write lots of new code or to deal with the headache of complex and time consuming data integration efforts.

And it also helps alleviate the worry around data consistency issues between your cache and your database. Uh, so ultimately it really gives you the speed of an in-memory database while alle alleviating a lot of the pain that comes with data integration or data consistency issues. Uh, now let's move on to Redis Smart Cache.

So, uh, it's likely you've heard of the cache Aside pattern for caching. It's the most common pattern implemented by Redis developers and architects, um, for optimizing application performance. Um, and while it's a very simple, uh, concept, correctly implementing, it can be far more difficult and more time consuming than it appears.

Uh, and these challenges are especially true for larger and more complex workloads. So that's why we built Redis Smart Cache. And what Smart Cache is, it's an open source library that seamlessly adds caching to any J D B C compliant application.

So it enables developers to quickly and easily identify their worst performing queries, enable optimize query caching, and observe and monitor performance. And it does all this with a quick configuration. It doesn't require, uh, you know, any query analysis or code changes.

It simply works right out of the box. So how do you know if you might benefit from Red A smart cache? The first question you'd want to ask yourself is, do you have other database systems?

And I think most of us do, and most of us, you know, will have Oracle and SQL Server and Postgres and all kinds of databases all over the place. Uh, the second question would be, what are you gonna do if another system comes up and says it needs to access your relational database and it needs 50% of the resources? Um, how would you then scale that database to accept this new workload?

Would you add compute resources? Would you swap out the node of the database and put in more faster cores? Just generally what would you do, um, to scale up that system?

And then finally, the final question is, if you're trying to save costs, how might you do that? Um, and Reddit Smart Cache is away. It alleviates traffic from backend database systems and it pushes them into red.

A Redis database, which allows you to increase concurrency without buying more licenses that are very expensive for your backend database. Um, so basically onto how Smart Cash works to make caching easier. So let's imagine a real life application.

Suppose, uh, you're an investment firm and you have a portfolio management application that has two components. It has an a p I powered by Spring Boot and a relational database. Uh, in this example we'll just say Oracle.

Um, so let's say the application isn't performing well, it's starting to breach the performance SLAs, uh, and the engineering team has decided to write code to, uh, implement a cash to address these issues. That team would have to, uh, you know, undergo a pretty expansive and time consuming process. They would first have to analyze the existing code.

They'd have to research how to use Redis for query caching. Uh, they would need to do the actual programming. QA would then have to run load tests.

And finally, once all that's complete, the application would finally be ready for production. Um, well, Redis Smart Cache offers a much faster and easier alternative to the days or weeks of work that that would take. So with Smart C, you can deliver the same outcome, uh, really in minutes with just a few simple configurations of Smart Cache and your application.

Um, and once set up Smart C wraps the applications existing J D B C driver and caches extensive queries in Redis. So queries are now served by both Redis and the original system of record. Uh, you can then set up rules for Smart Cache to optimize query performance based on your application needs.

Um, and with Smart Cache Cache aside and Query caching are really just that simple. There's no need for code analysis. You don't have to become an expert in Redis.

You don't need to learn the best ways to handle failover or observability or scaling. It's all baked into the cake with no coding. Uh, I mentioned that Smart Cache has built an observability for your right caching layer.

Uh, this is just showcasing an example of the prebuilt dashboards that Smart Cache comes with, uh, smart Cache and it's dashboards capture critical caching metrics like access, frequency, mean query, time throughput, and a lot more. Um, and all this data is exposed in prebuilt Grafana dashboards to help you decide exactly how well Smart Cache is working and which query caching rules to apply to optimize your application caching strategy. Uh, so onto some of the benefits, um, you know, how does Smart Cash help it?

It can help architects and operators reduce the burden on existing databases without the need for, you know, more expensive licenses or to scale up your database. It really helps enable quick growth. Uh, it also simplifies the complexity of the most common caching pattern, which is cash aside.

Uh, by reducing the need for businesses to identify slow queries, create caching rules, update application code, set up, monitoring, scaling, uh, handle failover, uh, smart Cash simply works to do all these things outta the box, which in the end makes for a much easier and much simpler to produce, uh, a much faster realtime application. So just to sum things up, Redis is constantly innovating to help our customers lives. We offer a core product that's incredibly flexible, highly performant, and easy to operate, but we're also building new needed functionality into Redis like R D I and Redis Smart Cash to address practical concerns from the market.

Um, invest. We're investing the time and the resources into making caching more efficient so that our customers don't have to, to make their lives easier. So if you're interested in getting started, we recommend you use the QR code to the left to speak to a Redis expert so we can work with your business and your application needs, and we can really help you understand how to best use our services.

If you're not quite ready for that yet, that yet, um, feel free to check out the middle QR code to learn more about Redis data integration or the QR code to the right to try out Redis Smart Cash. Uh, that's it for today. Thank you very much for your time.

Uh, I hope you found the session interesting. Please feel free to ask any additional questions that you have live in the chat q and a, or if you think of anything later, please feel free to shoot me an email or a message on LinkedIn with any feedback or questions. Thank you.