Charlie Cartwright on Automating Business Workflows with AWS AI Agents

Hello and welcome to the latest edition of the Techstrong AI Leadership Insight series. We're here with Charlie Cartwright, who's director of the Quick, uh, AI agents help You the workflow. And we're a little chat about, well, just how do we get all these AI agents to start working for us?

You know, explain to us how you guys envision all this coming together and how do I actually like tie something together across different products and services to actually drive the outcome as intended? Yeah, absolutely. I, I think with, with Amazon Quick Suite, you know, we're, we're going to reimagine the way that, that people do work in, in the workplace.

And what you're touching on is exactly what we've been focused on. I think if we rewind the clock clock, like in the last two years, I think we've all all had those amazing moments of delight where we were using consumer AI applications and it was able to do wonderful things for us. But doing that in an enterprise setting, like there were point in time, uh, benefits, but you didn't necessarily have those applications, those AI applications connected to an information ecosystem.

And, and that's really the key here to be connected to your data, um, like data stores, for example, right? To bring in all that structured data. And the same thing for document repositories, have access to all your strategic documents, your internal wikis and web search that can come in.

And then equally important is to be connected to all these applications for which users often, uh, operate with on a daily basis. And that creates this like, layer of context that's also relevant for humans to be able to get access to relevant information quickly to make decisions. But that is the same context that agents need to be able to execute on my behalf to interoperate with one another and to actually operate with autonomy.

And if they're connected to those applications to take actions and they have the enterprise context that is needed to properly perform those actions, then you start to get the real utility that we've, that we haven't seen in some of the enterprise applications For the uninitiated. Give me an example of how that plays out or something that maybe you're personally doing that the rest of us would go, wow, I didn't know I could do that. Yeah, yeah.

No, thank you. Um, so this is the, this is one of the amazing things about this product is, you know, me and my team, this entire organization, we get to utilize it and work with it and experience the benefits. Um, just recently I was working on, you know, something to look at, like a competitive landscape, for example.

And previously this is something that I would go to various different sources. One, like a competitive landscape document maybe that I, I wrote, uh, a few months ago, right? And then I would look across information, my, my email, various document repositories and, uh, dashboards and data that I use to understand maybe some things internal and external related to adoption, uh, product capability performance.

And then couple that with a lot of time searching on the web to understand what's out there, how these capabilities compare. And this is something I was able to do with our research agent in Quick Suite, um, which previously had taken days coordinated across several people, and I was able to do this in, in less than two hours. And, and so it kind of like this remarkable transformation where something had taken me so much time just to go across, you know, tens of different sources to collect that information.

And then I finally get to the point where I can start to analyze it and think about it. And I was able to short track that and, you know, within two hours, think about some of the opportunities that we have as we build our product, as we go toward general availability in comparison to, to what's out there, for example. So like a remarkable change in the amount of time that me and my team spend on a daily and weekly basis just because of how well Quick Suite is integrated into and has access to all the enterprise information that I utilize to do my job on a day-to-day basis.

And the same is true for people at various different, um, roles in the organization and different levels in the organization. How much faith can people have in the output of these tools? 'cause everybody's been a little bit concerned about hallucinations, so how do I validate what the AI agents are coming back with is what is actually in that data?

Yeah, Mike, uh, great question. And, and of course this, this is something like there's no bigger trust buster, then when you are asking a question and you know that information, you can see that information and, and here it is this like AI agent or, uh, assistant can't represent that information correctly. And so of course we're doing, um, benchmarks where we look at different question and answer pairs that we can ground truth against so that we can do things to make sure that we answer with the right relevance.

Also, um, when you have structured data, right, like actual data sets and things like that, and all this context across the, the various information ecosystem within quick the document repositories, uh, access to email information and Word documents, various drives, you're able to give the right context to the underlying model. So the, the need to hallucinate and the likelihood of a hallucination also decrease. And, you know, that can often be the case when that context isn't there.

And that model is essentially trying to answer that question the best it can that the user's posing. And in this case, we're able to equip these agents and the underlying models with that context. So not only can they coordinate, communicate, take action, but they're able to do that with, uh, users as well.

Hmm. Is there per chance any way that there's a setting for the AI agents so I can kind of determine how aggressive I want it to be to answer a question so that maybe some of those wrong answers get tempered a little bit because as far as I could tell, the AI agents are just trying to please us and they go to any Lent to do it. Yeah.

One, one of the features that we have, the capabilities that we have in the quick suite is the ability to create your own chat agent, like create your own chat assistant essentially. And, and with that you can, um, change and dial aspects of the, the persona, the instructions for that chat agent, just how creative that chat agent goes, how much it needs to stick to a script, so to speak, right? To to go in the bounds of a particular goal, to only look at context, um, for a certain set of underlying, uh, data stores or document repositories or a subset of other applications and actions.

So you can scope that down, but then also fine tune that persona and kind of dial that creativity. And what you can do is get an agent to respond very much to the objective that you're trying to achieve. So I can fine tune that.

How customizable is the overall experience? You mentioned the chat, but um, every company has slightly different workflows, so how does the platform actually know what my current workflow is? And for that matter, will it suggest ways to improve my workflows?

Yeah, so I think to the, to the first part of that, um, Amazon Quick Suite I is is highly flexible in terms of how this can be utilized specific to a user's workflow. So if that's certain daily routine task that I, you know, a user's taking information from one application or one source and moving that to another, those are things that can be automated with a, a wide spectrum of automation capabilities that we have from low code to more advanced enterprise workflow automation. And so really what it is, it just kind of depends on, um, the different applications and information that are available.

And that's what really drives that customization and the opportunity to automate some of those workflows that exist within an organization. But it's not that the capabilities themselves, um, are restricted or, or so specific that they can't generalize across different organization's workflows. It's again, back to that context and those procedure documents that can be given as contacts.

So the enterprise contact and maybe a procedure document or an SOP that can be used then by that AG Agen planner to create an automation workflow. Mm-hmm. How do I manage the level of permissions that I give to each of these AI agents?

'cause from what I've seen so far is in their quest to please us, they can be overly aggressive in accessing any and all data they can find. So do I have to think through what it is exactly? I'm gonna let that AI agent see.

Yeah, I think this is, this is both important for users and agents. So as, as with Amazon Quick Suite users now have access to information and so much information that, that maybe they didn't even realize, right? Or it wasn't used on a daily basis.

And so the things that, that we are very mindful of and, and take with a high degree of responsibility is making sure that users only access the information content that they have access to already. And that through quick we're not giving access to anything they shouldn't. And the same is true with agents and specifically with agents.

When I, you know, when I create a custom chat agent for example, I can actually equip that chat agent, I can narrow its context, it can start with the same context that I have access to, for example. Or I can further narrow that and I can give it access to a subset of actions, um, or just a few applications even. So the flexibility is really there to, to take, to take that agent and focus it on a very specific goal so that my interactions, I can limit the context and I can limit the applications and the actions that it can take within those applications.

And also on the reverse, the other side of that spectrum is also capable where I can take and and customize that agent or make sure that it has similar access to many of the applications, document repositories, internal wikis and data stores that I have access to. Mm-hmm. Um, a lot of organizations are experimenting with AI and they, some of them have even gotten so far as to build their own AI agent, but I kind of feel like we're at that point again where we're trying to figure out when do I build versus buy an AI agent.

And I think a lot of the use cases that people are coming up with are just gonna be things that are gonna be standard features of a platform like yours. So where's that line between build and buy in your mind? I think the, the build and buy discussion always comes down to a speci, a specific set of use cases, objectives, and goals.

And we look at Quick Suite, um, some of the, the one P agents that come with Quick Suite, for example, a PhD level researcher, right? A business analyst that cannot only look at information that's in a dashboard, but that can understand and extract and generate insights on the underlying data set, whi, which is very powerful. And then a team of automation experts.

So we have this broad set of capabilities that come with quick as part of our one P agents. And the other thing that we offer there, Mike, so let's say that a, a customer or a user, and maybe it's a power user, they want to introduce their own agent, right? And, and they wanna build that on like AWS infrastructure or somewhere else.

That agent can then be brought into and can be coordinated with, um, among the Quick Suite agents to maybe offer some specialized capability that is not there in Quick Suite today. And so what we're trying not to do is take, take this position that, you know, everything has to be here in Quick Suite and just like the applications, we want users to be able to connect applications and data stores, whether they're part of AWS or not part of AWS, we wanna make sure that we can meet users where they work and how they work. And that quick is extensible into those applications, whether it's a web browser or it's word for example.

And so that's a very, very important, uh, capability within Quick. And so we wanna make sure that for organizations that need to build a specialized agent, we need to enable that and we also need to support that agent to have access to essentially this enterprise comprehensive information so it can operate with the autonomy and the agency that the first party agents can within quick as well. What LLMs or platforms are the agents using to accomplish these tasks?

And are they permanently tied to those LLMs or will they change and swap them out as LLMs Advance and different ones are available at different price points? That's exactly it. We, we evaluate and we change models based on the different capability to achieve the best outcomes, um, or users for that given capability, right?

And so, and so that's really important because as models continue to evolve, we wanna make sure that our users are getting the best experience possible. And so nor do we wanna be, uh, held to one particular model and sometimes one particular model isn't the best for all the broad set of capabilities that are available in quick. Hmm.

So what is the pricing for quick look like? I mean, a lot of folks are, um, concerned that over time the these will just get more expensive and cost prohibitive and other folks are saying, well, a lot of these agents maybe are low cost today, but over time the cost will add up. How do I think about the total cost of doing some sort of AI workflow?

Yeah, so, um, we, we basically have have two, two SKUs or two tiers, um, a professional enterprise and in, in the, the professional or the $20 tier, um, users get access to all the capabilities that we talked about today. So they can chat across their data, they can use, um, you know, PhD level research, uh, business analyst, and they can also automate workflows like no code workflows with a capability that we call flows. And so they have access to all those capabilities and the, the things that we do there, um, for research and automation where there's these agentic hours that can essentially be accumulated.

Um, we offer agentic hours for those capabilities and for users that need to consume even more of those agentic hours for that research agent for example, there's a $40 skew where we extend those hours and then also offer consumption so that users can go into, you know, as many hours as they need for those power users. And the same thing with automation workflows. I think I had mentioned briefly earlier that we support the spectrum of automation.

And so one is this no code, natural language automation builder where you can think about like routine tasks being automated. Like maybe, maybe I want to create like a weekly business review. I wanna automate aspects of that or, or fully automate weekly business review or, or pull information to prepare for customer meetings and send that out to the respective, um, owners for those meetings to resolve some open customer issues.

Like those types of things can be automated and flows, but if we're looking at more complex automation in the creation of that complex enterprise automation for that, we have quick automate. And in the $40 skew, um, folks can author and create that enterprise level automation. And that that automation comes with an agentic planner.

Uh, it also has like the build, observe and deploy capability versioning that you would expect in more development like tools and also supports human in the loop so that users, um, so, so that like when the workflow maybe gets stuck or the designer of the workflow, the author of the workflow just wants to put in a, like a criteria for a human to be the decision maker if a policy, um, reaches above a certain threshold or something like that and quick automate that can be supported. So ba mike, so basically to, to reiterate, we have two SKUs, a $20 SKU that offers access to all the capabilities we just discussed. And then for power users that want to create these complex automate enterprise automations that's available in the $40 skew along with higher limits on the agent hours for research and automation.

So how will workflow automation evolve in the office going forward? 'cause I think, you know, there's a lot of requests that individual departments have put into it to help them build something and then there's a low code tool and it went back and forth and usually dies in the vine. And then there's also the whole notion that, um, I, I wanna be able to create a workflow that's kind of disposable.

I might only need it for a little while. So is that kind of how this is gonna evolve? We'll see more of disposable workflows because individual end users will be able to do things without necessarily requiring somebody who knows how to code something.

They do everything for them. Yeah, that, that, that's exactly it. And, and so we expect like all of these types of workflows that add add weight and complexity and undifferentiated work that, that users have to go through today just to complete their job, but they don't necessarily add that differentiated value to their goals and objectives.

Um, these are things that previously would probably be stacked up on like a center of excellent department, right? That may be looking at robotic process automation or even uh, teams of engineers that were looking to automate certain capabilities, but maybe it just couldn't be justified with some of the other things that were on the plate there. And now users in a no-code, natural language prompt can automate those workflows.

And so to your point, even if that workflow doesn't maybe have weeks or months of longevity or it serves for a a point in time, um, that can still be highly beneficial and a no brainer for a user to go in there and automate that workflow. And of course the same is definitely true on the other end of the spectrum for the more complex automation use cases where you can use agents and natural language and maybe a process document to get started to create what is a much more complex comprehensive workflow. And then the user can go in there and, uh, adjust or insert workflows as they need at any level and debug as necessary.

Of course that comes with test runs so that validation can be there, but it's absolutely the case, Mike, that we will see a level of automation opportunity that was just not attainable before. And so, and I'm really excited, you know, even in, in internally before we, uh, went with general availability with Amazon Quick Suite, you know, we had tens of thousands of Amazonians that were basically battle testing this. And so it was, it was energizing to see like the excitement and the love that they had in the utility they were getting across these capabilities.

And we were seeing exactly what you're talking about where we have folks that had probably never, not probably, that had never automated a workflow before that were coming to us and saying, Hey, look what I've done. I used to have this, I used to collect this survey across all these departments across the breadth of Amazon and that took weeks and weeks. I think they described it as months even and they were able to do that in, in an hour with Amazon quick.

And so, you know, these success stories are, are remarkable and it's so great to see them. So we are seeing that exact transformation that you described internally at Amazon and, and we're also hearing of these great success, success stories, you know, with our hundreds of external beta customers as well that entered the, the Quick Suite beta program before launch. Alright, Hey folks, you heard it here.

You can go after some big giant AI project all you want and you might swing for the fences and miss, or you can play small ball and just go after all these smaller little workflows that at the end of the day, collectively, I'm probably gonna make a much bigger difference. Hey Charlie, thanks for being on the show. Thanks Mike.

Thanks for having me. All right. And thank you all for watching the latest episode of the Techstrong AI Leadership Insight series.

You can find this episode and others on our website. We invite you to check those out. And until then, we'll see you next time.