Mike Miller on AWS’ PartyRock

ai video series. I'm your host, Mike Azar. Today with Mike Miller's, director of Product Management for AWS.

And we're talking about how to make AI accessible to mere mortals rather than everybody having to be a data scientist. What can we do here that's gonna put the power of AI in the hands of everybody? Mike, welcome to the show.

Yeah, Mike, thank you for having me. It's, uh, great to be here and look forward to, uh, a good lively chat. So how do folks get started?

Because on the face of it, you see all these massive large language models and they're huge on the foundational side. And, um, then I'm supposed to get a bunch of data scientists together and maybe some developers and a couple of data engineers, and I'll throw in some security people and hopefully something good will happen on the back end of that. Um, is there another way to think about this and, and how do I make this accessible to folks who may be used to, um, working with no-code tools, low-code tools?

Yeah. Or for that matter, no tools at all. Yeah, absolutely.

You know, and you, you really nailed it, right? AI has gone viral again, you know, uh, but the conversation still remains very technical. You know, um, you know, like you mentioned, if, uh, if a company wants to get in the weeds, they're confused, do I need data scientists?

Do I need a bunch of training data? Do I need to build my own LM How secure is it? Et cetera, et cetera.

And so, you know, um, uh, that, and that's true for both technical and non-technical folks, right? Uh, non-technical folks, in fact, are just left completely out of the conversation in a lot of these situations. Um, and so, uh, you know, AWS realizes that, you know, innovation can come from anywhere and we've gotta empower, um, you know, the greatest number of people to drive the most interesting innovations, given that generative AI is really one of the most powerful sort of profound technical changes, you know, in our generation.

So Amazon recently made an AI ready commitment, uh, to provide 2 million people with free AI skills training by 2025. We offer more than a hundred courses and learning resources on artificial intelligence, machine learning, and generative AI through our AWS skill builder and educate digital courses. Um, and, and that is, uh, tailored for learners of varying, you know, backgrounds and experiences.

Um, but one of the things that we found, uh, you know, getting people hands-on with machine learning over the last six years is that one of the best ways, um, to learn and gain intuition about AI is in fact to kind of play with it and, and be motivated to have fun with it. You know, people need a place that's designed with safety and security, but also has a fun aspect. So back in 2000, uh, 18, my team launched a product called AWS Deep Racer, which was a little, um, remote controlled car that we had built to drive autonomously around a track.

And you were introduced in a hands-on way to machine learning. You taught this little car to drive around a track. And then, you know, we had races, uh, and a league.

And so, um, you know, individual developers could get hands-on, uh, uh, and, and have a good time while learning some certain aspects of machine learning. Like fast forward, you know, four or five years, generative AI was all the rage. Um, you know, even internally at companies, um, you know, employees were looking for ways to get hands on with this stuff.

So one of our teams internally built something called the LLM Playground, and it was all about this kind of low code approach, you know, with no technical skills, no coding required. You could just get in and kind of in a safe environment, like play with prompting, play with turn, chaining these things together, wiring them up. And this thing took off inside our company like Wildfire, and we said, whoop, the light bulb went off, like, wait, if the employees in our company, who, by the way, almost half the users of this thing were like marketing people, um, you know, non-technical people, legal folks, business analysts were using this and getting hands-on 'cause it was so easy to use.

We said, Hey, we gotta turn this thing into something that, um, external customers can use. And so we launched a product called Party Rock in, uh, this right around the end of last year. Um, party Rock is an Amazon Bedrock playground.

It's an intuitive generative AI application building playground that makes learning how to build AI based applications and getting hands on with this stuff, uh, fun. So users don't need to know how to write a single line of code, uh, to start creating applications with Party Rock. And you don't even need an AWS account.

You can just log in with your favorite social media account, uh, and get hands on, uh, in a, in a freeway. How do I need to approach this from a cognitive perspective? Because at the end of the day, um, I may have prompts, but to your point, I'm trying to daisy chain prompts together to create some sort of action.

And I'm still constructing something that requires me to think logically to make things kind of flow in a certain order. Uh, it's not clear to me that we all naturally think that way. So, um, over time, does, does the Party Rock environment kind of teach people a different way of thinking about things in, in a way that a software developer might have thought about things all these years?

That, that's a great question. And it's something that internally, when we built this LM playground, the developers realized very quickly exactly that point. Not everybody thinks in this sort of logical, like, okay, this input turn, you know, the output of this turns into input of that and et cetera, et cetera.

And so we built this thing called an application builder, uh, which again, is, um, enabled by this generative AI revolution. We're literally, you just type in and sort of plain language what you want. I would like a travel recommendation assistant that takes us input, you know, a location and my travel preferences.

And this application builder will then generate an app giving you inputs, wiring things up together, giving you different outputs, uh, and it'll create a travel recommendations for app for you. It'll have like an input for like, you know, what are your travel preferences, an input for, like, which part of the world do you wanna travel to? And then it might have generated a prompt itself to say, okay, given these interests in that location, list the top 10 activities, or, you know, uh, places, sites to see, uh, for a travel itinerary.

And so users can kind of see an, an idea that they can express in natural language. It doesn't have to be even very well formed. And they can see how that translates to a set of logical sort of, um, widgets we call them, or logical inputs and outputs and prompts that get wired together.

And they can even go in and look at the prompt for each one of those things to start then gaining some intuition about how generative AI actually works. As we kind of start playing around with stuff, eventually we want it to actually go to work per se. And so how do I lift something from the area that you created for me to experiment?

If I build something I really like, how do I promote it into something in a production environment that I might wanna actually use on a regular basis? Yeah, that's a, that's a great question. And, and, you know, so Party Rock is an Amazon Bedrock playground.

I dunno if you're familiar with Amazon Bedrock, it's our foundation models as a service offering. So, um, with, with Amazon Bedrock companies of all shapes and sizes and all sort of generative AI skill levels, uh, can get access to a library of foundation models from, uh, some of the biggest, uh, folks in the industry. ai, uh, a wide range of models, models from meta, you know, meta's Lama models.

Um, and they can use these things as a service. So what my point is that Party Rock is built on top of Amazon Bedrock. And so as you're building apps on Party Rock, you actually have models Model Choice.

You can go in and experiment. You can say, well, I want, I actually wanna use, you know, Amazon Titan, or I wanna use Meadow's Llama, or I wanna use Anthropics Claude, uh, to do these different types of prompts. And you can start to, like I said, uh, get some intuition and gain some experience about like, well, this model might better at this kind of task, like language translation, or this model might be better at this type of task, which is like, um, you know, summarizing documents or having a conversation with you, right?

Um, and so you can use that experience to start, uh, getting a better sense of like, okay, well if I'm, if I'm building a generative a app application on Amazon Bedrock, which, because these are foundation models as a service, they're just API calls, right? And it's super easy, you know, we give you tutorials and sort of notebooks and things like this, so it's really easy to take the prompts that you've built in Party Rock and take those and sort of copy them over into your eight Bedrock API calls. So it's pretty, it's not necessarily like a one click export, but it's pretty seamless in terms of like developing the prompts and then sort of being able to take those over to Bedrock, especially since you're using the same, uh, set of models.

Will I mix and match various LLMs within the context of the same application? And do I need to kind of think about how to orchestrate all that? Yeah, that's a, it's a great question, and it kind of goes back to that fact that, you know, uh, our belief is that there's not gonna be one model to kind of rule them all.

There's not gonna be the one model that does everything, uh, amazingly well and is both cost efficient and, you know, has the right latency for your use case and, you know, performs accurately for your specific type of, uh, action. And so, yes, in Party Rock, you can mix and match, uh, LLMs from Amazon Bedrock to your heart's content. You could even have one input that generates a prompt, uh, and then sort of generates output.

And you can have like, let's say three outputs for the same prompt, and you can say different models for each one. And so you can have a nice little comparison there to see, um, how different models respond, uh, to different prompts, either different styles or different details. How do we educate people about what's really happening with these models?

Because, um, they're probabilistic, right? They're basically, it's a best guess based on the data that they have. And, um, that's an amazing achievement.

But it may not always turn out exactly as you hope, but I think a lot of novices think that it's deterministic and that whatever they put in there, they're gonna see the output and they're gonna automatically go, looks good to me without actually kinda validating it and putting this through its paces. So to what level of education do we need to have with folks to help them understand the difference and, um, you know, maybe not to just, uh, automatically turn the key on every time they see a new piece of code. Yeah, absolutely.

And I think this is something that as an industry, you know, we're getting better at. And I think the sort of zeitgeist, you know, I was just doing a presentation dictionary dot com's word of the year last year. Do you know what it was?

Hallucination? Uh, so, uh, I, I think it's starting to get, get absorbed, you know, across, across, um, users of all of all walks of life. But I think, you know, party Rock does a great job where when we present these sort of applications and sort of the app builder and even like the tone, the look and, and and feel of the tool, it's more about fun and creativity versus like, Hey, this thing will provide answers for, you know, every question you have, right?

And so, um, with Party Rock, we try to do a good job of sort of teeing that up upfront, you know, giving the user, we have a set of tutorials, in fact, uh, that are in fact built on Party Rock. So we have a Party Rock app that's a tutorial for how to use Party Rock and sort of have fun and how to explore these, um, prompts. And what we do with a lot of those tutorials is try to present all of these facts to you right up front.

Um, and in fact, we just did, um, a Party Rock Hackathon that I'd love to, I'd love to talk to you about, because, um, we used the Party Rock Hackathon to really sort of, um, both educate users as well as sort of inspire folks around the kind of things that, you know, party Rock and these lms, uh, could be good for, especially in the sort of creative and entertainment realms. Well, walk me through that a little bit. 'cause we too had our own little AI hackathon here, and as far as I could tell, I think, you know, over three days, we spent half of that just figuring out what data should go into the other one.

Yep. Uh, yeah, I, I, I can hear that. I mean, we should have, we should have worked together on this one, Mike.

Uh, so Party Rock made it super easy. We had, um, over 7,650 people from around the world to register for this hackathon. We had over 1200 submissions of apps across, um, a number of categories.

So we had creative assistance as a category, experimental entertainment, interactive learning, and then a freestyle category. Um, and so this hackathon was all about, uh, you know, get getting inspiration from your peers because Party Rock has this really cool way to share apps and then what we call remix. So, uh, you could take an app that somebody else built, kinda like an Engineering ling lingo, you fork it, so we call it, uh, we, we call it, um, remix, um, going along with the theme.

So you click remix and it basically makes a copy of the app for you, and then you can go and edit it and tweak it and kind of make it your own. So part of our category was like a remix category. So we presented a few different apps, and we encouraged users to kind of go crazy and remix these things and see what they came up with.

Um, and so the Hackathon, we ran it for a few months. Uh, it just ended, um, at the end of March. And like I mentioned, we had, uh, participants from all over the world.

In fact, we just announced our winners. Um, you know, we all awarded, uh, up to $20,000 in cash and a hundred thousand dollars in AWS credits. Um, the top prize went to, uh, believe it or not, an interactive crime thriller app.

Um, that was really fascinating. I used it, I mean, I was in there playing it for a good 30 minutes or an hour because it, he wove in, uh, sort of elements of like, here are clues, here are tools you can use to, let's say look up like user information in this fictional world, talk to suspects, talk to the detective, get hints and try to solve a case through sort of an interactive, uh, series of prompts. And he built this all, all in Party Rock.

And in fact, that guy was from India. We had our second place winner, winner was from the uk. Our third place winner was from Nigeria.

So, um, you know, party Rock enables users across all, all skills, all walks of life to really get hands on and do some fun stuff. I think, you know, we're still in the first innings here, and if I look at things, we're using AI to generate text or video or code or whatever it is, but it seems to me that as the engines get bigger and there's more parameters, there's gonna be a greater ability to orchestrate tasks. And you can do things like, say, build me a website, and it will organize all the things that go with that.

So are we at the beginning of this and, and how smart can all this stuff get? Yeah, I mean, uh, you know, if you're familiar with Amazon, you know, and Jeff Bezos and his, you know, shareholder letters even from the early days, we love to say that it's day one, right? And, uh, it's, you know, from the AI perspective, like we just woke up and, and we've barely had our coffee on day one, right?

It, we are so early in this journey and, uh, the potential is promising, right? Um, everything from, like you're saying, orchestration to agents, um, you know, a as well as the capabilities that you need to, um, ha to perform responsible ai. So that's one of the cool things about, uh, Amazon, uh, about Amazon Bedrock, this foundation models as a service, is that all the capabilities that, um, companies need to build really rich applications and sort of stay on the forefront are all either in there or are being added.

So there's Bedrock agents, which allows you to, you know, orchestrate more complex tasks and send, uh, individual, you know, foundation model powered agents off, um, to accomplish things and then orchestrate them when they come back together. We have built in, um, you know, responsible AI sort of guidelines and obviously policies for the usage. We have a tool called Bedrock Guardrails, which allows customers to sort of specify, you know, safety parameters and guardrails around, uh, output.

Um, there's a number of other like model evaluation and sort of all of the sort of, um, you know, model management and sort of infrastructure management that you would expect from AWS all applied to generative ai so that as companies, you know, really push the bounds, you know, bedrock is enabling them to do all these things without necessarily needing to have a giant, you know, organization of, of data scientists and of machine learning experts and things like this. There are a lot of choices when it comes to places to go play with AI models and build your applications. From your perspective, what ultimately is gonna make people pick one platform for versus another?

What do you think it will come down? Yeah, it, it's really about, uh, kind of choice and options and opportunities to find the right model to fit the job to be done. Uh, so Amazon Bedrock is really interesting from that perspective because it's a range of models and even from the same provider, we offer models across different price points and latencies and capabilities.

So take Claude three, the latest versions of Anthropics, Claude, for instance. There's Haiku, which is very fast, you know, low latency and pretty inexpensive. There's, uh, sonnet, which is sort of their mid-level tier, which is, um, pretty fast, uh, more complicated, allows you to like sort of engage and chat.

And then there's Opus, which has performance measurements that are off the charts. Opus, of course, is gonna have higher latency and it's gonna cost more to use. Um, and we offer those kind of choices across almost every third party, um, foundation model, whether it's text or image or multimodal that we offer.

So companies, uh, you know, want choice because they wanna be able to, um, you know, match the right sort of infrastructure and the right tool to the right job. Um, and that's where Bedrock really comes into play. Uh, along with sort of a, you know, our, our stance at AWS that, you know, security, uh, and and privacy are job one.

You know, you can be guaranteed that if you're taking a foundation model from Amazon Bedrock, maybe you're fine tuning it with your own company's data, that's a hundred percent yours. That is your instance of that model. It's private, it's in your VPC, no one's gonna get access to it.

And so, you know, at at AWS and Amazon, you know, that's job one for us. And making sure that companies can have the trust that, uh, you know, we're doing the right thing and that, um, you know, whatever data they, uh, share with us is gonna be secure, uh, but they're gonna have options. Um, and so that's really, I think, some of the key elements of sort of making a choice for where does a company wanna sort of bring their generative AI workloads.

All right, folks, you heard it here. Welcome to the AI supermarket. There's tons of LLMs.

You just go shopping down the aisles to find the one that's right for you, and hopefully, uh, you can get an in and out of there with as least amount of friction as possible. Hey, Mike, thanks for being on the show. Absolutely, absolutely.

Thanks for hosting me, Mike. I appreciate your time. Thank you all for watching the latest episode of the Techstrong AI video series.

You can find this and others on our website. We invite you to check them all out. Until then, we'll see you next time.