AI-Powered Innovation in the Cloud: Strategies for Breakthrough Results | The Six Five Summit

Hey everyone. Welcome back to the six five Summit. It's day one, we're in the cloud infrastructure track.

Really appreciate everybody tuning in and being part of this event. For this next session, I'm very excited to bring in Sasha Gupta. Sasha is a VP and GM of the infrastructure and Solutions group at Google Cloud.

And this conversation is gonna be all about innovation in the cloud, a big topic, something that every company's focused on, and we're gonna be looking at everything from public cloud to edge, to sovereign cloud, cross cloud, and so much more. Sasha, welcome to the six five Summit. Excited to have you here.

Thanks. I'm excited to be here. So I'm gonna get started right away, Sasha, and I think this is a really exciting time.

I mean, look, AI is on the tip of everybody's to, it is risen to the conscious of every enterprise, every business on the planet. And of course, Google is at the center of this. You guys are advancing everything from silicon to, you know, the next wave of applications and everything in between.

I'd love to get a little bit of a perspective on, uh, from you overall on kind of what are the advancements you're most focused on? What are you seeing customers get really excited about? And then of course, are there industries where you are seeing these use cases for the cloud rising quicker than other?

Yes, absolutely. So, you know, our, our approach is to understand what customers are trying to do with ai. Are they trying to do, you know, foundation model training?

Are they fine tuning based on a mainstream model? Are they inferencing and serving for their enterprise application and giving them an integrated system that's optimized at every single layer? So what we introduced at, uh, Google Cloud next is our AI hyper computer.

This isn't a, something just that came together very quickly. This is something that we've been working on through decades of research. And so at the very bottom layer of that, uh, hyper computer, uh, is our silicon.

So this is providing choice with the latest generation of Nvidia GPUs, but also providing the latest TPU technology. In fact, we just recently at IO announced Trillium. This is our sixth generation of tpu.

So yeah, it's not the first second generation. There are sixth generation of tpu. In fact, the new models we announced, like Gemini Flash, the new Imogen models, the the new Gemma two model have all been trained and are served on our TPUs.

So commitment to get the latest from Nvidia, but also provide options at that bottom layer with our own TPS and other silicon choices that will become available. And then on top of that, making sure that there's open software choices. You know, most customers access this through Kubernetes with GKE, but if they have other open source capabilities, like if they want to use Run, for example, if they wanna use, uh, other, uh, open frameworks of any kind and making sure that those are optimized on top of GPUs and TPUs, that's super important for us.

And then finally, they want to consume it in different ways. I need to run a training job for six months. I need to serve and I need burst capability.

I need to go up and down. So how they consume it, we provide great choices. And then finally, uh, we wanna make sure that they have access, not just to our Gemini models, the latest and greatest models that Google provides, but we will support over a hundred models through our model garden with Vertex so that they can incorporate third party open source Google models, build them into their development chain and deliver agents and enterprise applications on the other side.

So optimizing this at every single layer and then providing that integrated system, I think differentiates us, but provides tremendous value to customers. Yeah, Sasha, and that was a really good and interesting explanation when I was at Google Cloud. Next.

One of the things that really, uh, caught my attention was the amount of work that you were doing on your own silicon innovation. Uh, the TPU, you know, when I found out that Gemini's newest models were trained entirely on TPU, I shared that out, it actually created a ton of buzz across the social spheres when I was talking about that, because I don't think people fully realize and appreciate that there is an alternative. And I understand your whole perspective about homegrown versus obviously merchants.

And of course you have a great relationship with Nvidia, NVIDIA's doing really great things, but there is optionality there, and I think that's gonna continue to expand. And then of course, software is really important too, because you want model adoption, and of course you want people using Gemini. It's a very powerful model.

It's got a lot of mainstream attention, but there are all kinds of different model varieties that are gonna be out to the market. We're hearing more about small language, we're hearing more about industry specific. And then of course there are other open source, large language that people are building on, and that's gonna be critical, that Vertex and that Google doesn't limit, but really is democratizing access to all the tools and technology.

So great to hear all that. Definitely in line with the assessment that we've made over at Futurum Group, and of course on various conversations we've had on the six five. Now, I wanna go in a little bit of a different direction.

I wanna talk about the slower moving higher highly regulated industries. So, you know, we have customers that are moving to the cloud, but they're all moving at different paces. AI is accelerating, pushing them forward, but at the same time, in these highly regulated industries, there's all kinds of sensitivity.

Everything from the privacy of data to sovereignty to where data resides in residency. And of course, you know, we heard from Thomas Kian at, at next, you know that there's accreditations within Google. They're working across top secret and secret, uh, you know, workloads for the US government.

I'd love to hear a little more about that. You know, kind of what are you doing for government? What are you doing for regulated industries along around the globe?

And then of course, for this particular customers, For this particular group of customers, how are they thinking about ai? Yeah, so this is, this might be a bit of a longer answer, but the, I mean, the, the first thing I just wanna make clear is in our public cloud regions, we offer a set of controls like, where does your data reside? Uh, who owns the encryption keys?

The customers own those encryption keys, who has access to that data? So I need to keep it in this region in my country who has access? You can control all of that.

We also are very clear that when you feed your data into our models in our public cloud, we don't train our model based on that data. Your data is your data. Those enhancements are for you in your environment.

We're not learning from that. And so there's a set of protections there. But as you said, there are reasons like you're a government customer, uh, defense intelligence, or you're a highly regulated, let's say, central bank, uh, or an energy company.

And then what happens is there's simply no way that the regulations will allow you to move that data into the public cloud. So for that, we built a product called Google Distributed Cloud, and think of that as the ability to take our services such as translation, you know, translating from dozens or hundreds of languages, speech to text APIs, optical character recognition, vision prediction capabilities, and moving those into a hundred percent air gapped cloud environment. So it's your own cloud that delivers infrastructure services, database, data management services, AI services locally for you on, on-premise.

Okay. And so the, the latest actually example that we hear here, uh, is about I wanna be able to do search, you know, it can be multimodal on multimodal content, multimodal data. I need to be able to do search, but it needs to stay within my premise and it needs to be on my own data.

And that's what Google distributed cloud enables through a complete solution that we provide. We also use a product of ours, which is Alloy DB as a vector database. As part of that solution, we use our own open source model, Gemma, but we can use any third party open source model that, uh, we can run on this system.

So a very powerful AI data, highly secure solution that runs a hundred percent air gap on-prem. So that's the defense use case. I just did wanna add one other example.

And so on the defense side, by the way, you know, we're very proud. Like we have CSIT who provide services to the Singaporean government that's deploying Google distributed Cloud. We're also working with, uh, Proximus who's deploying this in Belgium and Luxembourg to provide highly secure private cloud air gap cloud services.

So, uh, seeing great momentum in those segments. And then you have a very different example, which is, you, you need your data on-prem, but it needs, doesn't need to be air gapped, meaning operations can be done centrally. So think of this when you have hundreds of sites or thousands of sites, and you wanna provide some kind of data AI capabilities in those locations, and maybe because of latency or survivability of the link, like the link may go down, you still need to have services locally.

That's where you wanna make sure that you've got cloud infrastructure in each of those sites. So one example of this is McDonald's. Another example is Oran, uh, orange, uh, where, um, with McDonald's they have, you know, I think almost 40,000 locations where they want to be able to, uh, get operational simplicity, get, you know, all all of their data points, make, uh, their tool smarter, but really make their customer experience better, make the experience of their crews better in those stores.

And so McDonald's is working with us to deploy Google Studio Cloud in all of those stores. And then they can build AI capabilities on top of that, such as automated order taking, uh, a a as one venue add that many retailers are are absolutely looking at. And then Orange has operations in 26 different countries.

They need to be able to keep data, analyze that, apply ML to it, but at needs to stay in each of those countries. And we're able to deploy Google distributed cloud in each of those countries to service as use case. So, you know, we're absolutely about enabling AI anywhere.

Okay, so the best place to do it, you get massive scale. All the latest and greatest technology is our public cloud regions, and we have a set of controls there. But if we're latency, survivability, cost, or for, uh, regulatory reasons, you need something that's outside of our public regions.

We have Google distributed cloud, Sachin, there's no doubt that Google has continued to advance its services capabilities to be able to address both the, you know, various complexities of highly regulated industries, and of course dealing with sort of the distributed and hybrid multi clouds. And that's going to be the norm going forward. Uh, you know, congratulations on, on, and, you know, receiving those clearances and being able to work with those various governments, I think those applications are gonna be important.

And of course, governments are gonna want to benefit from what AI can do. Uh, it's gonna accelerate every industry. No different to the highly regulated defense contractors and defense, uh, you know, organizations around the world.

Now having said that, you sort of, you know, you sort of lent into a bit, you, you were talking a bit about the kind of distributed options, and I wanna talk about that because we all know there's, there's significant technical debt. We all know that every workload is in, in the cloud. You know, one of the things we talk about regularly in the, on the pod, Patrick and I, is kind of about workloads on-prem versus in the cloud.

And I think over time the TAM is expanding. We're seeing more workloads go to the cloud, but we're also seeing the overall workloads grow. There's still a lot of demand for, for certain amounts on-prem, and there's also a certain amount of de demand for multi-cloud.

I mean, Google has accelerated really quickly in the era of ai, but companies have already had workloads in other clouds. So I would love to kind of understand how you are approaching and engaging those customers and really helping them deal with this in time because maybe they're seeing Google as a great opportunity to address the AI challenges and they wanna move more there, but they may very well have workloads in other clouds edges on-prem. What is the kind of current state of your approach for multi and cross cloud?

That's a, that's an excellent question. And especially with ai, you know, you're talking about BigQuery being leveraged for data analytics and ai, uh, as well, what you find is how you move your data securely between other clouds or multiple clouds. And on-premise, because in many cases it's still sitting in on-premise data centers is a complicated question for customers.

And so we're solving this by introducing something called Cross Cloud Network. And what the Cross Cloud network does is at the very fundamental level, is makes it simpler, makes it cost effective for you to connect all of these environments. So think of, uh, connecting your on-prem environments, your multi-cloud environments, your SaaS environments, all into the Google backbone.

And the Google network is now your network as an enterprise customer. So to make this real, there's some capabilities we reduced, like cross cloud interconnect, where we can take away a lot of the pain of putting routers and firewalls in, into colos to provide secure simple connectivity between clouds. Many customers are already using this.

And then we actually introduced something and we're, we're working with Scotiabank on this actually, where we can take all of the services that are sitting on prem or in another cloud provider or inside Google Cloud. Like for example, they, you, you mentioned BigQuery, that's a service they're using. They're using ISV services in our cloud.

They have wallet services and credit card services that they integrate with. And you can create a representation of that service in a, I'm getting technical here, but like a service landing B, B, C, and think of that as the ability to apply policy in terms of who has access, how do developers access these services, security and simply to build the apps that they're trying to build. So it's about simplifying data movement, it's applied offering a whole suite of security services because they have to secure that data.

Uh, and it's about representation of all of the different environments that customers have in one consistent way so that the network teams and the security teams can move much more quickly. And the developers are not sort of encumbered by those complexities anymore. And so Scotiabank is just one of the customers seeing great value in this.

We're thinking about really making the Google, uh, tremendous network the backbone that we have, our sub C table investments that we have. We l we run all of our great multi-billion user products on it. We want that to be the network that enterprises can use as their own Statin.

I really appreciate you covering so much ground there from the architecture of your chips all the way to the networking of multi-cloud. You covered a lot of ground. You were great.

It was really a lot of fun to have this conversation here at the six five Summit. Appreciate it. Let's have you back sometime soon.

Thank you so much. And for everyone out there, we appreciate you being part of the six five Summit. We've had a great day one so far, and we hope you stick with us.

We have lots of content here. Be part of our community. Stick with us for all three days if you'd like or be able to check the sessions out on demand.

But for this one, for myself, I gotta go say goodbye. We'll see you all soon.