Techstrong Gang – July 25, 2024

Hello, everybody. I'm Mike Baer and welcome to Textron Gang. Today we're gonna be talking about, well, double AI agents, a whole lot of interesting stuff about how to figure out how to, who are AI systems.

And then finally, maybe why is DevSecOps Seems like it's a lot like teenage sex, and we'll explain that in a minute. You're watching Text Strong. All right, folks, welcome back.

We got a full house today. Joining us once again is Mark Henkel. Mark, how are you?

Where are you? I'm great. I'm in Raleigh, North Carolina today.

All right. We, we, we might have a, a, a lot of action in North Carolina. Paul n joining us too today.

I assume you're home. I am home, yes. In North Carolina as well.

All right. And finally reaching out to Denver, we have Mitch Ashley, who's as always Mile High. I'm in the northeast part of Denver, if that counts.

No. All right. And then finally we have Bonnie Schneider or All Things Green It and Weather Expert.

How are you? I'm doing well, thanks, Mike. All right.

Let's jump in with this whole, um, AI agent conversation. Um, Salesforce announced a partnership with Workday. Um, what they're really working through here is some data virtualization technologies that will allow their AI agents to collaborate with one another.

And Mark, let me start with you, but I can't help but wonder if this is the beginning of maybe a thousand of these types of relationships as AI agents of all. Yeah. They, they are, but I mean, I don't think the, the integration is so much, the agent is just the fact that we're in an API driven world, and these AI agents have the ability to consume that and analyze these APIs.

So, um, you know, back a few months ago, or maybe few weeks ago in the internet time, it's moving fast with ai, but, um, uh, we, they talked about, um, uh, large action models came out from Microsoft, um, back in February. And that's the idea that the interfaces between things will be orchestrated, but by large language models rather than by, um, do or even voice, um, integrations that it'll, it's really gonna be, you're gonna talk to the large language model. It's gonna do stuff.

So you're gonna see that a lot. And that's sort of the state of agents today, is they are taking your, um, instructions and calling an a PIA lot of Zap. Zapier is having a heyday with that, as well as others to, um, take the, the output from one system and use it as the input for another.

And they're, in this case, they're, they're parsing a lot of data from the, from what their announcement says. I think that you nailed it, mark. It's about automation Plus, you know, yes, it's integration, but to, to accomplish workflow automation and that are driven by the models opposed to wiring up, you know, erector set created workflow processes, whether it's, you know, easy to do through a URI or not.

But more and more we'll see AI agents helping us. Like, I wanna set up a workflow for that, or I recommend a workflow. There's one that Mark set up.

Would you like to use it? Yeah. Yeah.

It's, I think it's sort of an evolution of robotic process, uh, automation. Mm-Hmm mm-Hmm. It's like the big companies like UiPath, they've, they've bridged the legacy by automating things, but I'm not sure if the AI story is playing in yet.

'cause they lack a lot of the reasoning and problem solving that would make it really interesting. But you're right, Ramit, it's all about automation. You know, you Have a research note on this, on toum, I believe.

So what's your take? I do, yeah. Thanks.

Thanks, Mike. And, and you know, Mitch and Mark, I, I agree. I agree that there's a lot of, um, stitching together that, that occurs.

But like, the thing that I found most interesting about this, um, announcement is the tighter integration to the overall Salesforce platform. I mean, let's, let's call it what it is, works Workday and Salesforce, they're both industry titans, right? They're both very powerful platforms that many organizations are using and they use instead of using them independently in silos.

Um, leveraging the Einstein One platform is something that Salesforce has been trying to kinda get in. And, and combining that work with Workday AI is definitely a, a, a focus for those LLMs, right? So Mark, I agree with you that there's some harmonization that needs to occur and, and Mitch as well, when you were talking about, uh, some of that.

But, but there's a, but there's definitely the, uh, desire to communicate that natural language, everything that we've been hearing about from an AI perspective, but not so much. And then, um, uh, I don't wanna call it like a me too kind of play here, but, but Salesforce is, is is powerful, right? And when we look at Salesforce, and not just from a tech stack, but from a position in the market, and, you know, data Cloud is also being tied into this announcement where we have, uh, shared data, data foundation platforms that are built as well as using the, um, the Salesforce zero Copy partner network is part of, uh, how Workforce is bringing this together.

So this is allowing for seamless integration between Workday and Slack for sure. Uh, as, as well as, as, as the overall, uh, you know, overall perspective. I do wanna also tie in some of the key points here that I find interesting.

'cause I, from my perspective, when I, as I, as you all know, or may know, I covered the, the DevOps and the, and the app dev space, uh, you know, I really looked at it from that lens. And when I look at it, I, I really kind of pulled out about five salient points from this announcement. Uh, one would be the AI powered, uh, employee service agent, right?

That that is something that's helping with, uh, develop and handle employee, uh, inquiries across AR and business functions. So that's something that's interesting, right? Because if you could, if you, if you can kind of go feed the data to, regardless of the platform that you're on.

So if you're on Salesforce, you're on Workday, and you can cross reference each other, that's really powerful in my opinion. So you have one, I I dare I say one pane of glass to look at. We know we don't have one pane of glass, but just in this context, I think it's, it's relevant.

Second data point, I think is, uh, that's relevant here, that comes to my mind is the shared data foundation, right? So when you have this unified data platform and the underlying power of the AI agent that kind of ties it together, it really improves that data accessibility. That to me is harmonization data is the new oil, right?

So to so to speak, right? And it runs the business, right? So if you don't have access or you have silos of data, that's gonna cause complexity.

That's something that I think is going to over, uh, be overcome with this. Um, a couple of other points I want to tie deeper integration. Mitch and Mark, you talked on this integration is key.

The API calls your key. I, I, I found that deeper integration between, uh, you know, Workday and Slack is also gonna help with collaboration and productivity. That's something that I think is gonna tie it together as well.

Um, and then a couple of other key areas that I, that really, uh, stood out for me was the employee, uh, experience that's gonna be improved with streamlining of onboarding employees, uh, using that, that one data source, having that, uh, you know, HR and, and, and career development kind of perspective. But most interesting, in my opinion, is the opportunity for developers. And this creates a partnership for the developer ecosystem to really expand across the AI capabilities.

Marked to your point, it's in its infancy, right? It needs to grow and it needs to change. And it's, it's going to, but this is where the developers can kind of come together and use one set of tools, one set of tech stack in order to develop across different, different aspects of the business.

That was a lot to say in a short period of time, but I hope that that kind of summarized the, the, uh, from my perspective. Yeah. Marco, oh, go ahead, Mike.

Well, mark, let me get your opinion about this, because, um, yesterday we had SAP as a topic on the show, and they were talking about how, you know, go with one vendor, one platform, we'll just take care of everything end to end. Um, is, and, and they were saying, you know, in the age of AI with all the data that's required, that that argument is more compelling than ever, but you know, we still have this kind of best of breed world that we live in. Is that gonna fundamentally change in the AI era, or is it gonna be just as we've always known it, and people will choose between one massive platform versus best of breed?

Well, I think you're the perfect example, Mike, of why we should go with, uh, multiple vendors, because it's really hard to be good looking and smart like you. It's very rare. So I mean, the, the, the key is that, wait, I, Where, where am I sending that 20 bucks to you?

North Carolina, Apparently. Uh, but no, I, I think that, that in the, in the legacy IT world, we definitely, you know, like that one vendor approach until you got locked in and you got, you know, massive Oracle bills, or you've got locked in and need a feature that just isn't on the roadmap for your vendor. So I think ever since we moved to the cloud, and microservices is sort of the de facto pa design pattern these days that it's best to pick best of breed.

Now there's like in a vertical and data or business analytics, yeah, you want to have, you know, things that work together, but that's totally, um, when we see Salesforce, they're, they're like, yeah, we can be very good at, um, serving customers, but we may not be the best at, you know, hr, so let's make our, take our customer service and make that the, the front end for employee relations. Um, so I, I'm, I mean, I'm, I'm definitely, I have commitment issues in general, but, uh, commitment to, uh, to a, to a technology has just bitten me too long. And I, I really, I, I think that we, we have like a very valid point in the fact that, that people want to integrate their, their tools.

So Slack with Workday, et cetera, um, it's all, all makes a lot of sense. You know, there's a point of integration or different potential differentiation, sorry, Paul, be short. Um, and that is, you know, Microsoft made a quite a bit of hay when they did their co-pilot for, uh, for 365 announcements that they have a, uh, context graph that really connects all these different data sources to help help the AI make intelligent decisions about how the, that information is connected.

Atlassian did the same thing at their Atlassian team talking about their team graph. Um, so, so there's a potential line of differentiation of here's the things we can do within our own monoculture ecosystem because of all our products and how, how we can tie them together. We'll integrate with other things, but it may not be as rich integration.

So there may be a, a kind of a two tier system here between, you get other advantages like you do in in, in other parts of working with a single vendor, but in the AI terms versus, you know, putting together multiple vendors. So this, that's a supposition on my part. Yeah.

Mitch, I, I, no, this is, that's exactly what I was going to kind of drive towards. But Mike, I do want to kind of comment some of that. You led the conversation with Mark here.

This is, I don't view this as a, uh, this or that. This is the way I view this partnership is it's, it's up to it's customer choice. It's where they're what, where the customer's using the platform, what to do.

And Mitch, I think this is echoing your point. If, if the customer's using a tech stack and they're comfortable with that tech stack, the key here about this announcement is the alignment of the data set behind each one of these platforms, these tech stacks. And that data set can be applied across the different, uh, areas, whether it's HR or, or finance or whatever it may be, or marketing.

You know, there, there's different areas where that data stack, if it's siloed, you have individual approaches. And that's where I think the connective tissue here of bringing it together really has the power. It's, it's, and the freedom of choice and best of breed is always going to be where customers go.

They're gonna use what they're comfortable with. Mark, let me follow up on that point, because this is the thing that, uh, has me kind of quote unquote, it's a mic scratch your head moment. If the data is normalized or harmonized between these different vendors, won't that ultimately lead to more merger and acquisition activity?

Because one of the barriers to that will drop, because I'm not gonna have to spend all this time organize on all this stuff. 'cause I'll take advantage of data virtualization. I'll throw some agents together and the customer won't really care where the data resides, per se.

Or is that a, a stretch too far? Well, I mean, I, I think that that's probably one of the biggest hurdles to m and a m and A is, you know, bringing the data together. But I think the bigger hurdle was the SEC and their, um, sort of anti-competitive or anti-competitive stance, um, against these, these mergers.

So, um, but I think what's, what's gonna happen, and I think Paul, there's a really fine nuanced point to, um, what Paul said, and I think it's the real story here is that where we are today is in that stochastic patch, um, parrot phase where we are really good at the natural language processing and using that as the interface. So I think what's gonna happen is, in the near term, we're gonna have interface, you know, advancements, but we're still gonna rely on ETL at extract, transform, and load for, for data normalization. But in the long term, what's gonna happen is that those models are gonna get better at taking disparate data sources and create a data, creating data pipelines that inform large language from the AI standpoint, at least the large language models and the, um, you know, populate knowledge graphs and do things that actually make AI get better.

But for right now, um, the AI story here, in my opinion is, and Paul nailed it, was just a hundred percent the, uh, um, fact that it makes natural language processing the interfa, the defacto interface in the long term. I don't think it's the data normalization, I think it's just the data interpretation and, um, making it consumable in a consistent ways. So one fire hose that everyone drinks from, It sounds like it's a more energy efficient as well if we're combining forces.

I, I agree with what Paul said about productivity and also, um, you know, combining, combining assets. But, um, it, it is interesting if you, if you do eliminate competition, that that could be a negative. It'll Be interesting, folks.

We're gonna have to wait and see how all this plays out. But as we've talked about in some previous episodes, we're trying to figure out ultimately if there's gonna be one agent to rule them all, or thousands of agents that we gotta figure out how to manage. We'll be back in a minute, Discover how cloud native is becoming the new compute stack at cloud Native.

Now on July 25th, we will explore the transformative shift towards modern containerized and microservices based applications. Learn my Kubernetes remains a dominant force in container orchestration and how WebAssembly is emerging as a powerful compliment. Don't miss this opportunity to stay ahead in cloud native development, register now and claim your spot in the future of tech innovation.

When you talk about ai, uh, one of the things that's not always part of the conversation is how much water AI is depending on to keep data centers Cool as it processes all of that ai Well, you know, it's interesting. If you look at chat GBT for example, let's say the average person is doing 10 to 20 prompts on chat GBT. Now imagine a hundred million users at the same time.

Well, how much water is a that way too? It could fill over 20 Olympic sized swimming pools. So we're looking at a tremendous amount of water.

I wanted to take a closer look at some innovative solutions to solve this problem. As more and more companies lean into ai, how can we reduce water usage and lower the water footprint? Hi, I'm Bonnie Schneider with your Ecotech Analyst Insights.

Today we're talking about AI's energy demands and using less water. As AI technology advances, so does its use of water resources. For example, training one AI model can consume as much water as several households in one year.

That's because data centers need water to prevent overheating and keep cool. But how can we use less water to accomplish this? One way is for data centers to use less fresh water resources.

For example, Google's data center in Finland uses sea water for cooling. Another strategy is to improve the efficiency of AI models themselves. By making algorithms more efficient, we can reduce the energy and the water needed to run them.

This involves using advanced AI to optimize its own processes, a kind of self-improvement loop. An example of this is found in Nvidia, despite their high power use NVIDIA's upcoming Blackwell, GPUs are designed to complete AI tasks faster, potentially saving overall energy. Solar and wind power can also reduce the indirect water consumption associated with electricity generation.

Google and Microsoft are investing heavily in renewable energy projects to power their data centers. For tech companies public awareness and transparency is essential. It's best to disclose water usage and communicate the steps you are taking to improve sustainability.

Reducing the water footprint of AI is a multifaceted challenge. It requires innovative data center designs, more efficient AI models, a shift to renewable energy and greater transparency. By addressing all of these areas, organizations can lean into AI advancement without draining our water resources.

Well, you heard about some of the innovative solutions that tech is coming up with to solve this problem. I think one of the interesting ones is using seawater versus freshwater, and then of course that's gonna come into play. Where do you locate these data centers?

Another method for cooling has been to put data centers in more cool cooler naturally, uh, locations. So water usage is an interesting topic, and I wanna also pose it to the panel because one of the criticisms that big tech is facing is that we are seeing, um, we're seeing a lack of transparency when it comes to water usage among big tech. So I'm wondering if we can, uh, bring it out to everybody.

What is your take on this? I'm gonna jump in and just say, I think there's a lot of areas where there's a lack of transparency, Bonnie, um, in the kind of the, the metric or the examples that you gave of how many prompts end users might be entering per day. Um, I think the real under the surface of that, that the iceberg that's the not showing above water, is all the API calls that are being made by applications to services, to their own AI models, whatever it might be, that that is, you know, probably who knows tenfold, maybe 20 fold, what end users might be, might be doing.

So I, I, we need something to surface to follow the, the iceberg model surface. Make that visible of how much energy, how much power, how much water resources that we're, we are consuming with ai. Something that we can just make some intelligent decisions about.

How much is enough and how much is too much. Mark, do you think that energy is gonna wind up being an inhibitor for AI adoption as we kind of start to Bonnie's point at billions upon billions of Carl Sagan prompts going everywhere? Do we have the energy resources to drive that?

It, it already is becoming the sort of, you know, to use the environmental metaphor is the, the underlying part of the iceberg isn't, you know, lack of Nvidia GPUs, it's lack of, um, data centers that don't have enough power and cooling to accommodate Nvidia GPUs. Um, even if you can get them, I think that, um, what AI is doing is exacerbating a problem that had existed before it hit, it became a mainstream thing and will exist until we fix it. We, you know, all of these power generation techniques that we use that aren't solar or nuclear are, you know, very, uh, water depleting.

We, we need water to, um, you know, do cogen. We need water to cool these, um, to do evaporative cooling. We need existing, uh, power needs to be, um, increased many fold to handle the increased heat from these, uh, um, power gulping GPEs in these data centers.

So I'm, I agree that, that I think it's probably, and I write about this in my newsletter quite a bit, is I think it's the, you know, the thing that's keeping Jensen Wang at Nvidia up at night is even if we make it, do we have a place that can handle the, um, the heat, the heat concerns in the power consumption concerns in general? Yeah, and I think that's one thing, Jensen when had addresses in a different way where when those questions are posed to him, he focuses on the efficiency of his models and the new Blackwell GPUs, um, being more efficient. So it's almost a way of saying, okay, we do are, we are using a lot of water, but we're going to be using less of it because we can operate more efficiently.

Hey, Paul, are you ready for a data center to be built in your backyard? I mean, 'cause that seems like where we're headed. Yeah, I actually am.

It would be interesting to see just pumping the water outta the ocean here, but, um, but no, you know, look, I'm gonna take a little bit of a different spin to this conversation and, and, and I think that it's important to note that, and it's probably the less popular spin. Um, you know, I write a lot about sustainability and, and, and the impacts to, you know, what, what's going on with regards to AI and application development and such. And, you know, ultimately in, in, when I, when I talk to CIOs and I talk to organizations about their monetization efforts and such, ultimately it comes down to a business decision.

There's, there's sustainability is, and, and just bear with me. It's a nice to have for most scenarios, even though a lot of con uh, uh, organizations, a lot of places, compliance regulations and and governance are in place such as the EU that says, you know, you have to do this. Um, but many organizations are looking at the bottom line.

It's how they can, how can they stay competitive and produce most effectively and efficiently, but they have to also look at it in the, in the context of, uh, responsibly as well. Right? So, Mike, to your point, building a, uh, a, a data center in my, in my, in my backyard, I think that's a great, um, opportunity.

A great advantage if it's to Bonnie's point, the right location, where, where do you put it? Right? Where is it that makes the most effective, efficient use?

But it also has to be cost effective too, right? And that's where, uh, you know, where we see solutions where we're taking the output from the hot water generate hot water in the building, for example, from the, so the output from the, the data center that, that creates the hot water for the building. But that's, there's only so much you can do with that, right?

Uh, I think that there's a lot here. Uh, and, but again, we also have to keep, I say keep our eye on the prize, so to speak. Organizations have business KPIs, organizations have results they have to drive towards.

Are they going to spend and sacrifice sustainability over results? My opinion is yes. I think though that that, that things are changing in that regard.

I think that, uh, the more they're made aware that they can do both and not have that maybe even be more cost efficient with making sustainable changes, the more it's gonna get looked at. And the other thing that I've mentioned on previous tech strong gangs, um, Paul, and that's interesting to note, is that we are in a time where we're changing, um, the rules and regulations. For example, we're seeing the, um, European rules will, that are general reporting that's in effect now people are gathering data.

But come next year, we're going to see sector specific rules. And that means that the technology sector will have its own set of rules that it's going to have to oblige by. Plus, ironically, this is, I'm talking about water.

The state of California is also implementing its own, uh, ESG reporting role. So in my opinion, the combination of the tide turning where we're going to have more regulation, um, over the next, let's say year to 18 months, and new opportunities to be more sustainable, but also be cost efficient, I think that's going to switch a little, the mindset where it's going from. Nice to have to, well, we need to consider this for regulation, and if there's a way to do it and be more cost effective, um, then they'll implement it.

And the final point that I've also expressed before is the, the interest in employees and the interest and the people that are working there, and just the company presence that of our, how are we being more sustainable? How are we more addressing it? I think those questions come up.

How much, Mitch, how much of this is an architecture problem? And I just want to ask this question to you because it seems like it's a software problem, at least in my mind. The fundamental is, let's say that data center that I want to call in Iceland, it's just too far away for that.

API call mm-Hmm. But I can't build massive data centers in the backyard because, uh, I can't move all that water around all the time. But isn't there there a way to architect this stuff so that, um, you know, it's a more distributed environment and we are being more energy efficient.

And maybe developers can think this through a little bit. Well, it's on, it's on a, uh, counter, uh, graph, if you will. Um, which is the kind of utility of the cloud, the elasticity of the cloud and resources, availability versus cost, or in this case sustainability.

Um, and unfortunately, we don't consider cost that much when we're designing software until we get into bigger applications. And then we start worrying about, okay, how much is this gonna cost to run per month? How many servers am I, you know, how many instances, how many Kubernetes clusters, whatever it is.

So there are definitely ways to design things to be more efficient. I think cost is always gonna be the driver of that, at least for the near future. Maybe, maybe sustainability becomes something like that.

I almost think we need like a lead standard for data centers or for software. So you can certi at some point, certify your software to be a, you know, a leads class two, uh, architecture or a data center that's this level of sustainability. Um, something like that.

And then that way, you know, you could decide where you wanna run your stuff to be more efficient and more sustainable. Mark, you like that idea? And where would I put that on my software box that I don't have?

Yeah, I mean, I, I think that, um, you know, I go back to the Charlie Munger of fism that says, um, you showed me the, um, incentive. I'll show you the result. And for the, there's two different, you know, things here is if you're a public company, the, the, the incentive is for better returns to your shareholders.

So I think what the first thing is, beyond anything else, is if you really want to see mass change, especially in a public company where they're beholden to their investors in a very transparent way, you're gonna have to show them a way to invest in, uh, reducing their costs via, um, green power. So we've gotta solve that problem. And the other side of things, if you're private, you could be like, uh, um, you know, Nova Nordisk is actually one of the most valuable pharma companies in the world, but they're a nonprofit.

So they can make that decision that says, you know, we can, we're gonna invest in, you know, supporting the, uh, the people of Denmark and we're gonna make profits around the world. So when it comes to these certifications, I think the certification probably would be something that offers some kind of tax incentive to a public company or some other benefit. But I think it all could be solved.

I think it's just, um, the way that we're, we're talking about it right now is hopefully AI will get mature enough. They'll come up with the answer and tell us how to, you know, reduce the cost, reduce the power, and not kill the environment. Uh, I mean, and I'm being a little funny, but I'm also being serious 'cause I think it's very hard calculus.

And one of the things that'll do really well that we don't do as a human species is parse a massive amount of data and find those key insights in that, like, massive triangulation of need for ai, need for power and saving the planet. I think it's interesting because we're talking about being cost efficient and incentives. So I mentioned regulations, but one of the other factors is, um, using renewable energy and leaning into that.

And all of the big tech companies are doing that for their data centers and partnering with some utility companies to do it. So there's an incentive for that. And then of course, there's government incentives because we're also seeing a tremendous amount of, of grants and money that's pushed towards using renewable energy, developing renewable energy.

So in, in many cases, and as this evolves, it is gonna be that cost efficiency as well as using renew, leaning into renewables, which seems to be across the board for big tech and objective. Paul, are we gonna send developers to, uh, energy efficiency camp, maybe teach them that the right software that is kinder to the environment? You know, I think it's an interesting perspective.

'cause uh, that's, that's where my head was going with this conversation. I think that there's two things. One I wanna start with.

Um, if we make, whether it's rules, compliance regulations that box out solutions that are so cost prohibitive that the solution's just not gonna be affordable for anybody to buy. That's, that's a non-SEC. That's not gonna start, right?

It's just not gonna start. And people are just not gonna do it. So I don't care, you know, how green we wanna be if we can't build it right with that's within a cost structure, it's gonna have to, it's gonna have to be adapted appropriately to what the market demand is.

Second thing is, I think I want to touch on what Mark was talking about here, and Mike, you kind of said it in a, in a different way, but the way we can look at creating these applications in AI is, is education. And a lot of o well, what I see a lot is of over provisioning is occurring for AI modeling and utilizing resources that businesses frankly don't need the full-time utilization of these GPUs. They're using GPUs and, and they could be using GPUs to do the job that the GPUs is doing because they don't really have a sophisticated AI model.

And that's an example. Or you might have this ebb and flow where you can do some more, uh, do some time sharing of that GPU. This is where we have to rightsize the provisioning of the resources for the applications that are being created.

And Mitch, to your point around the architecture, you know, one of the things that comes to mind is the folks that are building these applications, they're being asked by their leadership to create and push code out the door very rapidly, right? They, the cadence of delivering applications is very, very fast now, and they're asked to do this, but they're also saying, in turn, we can deliver that if you give us these resources they need. The challenge is, is a lot of the DevOps teams that are putting these resources in place, they're over provisioning.

'cause they don't want to be short when they deliver, right? So if there's an mike going back full circle to your comment, if there's an education on what is being used and how to provision the right amount of resources, we can reduce the cost for, for the, for the usage of these things, which it will help with the sustainability effort, and then also provide the, the, the rapid release and meet those business KPIs. So it's not the silver bullet, but there's definitely an education that has to, that has to occur.

You know, in addition to over provisioning. I'm sorry, I was just gonna jump in the addition to over provisioning. Uh, what you're talking about, Paul, is, is AI the right tool for the job, for every place that we're using it for?

Are you wasting resources trying to accomplish something through an AI model or machine learning when it could be potentially built on a, on a traditional, uh, architecture. So we may be using, we're still learning, right? Many organizations like, where should I be using generative AI and, uh, is that the right place?

So I think we'll probably have some waste that's coming from that as well. Sorry, Bonnie, I stepped on what you were Doing. Oh, no, no, that's fine.

I was just saying anyone that's watching that wants, uh, more information on that is the Green Software Foundation is a great resource and they are growing in membership, uh, rapidly with, uh, major organizations joining because of the interest from developers looking to, uh, wanna learn how to code in a more e energy efficient way. So I think it's something to keep an eye on. All right, folks, we gotta cut this off here, but it's pretty clear to me that saving the planet starts with that first line of code.

Go from there. We'll be back in a minute. All right, folks, we're back and we have joked about this a at the top of the show and probably in some previous episodes, but, um, we talk a lot about DevSecOps, but it's not quite happening as much as we think.

And there are a variety of reasons for that, that we'll jump into that. And I guess if you've ever seen any of the studies about teenage sex, that's what we're referring to. But all let me ask you this straight up.

Like if you look in this report that Jfr put together, they said that only 30% of the folks that they surveyed were even concerned about vulnerabilities in their software. And for all the talk about DevSecOps and all the issues that we've been looking at for years now, why do you think that is? And what is the state of DevSecOps in your Mind?

Yeah, it's a, it's a great, great question. Uh, you know, I, I, as a, as an analyst, I do just a lot of understanding the data in the market, the business challenges that are happening. And when I think of this, the overall CID pipeline and the SDLC, you know, I I, I've done some trending data over the last four or five years on, on, uh, the, basically the results of the CICD pipeline.

And, and you know, what's interesting to me is I found that in 2022, um, the, for continuous integration, uh, testing, only 29% of organizations responded that they were doing it. And, and part of the reason behind that was very similar to your comment and your question mark Mike, which is, um, you know, there's a misunderstanding that if, if they see or find a bug that's in production, that their sprint releases and the cadence of releasing these applications happen so rapidly that they can, uh, address it with a sprint review. And, and with it, you know, with the agile software development process and such, now when I look at that trending information, I see that in 2023, that number jumped from 29% to 66% of organizations that responded to doing continuous testing.

Now, why is that? Well, in my, uh, you know, and I was kind of going around talking on the network on different organizations and companies about what they're doing and vendors that are, that are addressing these challenges. And I was expressing my concern of saying that this is not, uh, not how you should be doing it, because obviously if you have a vulnerability or a problem with your application and it's not addressed, uh, you lose that, that that prospect or that client that's using that application.

'cause we're, we're a very, uh, uh, instant gratification kind of, uh, society where we need to make sure that our phone and our applications on our phones and such are working very, very rapidly. If they're not, we have no tolerance for that. And if they're broken, uh, even worse.

So from a testing perspective, but from a security perspective, that can be the life or death of a business. I mean, that could, and, and not just hurt your reputation or impact data issues or, uh, you name it, uh, you know, financial impacts. But look at, uh, all the things that we're seeing within the, uh, you know, the, the, the, the days today or what we're seeing, um, people's lives are impacted with airlines, with banks, with, with, uh, applications that are just being shut down because the testing's not being done.

And so I think that when we look at this, this study from jfr, you know, it doesn't surprise me that the numbers are they, what they are. Um, but I also think that it's, it's evolving, um, because there has to be some alignment between the CICD pipeline, the dev sec op pipeline, as well as the telemetry pipeline. So all this has to be fed into that infinity loop appropriately, but it also is putting a lot of demands on that whole shift left mentality of everything goes to the developer and the developer sitting there going, there's only so much I can do in a day, right?

So I think that's where we're we're at with it. I think that there's a lot going on here, and I did, I did enjoy that, that that JFR study, because it did, it did highlight a lot of synergies between my market research and, uh, and what they produced. Mark, you've built, uh, or at least led some software projects over the years, um, vulnerabilities.

I mean, is it top of mind when you're building the software, you're just racing to get to the deadline and, you know, you're hoping that mistakes just don't get made? I think that the, especially when it comes to vulnerability security, that that knowledge has always been siloed and specialized. And, you know, I came from, uh, organizations that build cloud software.

We, we built some of the biggest clouds in the world, and we did not have a lot of people that had operational experience and security experience. And as fast as things evolves, the attack face evolves. So, you know, from, you know, moving things to hosted data centers to the cloud, there was a new attack face.

So people were leaving their S3 buckets open all the time. And, you know, as we add ai, there's a new attack face. Um, I think, uh, it's a recognized problem when Shannon, uh, leads, who came up with the term DevSecOps back at Adobe in like 2013.

The, the process is obvious is that collaboration across the development, deployment of infrastructure. But the reality is that we still have people that are specialized and work in silos. And I think that until we get, like today, I know a lot of people who do CID CD identify as DevOps.

Well, I wouldn't identify you as DevOps unless you understand the full life cycle and are collaborative across that life cycle. And, and, uh, I think that that's, that's the, the reality of things. I think we just need, uh, processes that evolve that, you know, security needs to be a design point, not an afterthought.

It's been a, you know, battle cry for a long time, All to Mark's point. Um, when I talk to developers, one of the things that comes up frequently is they go, well, the number of vulnerabilities that are actually being exploited is pretty low as a percentage of the total number of vulnerabilities that could be exploited. And I wonder if we just luing ourselves into some sense that since the bad guys aren't really banging on every vulnerability that we know about hard, we're just kind of ignoring 97% of them.

And as a result, we're kind of creating a security technical debt that's kind of like a time bomb in our applications. And, um, is this all gonna come back to bite us? Yeah, I, you know, I think that this, uh, question, Mike, that you're asking, um, aligns nicely to what Mark was talking about as well, which is, uh, it is a little bit of the wild west, right?

It's a little bit of independence being, uh, you know, organizations are pushing a lot of, uh, independence and authority down to, um, organizations to push code out the door because of business cadence and because of require requirements from the, the organization's KPIs. Now, I will say, much like we talked about on our last segment on sustainability, there's a, there's ways that are, that are going to be put in place that are going to help address some of these security vulnerabilities. So it won't be such a wild west, you won't have that, you know, uh, turn a blind eye to the 97%, like you were talking about Mike, for example, EU is putting in place, the European Union is putting in place a cyber resiliency act, uh, that requires organizations secure the software supply chain.

You know, that's important, right? If you have that in place, or, you know, what we saw at the, uh, um, uh, the current administration put out an executive order around, uh, you know, SBOs for software to make sure that it's, uh, it's, it's protected in a, a specific, specific way. These regulations are going to be help, uh, they're going to help unify and kind of align everyone to the same kind of roles and governance for redu du uh, producing a releasing code.

That's important because, you know, as we were talking about earlier in this, in this conversation, it's not about just pushing the code out the door and if a developer forgets to do something or if there's a security vulnerability, it's like, oh, well, it just happens. It impacts everything. Uh, potentially we see, you know, like the what, what these organizations, uh, you know, our lifelines are running, you know, on these, uh, these software packages that are being pushed out the door.

And if we have a, a breach in those, in those, uh, software packages, it's a very fragile place to be, right? So we need to make sure that there is, uh, we're all marching into the same tune of saying, okay, if there's regulation compliance, I'm not saying again, that's like the perfect solution, but at least it's everyone, uh, is not just doing whatever they wanna do to release code because they're trying to hit their KPIs. Be a lot of confidence in software as you listen to this.

Yeah, no, I mean, it's, I think it's, it's all related. I agree with Paula going back to our last conversation. Sometimes the regulations can help and, and, and drive, um, everyone being on the same page with it.

I mean, everyone's gonna agree with that, but, um, I think that's an aspect to consider. Mark, to Paul's point about fooling ourselves, one of the things you'll hear from developers when they get that long list of vulnerabilities from some cybersecurity team is they'll go, ah, that's not even internet facing. I'm not gonna worry about it.

The problem is, is that piece of code or whatever it is that isn't interfacing today, winds it being interfacing to the internet nine months from now as somebody repurposes it or copies and past it. So, um, are we also kind of being a little myopic about this whole thing? Uh, I think that just because there's some developers that are sloppy in their practices that believe in securing it from day one, isn't the, isn't the biggest part of the problem.

I think it's, um, what you, there's a little nuance to what you say is today, it may be private, tomorrow, it may be hosted in a public facing cloud. And that's, that's really what the, the thing is, is we have an expanding attack face. And the only way to, this is why DevSecOps is so interesting, is it's continuous, um, deployment and improvement.

And so, um, even if perhaps that development code was privately hosted and secure at the time, there may be a new exploit, there may be a new, um, way to, uh, attack that code that wasn't even thought of at the time it was deployed. So I think it's, it's a, you know, security has to be ever, um, ever vigilant or else it breaks down and obviously it's breaking down. 'cause the new junk mail is the data breach notification from the myriad of people of companies that you've given your data to.

I mean, I get one almost every week now. So I think that's, that's the real thing, is that the attack face and the dynamic part of it is something that we have to be continually vigilant about. All.

Should developers be signing their code, and should we be shaming folks when there's an issue and put 'em in the town square on the dock? I mean, what's your sense of what is the responsibility we should assign the developers fields? Yeah, I mean, as, as Mike, as you and I have talked about many times, it's, I believe in accountability from the organization.

I mean, if an organization puts out code, they're accountable for that code. So the, the, uh, you know, the conversation of saying, oh, I used AI to create it, or I pulled it from some repository somewhere, that's not an excuse to put out a SWAPPY code. You have to test it.

You have to make sure it works. It has to align to your business compliance, regulation and governance. And that's just how it's, if you don't do it, there should be, um, re accountability and, and, and a reaction to that.

If there's, if, if, if there's something that gets pushed out the door that's not, that's not appropriate. It's not within compliance regulation, sloppy development practices and not, there's no excuse for it. With that said, businesses also have to have realistic expectations on developers.

You cannot expect a developer, again, I mentioned earlier about everything shifting left and the developer sitting know what the catchers make in the ball thrown at them left, right, and center. Uh, they have to have a relief. So that may mean you have, um, you know, additional resources or use service delivery partners to address certain, certain parts of your projects or whatever.

But whatever it is, there has to be some type of, um, relief for the developer in order to produce and, and deliver the appropriate code. Uh, again, realistic expectations and honestly, realistic expectations probably should be incorporated into that governance. Mark, he mentioned ai.

Well, things get better or worse because of ai, because, well, we're using AI to essentially generate code that was created by a model that was trained on code that was less than perfect. Yeah, I mean, I think it's fundamentally an arms race. It's gonna be the good guys with AI providing, putting counter measures against the bad guys using AI to attack.

It's just, you know, I don't know that it gives you a clear advantage other than it gives you a lot more leverage in the amount of checking and verifying that your code is, is secure. It's checking and, and verifying that your infrastructure has been secured properly. And that's, you know, in our earlier segment we were, we were, uh, you know, talking about natural language processing, just the ability for a security professional to tell it, and a large language model to do an audit and chat with.

Its, we're talking more about chatting with data before, now we're chatting with security. I think it makes you more effective and productive. But it, it, as I started out saying, is the good guys are gonna have AI and the bad guys are gonna have ai and hopefully the good guys have more AI than the bad guys.

But, um, I think it's gonna be pretty evenly matched. The incentives are there for them to keep trying to hacking. And the, the stats are undeniable.

You know, we have more data breaches now, um, set up 72%, I think from 22 to 23. And I don't wanna see the 24 numbers. 'cause my desk, I just threw out 10 of those data breach notices this week.

Yeah, I was, I was thinking, mark, what you said about, um, the checks and balances and checking. I think that when it comes to code and ai, that's a great point because sometimes AI makes it up so somebody has to be there to check it. Yeah.

So, So Paul, last comment here. On the one hand, it seems like we're gonna have more code than ever on the other side, to more Mark's point, we're gonna have better tools for discovering vulnerabilities because we're gonna use ai. Are you, uh, more optimistic about security or more pessimistic about securities these days?

No, I'm, I'm more optimi optimistic about security. Uh, you know, look, I think when we talk about AI and we talk about the use of AI within production workloads, nine months ago I ran a study showing that, uh, organizations indicated that they were running, 18% of respondents said that they're running AI in their production workloads. We ran that study nine months later, that number jumped from 18% to 54%.

So 54% of organizations are using AI and their production workloads. That's just going to increase, that's just gonna keep going up. Um, operational efficiencies are going to continue to be used.

Our AI is going to be, be a consideration for that. So with regards to the question, do I feel more op optimistic about security? I feel more optimistic about security and ai as long as the, the learning models that are being put in place, I ha have the right ways of doing things.

I don't, I do not believe the way we're doing a, uh, security today. Like that post checking, like, here's the list of things we found. Well, that's already because they found them.

What about the ones that are not found? There has to be a different way, and it's a real opportunity for the, uh, the, the security vendors to come out the door to have a, think of a different model, a different way to do the scanning or doing the understanding of what's going in these, these applications before they're, they're, they're compromised instead of reacting after they're compromised. We, we can't just sit here and wait with the catcher's minute and go, okay, well if something's been compromised, we need to react.

We need to understand ahead of time. I don't have the answer. I think that's something that the vendors that, uh, would be in the space, uh, more, uh, smarter people than I would know how to deal with this.

But I think that that's really the, the focus that we need to be starting looking at is being proactive, understand it versus reactive and say, Hey, we saw this happen. We, we need to react. And then that, that's how we're going to get in front of this, uh, the security issues.

All right? As the saying goes from Paul's lips to God's ear, meanwhile, I'm crossing my fingers and toes and hoping for the best. That's what, that's all we got folks for Techstrong Gang today.

We invite you to stay tuned for all the great episodes and other interviews that we have coming up right behind that. I'm Mike Biard. Thank you to all our guests as always, and we'll see you in tomorrow.