Applying AI to the SDLC: New Ideas and Gotchas | DevOps Connect: DevSecOps 2023

This is about AI and the sdlc. Look, it's a massive, massive topic. We're not gonna be able to cover everything, nor do I pretend to be an expert at ai.

I can tell you that I'm a software architect and that these are the things that I'm running into immediately. This is what I'm running into right now. So let's talk a little bit about me, how it worked.

Yay. Technology. So I, my name is Tracy Bannon.

I go by trace. I am a senior principal with the Mire Corporation. Mire is a federally funded research and development corporation.

What's that mean? It means that after World War ii, the government realized that they didn't have enough objective independent voices. They didn't have enough people on staff, and so they created these research and development corporations.

So I get paid to be objective, which is really cool. I've got nothing to sell. Um, so I've been a software architect and engineer for a couple of decades.

I'm not gonna tell you exactly how old I am. Go Google it and try and figure it out. And nobody hit up my husband and asked that question either.

Um, that background as, as an architect, has given me some interesting opportunities to mentor lots of people in lots of ways, from very small teams to having to scale. So always understanding the pressure of needing to go faster, make people more productive, help them with what they're doing. So, as I said, today is the first time that I'm doing this, uh, in a long time.

So I have my notes. So you forgive me for flipping through my notes to make sure I don't miss anything. Um, the materials we're going through today usually takes me about 45 to 50 minutes.

So we're gonna do some wave tops. I'm gonna skip overs slides. You're gonna get those slides when you download the presentations.

So let's go ahead and talk about ai. So this graphic, it's AI generated, and yes, I created this AI generated graphic, um, based on some other things. So kind of cool, but I only say that because it's everywhere.

It's infecting all of the stuff that we do when we think about ai. Unless you've been completely air gapped, it's been infecting your feeds, right? Everything that you look at, your Twitter, your email, your family, it's all over the place.

My son, my husband has an apprentice. The apprentice was over at our house the other day, and this kid was excited beyond excited about all the things that's gonna happen with ai. It's just gonna help us with everything.

Like settle down. It's in its infancy, right? It's at the embryonic stage right now.

So let's actually talk about some terms. Let's get some, some terms laid out here. So, generative AI is exactly what it sounds like.

You're generating new content. It is not backward looking analysis of statistical information only. It, it is intended to generate new large language models.

What's a large language model? It's a generative model that has been trained with gobs and gobs and gobs of text data. And what it does is it does statistical analysis.

It understands the distribution, it create, it figures out how likely is one word to appear with another word, or how likely is one phrase to appear with another phrase. Kind of cool when you think about it. Software development is a language, right?

We're using languages. So it's become a natural extension to apply it. So you'll also hear the term AI assisted.

You'll hear other people use, uh, AI replaced. We're gonna talk about AI assisted for now. It literally means I'm taking a large language model and I'm applying it to the sdlc.

Now, what you probably didn't anticipate was that I'm gonna bring up low code, no code. So remember I said I've been doing this for a couple of decades while I was with some of the large global consultancies, one of the things they always ask me to do is, can you figure out ways to black box stuff? Can you make it so that the developers can go faster?

So we were forever looking at different platforms. Now, those technologies have really come about, right? You can really get low-code, no-code platforms that do a whole lot for you, but they're not necessarily being used by your developers.

They're being used by what we call citizen developers, functional people. I take somebody, I sit them in front of it and they can understand it and they can create software. What's interesting about that is when we talk about generative AI and applying it, those low-code, no-code platforms have been doing this for a while.

They didn't just jump on the bandwagon in in November when chat G P T went live. They've been doing it for a while. And what we think is that in part, it's based on the personas of the people using those platforms.

They don't care how you do it, they just want help. They just wanna do it faster. So we're seeing a big surge in that it suggests that they're more open to that kind of technology.

Doesn't mean we're not doing it with custom depth, doesn't mean that at all. We are improving quality using generative models with ai. We are using it for repetitive tasks.

But what's interesting is that we're seeing a surge in something called hyper automation. Yes, that's another hyped term. I'm sorry.

You have another buzzword to add to your lexicon. Um, so when I think about custom development, when I started, everything that we did was custom dev, everything that we did. And then we got to a point where we were writing our own frameworks.

Like that was really cool. And then we started to use some components here and there. We would knit those things together, but that would be maybe a, a business rules engine off on the side.

Well, now what all, what I consider to be custom dev now is gluing stuff together. I'm gluing stuff together. Hyper automation, uh, takes that another level.

It means that I'm using AI ML event driven. I'm using rpa. Whatever I can do to look at any process of any sort, even our beloved DevOps and automating the bejesus out of it.

So I bring that up because we're gonna continue to talk about hyper automation. Hey Deb, I'm glad you made it. Um, we're gonna continue to talk about this and I wanna bring up a really cool stat for low-code, no-code platforms in 2023, Gartner estimates 10 billion in growth for low-code, no-code platforms for assistive technologies.

But check it out for the hyper automation area, 720 billion, that's billion with a B in 2023 growth around this area. So you gotta believe you're seeing it everywhere, right? We're seeing all of this all over the place.

What's it mean though, from a security perspective? It means that a lot of stuff bang is being done outside the IT department. This should scare anybody here who is security first should be hellaciously scared.

Actually, anybody who's in the IT department should be scared. Used to be shadow. IT is what we called somebody doing something outside the IT department that had to do a technology.

They went out, got a credit card, somebody found a nice offering, a SaaS offering online. Great, no problem. We're gonna get it.

This means that our, our um, footprint, right? Our cybersecurity footprint is now huge. And then we have to be, we have to be really concerned about it.

And again, I'm gonna skip over a couple of slides. So what do we use it for right now? Y'all know what we use it for right now.

Let's talk about it within a generative, uh, generative within low code, no code. It would literally take us a week of sessions to go through all of this. So I just wanna hit the wave tops on a couple of these items.

Automated app generation in a low-code, no-code platform using generative ai. It works. And you can see some of the different vendors that are there.

I'm not endorsing any of these vendors. I've played with probably about 70% of these tools, and I can tell you they're doing amazing things. Consider that low-code.

No-code platforms vary wildly. So in the same, just like humans, the way that we, uh, we vary wildly the way that they're applying AI and ML is wildly variable too. So when it comes to interface design wrapper in particular has some cool stuff going on.

Point your workflow at it, have it optimize and make recommendations on what your workflow should look like. Um, there's one on here that is a, uh, integration support with Zapier. You've probably seen this before.

My point is the low-code, no-code platforms that we all love to hate. They ain't going anywhere. They're gonna get bigger.

We're gonna have to learn to live with them. We're gonna have to understand that they're using these generative techniques. But let's talk about the fun stuff.

So for me, the fun stuff is getting into how I'm gonna use this with my code base. LLMs, again, large language models, programming code is just code. So we've probably, is there anybody here who wasn't really excited the first time you used an IDE and it had IntelliSense and it finished one word for you?

You started to type a word and it finished the word and you were like, holy crap, this is awesome. Well, code completion is just that. But at a bigger scale, you start the sentence and it gets two words of, or two phone names or two bits for it, and it finishes the whole sequence for you.

Now it's not generating your entire code base, but that's wonderful. Tab nine's been doing it for a while and it's getting better and better and better. It's exciting.

Now I can go to the next step, which is code generation. I can use natural language and I can say, I'd like a function written in Java that does this. I don't actually have to give it any real code, and it will generate that for me.

And I'll show you some real live examples here. I also use it for code review. I love it for code review for one reason.

It gives the more junior staff a leg up. So they're looking at this stuff in advance, they're taking a look at it, or I'm able to train up mid-level staff as they're learning to review things. So it makes it so that I can scale.

So I'm really selfish is why I like the code review piece of it. Style checking. That's true.

Um, issue identification and debugging. You can point tools at your code base. And this is not brand new again, not brand new.

And you can see where you get different kind of errors, different kind of bugs. But I wanna talk about this last one here. Code refactoring.

The architect in me gets shivers at this one. And not good shivers, bad shivers. So if you look at the code, you take a look at it and it says, I'm, you know, you, you have two different functions, which function should you use?

Or we've analyzed your code base, you have some repetitive waste in the code. For me, I look at that and go, okay, great, I'm gonna dig into it. Who should be able to do that code refactoring.

If you're leading teams, if you're, if you are enabling an organization, you need to be thinking about who should be doing the code refactoring and at what scope. It's one thing to refactor a function. It's another thing to refactor a bigger code base.

And some of the tools are getting big enough that, and advanced enough that I can point it at a large code base. I can actually point it at my models for my code and it'll show me where that repetition is. It'll show me where there's inefficiencies in the code.

So I'm not saying don't use them, I'm saying be practical when somebody's starting to use these kind of tools. Be thinking about what the ramifications are of that, of that tool. All right, so let's go through a couple examples.

These are real examples. I type these in. I use these tools.

This is tab nine. And this is the fast code completion that we, I had talked about before. I said create, create a function, um, that sorts an array and descending order.

And what you see below it is the gray, the gray text is its suggestion to me. That's it Seems cool, right? Kind of novel.

Help us, help us, uh, jumpstart things. But I do wanna give you some ideas, some things to think about in a couple of gotchas. I'm starting to see people become more and more dependent on this.

I start to type it in. I take whatever suggestion comes at me. And if I'm not really paying attention, I'm just taking whatever suggestion comes at me.

It's an issue. It's an it, it really is an issue. And there are vulnerabilities.

Now, I'm not talking specifically about chat G P t, this is tab nine. I'm telling you that these tools which have been trained using prior G P T versions, I think that the tab nine was using two five and uh, co-pilot I think used three. And, and, but they're, they're jump start.

You know, they're jumping over the versions of this really quickly. Tools lack context. So they don't understand.

They may be able to give you syntax correct, but semantically not what you're looking for within that environment. Again, none of these things are bad. I just want you to be aware of them and how they're going to impact you as you go along.

Um, one thing that I think is, is really important is that especially when you're using this is code completion. It doesn't know that I have another function named somewhere, somewhere else that's called, I don't know, check age. It doesn't know that.

So we're talking about duplication. Duplication. You're actually talking about a code proliferation because we're doing code completion.

Um, one thing to be mindful of is it can be a little performance hit depending on your environments. You're just sucking down some bandwidth. Depending on how much people are doing things, it isn't necessarily or a big deal.

But if you're a smaller corporation, if you're a bandwidth challenge, this is gonna suck you down even further. If you're going to do this, make sure that everybody knows how to use code scanning tools. That's it.

Not saying don't, I'm just saying I'm being cautious. Take a look at another one. What's cool about this one, and this is co-pilot, is that it provides me with multiple alternatives.

So this is a code completion. Um, fully generating, I'm sorry. It's a, it's a code, uh, generation, not a code completion.

So I really did type in def max, some slice I just typed in the top and it provided that entire function. Kind of cool, right? You wanna know where the, the devil is in this, see that bar at the top?

It says accept, accept word. If I click on that where it says one to three, it'll give me the next full different version of the code and then the third time it'll give me another full different version of the code. So there's come some nuances with this for junior staff for learning purposes, awesome.

If they're learning, oh, that's one way I could write it. Here's another way I could write it. A lot of times doesn't come with any kind of commenting.

So they might be spending a lot of time trying to figure out what's the difference between option one and option two. Again, not bad. Something to be aware of.

Check out the stats on the, on the side there though, right now, and this is as of Friday. So these aren't, they aren't last year's numbers. As of Friday, users accepted an average of 26% of the completions.

That's not real high. So what does that mean to you and your organization? That means a lot of people are clicking and a lot of people are spending a lot of time going down this rabbit hole.

You need to make some decisions about how much time people should go down that rabbit hole. You need to be tempering people. If you are spending more time to generate a prompt or to click through the options, then it would take you to write that code.

Then it would take you to ping up Bob, ping up George, talk to Mary and say, what are your ideas for this? Stop, stop throttle back. And it's okay.

The technologies are gonna continue to mature. This is where they are today. Now GitHub makes a recommendation.

If you read the fine print, they make the recommendation that if you use their code generation, this is from their site, you better do rigorous testing. These are their words. You better do IP scanning.

You better check for security vulnerabilities. So the tools themselves that are there to help us also come with a lot of lot of cautions, right? Makes sense.

All right. Now that this is a wave top for you, cause I know that I am gonna run over on time here. Unit testing.

You can do unit testing with uh, co-pilot as of I think the end of February. I have not played with a yet. Tab nine has put out a new unit testing framework.

The problem that I'm seeing right now at this moment, right time sliced when you listen to this next fall and say, ah, she's full of crap. Well, yeah, this is because this is right now, this is today. It's how recent this is.

Most of the test, the test case that are being generated are very contextual. They're not, these are not your full-bodied unit testing frameworks, your unit testing tools. But they do let you test in line.

They do let you have an two pains on your current IDE to check out that one function. So I love it for learning. I don't want you to immediately jump into it.

Um, for writing all your unit tests, that's if you have people that are writing unit tests. So it's not quite ready for prime time. I want you to see my words very clearly up here.

There are so many use cases. It is absolutely exploding. It's absolutely exploding.

By late summer, you're gonna see a dramatic difference. It's changing every day. The models are getting better, the tools are getting better.

Beware of the pricing. Pricing is changing dramatically as well. So things you need to worry about, some things you need to worry about, questions to ask.

You need to talk to your vendors to understand how does their platform ensure security and privacy, right? If you are sending code into their model, are they keeping what you sent in? There's an article last week, uh, that I have not yet had a chance to report on for 5 0 5 where I believe it was a Taiwan, uh, company in Taiwan who they had multiple bits of their proprietary software and in a G P T engine that wasn't theirs in an external one.

Somebody was looking to do some debugging, be cognizant of where that information is. Um, figure out a little bit more about their models themselves. What are they doing to prevent malicious input into that model?

Except for the biggest companies, most people are using publicly available models. That means it's shared gang, it's shared. So if John Willis goes out there and goes, huh, I'm gonna put in some nefarious code out there.

I just wanna do a little Deming experiment to see how I can just muck with everybody. Well he may actually poison that model. It's possible.

It's very possible to poison that model. Um, the company that I work with has a lot of folks who are highly invested in figuring out model assurance. It's a real thing.

But some of the bigger companies are starting to train their own models. Um, how do they manage control to their models, especially in the low-code, no-code environments. Where I can, this is kind of cool, I can, within one of the platforms in particular, I can actually say I wanna embed a model for my end user.

And the models are already pre-trained. Think about that. It's like using a piece of open source.

But now I'm using a model that's open source. Sort of think about the ramifications. Where did the data come from?

Who has access to it? And how are changes going to be evaluated? If they're making a change to the model and you are subscribed to that model that is now going to your end user, how do you know that that's not gonna be a breaking change?

Just like breaking changes for any other code that we have inherited in the past. Now we're inheriting data the way that we used to inherit code. We're inheriting models the way that we used to inherit.

Um, let's see. Let's keep going here. Click lots of stuff here.

Ya thoughts You were gonna get away without me saying chat. G p t wrong. We're gonna talk about chat G P T because it's everywhere.

It's polluting everything. I think I heard John at the beginning of your talk earlier today. First thing you said was chat G P t.

So you can get it done with. Y'all know this. We don't need to really cover this.

The big thing for this right now is that chat, G p t is spitting out copy and code with this wonderful flare. It is so authoritarian. We look at it and we listen to it.

That's the attraction, right? The attraction for chat g p t is that I can use regular language and I can get out of it. Things that look and sound appropriate and accurate, I can tell you that I've made it say some pretty bad things that are really not accurate.

I like this bottom sentence here though. That's eerily entertaining and oddly educational. Now if you've never played, but chat G P t I urge you to go out and get a free account and I just want you to type in one thing.

Act cuz you always start by giving it, tell it who it's going to act as. Act as a witty dad and create 10 chicken jokes. I'm not joking.

Actually. Ask it to do that. And you'll get 10 kind of witty chicken jokes.

The reason I say that is it, it's worth your while. If you've never experimented with it to at least look at it and understand it. I wouldn't recommend subscribing to it unless you're gonna be doing some deeper research with it.

So I did try. Let's do a little bit of, can I use chat G P t instead of using co-pilot, instead of using some of the baked in APIs or the baked in ai, can I actually do something with chat G P T? Yep.

That's a real prompt that I entered in. That's the exact prompt that I entered in. Write an open API spec for a crm api, crm, customer relationship management.

If you're not familiar, I had to truncate it cuz it gave me a lot of lengthy bit of code. Code wasn't terrible, it didn't work right away. The errors that were there were simple so that I could get past them reasonably.

But what was really nifty about it is it understood CRM already. Now I'm saying understood, what does that really mean? It meant that it from a statistical analysis of the likelihood of words being together, that CRM likely means this.

And so that it could then go through the various parameters and figure out what CRM meant. It understood automatically because I said I wanted an open api, it understood that it needed to have, get post, put, delete. It did have some syntax errors.

Again, to repeat myself from earlier, I was able to monk around a little bit with unit tests as well. I do not advocate that. Don't waste your time with that Commonly claimed beliefs.

Why am I saying it that way? These are all, this is all over your feed right now. All the developers are gonna lose all their, their development time.

They're not gonna be able to or they're gonna be able to be so much more productive. Um, one thing to realize with chat G p t, it has a repeat button. What does that mean?

The regenerate button? It means it looks at your prompt and it tries to create something that is dramatically different that was there than what was created before. So if you're thinking about that with reference to code, every time you're creating code, it's dramatically different than it was before.

Is that good or bad for your organization? All right, the way forward. There's a ton of stuff here.

Rapid growth areas we don't need to talk about. Are we going to be replaced? Not yet.

In 10 years, developers are going to have to have a completely different skillset. We must know how to prompt engineer. We must know all of these different things that we've talked about today.

It is gonna free us up from some routine tasks, but not till it's a little better. It's not quite there yet. All kinds of considerations.

Look, guys are literally probably another 10 slides here. Let me take you through a couple of these. Sbam and lineage.

If I'm generating code, what's my sbam look like? Anybody know? Hmm, but we're gonna talk to DJ Chale later and figure that out.

Um, the free tiers right now of all these tools, they are just baiting and switching. You get sucked right into that. And the free tiers are having are getting featured pour pretty quickly.

Um, let me jump straight to all the, you're gonna get all these look things we need to understand. Ask your vendor more questions. Ask your vendor.

Lemme go back. There's another set of questions to ask your vendors and I apologize for rushing through this guys call to action and I'm gonna make, I'm gonna run over and I'm just gonna ignore Mark on this one. You need to talk to your tool vendors.

You need to talk to them and understand how they're managing models. How are they managing quality? Do this now.

And I also ask them about their AI roadmap. You need to know what they're thinking about and what they're doing. Then turn around and pulse your organization.

Just ask 'em what they're doing. This is not heavy handed. You wanna figure out, there's gonna be some smart people doing some really smart stuff.

Find out what they're doing and then turn around and figure out collectively what your policy should look like. Cause that's what you're gonna need to do to go forward. Then what I need from you, I work for federally funded research and development.

This is what I do for a living. I need to know your use cases, the tools that you're running into, the cool things that are happening, the bad things that are happening. So that's your call to action gang.

This is my contact information and I'm here all week. But at the same time, if you have any questions right now, You know, step up. I I want you to back up one slide please.

One slide because it's important in order to really use what TRACE is saying is to provide the use cases to compare again, since she's asking you to provide your use case. What is your problem, right? And if we can start working on those things that are gonna make an immediate difference for you, that makes her more valuable to you.

So thank you. You're welcome very much please. Thanks to Tracee.