Infusing AI into Software Engineering with Trac Bannon at AIE 2024

Hello. Today I wanna talk with you about how we can go about infusing AI into software engineering. My name is Trace Bannon.

Let's get to it. Looking back over 2023, there really was a mad hype that occurred. I'm sure you remember maybe where you were when you learned or heard about chat GPT.

It quickly got to over a hundred million users and we got chronic fomo, fear of missing out. You see it in your social feeds, you see it in your emails, your Slack channels, every place you go, there's the mention of generative ai, but please don't get swept away in the hype. There is groundbreaking potential and there are limitations and challenges to be aware of.

So where are we right now when it comes to infusing AI into software engineering? I just love this graphic. This is the hype cycle.

Are we at the technology trigger? Are we at the peak of inflated expectations or are we moving towards the plateau of productivity? Well, let's see what Gartner has to say.

They're pointing out, are we really two to three years away from reaching productivity? I would say yes, as would Gartner. These numbers come from 2021.

My research and my experience both reflect this. We are just getting to the point now where we are at inflated expectations, but it will take us two to five years to get where we need to go. But I wanna talk with you a little bit about AI in general.

There are so many different types of ai, generative ai, that's just the new kid on the block. I met with a number of Mitre employees. I work with the Mitre Corporation and about 200 data scientists and data experts.

There are so many different types of AI machine learning, and this is meant to be just a, an representation. So though I've thrown the Legos out on the table, uh, and you can see all of these different pieces, my goal in showing you this is simply to say there's a lot more to AI than generative ai. And we've been using it in software engineering for decades.

So my name is Trace Bannon. I'm a software architect. I'm a researcher, I'm a software engineer.

I work with a company called the Mitre Corporation. We are federally funded research and development corporation and my goal is to figure out and help the government and the US government's allies to get technology right, to lay the direction and to help us reign in some of these amazing technologies. So where can AI be used within the software engineering lifecycle within DevSecOps?

So the next chart I'm gonna show you is going to be an eye chart and you'll be able to download these materials and get after the next chart. These are not all generative ai. This is across the continuum of the different types of ai.

These are the ways that we infuse it. There are multiple different types of ai. As I mentioned before.

This is updated about every six, four to six months and there are at least a half dozen entries that have been added to this over the last eight weeks. What's important about this is to realize that every aspect of software design delivery, um, has some aspect of algorithmic benefit from ai. So guys, what's important is to treat generative AI like a young apprentice.

And I'm not talking about a college graduate, I'm talking about that 15-year-old. That brings a lot of energy to what you do. You're left sometime scratching your head wondering what were they thinking?

But you need to pay close attention. And this isn't my opinion, this is what the research shows. Even the service providers like Microsoft providing a GitHub copilot are recommending humans in the loop.

GitLab recently had added a post onto their site that says, incorporate human oversight. Ensure humans are approving critical decisions. They're required.

And there's a study by Purdue University that shows currently that if, if you're using chat GPT, that about 52% of the time the code that you get, the answers that you get from a software engineering perspective are wrong. So that's something that we just need to be aware of. We need to have humans in the loop.

But does generative AI contradict DevSecOps? Well, the answer is sort of, so you have traceability, auditability, reproducibility, some of those core concepts of DevSecOps. But if I'm using generative ai, which is non-deterministic, it means that I may not get the same result.

So how can I trace that and explain that auditability means I can go back to the source and always know challenges to be aware of. It doesn't stop us from looking at inventive ways that we can leverage generative AI to help us. You know, there has been some challenges as well with DevSecOps and generative AI in security.

There have been a survey of 500 developers recently that found that 56 are encountering, 56% are encountering security issues with generated code. And that came from a recent survey by sny. So what's important about this is realizing that there are challenges.

Uh, there can also be reduced collaboration. Why would that be? Well, a premise of DevSecOps is that you get fast feedback and they're constantly collaborating.

But if you're now focused at dis having your discussions with AI as opposed to the humans, you can have reduced human collaboration. You might be just too busy talking to your AI friend. Does this mean don't do it?

No. It means that there is groundbreaking potential and challenges to be aware of. So where are people using generative AI right now?

Well, stack Overflow did a survey recently and they said that about of 37,000 developers, 44% are somehow using AI for work. And 25% said they're about to. That's a lot.

That's quite a lot. But let's look at what the breakdown of what they're doing with it. 80 82% are trying code generation in one way or another.

We're actually starting to use the word code completion over code generation, and I'll talk about that in a few minutes. 48% are using it to debug. So they're providing the rich context of their code base and using it to help them to determine what that text says.

34% using it to generate documentation, a very excellent use for this. Um, and 30% are using it to explain their code base. But what's really interesting here is that less than a quarter are using it for testing, and that's actually a very exceptional place to use, uh, generative ai.

But let's jump and talk about the elephant in the corner. And that is code completion, code generation, AI assisted coding. Uh, you'll hear the term code generation quite a lot, however, we're moving towards the term code completion generation has an implication that you can ask one or two questions and have a, a large bulk of code created for you.

That's just not the way it is. What actually happens is that you can, um, make a request for a certain amount of code, a smaller amount of code, and it will complete for you. Much like if you're using Microsoft Word and use word completion.

Even if you do, you uh, have some functions generated for you, a larger code bot code base, maybe 50 to a hundred lines, it still is, takes a lot of human oversight. It is fantastic. By the way.

Code completion is wonderful for explaining your existing code. Pointing it at your code base is one of the strongest things that you can do to have it help to explain what's going on within that. This is going to become immensely important as we look at modernization of prior of existing systems.

Now, something to be aware of, uh, is that it's generally really, really well structured and well formatted. And as I jokingly said before, it's sometimes right, all of these neural synthesis tools have some challenges. Remember, when you hit the regenerate button, it actually creates a new set of code.

So you have decisions to make as a developer or as an architect. So in IDE help, uh, means that generative AI is actually linked into your IDE, your, your development environment. Uh, in this case, the blue arrow points to the fact that there are three different choices for this particular piece of code.

Understand or might not have the experience yet to know which of those three choices might be the best one. And in fact, whether you're new in career or whether you are well seasoned, there's a phenomenon that is occurring, which is called decision fatigue. In the past, this was something that we studied when it came to medical professionals, when it came to the military, when it came to the, the judiciary, when people had really, really heavy duty, uh, ramifications of the decisions that we're making.

In what world should our software engineers, should our developers be suffering from decision fatigue. So something to be aware of that it's not always what it's cracked up to be. We have to go into this being very pragmatic.

There's some other things to be concerned about, to be aware of. Again, ground and breaking potential, but there are some limitations. There is something called unequal productivity gains.

I mentioned that on the prior slide. Someone new in career will not benefit the same way. Somebody who is very experienced will because they can look at the same code base new in career doesn't yet have all the tooling that they need to understand how to make decisions between the code that has been presented to them.

Um, as a matter of fact, someone who is very senior in career, it may be very quick to help them augment, but remember it's a team that delivers, not an individual. There's also code churn. There was a study done, um, by a group called GI Clear looking at at GitHub, and they found that from 2019 to 2023, this metric called code churn was relatively consistent.

Code churn is how frequently you check in and then check something back out, make some tinkering to it, make some changes, do some debugging, check it in and move it back. And around refactoring, if you're doing that, um, an elevated amount of time, your code churn goes up. Well, what they have deuced, what they are seeing right now is that code churn is set to double in 2024.

Now is that correlation? Is there causation to code completion or code generation? We don't know, but we're watching that to see what's going on.

There's also this issue of less secure code. Is that always the case? Well, the studies so far, and they're all referenced in the, uh, appendix for this in the bibliography for this presentation show that the code is less secure.

Oftentimes, I was in a workshop in the middle of March and it was an entire afternoon and we were able to very easily generate code segments, code hunks, but as we looked at them, we could identify multiple oasp top 10 issues such as SQL injection issues. So without having the knowledge ahead of time of all of those security concerns, the code that's generated, depending on the model, depending on the technique, will be less secure. And you know, that drives me forward to say to you, what do we need to be concerned about?

Well, we need to make sure that if you have AI in the loop, that you have to have humans in the loop to help take care. So this is the same picture that I showed you before, but you'll notice the emphasis is on the watchful eye and not on the apprentice because generative AI can be unreliable. So you need to pay close attention.

Two different options for that. One is through your code reviews. This is one of my favorite ways to go about this.

It also helps me to mentor others on my team. If you are doing the pair driven development so that you are side by each, as you're doing this, you may want to more frequently change partners. You may also want to schedule to have someone come in who's a domain expert to be a part of evaluating the code base.

But no matter what, pay attention, this is groundbreaking potential with some limitations and some challenges. Just a word of wisdom. If you're generating the code, don't generate the test.

If you generate the test, don't generate the code. The reason is that lack of independent verification, there can be biases, there can be blind spots. Remember I said earlier that you can have hallucinations, uh, when you're using it for pros in the same way that a legal case can cite an in inappropriate or make up, uh, a potential case citation.

The same thing can happen with generating your test cases or the code. And there's a condition called overfitting. And that noise in the data causes the model to hyperfocus overfitting in one area of the code base so that you get unequal distribution of your tests.

So just something to be aware of. But let's ask the big question. Is your organization ready for this yet?

Are you prepared? I'm gonna ask you how you're doing with your SDLC, your software development lifecycle. If you have challenges right now, if you are not having a very smooth entry into DevSecOps into using those modern software practices, sprinkling a little bit of AI on top of it can actually make it worse.

So let's go back to basics For one hot moment, I recommend doing the minimums. org and to Dora Dora, uh, are the, uh, the DevOps research, uh, and assessment organization that was originally started under Dr. Nicole Forsgren, uh, and a number of others.

Uh, and it will give you beautiful sets of metrics and indicators. They have continued to do industry surveys and move forward, telling you how you can help, uh, and how you can achieve, uh, better software delivery. org will actually give you the details of, um, how to apply minimum CI cd.

It grew out of a number of us at the DevOps Enterprise Summit in 2021, bellying up to the virtual bar and realizing that we lacked, uh, a very distinct and specific set of taxonomies and criteria. So I'm not going to go into that for the sake of time, but please do visit both of those. If we have more time, I would take you down a beautiful rabbit hole and talk to you about ways that you can adopt to CICD and do the minimum.

So let's get into some gotchas. I just love this picture. When you are adding in ai, whether it's generative or not, you are need to adapt your workflows.

Something that people don't realize is that if you are changing your workflows, you are in fact going to have wavering metrics and measures. If you had a consistent measure before, you're changing the workflow. When you change the workflow, it's like changing the temperature on the water.

You've got to be very consistent in understanding where you are. Training is a must and humans are often resistant to change. That's nothing new, but always bears us to, uh, to bring it forward in this kind of conversation.

And what about productivity? I mentioned unequal productivity gains before, but I wanna talk about perceived productivity. If you ask someone who's excited about the potential for using generative ai, if it's helping them, oftentimes they will say yes.

Oftentimes they will perceive a benefit or gain in their productivity, but it's not necessarily measurable. So we need to be cognizant that while it can boost morale because there's excitement, it may actually take more time for them to learn about it, to tailor how they use it and to really be the humans in the loop watching what's going on. But there's a second reason that we have to double down on focus on productivity.

There's more to the game than individual productivity. We don't measure individual productivity when we're talking about software delivery. We measure team productivity, we measure team productivity.

So many of these surveys, many of the industry studies that are happening right now are hyper-focused on individual productivity and on perceived productivity. Something to be aware of as you are navigating this and bringing it into your organization. Space is a framework that was published in 2021 by Dr.

Nicole Forsgren and other researchers with Microsoft. Um, GitHub, I'm really excited about it because it, it addresses the human element, but it doesn't just look at the lines of code or the number of commits that I have per day. It looks at the human element.

I recently had a chance to talk with Dr. Forsgren about the potential to add another dimension to this, which is trust. Uh, although we don't have time for this in our short half an hour together today, trust is an area for generative AI in specific that we need to understand how humans can trust something that we know has the potential to lie to us, has the potential to hallucinate, has the potential to get things wrong.

Context is exceptionally important. Um, so this is another gotcha why if you're using an externally hosted service, you need to provide as much of your context as possible. Context.

When we're talking about software engineering, are your requirements, your diagrams as much as possible, your entire code base, prior code bases? It's okay if you are ready for that and you agree that you can share that with an external organization. If you realize that for example, chat GPT recently announced in the April timeframe, that your conversations will always now be saved.

That doesn't prohibit you though if you're going to self-host, uh, or or work with another organization that would have a a model hosted on your behalf. Did I mention before? I think I did.

No matter what you're doing, keep humans in the loop. So I'm gonna talk about the big picture for just a moment. How to add AI into the enterprise because it's not as simple as saying today we're going to do generative ai.

Today we're going to do this. There's a very quick way to move forward with it though. Start with a needs assessment.

This doesn't have to be a six month, uh, science fair project. It could just be a few folks sitting around the table talking about where potentials within the organization that might benefit. Whether it's reducing risk or increasing quality from leveraging ai.

Then identify a pilot program. Not many, not scaling the whole organization. Pick one.

Make sure you are looking at what your skills development needs to be and start out with the initial bit of governance. Do you want people using an internal model? Only if they're allowed to use an external?

Do you need any kind of buffering, any kind of white listing? What are the stop gaps? What are the um, governance that you're going to put in place to help your organization to protect your IP and to help you move forward more quickly?

Monitoring and feedback are obviously a part of this because you're using a new tool, you're using a new technique. So you want to understand how to improve that. Apply DevSecOps mindset.

How do we continuously improve on this new technique? Models techniques are coming out so frequently that there's an additional layer of fatigue attempting to keep apprised of it. You don't want your enterprise to become hobbyists trying to learn on this all in their free time.

So you do need to have dedicated individuals looking at thought leadership and keeping apprised of what's changing. So choosing when and where to start. Well, what should be that first, that first line in the sand, the first place to go pick something small potentially.

Um, look where you have, uh, ability to have it very contained and to lower your risk. Uh, you want to do this in a way that allows you to back out. Uh, and you don't want to make this mission critical on day one, but choosing when and where to start should be, uh, a prototype or a pilot.

So let's talk about designing your AI assisted software engineering tool chain. I love this graphic. It really shows the gamut of your options.

It could be that you use something that's completely off the shelf because it suits your needs right now. It helps you get a jump on potentially a market offering, but it might not be tailored expressly to your needs. It might have more security challenges.

There might be some other risks. And when you're weighing those, you may opt for and as the service off the shelf capability or you may choose to stand up your own model. You may choose to take an open source and to tailor putting additional rag augmentation, uh, embeddings so that you're very consistently tailoring it to exactly what you need.

However, it may take you longer to do that standup. You may have additional learning that's necessary for your architecture, engineering and operations group. Fact of the matter is we have to apply our architectural thinking, our architectural trade-off mentality to our tool set the same way we do to the solutions that we design.

Couple of leading practices. Imagine this guys. Keep humans in the loop and keep everything in source control.

Even your prompts, when you put your prompt into source control, make sure that you are including the model that you are leveraging. Um, and if there's a model version to put that against it as well, that will help you when you come back. If you need to regenerate, re-execute or are looking for some additional auditability, always secure your vulnerabilities and be careful that you're not putting your special sauce, your ip, your private info into public AI engines without knowing what the legal ramifications are of that.

And make sure that you manage your risk. I love this picture. I adore this picture.

'cause when you first look at it, it makes your heart pound. You see somebody on a tight wire, but when you look more closely, you see all of the different ways that he has secured and mitigated his risk. The same is true when you are thinking about leveraging generative AI in specific non-deterministic new technologies often being hosted external to your organization.

It doesn't mean don't do it, it means that it has groundbreaking potential and a lot of limitations. So in the appendix, I'm not going to go through this right now, are two or three different pages of questions for you to take forward to ask your AI provider. And whether you are the AI provider for your organization or you have somebody external, there are questions that you would want to have answered.

For example, if what are the, what are the changes on your roadmap that you're anticipating, how will those be communicated to me? And if there is a vulnerability that you need to patch in that model, how will you let me know? So again, many, many questions that you need to take forward to those people who are providing your model support to you.

Or if you are that person or that organization, providing it to your enterprise questions you need to be prepared to answer. So let's look ahead. This is the fun stuff.

What does it look like over the next 12 to 24 months? Well, this is not just my opinion. I had the chance to talk with software engineering Institute with Microsoft, uh, with some folks at IT revolution with Yahoo.

And we all agree that in the short term we're seeing more data silos that is kind of runs against the principles of DevSecOps and yet it is a known, known we'll see some slower flow, not faster flow, slower flow because we are changing workflows, we're incorporating new tools and there can be some quality issues. This is not the permanent state. This is what we're going to see as we're adding in AI as a tool.

Now consider that for each one of those dots, each one of those bullet points that you see, that's a different invocation of an AI tool. If every step along the way, I, Tracy, Ben, and am interacting with an AI tool, yeah, I may not be able to share that data from that interaction with the person to my left or the person to my right. So it can cause more silos in the short term, groundbreaking potential friends.

But be aware in the short term, we are going to see and we're seeing already a groundswell a, an absolute surge in platform engineering. I love a new term that I heard. I believe that we are at the, uh, point where we're seeing the intersection of low code, no code and platform engineering.

But the new term that I heard was low code, no code and pro code meaning platforms to help the professional developer, the professional software engineer. Ultimately, what are all of these platforms attempting to do? Well, they're trying to make it hard for the humans to make mistakes.

Codify our leading practices help us increase, uh, our ability to deliver value quickly. But what about this slide? This is the, the leader of Nvidia.

He recently said the future of coding is dead. Do you believe that? Well, some experts are saying, eh is only going to be those developers and engineers who are not qualified, who lack experience and lack certifications and, and qualifications that are going to be at risks.

The optimists say it generative AI will enhance the, uh, job opportunities for the qualified software engineers. The pessimist says might replace. And what about, uh, Devin, the AI software engineer in late March?

This caused a bit of a hubbub. They've been saying for years that there's going to be a rela a way to replace software engineers. Do you know that in the sixties, Grace Hopper created COBOL and said that we will need fewer software engineers.

Instead, what we needed was more cobolt developers. COBOL was intended to help business people to create their own software. It will we see a change in how things work?

I would say yes. I wanna pause for a moment. I wanna take you back to 1939.

I wanna take you back to when the Wizard of Oz was filmed. It started in black and white and there's a point where Dorothy's house is picked up by a tornado and it is whisked over over the rainbow and it lands in Oz, it lands in munchkin land. And as she opens the door, she sees just a future that she has never experienced before and it's completely in beautiful technicolor.

I believe that that's where we are with generative AI right now. We are at a point where we have to understand ai, human teaming today, AI is a tool. It is a tool that I invoke as a developer, as an engineer.

It's a tool. As we move forward, we are seeing the experimentation. I'm going through the research that we're leveraging.

Uh, we are going to begin leveraging it as an agent so that it becomes more like a team member. Think about agility. Agility is premised on individuals working one item at a time to reduce cognitive overload and limit work in progress.

When I have AI agents, one or more that are part of my team or per or perhaps making up the bulk of my team, how will humans and AI interact then? Will there be a trust that's necessary? Will we need to take new steps?

This is the future and I'm super excited to be focused on researching this with some of the best in the industry and in academia. But you know, we can't put the genie back in the bottle going back to the black and white of where we are right now. Um, prompt engineering as a discipline.

We need to understand it. We need to understand the ethics of prompting, the ethics of prompting. Imagine that human machine teaming.

What changes on a software team's performance will. We need different measurements, different metrics, trust in the reliability of the software that's coming out of something that's leveraging generative AI needs to be known. But why am I showing you the picture of a horse's backside?

It's because I was sharing this concept with a friend of mine named Lonnie Rosales from the great state of Texas, and she said, trace, you know, you can sometimes trick the genie back into the bottle. She said, perhaps a better way to talk about this is that you can't put the poo back in the horse. Now she probably used a different word than poo, but then I wanted to give you that visual meaning.

We are at a place where we cannot turn back. We will not turn back. And we need to be prepared for what's coming.

We need to know that there's groundbreaking potential. We need to understand the challenges and we need to prepare for the risks and the limitations. Call to action for you.

Ensure organization. Understand if and when AI is being used. And when you do that, you want to enable people to be doing research, but research safely.

Put your cybersecurity rules in place and make that your highest priority. As you're establishing guardrails for your organization, connect with those who are providing your model to you and ask those security questions and other roadmap questions. Um, and make sure that you are sharing your stories with me.

Um, I am focused on this for the next year wholly and completely on understanding and helping all of us, all of our practice, whether it is academia, industry or the government to better understand not just how we can use the models of today, but how it's going to impact us going forward. I'm gonna leave you with one final thought and now you'll see that this is in color. What really does matter is the humans in the loop.

We're writing software to help humans, to help humanity, and we're leveraging all of these new technologies and techniques to do it. I'm gonna leave this up for just a second. You can use this QR code and that will take you to the materials, including the bibliography and citations for all of the images that were used.

Thanks much.