AI-Assisted Development and Cloud-Native – Techstrong Con 2023

Hi Welcome to our session at techstrong Kong. We are talking about AI assisted AI generated software development and also in a cloud native environment particularly around microservices and distributed applications kind of seeing where we are today. And where we think this is going to go as part of our digital transformation.

My name is Mitch Ashley. I'm CTO with Textron group also principal analyst with Textron research and I'm joined by two colleagues who I both know well, and it's great, but they haven't been on a panel together. So I'm excited to see the interaction and you know experts in their own rights.

So Tracy Reagan would you do an introduction of yourself? Absolutely. I am the CEO of deploy Hub.

I am the community manager of a project called ortilius and what deployab and artillas is doing as we are building a central evidence catalog to track and store all your devops and security data so that you have that organizational profile for your security. As opposed to one container at a time. I have done I do work with the open ssf I served on their board for about a year.

I'm also on the board of the technology oversight committee and the continuous delivery foundation and I have the honor of being a host for Tech strong women TV, which I absolutely love doing it is the best show we do trust me. It's a great show folks. com Donald Lewis could be talking with you my friend.

Did they introduce yourself, please? I'll do it. Simply.

My name is Donald Lutz. I work for a company called Taos. We were number one in Cloud by Frost and Sullivan where Matt my Gartner.

I'm transitioning more into platform Engineering Services. And how do you Make everything Cloud native and that's an infinite topic that we won't get into right here. Yeah, that's a whole another maybe two day panel not have 45 minutes, right?

Well, so first of all, you know, I'm I'm not one to succumb the hype of chat GPT and you know blockchain and everything else that's gonna totally transform the world and eliminate the need for anything else. You know, that's sort of hype bologna BS whatever I really want to focus on is is in the software development world, you know the role that Us today I for AI assistant type development is today, whether it's code completion, you know in lines of code that we're doing in our Ides or other forms of that. Maybe if you want to start out Tracy kind of where do you see we are right now with assisted development.

We're using AI technology. Well, let me first say that I have been a huge Enthusiast of AI for most of my career. I worked for Farmers Insurance for about a year and we were trying to implement a system called IBM tears to build their actuaries and what we discovered after a year of hard work and Rule and building a rule space is that we didn't have enough computing power to make AI possible.

It was too it took way too long a person that who has the expertise could come up with an answer a lot faster than our software could so Cloud native actually has a lot to do with AI because Cloud native gives us the platform in which we can immediately expand. In order to go to massive pattern matching and then come back with an answer without having to have a humongous Farm of server sitting there in some giant array to do that processing. So if it wasn't for AI if wasn't for cloud native we wouldn't be where we are right now with AI.

So they are very very there there entangled. Let's put it that way one is not possible without the other. So in terms of what we're starting to see in.

AI applications and software development in particular. I think it has the potential of disrupting how we hire who we we bring on as programmers and it will help people who it'll democratize software development. Let's just put it that way.

You can probably do you do a two year, you know certification program at a university and begin using these AI tools to do some pretty sophisticated development. So for one thing it's going to change the way we hire people and the way we train them. Interesting.

Okay down. I'd love to hear your perspective too. Well, I thought about it a lot.

We've been talking about lately because everything AI gets thrown at platform engineering and because of that because I'm very interested in design patterns and microservices and kubernetes. I think it helps with a lot of the low hanging kind of things to take on in terms of development. But in terms of the more complicated Cloud native 12 Factor apps exonical architecture, all those kind of things to take on it isn't quite there for that level because there is a level of understanding and judgment that has to be applied.

It isn't a straight pattern recognition methodology, you know kind of related to that and you are deeper into development than I am these my own world of working with python or whatever development tool. There's so much housekeeping. We still have to do in code and development and writing software.

I mean I think about the things that I have to write routines for to you know, initialize this and open this and whatever it is and handling exceptions and kind of just goofy stuff that we have to do to make our applications work on like why am I still writing this again? Why can't that stuff be sort of constructed For Me Maybe with some may I assisted so I could then work on Okay. Now what's my Logic for doing, you know pattern matching of the text coming in here with my analysis that I'm trying to do.

Or data analytics or whatever it is. That seems to me maybe to your point three C. If you took a lot of that sort of complexity The Drudge work out of it would make it easier to get into software development.

Or disagree. Feel free to disagree. Oh, absolutely some of that housekeeping work, you know.

For most companies I think we've come up with ways to use templates, you know, a junior programmer will be given examples of how they can get started in their code. So we've always done this kind of Manual AI I guess here's a template. This is how you should moving forward.

The problem becomes becomes especially in some of the other components. So you have that you have a lot of different kinds of code, right? You have code that writes software.

You have a code that delivers software you have code that builds software. So there's a lot of different kinds of code that we could actually put more guardrails around with AI and provide the standard templates the problem with the way we have done it in the past is that if you let's say let's just take something as simple as a make file you put something to make file and somebody borrows that make file and starts to use it for their own build. They might be linking in libraries that they don't need because they didn't know to take them out, right?

So with what I'm seeing now with AI we haven't really gotten to that that place if you think about open AI codex you can give it English word commands and it will go off and generate python script. If you haven't already done it you should check out they have a video that shows you how to build a little game and it shows you how it generates that that the code but what I think we really need to get to is a place where we can have a library essential library for each organization where they can start building these codes Snippets and those code Snippets then can be applied in a similar way where you know, maybe it comes from a codex sort of platform. But the code Snippets are what you're building and code Snippets can relate to microservices.

So you could potentially have a ton of microservices that are running out there that are built on these these code Snippets and what you end up writing is just very small amounts of code in the first place and then english-driven sort of AI assistance process is going to be phenomenal because then you're only telling it what it the logic you're building the logic you're not building the infrastructure and I believe in the long run. That's where companies are going to want to go because most companies are going to want to have some of their own their own Snippets that they can use so somebody is going to end up building an AI platform that organizations can build themselves. But it's kind of the AI Ops platform idea.

Right as you know, the environment where your models and your data and all of that stuff is managed as opposed to people's laptops and PCs and exactly. Nobody does my version of the model. We're trying to you know enhance her run for an AI application.

What's your perspective on this Donald it is I mean since I'm moving more into platform engineering, you know, I see where the AI Ops stuff can work. It's not there yet for the level that we need because you know historically I've built all that through my building Frameworks. And if you build the appropriate Frameworks, you don't you basically don't have to do any of that, you know other low-level group.

It's covered by the stuff you've built. It's just that the the way it's happening right now is a lot, you know getting all of that bill in mapped. In control interviewed is complicated because it that itself needs to be a platform that it needs to be an abstraction that can be brought out.

That's not you know. Oh, let's run chat GPT and generate some stuff because it's not sufficient. you know, and it's just I'm still kind of on the bubble about it because of some of the things I've seen.

What's this a new tool set right? It's a neutral set. Yeah.

It takes you know and one things I know about Donald and and I know you do this too Donald lives in a very large Enterprise world and extremely complex architectures. Of course, none of us clean, right? It's everything from what we had 40 years ago to what we're building today and and how you mesh that all together speaking of which Donald's writing a book about service mesh, which is on another topic.

So so I'm curious Donald from your perspective. Is this a IDE build it into your development process? Is this a code library and a smart attend assisted agent that tells you there's code to do what you want to do already or is it something more sophisticated than that?

I think it's those three things. I'm not sure it's more sophisticated in that right now. You know because I've seen examples where people have used.

These tools to generate things and because they don't have the judgment to evaluate what they generated. It's really not so awesome. So then it violates various rules that we've put in place that relate to how certain platform things should be done, which is also interesting because it's sort of getting to you know, it's like, oh I generated this thing, but do you really understand the right way we're doing exceptions.

Do you understand the way we're doing all of the above the way we approach this? What does it mean when we containerize it does it map and effectively do you really get the idea of what we're trying to do? And I think that that the reference that Donald just used the Judgment.

It's kind of like I was talking about, you know, and we used to copy it a build script and find out we linked in an entire library that a bunch of libraries that we didn't need because we didn't have the Judgment in order to understand how to use that that template and when it comes to what we're doing now with assisted AI like open AI open AI codex you don't necessarily have to have a lot of judgment if it's basic stuff. but if you're really building it more complicated system, you're going to have to have that backend knowledge to really Know if this is right for you. It's sort of like you can't, you know, if you want to write a blog on some let's just say an AI topic you can't go to chat GPT and it's not going to generate this massive report for you.

But it's a good starting point. It goes and digs up research that you may have had to do manually and it's like like Mitch and I were once we're talking about this and it's like having an Insight, you know, it's like it's like having encyclopedias back in the day, right? Nobody said you can't use encyclopedias to go get this go research.

So now we are just automating all that and it's bringing it Forward easier faster to use and that's exactly how these coding platforms are gonna look they could have give you a good framework, but you need to have the judgment to go through and make sure that that's exactly what you want to do. You have to go through all the testing you have to go through all of those pieces. So it's not going to fix that.

It's just gonna get us to where we want to go a little bit faster and maybe a little cleaner too because it'll understand not to put There that has security leaks or memory leaks for that matter. Yeah and fascinated too by the thought of you know, just right to your point Tracy just writing code, you know using a chat GP or the Codex, right? They open AI codex.

It does not an application makes for most for most significant applications when I was running it as a CIO sort of the Shadow it loomed out there and people are all freaking out about it. I might look there's value that they're working on they're doing this because they have a need and but by the way, they're gonna they're gonna come to it and ask for security and data and integration to other applications and that's where this to me gets Beyond you just can't have a generic AI engine writing code for you. You've got a structure.

You've got an architecture. You've got a framework and then you've got integration into internal external Legacy current all these different environments and and maybe there's an AI assistant down the road that helps you with that but that's you know to your point down. That's part of that framework of how we're building our Work because it's got a mesh with no pun intended mesh with the other elements of your environment.

Yeah, I mean part of it is just when you look at it because a lot of the apps, I build are really big and they involved a main driven design and which gets to event sourcing some other really powerful design patterns and those Are hard to just generate out of what we have right now? Those are much larger issues. Okay, so, you know because you probably gonna get rid of middleware and then you have to figure out well do I have grpc?

How do I tie it in there? A lot of Concepts? It's great.

For sort of like I said the low hanging fruit. It isn't so great for The higher level stuff yet. We're not quite there, you know just because a lot of that is Judgment call and how do you map it in?

You know, do you take the gang of Four's book and just scan it in and it gets applied correctly. but Donald don't you think we're going to learn does she think that sooner that will over time will learn and we'll start those patterns will stop being relevant and obvious and we can start building AI around some of those constructs like service mesh, for example, so healing I do but I'm also still skeptical because of the large. Corporations I've been dealing with the past two years.

I'm not sure it's that's the exact answer. Say some more about Wayne why that why that man they have such complex environments and they they've got mainframes and they have everything in the planet and they want to move things and you've got some language that don't exist anymore. And there's so many things that have to be put into that level.

It gets really complicated because you can't really, you know, you just don't lift and shift them and say, okay we've got this going on and this happens over here and You know and and you know, you have to figure out well architected Frameworks that all the three major Cloud vendors have and how do you apply those principles? Right? How do you apply finops and how you actually accounting for cost?

There's a lot of Deeper things that I I see it as sort of like me, you know, the oh I have this co-pilot. It helps me with building scripts or deployments. But when I get to these larger larger things it's not quite there yet and you might be right Tracy that we're going to get there, but I just don't think it's there yet.

But it's just it could be a starting point. It could be a starting point. That's what I think is being used for now.

Just like, you know any of the stuff you get a starting point and then you have to take it and make it your own. It can't take it all the way but it could give you a good a good running start. And I think it'll get I think it'll get smarter as we get smarter about how to build AI systems because really we're just now beginning to see this.

This is the early days. We are at the beginning steps. Yeah, I don't disagree with that.

It's just that the issue I see is a lot of Executives now think that this is fixed their complicated movement across the board because they've read some article and now they think that they could just move this over and you go well, you know, you can't really those 12 people still need to exist because they only 12 people that understand how it works and therefore we can't suddenly just say, oh you're gonna move that and have it just go magically, you know, it helps with new systems. I'm not sure how well it works with old systems Integrations. Oh, yeah, I would think that you'd have to do a lot of peeling off a lot of duct tape on those old systems to make any changes and more layers right more abstractions.

Lots of lay lots of lots of skin to peel on that onion. Well, that's why I think you know, we we can't AI assisted or code completion kind of tools like that until the sense and for Microsoft and and co-pilot with GitHub is that seems to me the starting point of just helping the developer helping them be more productive reusing, you know building into libraries of reuse code that we already have that we can yeah, you didn't know this been written here you go, you know add this to this and it's not the make file that's got the 50 things in it. You don't need in the one you do.

It's got just the one but you know to your point Tracy. And that's why I started this by saying I believe that right now tools like, you know co-pilot based on open AIS codex. There's something about encouraging people to get into software development.

I was just on a earlier this morning. I was on a call that with a young woman who I'm trying to recruit to come and join our ortilius open source project. She's two years into college.

She knows how to code in Python, but she's afraid there's a step. She's afraid to take that next step and when somebody at a junior level like that if they can sit down and play on a tool like co-pilot and get some immediate success. That encourages them to take the next step.

So right now I feel like these tools are very there there serving entry level programmers to get them going and get them started and get them excited about what they could potentially do. They may need to mentor to look at their code though that they've generated to take that code to the next level, but that's okay because they have been successful at code at getting it at least to phase one and that to me is extremely exciting because it opens up a whole world of developers and people who may never have gone into this industry because they thought it was too scary and we need it right now, you know software development, even though we consider this industry in in a minor recession. There's still a lot of programming jobs out there and there's still a lot of people that we could be employing that don't believe that they can do this work and tools like co-pilot are going to tell them.

Yes, you can and that's the exciting part to me. Well, I would be remiss if I don't these cracked the kind of deployment and platform engineering and just a little bit. I know it's a big topic and Tracy.

I know you're you're Very much astute in the whole delivery aspect you continuous delivery and get Ops and wait the ways to do that. It seems to me another area. We might not just have code assisted help from AI is managing or configuring that infrastructure layer the soccer infrastructure layer, whether it's you know kubernetes on up or containers on up or whatever the mesh of mash of all the software that we're using, you know, because there's more build script there's more configuration scripts, even if it's declarative, right?

It's still it's still a lot of kind of glue and and manual tasks to kind of figure out how that all is going to fit together under whatever configurations. What do you think the role of AI Tracy is and sort of that the infrastructure code layer of our apps. That's hurry with it.

Please don't know the answer. I mean just to build it like a container right? I would like to be able to just type all the things I need in my container and I go get it download it or build my environment.

I need this version of Internet. So I want to use this version of istio. I want you know this version of kubernetes.

And for it to go and do it for me. I don't want to have to do the hard work. So let you know.

Let's bring it on this stuff. This stuff is I'm sure Donald will tell you is very very complex and and as a lot of moving parts and a lot of places where you can fat fingers something and you sit there until three in the morning trying to figure out what it was and you missed something. So it's time it's time that the that AI start focusing on that and the devops pipeline for that matter.

We should be able to run a pipeline enough that we we should be able to gather enough data to determine where a risk levels are if you deploy this particular object, you know, how many times does it fail should it go through a different process? And I want AI to come back and tell me that because it's not something that I have time to track. I'm a developer.

I'm doing the best I can I need to know if I'm consuming a component that may have a vulnerability before I consume it. So there's a ton of stuff that I feel like in the life cycle process where AI can contribute in a big way, but we are what we've not seen any of that yet and part of the reason is we don't gather a lot of the data right? So if you think about codex it can go query the entire GitHub all the open source GitHub repositories for data.

And that's where it gets its code Snippets. It's learning from the data that's already out there, but we're not doing that for platform engineering or for the devops pipeline. Yeah, there's not a lot there because there's so many elements that you know, you have to figure out what your cicd process is.

How do you do pipelines? What does it look like? What kind of infrastructure code thing are you using or you're really using one of those?

You know, what are the methodologies they use to get that to happen? How do you track it? You know, there's a lot of stuff you'd have to you need it to pull from all those areas you need to pull from the different clouds.

You have to pull whole host of things. You know that what I see it right now. It's to me.

It's really just focused on programming. It isn't necessary focused on platform or infrastructure things. It's not been applied there in a lot of ways and it'll get there but you know, there's just it's not quite there.

You have to take advantage. What does it mean to get Ops, you know, even though that's you know, another one of those bastardized terms that bothers me because you know, it's you know, like just because you had to get in here operations that I mean it's working correctly because you know, and then you have to get into how do I know what the right testing is working? How can I validate what's happening?

There's there's a whole bunch of companies that are creating ephemeral environment so you can spin up environments really quickly. And I know the CEO one of them really well and that's the whole concept is what does it mean to have an environment? What does that encompass?

Hmm. Yeah, you can put any word in front of Ops and have some have to have a new term but not really a new chair. I actually did that already right?

I don't know what else we can put in front. They did Revenue up. Somebody said that I'm like what the hell is revenue out?

But it's the data. That we don't. we intelligence around devops and I mean let's we can say we collect data but we collected one container at a time.

So all that data about it about how a container is configured. For example, is it locked inside that container with it in their s bombs? We don't have a broader.

We can't step back and look at an entire organization and have all that data in one place. We don't collect it. So as long as we're not collecting that data, we could never have ai apply against it because it it requires the data and we just don't have it.

It would be great if we did but each organization would have to have that data, right? Yeah, they would have to have it and then it gets into distributed tracing and how do you do your tracing and you know, you know open Telemetry is getting there because it has the right things but then you have to really think about what unique keys. Do you generate if you got a hundred microservices, how do you create the right key that allows you to track that effectively which requires people to really think about that because that's what distributed tracing is about.

It's not just turning on Prometheus and collecting stuff and sending to your duty which isn't really a collection of anything. no, no, we're just not we're just we're not consuming and collecting that data and one of the reasons why we're not is because There's no tools to do the reporting on it. It's just not something that we're thinking about.

But I I really believe as we grow in in the world of AI that organizations will start realizing that there is a lot of intelligence that we are leaving in logs and spread across multiple tools that if it was centralized. You would be able to do something with that data. You know, they have to centralize it.

I think that I think is is interesting about this is that as developers as we learn new things new technologies new tools, whatever oftentimes we apply it on our own work, right? So now that I know enough about a yeah. Yeah, I'm gonna go do something with it about making my world easier for my work.

And maybe that's maybe we'll hit a Tipping Point where you know enough developers say look. This is now that I know how to do this and how it's working here. Let's build this for ourselves instead of all that data would just be exhaust that evaporates.

Okay, we can feed that in we can feed that across multiple projects. We can combine it with what's in HubSpot or another AI engine. Hopefully that's one of the Sparks that happens that helps lead this not just some researcher or scientist writing kind of a unique thing that we all say, that's the that's the answer.

Yeah, and I I question too. How could it help us with open source? can we get more sophisticated in terms of how we're creating this generated this generated code that says Instead of using these open source packages that may have vulnerabilities.

It's going to generate its own when it know it doesn't ever have vulnerabilities. I can't tell you how frustrating it is when you upload your package to your container to artifact tub, but it has an a today and an F tomorrow. And it would be great.

If we you know, if some of these new platforms that we're seeing come up just generate the code for us instead of using a package. The interesting. Well, we probably need to wrap things up.

So let's kind of go back to our topic of AI assisted or AI generated code in a in a cloud native and I kind of think of cloud native as microservices and service mesh and that part of the architecture This Cloud need of the infrastructure right to kubernetes and the underlying technology any final thoughts and maybe we'll where do you think we'll have the advanced in the next year two year. Where do you think that'll be Donald law for engineering? It's gonna be there first.

That's where you're going to see the changes and maybe you'll get some of the patterns in there. Then when you get to the platform stuff that's going to take a while longer because of the level of complexities that have to be addressed effectively. You know, it's it's not going to happen that quick.

Yeah, it's always just a lot longer than you know, yeah. You types as well Tracy, what are your thoughts? I agree completely.

It's gonna be a programmers tool and the more we decouple monolith applications the easier, you know, if you just given a smaller function a smaller set of logic that you have to work with these code generation tools are going to be more appropriate for that as opposed to trying to build an entire application. So I believe that the the tools like that we're seeing from open AI codex will be useful in creating microservices because they're just not as complex as an entire application. Wonderful.

Thank you both for joining together to have this conversation. It's great to have you two on the same panel. Hopefully we can do that again soon and thank you for everyone.

Who's that attending Textron con the virtual conference and checking out this session. So hopefully we are benefiting not from the hype, but the it's actually helping me as I create generating just a system my software development and that's probably where will be for a while. At least that's where we're at.

Thanks. Again, Tracy. Thank you Donald and we'll thank you.

Thank you. Thank you.