Harness Field CTO Nick Durkin on How AI-Created Code Will Strain DevOps Workflows

Hey guys, thanks to the threat we're here with Nick Durkin is the field CTO for harness, and we're talking about bottlenecks and DevOps and AI coating tools and lines and tigers and bears of mind. Lots of things are going on. Nick, welcome to the show.

Thank you so much, Mike, for having me. Always enjoy coming on here. So again, thank you.

It seems to me that we always had issues with bottlenecks before ai, and now we're adding these AI coding tools and we're generating more code than ever. And it's not clear to me that our existing DevOps workflows are up to the tasks, but what's your take on what's going on here? Look, I think the last 10, 12, definitely the last eight years we talked about automate everything.

And I think it was something that was important to people, but it hadn't been driven by anything that was actually truly backing it. And now AI has put a massive amount of pressure and what we're seeing is if you look at the Dora report from Google, you know, 25% more usage of of AI coding, but it's not actually getting outta production, it's getting at, it's getting out slower, you know, one, one or 2% slower. And it's also having more bugs.

And so I think to the point is, is now there's a huge pressure, and we're seeing it specifically is about automating all those pieces, making sure it's easy for people to do the right thing, but it's hard for them to do the wrong thing. So putting policies in place to prevent, whether it's AI generated code or engineer generated code, making sure that it doesn't do the wrong things. Is this simply a matter of we're gonna have to manage more pipeline simultaneously, or is it also the fact that a lot of the pipelines are gonna have more dependencies on each other and things have to be orchestrated in the right order because otherwise it'll just all pile up at the back end?

Yeah, I, I see it, it was done in an interesting way. I'll, I'll, I'll give a very clear example, Mike. If our job today was to build an airport, would you let me build a runway for every airplane?

Well, probably not, right? And the idea here is very simple, is that we used to actually allow all of our engineers to build a pipeline for every application. And you wouldn't do that in the real world.

There's no way we did it to, to, to speed things up to give 'em freedom. But the reality is now, and we see with our customers, you know, instead of having, you know, 3000 pipelines or 30,000 pipelines, it's actually only what 19 to 30 templates are being used. And so the intention is to actually make the reusable, but more importantly it should be like playing a video game.

So my intention here is this, is that, you know, when I hit that critical vulnerability, I don't want to know about it two weeks from now. I wanna know about it now I don't wanna fix it. And so it should be like just every time that I fail constantly doing it and allowing the pipelines to truly allow us to go and move faster.

If we've got all of the same rules and all the same policies in place, now we can actually go forward, Hey, I failed because of resiliency problem. Great, let me fix that. Hey, I failed 'cause this is gonna cost me too much.

Great, let me go fix that. And so now we're seeing it in the pipelines and it's allowing our engineers to focus on what they're good at. So let's take that analogy a little bit further.

If I, um, eliminate number of pipelines that are being shared by all these different planes, at some point, do I have to wave off some project because the pipeline is being used by some other project, kind of like how I do at an airport? Great, great question. So I think that's one of the neat things that we have with scalability in the cloud, is that we can actually go ahead and burst those pipelines.

So even though there's one pipeline template, it can be leveraged by all the people and we can scale it out infinitely. And I think those are the benefits is that leveraging the cloud and, and leveraging, you know, whether it be the build and the deploy pipelines for what they are, uh, the intention should be that unless there's an actual blocking reason why they have to, you know, wait for each other, right? Because they, they have a dependency, but then that's not a, that's not a pipeline problem or, or a harness problem.

That's a science problem, right? If there's a dependency, but if there's no dependencies on each other, they should just move along and scale out and fill it. Are a lot of organizations gonna have to rethink their CI/CD platforms as a result of all of this?

Because a lot of the stuff that historically we built was for frankly a different era. I think that's what we're seeing across the board is everybody thought they could go build it with 17 different tools, right? We've had tons of people up on stage.

I'll tell you that's that's not the case anymore. Why would we spend an ounce of time building spinnaker or something to the effect of, uh, when that would mean, you know, not not beating their competition. And so now that you've got a pipeline that makes it easy to do the right thing creates the templates, you've got, um, you've got policy in place to make it hard to do the wrong things.

And now we're starting to automate all the parts that people hate. No one likes waiting for tests to run. No one likes creating those tests.

No one likes, um, you know, waiting and babysitting for babysitting deployments. No one likes turning on and off non-production infrastructure. So now when you're using AI in that pipeline to remove the worst part of people's jobs, now people come to it, they actually clamor towards it.

Versus we've seen a lot of AI in a lot of this area where we've started taking the best part of the job, you'll actually find revolt, right? Your, your engineers kind of, uh, push that aside because they look at it as taking the best part of their job. Mm-hmm.

It almost seems to me there's a little bit of a paradox going on here in the sense that you will hear people say they like using these tools to handle specific tasks, but at the same time they'll also tell you oddly enough that they're going slower. And maybe that's 'cause I don't know, they're spending more time on the code and less time on scut work and they're fixing more issues earlier. But, um, what's your sense of what's driving that little paradox?

I see it in two ways. 'cause we actually see the numbers. So you see the generation of code, but then you also see the quality of it.

And I think that's one of the areas where, you know, almost like Paraprogramming was in the day where we would actually go and have two engineers working together, you know, looking at the code and extrapolating from it. And I think it's, it's causing a new way to think about things. So yes, it might take a little longer.

What we are finding is the people are leveraging it to, to clear the white space, right? So I don't have to even just start from nothing. I've got a place to start with or to help me with the challenging tasks that are outside of my boundaries.

Maybe a security oriented, uh, function or, or something that's quite a, I can at least have a starting block. And so to your point, yeah, there might be some troubleshooting and there might be some areas, but what we are finding is qualitatively, uh, we're seeing people actually enjoying it. Uh, the ones that are, that are able to skill up and move forward.

Sometimes the ones that have massive knowledge, like in depth been at the company for 20 years, sometimes their knowledge actually outpaces some of the AI today. And so it is irritating, but for the most part we're seeing a qualitatively beneficial Quantitative is still a struggle for a lot of folks. Mm-hmm.

As you kinda wonder a little bit, we've been talking about platform engineering and various shows and previously and all kinds of places. And, um, will the rise of AI coding tools kind of force more DevOps teams down that path? Because I do need to find a way to manage all this at scale.

So I think that's it. If we're gonna actually take on and become supporters and we're gonna, we're going to support our engineers and we're going to provide them templates that work, we have to work in a more efficient way, right? We can't have 10,000 copies of the same thing.

We have to, to be able to do it efficiently. And so whether that's through, you know, our own, our own work or ai, what we're seeing our customers is removing. I don't want to go and have to create a new template for Java when I've got one for T net.

Can we, you know, have AI kind of help me create it based on my policies, my templates that I already have? I want to, you know, when I have to troubleshoot why a pipeline's down or why it's not working as an engineer or as a developer, I might not know all the 17 different systems that it's being used. So I want help in actually understanding what failed, why it failed, so I can get to it quickly.

Those are the areas where we're seeing people focus because regardless of, of whether they love their DevOps job or they love their developer job, those are the parts that people hate doing. And we've literally, since the beginning of time, you know, this have focused entirely on that. We started with making sure people weren't babysitting deployments.

We then said, Hey, instead of, you know, if I change the gas cap on your car, why would I, you know, test your entire electrical system, do the same thing with unit tests. And now we're adding that even to the point of I don't want to have to go and create tests like, let me take and copy straight from Jira and, and let me put it into a test automation that can actually go and figure out what it should be. Let's have the tools that have the context of your pipeline and the context of your application actually recommend tests that you implement for, for full functional tests.

So I think the idea is let's unburden those people so they can do higher level things, not remove their jobs in any way, shape or form. Going back a little bit to your earlier point as it relates to that, we all have deadlines and people rush to do things and when they're rushed, they tend to short shrift things and skip things. And if it's simpler to complete a task using ai, is it more likely that we're gonna run that scan or that we're gonna run that extra test that we'll have the time to do it, right?

Versus I think there's a tendency these days where we're doing things that we know, know are suboptimal, but we're doing it because we're trying to meet some sort of deadline. This is where I see people implementing and specifically harness our policy and using OPA to write policies that maybe a change advisory board would hold or maybe a governance team would hold or security team would hold. So that those things actually just can't happen.

And so the idea here, and I know I've talked about it a thousand times with you, but making it easy to do the right things. That's one part. Make it hard to do the wrong thing.

So even under that deadline, there are certain things that we cannot miss. You know, we can't skip, right? Especially say we're critical infrastructure or regulated industry, those could, could, could delve into fines and, and much larger problems.

So instead have that be known. I think one of the things that's important is that we all have to be playing the same game. I use a horrible analogy and I'll, and I'll use it today, but I think software's like playing football and just hear me out for two seconds, right?

Most people in the world play football with their feet and America, we play with our hands, God knows why we call it football, uh, in Australia, they bump it and they kick it and they jump all over each other. The problem is no one knows how big the pitch is, what the score is worth, where off sides is. All of these things are different, but that's how every one of our software teams are working.

And until we can actually have the same rules of the game till we know how things can go to production and it's okay to fail as we learn through that process. It's okay if you failed because you didn't do that test, it'll go out next week, right? It's okay if you failed, but I'm gonna give you that response now, not two weeks later.

And so the intention is to actually put that as part of the process. And I guess that brings it back to even playing the video game. Make sure you have those opportunities and abilities to fail.

Um, because we all know the rules of the game. Mm-hmm. As I listen to you, it occurs to me that historically every time there's been a great innovation, we typically use it to do what we're currently doing slightly faster for a little while.

And then one day somebody wakes up and just blows up the whole thing all together and has this epiphany and we do something entirely different. Are we on the cusp of that with software development or are we gonna be rethinking the entire software development lifecycle end to end? I think right now we're in that phase where, where I would almost call it like the renaissance phase where you actually, the reason we were doing paintings and we're able to go and create shows and, and spend time in art is 'cause we had an abundance of everything.

And I think that's what it's providing for us right now is AI is actually taking a lot of those tasks that were menial, the things that we hated. One of the things I actually talk about, Mike in a lot of my talks is, you know, a lot of people say that AI is gonna mimic human behavior. And I actually think it's a little bit opposite.

I think human behavior is going to mimic agentic AI in the sense that we build agents to handle the tasks that we dislike, right? And then we can focus, our teams can focus on the things that they love and more importantly, the things they're passionate and good at. When you put someone who's phenomenal at a task or, or has a passion towards it, there is no time at the end of the day.

Um, yet there are tasks that we all have to do that are minna and we'll do them tomorrow. But if we can remove those, I think what you get. And so I'm not, you know, I'm not necessarily directly answering your question, but I think right now we have an opportunity to be in that renaissance state where we can actually go do the things we're phenomenal at.

I'm also trying to figure out, um, how these AI agents will manifest themselves. And I ask this from this perspective, will I as a developer or software engineer have my own personal small army of AI agents that I am assigning tasks to? Or will the team have a, a single AI agent optimized for a specific task that we're all sharing and sending jobs to, and then it is kind of getting trained and better optimized for that particular task as it goes off.

So I think you'll see both options, but if we actually go extrapolate where the benefit would come from, you know, you can take it a simple medical practice. Would you rather go to a doctor who's seen one version of, of the things you have wrong? Or would you want to go to a specialist who's seen it every single day all the time?

That's one example. Now we take an AI context is what's important. What drives the value in an AI is its context.

It's the same as humans, right? The more experiences we have, kind of, the more it allows us to deliver. And AI is no different.

So if the AI has context amongst your entire team of, through your entire organization, more importantly through the industry, right? Being able to understand your code, your build, your artifact, your deploy, your infrastructure, your database change all the way through, right? Even your APIs and how they're operating in, in runtime.

Now when that AI has that context, it's a very different story than my individual one works for me. And so I think sure, people will create them that work for themselves, but where you'll gain value is where you actually had the most context, which is beneficial to the organization that's using it. And that's, that's our firm belief.

That's why if you look at all of our ai, it includes the context that we have of your organization. And so now it's creating responses. Potentially it's the PR that you can go fix something because we know the line of code, we know who committed it, we know it's part of this deployment.

And I think that's where value drives in any ai, it's the context that it holds. Mm-hmm. Well, let's walk through that for a minute.

Let's use your doctor metaphor. So if I go to the hospital today and I go see all these specialists, each of these specialists tends to think of whatever the cure is gonna be as something that's related to their field. AKA, you see a surgeon and they always want to cut.

So how do we kind of figure out like within that context that the AI agent in particular may have a, a bias towards a particular function or motion because of what it does? I think that's a perfect example of one of the reasons we wanted a platform that doesn't just include the developers, but includes security, it includes finance, it includes governance, it includes DevOps. And I say this because we don't wanna look at through one singular lens.

So I I'm a hundred percent with you. I think we have to look at this through outside lenses and it shouldn't just be about one focused area. Sure you can, if that's your area of, of expertise and, and of course it's gonna build stronger notions, but I think to your point, the value is actually a platform that allows all these teams to work in harmony.

So often you get, you know, finance will buy a tool to go beat infrastructure or security will buy a tool to go beat engineers, I think long are the days that that's actually successful. And now you have to actually operate to that point with a larger mindset and a larger amount of people bringing information into that platform. And so to your point, the more you can get a diverse understanding of the business problem, of the finance problem, of the infrastructure, uh, the application problem.

Now I think to your point, you're getting, you're getting closer. Is it all of it? Maybe not yet, but I think your, to your, to your purpose, that's that's exactly what we believe, right?

And then what is the definition of a developer going forward? And that may seem like a trivial question, but it's not because it seems to me we're gonna see all these so-called citizen developers who are gonna be using these vibe coding tools to rapidly prototype stuff and all that code is gonna come cruising through our DevOps pipelines at some point, hopefully. Um, so do we need to kinda come up with a different way of thinking about developers in their, I guess tiering them based on their knowledge or experience and how do we manage all that?

Sure. I think I'll start in reverse. How do we manage all that?

One of the things that I believe wholeheartedly and is incentivize the behavior that you want. And so if we're gonna incentivize engineers or developers to write, you know, lines of code, maybe that's the wrong incentive. And so I think what changes and where the titles change we've had, you know, um, you know, you can go through DevOps SRE platform, like we've had adjustments on that side of the fence.

We might see the same type of name change in developers. We may or may not. I think the functions that will change are gonna be based on how we actually incentivize people.

And I think that's the key is that to constantly make sure that what we're asking them to do, we're also incentivizing them in the background. If we don't do that, you know, you will cause to your point, a problem where I'm being incentivized one direction and doing it that way, but what the company and the business actually needs is something else. And so I think it's actually upon the business, um, and upon, uh, not necessarily on the tech, but to actually really take a look at what that does look like, to your point, whether it's generated by ai, whether it's generated by a human, whether it's vibe coded or it's built by someone who knows our business for years, I think it still all has to follow the same policy regardless because we can trust, uh, any and all of it as much we all make mistakes and, and putting those guardrails in place is actually will make sure we all are confident in what's coming out regardless of where it comes from.

And so I think that partially answered your question, but um, hopefully that makes sense. Is this in the final analysis becoming a bigger conversation than just something that occurs among US software engineering minded folks? And we need to be able to go explain this to the business leaders who the organization itself is more dependent upon software than ever, but it's not clear to me that they think in these terms or even understand these issues.

So how does that conversation get started? Well, I will tell you, in the last year we've had a massive change. So this used to be a CIO or CTO level conversation, and now this is a CEO, uh, COO and board level conversation.

So to your point, I think the industries that are realizing that either A, they're behind or b, that are pushing forward, um, quickly are all in an understanding that we have to take this on regardless of what we believe our business is. You know, we said a long time ago that, you know, almost every business is a software business at this point. And the intent here is this has become a much higher level conversation.

Um, I'm now having all of those conversations with the highest level executives because it matters to them as well. And they realize that to gain their investment, they put in, say in the code generation space or in the audit, the AI that they've already used, if they don't protect it on its way through a pipeline, it doesn't do 'em any good. Alright, folks, you heard it here.

I think it still holds truly, even in the age of ai, every time you have an action, there's an opposite reaction, sometimes more powerful than the actual action itself. And we need to figure out how to plan for that because it's happening right now. Hey Nick, thanks for being on the show.

Thank you so much, Mike. All right. And back to you guys and Steve.