Feature Management’s Role in DevOps with Trevor Stuart

This is Textron tv. Hey guys, thanks for the throw. We're here with Trevor Stewart, who is senior vice president and general manager for Harness and one of the original founders of a company called Split Software.

And we're talking about features management and the integration of the capability that was originally developed by split into the harness platform. Hey, Trevor, welcome to the show. Yeah, thank you for having me.

Good to see you again. You know, features and the management thereof. I mean, it's been around for decades and yet it feels like in the last four years or so, it became a real thing.

Yep. So what is driving that these days? Are folks putting in more features, trying to deal with more branches?

Are things just getting more complicated? Or why is suddenly this is like on the top of everybody's mind? Yeah, I think it's a, it's a combination of looking to move faster, right?

Everyone's looking to move faster. When you do that, you need to find ways to move safer, right? And that's kind of at the core of what a feature flag is.

It gives you that, that insurance, that ability to turn a feature off quickly if needed. So that's kind of one big component is people are looking to get out, move out faster, innovate faster. I'd say the biggest thing is really the kind of the explosion of AI generated code, right?

When you think about code that's being, we're seeing 14% increase, right? We're seeing massive explosion of code being generated. And so when you think about how you, how do you put the guardrails in place?

Feature flags become those guardrails, right? So we're seeing kind of the, this rapid explosion of feature flags and experimentation. I think in between there, we saw the rise of microservices too, and people are trying to manage things a little more in a modular fashion, shall we say.

So, um, and then ai, just the, and the coding seems to have just exploded all of that. So I feel like there was a progression of things. Yeah, a hundred percent.

Uh, you know, we started the, we started split about 10 years ago. Obviously we sold the company to harness about six months ago. But when we started, you know, everyone, I think the concept of a feature flag was new-ish still, right?

People are like, what? I've heard of a feature flag, I've been doing it for a couple years. What, like, I, I've heard I'm using config files or config changes, you know, what's the value of a feature flag, right?

And so when you get into kind of the real-time capability that that ability to segment your audience, to, to turn something on for a percentage of customers, to test it internally, to be able to get code to production quickly, safely turned off, that's really kind of been the compounding effect. I think, candidly, what's happened though in the last couple of years is we've seen people say, yes, feature flags are incredibly valuable, but how do I measure the impact, right? So when I turn a feature on, I roll it out to say, five, 10% of my customers, how do I make sure I didn't break anything, right?

So this is where our product obviously has release monitoring, but it really comes into kind of best practices around experimentation, right? So you can kind of see the evolution in kind of the arc of the industry over the last 10 years. To your point, it seems like early on it was all about, well help me build software more efficiently and maybe safer.

But, um, I'm starting to see more features in production environments because people are using it to segment tiers of software and usage, and it's kind of core to their business model. So, um, has this become a foundational element of these so-called digital transformation initiatives? A hundred percent.

I think every team is looking to do feature flags. I think every team is looking to run experiments. You know, I, I can't tell you which customer, but one of our customers, they shake their phone and you can actually see all the feature flags that are currently being tested within production, right?

That's, that's mind boggling, right? That you have teams within, within some early access customers who are able to do this. But that's really kind of what we've gotten to, right?

Is how do you get features out to your customers? They don't have to be completely baked, right? How do you get them on for customers?

Give them that ability to test and be able to get the signals as to whether it's working or not. And so, yes, it's a, it's been a really fun category the last 10 years. Anything that kicks off signals theoretically can inform some type of AI model or something.

So how will this all evolve in your mind? Will features eventually become tightly integrated with some sort of AI framework that's adding a level of intelligence or what's next? Yeah, it's all kind of answer that in two ways.

So we have a part of our product called dynamic configurations. It's, it's essentially A-J-S-O-N payload that you essentially ship to the, the ui, you control it through our product, and you ship it to your, your kind of client side or your server side devices. Right?

Now, within that JSON payload, you can have your AI models, right? You can have tokens, you can have the prompt, you can start to iterate like more dynamically, right? We have examples where we've changed out AI models behind the scenes just by changing the feature flag, right?

And so that's where you can kind of start to see the category evolving is how do we fine tune our widget, our, our our agents out there, right? That's one component. I'd say the second component is you're actually gonna start to see as these new capabilities as these agents, as these new workflows get, uh, rolled out to production behind feature flags, you're gonna start to see some of that AI specific data come back.

Let's think at as simplest core, right? This could be, uh, someone's thumbs upping the answer, right? And saying, yes, this is right, or I'd like to, I'd, I'd like to see more information there, right?

So being able to get those signals, we'll start to fine tune those models and help you understand, is that the right model to roll out? Or should we divert traffic to this, this treatment, et cetera. So what's your sense of how many organizations are not using feature flags and trying to manage that?

And what's the hurdle? Is it a is it a cultural issue, a technical issue, or, um, when you talk to folks who are brand new, what are they saying? Yeah, it's funny, I, I talked back 10 years ago when we first started the company.

I'd say everybody knows what a feature flag is today in some capacity. Um, most people are doing it though either through database config file changes, right? They essentially have a config true false, right?

Boolean, or they've built their own homegrown feature flagging tool. We have a, a customer that had 14 in in-house solutions that they built, built right Now, over time, one of those in-house solutions was not great, kind of caused downtime caused and issues. That's why they look to a third party solution like ourselves.

So generally speaking, I would say everybody kinda knows that they need feature flags. Everyone is trying to figure out what is the right way to solve the problem. Do we move from our, our database management or do we get to kind of a third party solution like harness?

Mm-hmm. Now, there are some people out there who would say, you know, I need a dedicated feature management platform that is entirely separate from the CICD DevOps platform. Or there's other folks arguing that this stuff needs to just be integrated into the CICD platform.

So where are we on this spectrum? Yeah. I, I think it depends on kinda how you're approaching the problem.

To your point, I, I think there are kind of three vectors to approach it from. One is the release side, right? And that's really where we're deeply integrated with harness.

It's why we sold to harness, we see feature flags or what is colloquially now becoming known as progressive delivery. We see progressive delivery kind of showing up where CICD stops, right? Get my code to production.

Now I want to gradually release that capability. You also see other vendors approaching it more from the say, experimentation lens, right? So think product analytics vendors, um, those vendors aren't necessarily thinking about it from an automation standpoint, from a, a deployment or a delivery standpoint, right?

Which means they, they aren't kind of building the workflows and the automation and those types of things. The other category is more the monitoring side, right? Which kind of goes to what I was talking about earlier.

I turn a flag on what happens, right? So now you have kind of the monitoring space looking at it as well and saying, okay, well we're gonna offer some lightweight feature flagging capabilities here. However, again, you're coming at it from a data first perspective.

You're not coming at it from a orchestration or workflow perspective. So I I, I'm a little biased, right? I like to see it more integrated deeply into the CICD workflows.

Um, but those are kind of, kind of the different vectors who are kind of thinking about feature flagging. We see a lot of folks these days talking about platform engineering, and I can't help but wonder is as I go down that path, am I not gonna trip over this feature flag management issue? Yeah.

Yeah. At the end of the day, everyone, I think there's a bigger trend here though, which is more around the platformization, right? I think everybody is looking for, um, a kind of best of breed platform that brings together kind of full visibility.

Our view is that we see that kind of best attached to kind of A-C-I-C-D progressive delivery, software delivery type of platform. How ultimately dynamic is all of this gonna get? 'cause I think, um, maybe we, you know, if I look back in time, software development is almost feels downright leisurely back a few days ago.

And, uh, here we are looking at the future with AI coding tools, and tomorrow it seems like everything is, um, moving at a much faster pace. So are we prepared to kinda make that transition and, and, and what needs to change? Do we need new CICD platforms entirely or do we add this in over time?

You know, what is the impact? Yeah, I think there's a lot of ways to answer that. I think if you look at it from, you know, if you think about personalization, right?

Think of, uh, again, let's go to a banking application. Uh, my banking app is gonna look different than yours. Not, you know, purely because I'm using different products, but because they're trying to sell me different products, right?

They're looking at me and saying that you don't have a mortgage with us, but you have a car loan, right? And so they, you're starting to see more personalization show up, and companies are building for that personalization, right? Based on what they know about you as a customer, how do they essentially build the product that's best for you?

And that's where I think you're starting to see feature flags, configurations. You're starting to see experimentation come into that. Uh, not every, not every feature is an experiment, right?

You know, some of our customers, 10% are experiments 20%, but not every feature is right. But they are trying to figure out how do they build the right product for you, for what you need to accomplish, right? So you're starting to see more of that.

Now, I think what you kind of, the other part to your question is more as the industry is evolving, you know, I like to think of it like a funnel, right? We're seeing a lot of code, we're seeing a lot of features, but the funnel isn't getting, it's, it's opening up the aperture at the top, but it all still has to go through a pipe, right? And so it still has to go through, get checked, look for bugs, look for security vulnerabilities, all the software is doing that as it kind of goes through those deployment cycles.

And so that's really where kind of the best in breed kind of software delivery products come in, right? So again, to your question of like, we're seeing a wider aperture at the top, more features, more code, how do you get those out safely, right? Whether that's, how do you think about reliability, speed, efficiency, et cetera.

You've seen people talk about this in the past about how the pipelines themselves are fairly brittle. So as we kinda look at these legacy platforms and we see all this AI stuff coming down in the volume of code at the top of your funnel, is it just a matter of time before something breaks? Yeah.

Either brittle pipes or a lot of pipes or a lot of confusion over what pipe. And I think that's what we're seeing in our customers is, you know, if I kind of think about my, the feature flagging base of harness, you know, the thing they're looking to us for as part of this acquisition and part of this integration into Harnesses Harness has best in class orchestration pipeline capabilities, right? How do we bring that to feature flag releases?

And so that's really kind of what you're seeing our customers are asking for is templatization, right? We wanna roll a feature out, here's the process by which we release a feature, we wanna run an experiment. Here's the process by which we do that.

And the reason I mentioned that is that I think that's actually kind of how you're gonna see the category continue to evolve, is you're gonna see best in class templates, best in class pipelines, people trying to get to, can they get to a single pipeline, which is here's how we release our features, here's how we deploy our features, et cetera. Single pipeline. I don't think I ever would've imagined that being possible, but here we are.

Um, so what's next for you guys? Where do you go from here? Now do you got the, the two platforms integrated?

What are you looking for for the rest of the year? Yeah, so we got, so yeah, we did announce the integration, uh, got the two products together. Um, we're currently in the midst of moving our customers into harness.

That's kind of obviously a big step, right? We can, we can build the two products and kind of bring them together, but we're bringing our customers in right now. Um, after that, you know, we're really about kind of delivering on the acquisition vision.

You know, when we, we sold to Joti and the rest of the team here at Harness, our vision was how do we bring the best in class automation, orchestration, workflows of Harness and back that by data, right? And so what I mean by that is imagine a pipeline that says we're gonna turn this feature on for 10% of our customers. We're then gonna make sure that we don't see a spike in page time or a spike in errors.

Well, if we do roll that back to internal testers, only if we don't, let's turn that on for 20% or ramp that up to 50% and then let's run an experiment. And once that experiment is done after seven 14 days, publish the results of that experiment to a Slack channel so that the team can review that and approve it going to a hundred percent, right? So that's really kind of what we're trying to deliver here.

Uh, hopefully by this summer we'll have kind of the, within 12 months of the acquisition, we'll have delivered on the vision of pipelines and automation plus data. Now, that gives you a sense of kind of the acquisition vision coming to life. I'd say the other big thing that we're focused on is there's a broader trend in experimentation and analytics around warehouse native.

And what I mean by this is, you know, every customer today, as they think about that data side of the equation, right? Uh, return a feature on for 10%, being able to understand page load time or error rate spikes or impact on checkout rates. Well, they're paying a data tax, right?

They're moving that data to me from their own data warehouse. And so we're looking at building warehouse native, which will essentially allow us to do all that computation all within our customer's data warehouse. So that helps from a cost perspective, that helps from a security perspective.

And so that's probably the biggest thing you're seeing us focus on outside of the app acquisition. I think a lot of DevOps engineers are not sitting around necessarily worried about whether they're gonna get replaced by ai, but they have this nagging feeling in their guts somewhere that says that this is gonna be a, a significant challenge as the amount of code going through those pipelines starts to increase. So what is your best advice to these folks about how to get in front of this?

I think there's a few thoughts. I think one is, you know, looking to platforms like Harness, candidly, right? When you look at Harness, uh, harness is, is continuing to build capabilities that are trying to make that easier, right?

Whether it's, um, thinking about QA and tests, automation, right? Making, so you don't have to update your tests constantly, right? Can ai, I just update my tests for me and catch bugs for me, obviously with our merger, with Traceable, being able to think about from a security standpoint, um, being able to catch those security issues automatically on behalf of customers, halt pipelines, tell 'em, Hey, you have something going on here, don't turn that feature on.

Um, that's really kind of where I see the, the, the market starting to look to companies like Harness and saying, Hey, if we can get all that automation done, then it doesn't matter how much is coming in, if we have the right tools going through the pipeline. Folks, you heard in here, there's a little bit of an irony in the fact that the future of software development is really hinging on how we manage the technology that we built about 30 years ago. Hey, Trevor, thanks for being on the show.

Awesome. Thank you for having me. All right.

And back to you guys in the studio.