The Role of AI in Agile and DevOps with Matt Schvimmer at Atlassian Team ’24 Europe

This is Textron tv. Hi, I'm Mitch Ashley here in Barcelona at Atlassian, team 24 Europe. Another great pleasure.

I've talked to some folks that I've, uh, known for a while. And another gentleman who I've met, talked on several occasions is Match er, who has had a product for Agile and DevOps. Good to be chatting again, Matt.

Great to see you. Feels like yesterday we talked to, like yesterday team in Vegas. In Vegas.

Well, you never know what day it is in Vegas, so it probably was yesterday. I hope we're not still in Vegas, right? Yeah.

Yeah. Barcelona. You did a little shopping here.

Uh, I did. I did. I, uh, I like to remember where I am, but uh, I use this, I had a presentation today on, uh, developer experience, Uhhuh, and I was trying to weave the home team being top of the table, being top of the table in LA League.

I was like, we're gonna get you top of the table here and how you think about developer experience. There's a good analogy. You know, I try.

I'm not, I'm not beyond pandering to the crowd. Yeah. So, Yeah.

Not, not lower than pandering. I'll, yeah, exactly. Well, tell us, I mean, a lot of, a lot of stuff happening, you know, robo AI going, going, uh, ga um, and some unique approaches to ai.

It's not just about code gen, code completion. Yeah. Talk about your, what, your thoughts on the announcements.

Yeah, a hundred percent. That we had a, it was a pretty broad set of announcements we did today, at least in, in my world. And so covering the agile and DevOps space, really thinking about end-to-end, SDLC, not just the cogen as you, as you, as you put it.

Um, really starting with like how we use AI to empower each teammate across the workflow, not just the developer. So we started this in, uh, just the level set on the portfolio, right? It starts far left of code in Jira.

Product discovery is the first thing we started talking about of when you start thinking about what you want to build, right? Ideating, coming up with ideas that goes into Jira, where you plan and track your work, of course. And then, you know, on through to the write of code, which is, uh, when you talk to my friend Andrew about, about Compass and, and, you know, once we get things into production, like the whole end to end flow.

Uh, and so we've really invested in AI across that whole process starting in JPD to help the product manager be more effective and more productive, and help them drive clarity in what they want to build, uh, through to, uh, which you talked, which you heard from Mike through to, into, well, actually before Mike into Jira, how we can help get, uh, requirements ready for dev faster and next into the developer helping him build test and make sure he or she's code is of high quality all the way to making sure that we can continuously improve and make sure what's in the environment stays, stays, uh, stays safe. And so it's, the whole thing is about you, you know, AI as a trusted partner than the keeping the human in charge at every step of the way. That is an important part of it.

It's not bot go do something and I don't know what happened, or I don't control it. It is a, I call it augmented, but it is an interactive Yeah. Kind of human oversee led whatever process you wanna think of.

I think it has to be a partner. Like it's, um, uh, especially when you're dealing with code, the, uh, that, that would be a massive amount of trust that I, I don't think I've seen many customers willing to do, to just let this thing go out and, and build, like, and no matter what, that goes well beyond Atlassian. That's just like AI in general right now.

Uh, thinking about it as a, as a productivity improvement and a guided, a a trusted partner to help me be more productive, but not an autonomous thing that we don't know what's happening. Yeah. It's so important because, you know, for me, like gender with generative ai Yeah, it's interesting, helpful, some good things we can do with it.

Super helpful. Yeah. But when it moves beyond the individual productivity or individual aid into a team setting, that's when it makes a massive difference, right?

It's not just empowering each individual doing their own unique stuff. Yeah. Yeah.

It's that, but helping them do work together Yeah. Be more effective, efficient, whatever it might be. Yeah.

A lot of the that's multiplier effect is Yeah, that's right. So you start getting this productivity at scale almost. Right?

A lot of the VO conversation was around that, right? Of like, uh, I think when we were talking offline about virtual teammate, right? Is a virtual teammate that's helping the entire group be more productive.

And, uh, a lot of the robo agents that you saw launched were about how can the teammate participate with the rest of the group, whether it be, you know, simple things like once the entire team is done checking and code, when we're ready to release the production, robo can be the teammate that says, Hey, here are your release notes from everything, and put that kind of thing together. That's a Yeah, I Don't have to write 'em. Yep.

Yeah. It's, uh, you know, it takes away a lot of the tedium is what it does now, uh, which is pretty cool. 'cause it's probably the least favorite part of people's jobs, the Toil, right.

We call it toil oil and Yep. Reducing toil Ary. Yeah.

Interesting. Um, talk about the reception of ai AI as a teammate. Yeah.

Versus AI as a bot, or AI as a, you know, chat or something else. That's a unique concept that Atlassian has. I think, um, AI is a teammate.

The, the nice thing about it is, uh, uh, developers are a skeptical bunch, and they don't naturally trust, uh, they don't, they don't like to just sit, believe that something's gonna happen. They need to see prove it. And so as a teammate, uh, and allowing the, the developer or the knowledge worker to invoke that teammate helps build confidence over time that they can see, you know, four point use cases, four x you know, teammate does, does exceptionally well.

And so that trust basically earns equity to bring it in more and more and to trust it, just like you would a new hire, right? And you give the new, hire some tasks, you feel better, you start giving it a more advanced tasks overall. And that's, uh, that's honestly how I've seen AI get adopted, you know, across our products today and how our own employees have adopted it, frankly as well.

Interesting. How is the human AI agent interaction is, you know, the, the conversation or the checking in, do you tell, do you tell the agent, do these things and check with me and then I'll tell you what to do next? Or is it kind of built into how agents operate?

How does that work? Yeah. You know, I'll use the example of, uh, the auto dev agent is one of the agents we announced today.

This is the one that, uh, at a high level, it takes a Jira issue that's like the main unit of work that's inside of Jira and allows, you can take it from an issue to code. And there's multiple steps along the way. And we've had, we had to learn over time how to build this that we had to go, we couldn't just say, click a button and here's your code.

The developer wants to understand step by step what you're gonna do and why. So the first thing it does is, uh, and, and it only shows up by the way, when it feels confident if, because you don't wanna become clippy. I Forgot about that.

Yeah, exactly. You don't wanna be clippy, like go Away, right? It doesn't show up when it's got something real to say.

Right. It's gotta have high confidence, otherwise I'll never look at it again. Right.

Right. Trust is lost and, uh, what does it say? Trust is earned slowly and lost very quickly.

Right? Yeah. I, I butchered that analogy, but you know what I'm saying, I we got the guy, we got it though.

But yeah. So the first thing, uh, this auto dev agent does is it, when it shows up, it first will tell you, it gives you context to say, this is what this issue is about and, and, uh, what we believe the requirements are. And so first, so the developer first gets the buy off on of what it thinks the agent wants to do, and then it's like, okay, I trust it.

Now I'm gonna tell you the test plan of what I'm gonna build for you. Trust that, okay, now I will build the code for you and trust that want to edit each along, each time along the way, the developer gets to tweak and interface back and forth until he or she's comfortable all the way to, okay, I trust it. Create a poll request.

And so it's, it's a guided step along the way, continuously earning trust that I know what I'm doing. And, uh, and you know, the, the human is actually a coach kind, helping correct it and educate it. So that's a good example within auto dev, which is something we announced.

Interesting. Does the AI agent, does it kinda learn as you go? Like, I, it's told the human has told me three times to change.

It's getting smarter changes differently, right? Every time you interface with it, you're training the model behind it. Right.

And the other important thing is that model is trained based on you and your company. It's not trained on, you know, everyone else. So it's really tailored to how your organization operates.

But yeah, it's constantly getting smarter based on what tweaks you tell it to make and then realize, aha, I I fool me twice. I'm not gonna do it a third time. Yep.

Well, and then, I mean, that's to your advantage between the agent versus going back to everybody benefits. Well, It's not Competitors benefit, you Know? Yeah.

And you want it, and one you, you know, everyone's concerned with, they don't want their data being used outside of their instance and being trained to someone else because of the security concerns with that. So it's responsible to, to make sure that people are very clear that, yeah, the agent is gonna get trained, it's gonna continue to get smarter, but only in the context of you, which, uh, uh, gives you kind of that peace of mind. And actually a lot of regulatory, uh, like especially in Europe here, require that.

So yeah. Very specific. Yeah.

How About Kind of looking forward, how, how do you think, um, AI agents might help us with security debt or technical debt? Yeah. Or more secure code things, you know, things that are kind of vexing problems, right?

They're not easy Oh yeah. To solve. Oh yeah.

So I totally think this is where the agents are gonna pay off a lot. Um, you know, think about, uh, we've been talking about this a lot internally, by the way. Uh, think about all that code that's in production already that is aging in place.

And as you know, uh, a lot of it doesn't adhere to code quality standards or, uh, there's vulnerabilities on that. Like there's a huge opportunity for agents to automatically fix those things that are, or at least automatic flag things that are in violation to see the same human process. We talked about auto dev to flag and say, Hey, these are outta compliance with the standards you've defined.

You know, here's what I suggest we do. Should I kick off a PR to go and do these changes? You know, that's absolutely where we see things going.

Um, we didn't announce anything around that yet, but something tells me when we talk, Mitch, next time, uh, in Anaheim, mark the Tape right there. Mark The tape there. Yeah, mark it.

Uh, but that's, uh, a lot. 'cause we've talked to a lot of partners that that's not gonna be a one part, there's so many things that are unique to that. Like there's security expertise, there's test expertise, there are different, there's code quality standards.

There's like a bunch of different partner contributions to that that will be like an ecosystem play, I believe, to create these really smart agents that can understand all the different things that are wrong in production. I think you're spot on. Well, let's Go.

It's a really important point. It isn't like, well one day we'll flip a switch and it'll go fix all the technical. That's right.

That you can see the, you can see the steps to get there, like the, to end the demo showing the brand guardian, right? Yeah. I wanna see that this is being enforced on content.

Yeah. Um, removing feature flags, right? Feature flag that goes out.

We don't need those feature flags anymore. We're always on. So you can kinda see, alright, there's a lot of that stuff Yeah.

Where you'll have specialized agent at fixing specific kinds of debt, maybe more upgrade agents that already know how to upgrade code. Yeah. You know, with new versions of things.

We actually built a feature, we talked about that, didn't we? We talked like the feature flagging, we built that internally for our use case, right? We had just hundreds of stale feature flags.

So we built an AI that understands that can find when a feature flag goes scale stale and automatically, automatically, automatically retires the feature flag. You know, this would've never been removed, you know, what's that? It would've never been removed.

It, no. It just continues to, theres no Value Continues to just take on debt, right? Yeah.

And people start forgetting why it was there and it becomes, ultimately it becomes a, it becomes a risk. How about in Production? One of the challenges in software creation is just complexity.

We're dealing so many Kubernetes here and development environment here in cloud this, and you know, it, it's not just code. It's putting all that together. Yep.

You into some kind of a solution. Search obviously is one of the things that helps with that. And the teamwork graph giving context to what, you know, it's being presented back to you.

How do you see AI will help us manage some of that complexity or maybe even reduce it? Hmm. Yeah.

The, the downside with AI producing a lot more code is suddenly I have a lot more code. Right? Yeah.

And the complexity and what it all does, making sure it's under control, that it's compliant becomes bigger and bigger challenge. This is, uh, actually one of the reasons that we have Compass. So Compass, I, you know, if we talked about it, it's like our internal developer platform that, uh, is essentially your microservice component registry, right?

And so it has this map, it connects all your repos and it knows what are all the things that are out in production, what are all the units of code that you've done? Who has access to it? Who owns it, what it does, what are the APIs about it?

So we have all that information. And so the next step on that and, and we also present a very clear message to the team on what the component health is, where their component violations, where their security vulnerabilities. So we can point you to those things.

Um, the next step on that is applying AI on top of that. And like, think about, you just talked about code search, code search, but with like a robo chat type interface where I can ask a question. So show me, show me all the services that have a potential security vulnerability related to X.

Um, and then the next step would be invoke that auto dev like use case to go go, you know, go make this change across all those that I've done now found, right? So that's, uh, that's uh, absolutely the way this thing is going. Interesting.

You know, you mentioned about more regener creating more code, right? 'cause we're more efficient at doing it. The other flip side of that is we also have to be more effective at testing it.

Yeah. Otherwise we're just building up more code that isn't tested or gets delayed. 'cause we can't test it fast enough.

How do you see AI helping us in the testing side of it? Man test is, I think, um, code and tests are the two most obvious initial use cases for ai, right? Yeah.

We updated it says that. Yeah. And, um, yeah, I see it a, like one of the other things that we did with the feature flag process that I talked about, the other agent we had built internally we're not, we we're using it ourselves right now is to find code in production that doesn't have sufficient test coverage.

Um, to flag that as a policy violation. Okay. Um, I, I, I think the hardest part, I think the unit test is very easy for AI to generate today.

The integrated test use cases are like the next frontier that's gonna really, um, uh, be game changing for this. 'cause that's the hardest thing. Variables explode of the Variables explode.

Understanding where this service is used in context of all these other services. And, you know, that integrated test coverage of, 'cause you don't, it's hard to understand that a may break f down there. Uh, that's the right, that's the, that's the biggest value add of, you know, of, of automated testing.

Yep. And, uh, once it gets to that, and it's only a matter of time as it learns the connectivity and almost you, you know, can understand the relationship that makes, uh, some people will call it a value stream. Uh, you know, if you think understand how the value stream all connects, then a AI is gonna really blow up in value for quality code quality.

I Think imagine being able to say, if I fix the code this way or this way, which one is the least testing? Or Oh, then you can have impactful Or Yeah. Yeah.

Give me a give, you know, give it two scenarios and say what's the most resilient approach to this? Uh, that Yeah. We're, we're, we're a little bit ahead of, of we're magic of where the product is.

But no, we're imagineering. Imagine I had a boss once call it imagineering, imagine, which, uh, uh, I, I loved 'cause Imagineering never crashes. That's that's right.

It's really nice And it it's always moving forward. Yeah. Yeah.

But I think this is, uh, I I think this has come, this is gonna come quick. This to this test management space. It's ripe for this.

The security vulnerability space is probably, you know, the fast, the fast follow third leg of that stool, I believe. Yeah. The hardest part is gonna be personally sorry, is uh, uh, the front end ideation.

Part of that is still a very, is the most human, uh, artistic side. You know, like it's, um, it, it's not because a lot of times tell you what to build. This is the old Henry Ford of, you know, if I ask my every customer what to build, they say a faster horse.

Yeah. And so sometimes there's the vision piece of where things are going. Like, I don't know if anyone, I don't think AI would've envisioned the virtual teammate that you just talked about.

Yeah. It took a little more of a, a delighter forward thinking, uh, you know, visionary to build that That's a very innate human capability that, uh, I'm glad to say is gonna be there for quite a long time. It's like, uh, I dunno if you, Alan Kay, who's an Apple fellow came up the idea of the laptop.

Yeah. He is saying, I don't know who discovered water, but it wasn't a fish. That's Right.

That's right. Yeah. That's funny.

Yeah. Yeah. It's, uh, well, don't tell the people at Xerox either about the use Interface.

Okay, well that's all another issue. Exactly. What, let's talk about the adoption for people to customers to start to use AI capabilities and development.

What does, what does it take, what do people need to be thinking about? You know, you don't wanna say that's different. I don't want to try it.

You also don't wanna just like head head over heels and, you know, find yourself in trouble 'cause you didn't think about how to you want Yeah. The ai what's the adoption men mindset need to be? You know, it's, uh, there, there's, there's two things that we've been, that we've learned and one of 'em seems like really obvious, which is, uh, just the, the, um, uh, the material value, the utility of it.

So the utility of it people, if the utility is not there, if you haven't validated the utility of it, people tune it out really quickly. Like, we've seen some of this where we tested, we put a feature out there and you know, like we talked about with the auto dev use case, if it doesn't work that, and I like, you know, if it works 10% of the time, boy, you're never gonna click that, click that again. It's, there's no utility.

The other part that I found that I was, that I was more surprised about was the usability, the, the, like, the visibility of it, how it shows up in context was a lot more difficult to get right than I thought. Like, it wasn't like we first, for example, we put a button up there, it was like a little glowing button. Yeah.

Nobody clicked the button. Everyone, because it, It's like idiot light on your dashboard, like turn it off. Yeah, that's right.

It's that car service light that I, I'm 90% sure my car's not gonna break down. Yeah. But, and I wish I knew what that thing meant, but I Have idea.

Exactly. But people saw it and you know, like, well my work is here. It's not this little button up in the right.

It was out like context matters of where it is. And so then we started playing with how we, how, how it appeared, how it started communicating, what it was doing. And so the, when you married the utility with the usability of it, that was like the secret sauce of, of putting this together.

And, you know, I wouldn't claim that we've, you know, cracked, cracked, you know, the Fibonacci sequence on this yet, but it's, you know, that's, that was a, I like that was a tough analogy. I, I'm with you there. Um, but yeah, we haven't really fully cracked it yet, but, um, we've been learning a lot about having you have to win both for the customer to really adopt.

And So yeah. Like that's something you don't solve once as work evolves. How you do that work that will also change.

Yeah. Yeah. I mean we, um, one of the things I showed in my super session earlier today was this thing called AI work suggestions.

And, uh, cleverly named, it's an AI powered list of what we think the most prescriptive tasks are that you need to do. And it, it pulls in, it's in Jira and it's got like, um, you know, it's not just the backlog you work on, but it also pulls in security incidents, prs, build fails, it shows all this stuff. But we've played with it a number of times now as to where we put it because people didn't find it.

People didn't find it, people didn't believe it. Like it was, uh, it became unnatural outside their workflow. And so we've had to learn how to fit it within more natural context of where they think to look how they wanna operate.

Yeah. So you just can't put it in the middle of the screen every time either. 'cause then now it's in the way what I do.

Well then it becomes clippy. Yeah. Then it becomes clippy again.

You're like, ah, Just Close the paperclip. Yeah. That is the famous don't be clippy.

Right. It, you know, it doesn't go, it's never old. It's always Works.

Yeah. We won't forget it either. We May be getting old, but it's, that doesn't get old, You know, speaking of getting old, I think we're gonna close the show here pretty soon.

Yeah. I'd like to keep talking, but they're gonna cut off, off on power here. These guys need to catch a plane sometimes.

So. All right. It's been a pleasure talking you.

Always a pleasure to you. We won't shake hands. No, we should do that next time I'll have a voice.

Yes, someday. Yes. And we'll get to have some more conversation.

Always. Thanks Pat. Till next time.

Till Next time. Like what you wanna know what people are thinking about the create the product, the technology and how we use it. This is kind of the behind the scenes look for Matt, so we appreciate you sharing that with us.

Thanks for joining us here as part of a part of Atlassian team Europe in Barcelona. And it's been our pleasure to, uh, talk to Matt and all the great folks that have joined us here on video. So thanks for watching and we hope you'll check back again soon.