AI-Powered Testing and Modernization: Insights from Diffblue’s Andy Piper – EcoTech Insights

Hi everyone. We're at Code Remix in Miami and joining me now is Andy Piper, who is the Vice President of Engineering at Diff Blue. He came all the way from England for this conference.

So Andy, it's great to speak with you today. Thank you. Very pleased to be here and enjoying some of the Miami Sunshine.

Yeah, Exactly. Exactly. So tell me about Diff Blue and your role there.

So, diff Blue is a spin out from Oxford University, so it was started by a couple of, uh, ex academics and we use AI to write tests, Java tests automatically for Java code. And uh, we've been going about eight years now I think, and uh, there's 45 of us, mostly based in Oxford. So how has that testing field changed?

Well, uh, we used to be the only product in the space until Gen AI came along. So, uh, there used to be no AI products at all. Uh, and originally people were basically generating tests in a very standard way.

So just sort of template type tests, not using ai, but we actually, because we're able to run the code, we're able to analyze what the code does. Developers are a crowd that are hard to please, and so they like things like, does the test look good? Does it, is it understandable?

Is it named well? So we are able to do all of those things. Um, but the other thing that we do is we test what the code actually does.

So Gen AI tends to test what it looks like. So if it'll read the comments and based the test based on the comment rather than based on what the code actually does. But we, because we're able to execute the code, write the test based on the code execution, we can write tests that are a hundred percent accurate.

Well, where, where does AI come in? Because there's obviously a lot of controversy with that, with people using it and trusting it too much, it needs to have a human overseeing it. So how do you balance that?

Well, so one of our value adds is that you can trust us. So the tests that you get are always correct. We tend to, one of the things we do is discard tests that don't compile, run past all the rest of it.

But also because we're, we're not using Gen AI at all. We're not using lms, we're using reinforcement learning. We're able to take a, uh, very deterministic approach.

So you always get the same test, same test that is right. So with testing, you want to be able to trust it. You don't want something that you've gotta keep checking, otherwise you're testing the test effectively.

So What have you noticed in recent years that developers are asking for that you've kind of had to maybe shift or, or adjust to keep them happy? Sure. So, um, developers very much want idiomatic tests.

So they want, so for instance, we, we have a lot of support for spring. They want tests that look like spring tests in the way that a developer would write a spring test. They don't want just kind of a bog standard Java test that uh, doesn't understand the context, doesn't understand the domain.

So we've had to respond to those sorts of developer needs. Developers always want to, they kind of want to fire and forget when it comes to testing. They wanna focus on actually doing their, their day job, which is writing code.

They want the testing part to be handled for them. And often if they use these sort of gen AI assistance, they have to stay in the loop checking things. Whereas what we do, we, we, you know, we're able to do that a hundred percent correctly and that's one of the ways that we sort of responded to developer needs.

So what Do you see going forward in terms of testing? Like where do you see the space going? Sure.

So, um, I, I mean one of the trends is that the one we've announced here, so we've announced a, a sort of, uh, hookup with modern where they're using customers can use modern to do automated upgrades, and then they can also use us to validate the output of the automated upgrade. And that's a very powerful thing. A lot of customers are dealing with issues where they've just gotta upgrade all the time.

How do they know that that upgrade is gonna be safe? And testing is the way that you can do that. And if you can automate that part of the process, then that's, that's very powerful.

Suddenly you've released a lot of, uh, developer productivity to, to spend on other things. Right. And also do you feel that it's also helping with efficiency in their productivity?

Uh, it definitely helps with efficiency, but you know, developers will fill the time available. Right. It's also about unleashing their creativity to some degree.

So there's a lot of rules and regulations imposed on developers today. They don't really wanna be doing that stuff. They don't wanna be doing the security checks and balances all that.

If that can be taken away by tools and ai, then they're quite happy about that allows them to focus on their creative passion. Absolutely. And since you're based in Europe, the movement of green software and being more sustainable is bigger there than here.

I was just wondering what you've seen in that space, people's interest in that? Good question. I, I, I mean I've seen it in conferences.

Um, I think, uh, in terms of what we do, uh, one of the values of unit tests is they're supposed to be fast and light and, you know, that's one way of saving energy. For particular, you know, lot, lot of people, their testing infrastructure is very sort of end to end oriented and that can be very, very expensive computationally. Whereas if you have a unit test approach, then you can save on some of that output.

But it often people don't write the number of tests that are required in that space. 'cause it's just too hard, too, too time consuming. That's true.

Well, have a great conference. Thank you so much for joining me. Thank you.

No problem.