Navigating Software Development with Sonar’s Andrea Malagodi

Hey, everyone. Welcome back here to Techstrong tv. My next guest is joining us from Switzerland.

Today, I, you gotta love the internet. You just talked to people all over the world, like they're sitting here with you. Uh, it's, his name is Andrea Medi.

Andrea is the CTO slash ccio over at Sonar. Andrea, welcome to Techstrong tv. It's great to have you back, Alan.

Thank you very much. Uh, very happy to be here. Absolutely.

So, Andrea, before we get into kind of our topic of discussion today, let's spend a few minutes or moments talking about you, right. C-T-O-C-I-O. This is a combination we seal more and more of, I'm not gonna say it's common.

Not every company does it this way, but it's not unheard of either, right? Yeah. Where the CTO and the CIO role is, is under one person.

Yeah. But it takes a special kind of person. Let's hear a little bit of your, of your journey to being and why you are the right person to be both CTO and CIO.

Yeah. Um, I guess prior to joining Cerna, which is about four, four and a half years ago, I think, uh, I spent, uh, a number of decades at JPMorgan Chase, uh, working in, you know, a number of different roles in technology. com in the us, which is probably familiar to a lot of your, uh, American viewers.

And also me. I'm a, I'm a customer. There you go.

It's very familiar to me and also helped, um, you know, start the Chase uk. Um, and then I said, okay, it's been a couple of decades. I think I need to do something else.

But I guess throughout all of that, um, you get kind of an exposure to, uh, a lot of fast moving technology, a lot of governance, uh, a lot of compliance, a lot of risk management, and those type of things. And I suppose that's maybe why, um, you know, I have a balance between, um, these two different dimension of Okay. Leading a tech organization and get those things done, and also have more maybe of the information officer, uh, type of, of, uh, considerations in what we do.

Sure. I mean, it doesn't come any more highly regulated and mission critical than, than a large Yeah. You know, a top 10 bank like Chase.

Absolutely. Yeah. No, it's been a, was definitely an exciting place to be.

I cherish all those years and all the learnings. Uh, some of it was maybe tougher than others, but, uh, you know, uh, you, you, you, you learn as you live for sure. It takes, it takes, what is it, iron on iron to sharpen or some, there's some Yeah.

Saying about that, but you, you know what I'm saying? Um, Andrea Sonars, look, to me, it's a well-known brand in the, in the tech world, and I think for most of our audience it is as well. But there might be folks out here who are not familiar with Sonar.

Yeah. Um, give them, you know, what's the sonar kinda story? Yeah.

So I, I think what I, I did for a while when I came into a group and said, you know, I'm from Sonar. Uh, and I would say, and you may not remember or know what that is, but if I say Sonar Cube, which is the product that most people have been exposed to, it would be like, oh, yes, no, I know what that is. And the origin story of that, uh, is, is, uh, is pretty cool.

Uh, it's about 16 years ago, and it's three gentlemen that get together and they find through their shared experience that they basically have the same problem. They're leading engineering teams, and they keep having to go back and re-explain what good looks like, be it in terms of quality or in terms of security. Uh, and they were wondering, why, why isn't there a product around that tries to help engineering managers and engineers to kind of get consistency?

So we all sit around the table, we agree on some type of standard, then we all walk away. But the result two months later is things don't really look the way that we had just agreed. Um, and they basically built SonarCube on this basis, but they had, uh, a few key principles, um, right out the bat.

And, and they were that there was a bias to action. So it's not a reporting platform, it is a platform to help the developer get an actionable insight that they can do something about right now. Uh, and that's, this, that's probably what makes us stand out a little bit, um, certainly from the past where when we come up with an issue, if that's a security observation, a code smell, you know, another type of blocker, um, we give people an explanation as to why that is an issue.

And we also give documentation next the description of how you can solve this. And these days, even we have an AI code fix that if you'd like to, you can also get an actual detailed proposal, change of code that you can go and fix it. Um, and, and that means that we were very early on, um, kind of not really in the shift left, but in the start lefts game, uh, I'm writing code, I'm raising a PR already on my IDE, I could have Sonar cube, IDE plugged in.

I can get some feedback there. I raise my pr, it gets reviewed by the SonarCube platform, and I get actionable feedback, and I can kind of protect my code base, um, from, you know, unintended consequences of poor, uh, code in terms of quality or security. So I can block the pr and in that way I can make sure that, uh, my overall code base is safe.

And the reality is, of course, uh, nobody just works on new stuff constantly. We are also having to do a lot of enhancements, et cetera. So you can imagine you take a piece of code that was maybe before you had SonarCube, and you start taking that code into your IDE and you make a change, well, then you're now also fixing the things that are trespasses from the past, uh, because those you also get feedback on.

So in that way, it, it's quite a wholesome, you know, approach to it. Uh, and whether the developer is, you know, a human developer or an ai, uh, generated, uh, code base that you've made it edited, I mean, it really has no, um, makes no real difference. Um, if you're using an LLM, you are, uh, most probably, um, getting code that is non-deterministic in terms of the outputs.

I, I saw that recently at a, at a big session. Uh, i, I think it was Stripe Sessions in San Francisco where they wanted a demo versel on stage. And it was kind of all cool.

It was a little feedback form. And, uh, you know, one of the founders was up there doing his commit and, you know, they had the feedback form, which was to get some confetti on a screen, and the confetti came down on one side of the screen and he sat down very proud of him himself, and he said, oh, that's funny. We've done this 200 times probably in rehearsal, but has never come down on that side of the screen before.

So the non-deterministic output that you get, um, having a deterministic platform like Sonar, which is based on thousands of rules, uh, is a good balance, right? And, and it is kind of a, a little bit common sense that you want to have this type of, um, um, friction on that code because some of that code can be generated quite quickly. And I think most people will recognize if they've done some development that, you know, if somebody comes with a two line change change, you can kind of argue about that for five days.

If somebody comes with 500 lines change, it takes about three seconds to say, yeah, sure, I'm gonna approve this. And you know, what we see, and I think everybody recognizes is the generation of code is getting quite a volume less in terms of what we see the LLMs generate. Um, so having, um, you know, a trust and verify approach, which is kind of what we are calling it, uh, is, is probably quite wholesome.

And especially if you're working on critical systems, it's obviously much, much more important and really something you must have. Um, and, you know, uh, lots of other systems, obviously yours who use it, but certainly for crucial, uh, critical systems, it it's a crucial, um, element to have in your developer workflow. Absolutely.

You know, we had this conversation yesterday on, uh, a, uh, uh, a webinar that I was involved in. I was moderating it on panel, which is, you know, at some level codes code. And whether that code was written by a human or by, uh, ai, we shouldn't treat any code as secure or Okay.

Without having testing done, right? Yes. And, and, and, and so it only makes sense.

I mean, AI code doesn't necessarily have a, shouldn't have anyway, a scarlet letter, you know, attached to it, just 'cause it wasn't generated by humans. Now, we could look at code by humans and, and, you know, within an engineering organization, we could say, you know what, Andrea usually turns out amazing code, much less, you know, much less bugs, much quicker, uh, deployment versus, you know, some junior developer that really, you know, we need to watch everything he does and go through it. And, and there is, I'm not saying that's not the way things happen in the world, it is the way things happen in the world, but it, at some level, we should be saying, all code needs to be checked, all code needs To be tested.

No, I, I agree with that, and I, I think, uh, I mean, I think we're over this, but, you know, I, I don't, I mean, acting out of fear, um, in terms of, you know, having AI as part of your productivity, uh, efforts of, of, of, you know, yeah, just being productive and being able to put out more, um, I mean, everybody should embrace that. I mean, it would be kind of totally silly not to I agree with you. Right?

Yeah, totally. See, not two. I think what was interesting, and, and we actually issued that this week.

Uh, we, we had done some analysis where we basically said, let, let's generate, uh, a set of tasks. This happens to be about four and a half thousand tasks, and let's actually put those tasks through LLMs to see how they solve those tasks in terms of coding. And we then took SonarCube and analyzed this and did some other analysis.

And, you know, uh, the trust and verify is kind of from this, uh, uh, experience we had and the evidence that we put out is pretty important. And I, those some striking data, um, I felt in the report, and it's available on our, on our websites, uh, to, to pick up for free. Uh, don't worry, there's no kind of form you have to fill in.

You can just download it and, uh, and read it. Um, that's great. For example, when we produced, uh, code from some of these different models, the amount of code that got outputs and created was really, really, really starkly different.

I mean, Claude Sonnet, uh, four was producing something like 370,000 lines of code where open coder was producing about 120,000 lines of code. Now that, that's it, like a huge difference in terms of that. And then you come into, well, okay, is that then more complex code?

Is there a whole issue here of saying, well, you're sitting as an engineer, you're trying to review this output. If you constantly have this volume less output, um, you know, what, what does that type of mean? Um, then we looked at security and, you know, we saw about 70% of the, the vulnerabilities that came out of example outta me Lama were considered blockers.

Right? Again, you know, another very, very big number. And, um, across any model, we found that, you know, there was a very, very, very high propensity for code smells, which is, you know, not necessarily big issues, but long term you're gonna get problems and issues.

And I think, you know, you have to technical building up, yeah. Basically technical debt building, building up. And I think every organization needs to figure out how do we make sure to balance out this?

I mean, as we, you know, as practitioners, we know, I mean, there's no, the, the perfection is, is, is is one thing. Progress is probably more important here. Um, but keeping an eye and figuring out how do we put in the, I call them guardrails, if you will, but you know, more this trust and verify approach, specifically as we know we'll have a higher and higher propensity to use LLMs in our software development work.

Um, because you should, there's productivity gains and you know, they're real to a large extent. Um, but figuring out just how, how we get a good seesaw on, on balancing non-deterministic with deterministic. Um, and then obviously we try to help also, not just to find the issues, but also to solve them, um, and using AI for that.

And there we may have a little bit of an advantage because, you know, we have your code, we know exactly what rule it is you triggered. So we have the context, we have that rule description, we have the solution. When we package that together and send it to, you know, whichever LLM it is you, you choose to use that we offer, um, we get, you know, quite high acceptance rates on all of those because it's very precise.

Um, and, you know, the output is very, very, uh, strong in that sense. Yep. Couple, couple of observations.

First of all, you know, how does the amount of vulnerabilities per x lines of code or however you want to, you know, measure that compare to code generated by humans? Is it that much worse? I think it's not, uh, I mean, at the end of the day, the code that the LLM is producing is based on the code that it has access to, right?

So you have to ask yourself, where is all that code coming from? Well, it's coming from humans. Mm-hmm.

Uh, the other question then to ask is, well, if it's, if it's sourcing code from the worldwide, you know, uh, internet, so to speak, and that code then to some extent must be public, is it then the best human code that's being produced, uh, and the private source code that it obviously cannot get hold of to be able to generate what is generating? So this is a little bit of question, but, but to me, but it's a mixture of everything, right? It's a mixture of both the, the worst, the worst of in the best.

Yeah, exactly. You know, I went to law school a hundred years ago, there's an old saying when it comes to evidence, the fruit of the poisonous tree is the term in evidence, right. But crap in his crap out.

Yes. Right? And, and that's At the end of the day.

Yep. And, you know, this is why there's an argument to be made that we need to develop. And, and maybe the anthropic cloud folks have done this a little more than some of the others, but we need to develop LLMs that have higher quality code to train these ais on.

Yes. Right. And make, make sure we weed out some of the really crappy code, because that only comes back to bite us.

But, but here's another lesson. Andrea. I, I've been in security 30 plus years, and unfortunately I, you know, I, we've learned this the hard way in security.

You can't throw yourself on the tracks and expect the train to stop. And in many ways, this whole vibe, coding, whatever you want to call it, ai, LLM, you know, AI generated coding, the train has left the station. We've gotta figure out how to keep the passenger safe, right?

And how to make it better. How to put in, you know, the guardrails you're talking about and, and so forth. But don't, let's not fool ourselves into thinking that, you know, we could pull the handbrake and, and bring this thing to a screeching halt that's not happening.

No, I totally agree with you. And, and in all honesty, we, we focus a lot, you know, as we do right now in the whole conversation around software code and, you know, actual code being developed. And that, as we all know, the software development lifecycle has a few other steps.

You know, let's figure out what, whether we know what we wanna do, let's write this, let's do diagrams and other things. And again, the LMS can be super helpful in many are many of these stages. Um, and I think the, the, the thing that can sometimes be hard is if you're sitting with a responsibility for a carra of maybe, you know, a couple of hundred developers or thousands of developers, you know, at the end of the day, the buck of mistakes somewhere stops, right?

And so there's somebody that's accountable for what it is that's being done and how it's being done. Um, and, you know, uh, you have developers coming up and say, Hey, I wanna try this. I wanna try this, I wanna try this.

And you're like, you know, I, I want you to try this. 'cause I want to encourage the innovation, the creativity. And yet I also need to know that we have some balancing because I have, uh, responsibility to the organization, whatever organization is I'm in, to make sure that we, we, we stay safe, secure, and specifically if you're an organization that deals with customers, that they are also safe and taken care of.

And it's a super challenging, uh, situation. And some organizations are comfortable, have a risk appetite that's somewhat different than others. Um, you know, we try to cater to, to everybody, but obviously, uh, service obviously, uh, very large customers.

Um, and it, it is definitely a challenge for the CTO today because they, they, they have a, a challenging role of trying to figure out how to strike the right balance between creativity, efficiency, productivity, and yet knowing that systems are gonna be up tomorrow and we're not gonna have some, you know, uh, terrible leak or whatever, uh, in, in, you know, in the middle of the nights. I get it. I get it.

Andrea, we're about outta time. Unfortunately. I could sit and talk to you about this stuff all day.

You know, we do a lot. Me too. We do the Me too.

Yeah, we do Textron Gang, and we do a lot of podcasts on this. Control Alt Deploy is one because, you know, a lot of what you're talking about is, is platform engineering related too, right? With Golden Baths and, and, you know, guardrails, we do a platform engineering podcasts.

I'd love to invite you on at some point as well. Um, we will continue this conversation, but for now, I gotta, I gotta go to a break here and, and bring on our next guest. All right?

Okay. Thank you very much, Alan. Thanks for taking me on the show.

My pleasure. It's always a pleasure to have Andrea Maldi, C-T-O-C-I-O, it's Sonar here on Trunk tv. We're gonna take a break.

We'll be back in a bit.