The Messy Reality of Vibe Coding

Hey guys, thanks for the throw. We're here with Tyler Meredith, the CTO for Unique, and we're having a little chat about, well, security in the age of AI coding, because maybe we're all obsessed with the wrong thing at the wrong time. Tyler, welcome to the show.

Thanks, Mike. Thanks for having me. Everybody and his brother is talking about, well, my God, we're generating more vulnerabilities than ever because the AI coding tools were trained on, well, surprisingly flawed code, and garbage in is still pretty much garbage out.

But there are other concerns here, and it has to do with what data's being accessed, and there's just a whole host of other things to be thinking about. So are we kind of spending too much time thinking about the obvious and maybe not working through the whole issue? Tyler, what do you think?

Definitely, that's my position. I take the position, and have for the past few years, that we are all operating out of ignorance right now. The tools are so new, there's just no baseline, and if you don't know what you don't know, the right thing to do is design and experiment and learn, and then operate from a position of data and understanding.

And so for me, the reluctance to use these tools because there might be issues is the same reluctance you might have to hire a human because they might mess up. Right? We don't know until we try, and there's trying and learning, that's where I think we should be putting our focus, and then we will learn from that data if the worriers were right or not, but so far, I think they haven't been.

Right. Well, it seems like there's two extremes here. There's those that are overly worried and doing nothing, and then there are those who are just diving in the deep end of the pool with little or no regard for anything.

So, are there maybe best practices? You hear the phrase secure by design, and do these things still apply, or did they just get tossed out in a rush to AI enthusiasm? No.

We definitely don't toss out everything. But we take the position of making sure everybody agrees on what is actually valuable. So vibe coders and AI, it can construct things very quickly.

The metaphor is like a construction crew building a house. Before you've moved in, before you have a safe with your passports and your money or anything that's actually risky inside it, go nuts. And if you've ever been to a construction site, it's chaos, and then it all comes together.

And so understanding that the build process might be a different kind of mess. But I've been in technology my entire life, and it's always been messy. There's always been chaos to wrangle, and we have good processes in place for that.

Vibe coding or these AI tools are just like another member of the team. They should obviously fit into the standards and the policies, but worrying that they're somehow going to go rogue and renegade and then not even allowing people to use them, I think those companies are going to feel the consequences of those decisions. So I'm definitely more on the side of go fast, dive in, but maybe not with total abandon, if that makes sense.

Mm-hmm. So it's trying to find the middle ground there. So as you look across at customers and folks that you know, and who's doing it right and what are they doing?

" Yeah. Well, the big dog right now, of course, is Anthropic as the creator of the foundation model. But the way that they've implemented their own creation with the tools, I find that they're patterning the rapid expansion, in a way that I vibe with, if the pun lands.

And then I think there are companies, to name a big one, I saw the way Microsoft rolled out Copilot and just it's an add-on, it's a sidecar and kind of an afterthought, and I don't think that's gone very well. So as two companies everybody will know, those are the two big ones that I think any person who watches this will have some sort of opinion about. But there's a lot of smaller companies.

My personal opinion about Cursor, they were really hot, and they went big and got of a bunch of users, but very early on last year, I said these guys have just kind of sprinkled salt on top of Visual Studio Code, and they're just mashing in some foundation models, and there's not much to glue it together in a way that I think persists, and I think that that prediction is coming true. " And so I think that a lot of the correct companies are using tools to create solutions to problems that are highly valuable, and I think the ones that are not doing it as well are just kind of sticking two buckyballs together, and very quickly, one's going to absorb the other. Right?

And so the stickiness that they've built their business on is going to collapse. Do we need to have kind of multiple models underneath some of these tools? And I ask the question because there's this notion of jagged intelligence, I think is what they call it, where not every AI model is good at every little thing.

And so maybe what I really want to have happen is some model writes some part of the code better than others, and another model may be better at security, and these things are starting to be used in a more collaborative fashion to get to the outcome we want. I wholeheartedly agree with that, and I do that every single day. " And then we might go to Codex and get an outsider's opinion.

And then sometimes we just go to Grok and we say, "Just talk trash about all of these guys," because that's what you're good at, apparently, as a model. And so I find myself... ping ponging off a bunch of things.

6, and things are just happening at light speed. And so yeah, I'm telling my team, "Never get comfortable with just one. " And that's what a mixture of experts-- The models have these MoE designators, right?

And some of the models only activate certain parameters. It's the same thing with humans in positions of leadership. You should have a mixture of experts telling you, challenging you, giving you a different perspective, and you should listen.

You should be listening to all the opinions, because with more data, we can make better choices. So I totally agree, you shouldn't just use one. " I try and live my life by that, right?

" And we do get some highlights where one thing did something stupid, but they're very quickly becoming very capable across the board, right? So the latest Codex, I'm sure you've seen the news about Mythos from Anthropic. The benchmarks are crazy.

I think we are converging at a time where we don't need to distrust the model anymore because it's certainly better. It's made less mistakes over the past 12 months than I would have on my own, and that gives me a level of confidence. I could quickly understand a mistake when it's highlighted to me.

Sometimes bugs can get out in the wild because you're just focused on something else, right? So it's less about the capabilities of the people not being up to par and just context overload for humans, and that's kind of how a lot of bugs, in my opinion, slip through excellent organizations. People are just overwhelmed, and the models can just hold much, much more and make those associations.

So I'm gaining a lot of confidence. Yeah. Is this just going to mirror the way we humans work together anyway, to your earlier point?

" And isn't that just what we've been doing all the way along? I think so. We're implementing something at Unique we call Mosaic, which is an agentic framework of decisioning at different stages of the pipeline, which is basically what you've described.

The more widely known is the AI agentic stuff that AWS has released maybe in the last week or so. It's a very similar concept. Of course, AWS is focused only on their cloud, their infrastructure.

But we are going to have different models responsible for different parts of the pipeline, and there will be a model that's security-focused, and there will be a model that's just efficiency of the code. Is this the most performant way to write a function, et cetera? And they're good at different things, to your point.

And so we are looking to optimize for cost. We're optimizing for the throughput capacity. I like to think that the world we all live in in software now is more like Formula 1, where you're operating on margins of seconds.

Your competitors are going to be pedal to the floor as fast as they can, and if you're going to compete, that's kind of what you need to be thinking about. It's going to take a lot of resource and investment, which is we've all seen with the prices of everything going up and infrastructure and everything. But yeah, this is the new normal.

That's the bottom line, right? AI is here to stay. It's definitely not going away.

There is no bubble. There is no collapse. Some companies will fail, and some will succeed, but this is the new waterline that all of the ships, all the companies of the future are going to have to contend with.

Do you think ultimately that we might wind up with more secure software? And I'm asking this question because the models are getting better and smarter and they're reasoning and they're showing learning different codes, and heck, AI agents are learning from AI agents, and maybe that first wave was a giant experiment because it generated a lot of vulnerabilities, but that's just the first wave, and the next wave's going to be a totally different thing. I agree.

Trust is how we build relationships in sales and marketing, and what I see from Anthropic and Codex and DeepSeek and all these things, they know that there's a lot to lose if their models are just horrifically insecure and big giant holes in terms of the kind of quality code that they can produce. They know that the consumer will pivot to some other system. So it's one of those houses built on blocks of trust where the team inside these foundation model companies, if they screw up and they just are fast, loose, and reckless, the market's going to surface that very quickly, and then it's going to be all over Reddit and all over Instagram and whatever, LinkedIn, and people are going to pivot away from it quite quickly.

So they've got a lot to lose by rushing and doing it wrong, and I have to believe that trust is built on that, just like it is with the operating system of Microsoft or Linux or macOS, right? Those companies have given us a foundation on top of which to build applications for a long time, and we have a baseline of confidence that the OS is capable, right? But none of them were ever perfect, and neither will AI be perfect, but people are going to gravitate towards the ones that are stable over time.

So yes, they will be better and better and better, and more secure code will come out of it because what's at risk is so great if they do it wrong. So let's take this to its logical conclusion. Let's look back in time at all the software that we humans have built.

Is it not already rife with vulnerabilities? It seems like we're learning that the hard way with these new models are surfacing all kinds of vulnerabilities that nobody knew existed. So are we going to have to go back in and rewrite all that stuff because we know that it's inherently flawed?

Well, hopefully, the tools will help us do that, but yes, in some cases we will. I think the lesson to take away from that, though, is that all of these things were out there for a long time, all of these zero-day vulnerabilities that Mythos found or whatever. I saw the news article about it.

And the world didn't collapse. We've been getting on with our business and building our markets and our communities, and it's never been just perfectly smooth sailing, but also it's not like economies and countries and governments just fell apart because there were bugs in software. So it's a mind shift, right?

Like not being perfect is a feature. None of us are perfect, and the AI won't be perfect, and that's okay. And so what I try and encourage people to do is don't operate from a place of fear, operate from a place of curiosity, ask questions, design experiments, learn the things that you may not know, and make the best decision you can with the data you have at the time.

Right? And that echoes industry leadership from many people, not only this century, but for hundreds of years into the past, people have been basically saying the same thing. We have to push forward, and we don't know what we don't know.

And so for me, we're going to get better results with AI 100% because somebody out there desires a better result, and they won't rest until they have fixed that problem, right? And that's the humanness that AI doesn't have, right? It's our desire to drive forward and create and make better.

That's not in the machine, and we'll put it there, right? And so the software will get better because of us, right? Not in spite of us.

So- I think part of the- Yay, people ... part of the concern, though, is that people are saying, well, this model that can find all these vulnerabilities won't be kept secret for long, and somebody else will do something similar, and the open source folks will have it, or somebody in China will have it, and the whole cycle of vulnerability discovery and exploitation is going to collapse to near zero. And so I'm going to have to fix things faster than ever, and so the patches need to come faster than ever, but the amount of time and effort required to create the patch will be also shorter.

So is this whole thing going to spiral into some sort of near real time process? I think so. I think we're a ways away from that.

I think we're compute constrained for near real time, but I believe that is more what the future looks like than not. And the companies that choose not to or can't because they're resource constrained, supply and demand will take over, right? The free and open market will decide who has the solution and what value that solution has to that business, right?

So people will acquire tools, or they'll sign up for subscriptions that make sense, and that thing we've been doing for hundreds of years of bartering and trading back and forth between tribes, between communities, whatever, that's just going to keep going. It's kind of what we are at our core. And I do think it'll spiral to a certain...

Like the water is rising in that watermark metaphor, and we are going to have a certain level of increased quality across the board. And I'm excited about that, right? I may be a diehard optimist, but I see the good in people, and I think that they want to create, and I think that on mass, they create good stuff.

And so I believe that some companies will fade away. Maybe big enterprises that are slower to pivot, they're going to take a hit. But humans don't change, right?

Just because I get fired from a business, I'm still me. And if I'm a hustler and I want to create something, I'll do it. And that's just the human cycle, isn't it?

Right. So yeah, I do think we're heading to that moment. And nobody got out of bed this morning and said, "Yeah, I know what I want to do.

" Nobody. It was just that we made it too hard for them to build a secure one, but if it becomes easier, then people will do the right thing, right? That's what I see on balance, right?

There's always some bad actors, but I just try and... The phrase I like to say, use it or don't, but I've never met an evil baby. We don't start out when we're born in this way that we want to just create chaos and strife and destruction, right?

So I think that most people are just good at their core, and I think people, if given the chance, and they're uplifted, good leadership, I think they want to create good stuff, right? And so I just don't believe that there's all this bad and ballyhoo and scary and fear-mongering stuff about insecurity and oh my God. I'm not with that, right?

Because it kind of runs counter to what I see in the world. I see people helping, creating, lifting up their communities, helping their neighbors, et cetera. That's the mass of us, right?

And so, yeah, I believe that people are going to get up and create better apps and better solutions, fix these problems, and it's just going to get better and better because no one's out there trying to make it worse and worse. There you go. Folks, you're hearing it here.

Hey, when it comes to AI and application security, we just need to remember to speak to our better angels, right? Tyler, thanks for being on the show. Thank you very much, Mike.

Appreciate the time. All right. And back to you guys in the studio.