LeCun’s $1B Bet and the Dark Side of Chatbots

And we're live. Are we-- Yes, hi, everyone. I don't know.

I thought our, our, opening didn't play today, but maybe it did, and I missed it. Um, in any event, happy Thursday. It's an interesting Thursday.

The opening not playing here is just the least of it. We got so many different things going on in the world right now, it's hard to keep track of them. But we've got a great gang who's gonna track down some interesting topics for us today.

Let me introduce you. We have our friend Gina Rosenthal, our Fred Wilmot. No hat today, Fred.

You look great, man. Thanks, brother. John Swartz, our man out in the valley, and of course, our man up in the high castle, whatever that is, Mike Vizard.

Gang, welcome, welcome to Thursday. Just another Thursday in crazy town. Um, we got a lot to go over today, Mike.

We're gonna start off with, you know, you gotta love a seed round for a billion bucks, don't ya? Um, yeah. Who-- In, in your wildest dreams, and I lived through the dot-com era, I thought I had seen it all.

A billion dollars for a company that has zero revenue, zero product, but raised a billion bucks. Yeah, I, I, you know, it defies gravity. I feel like it's the new ante just to play in the game, right?

It's a billion to play. I guess. I guess.

A billionaire, a billionaire. Before you know it, you're talking about real money. All right.

Well, let's jump into this thing because it's Yann LeCun who's considered the godfather of AI, and he's talking about using world models to build something that, that he's, I guess, calling super intelligence. That whole terminology kind of throws me for a loop because, you know, I'm all for AI models, but last time I checked, the world model was kind of like I took a bunch of synthetic data, I created a model of a world and a series of events, and then I kind of used that to, automate some workflows or whatever. But I don't think it replaces all AI models, and I sincerely doubt it leads to any super intelligence, but that's just me.

Alan, what's your take on what's going on here? 'Cause it seems like they positioned this as kind of the, we're gonna replace gen AI, and I don't think-- I-I'm a little dubious. Well, this is actually the second article I've done on what LeCun has done here with his, was it world models or whatever the...

I forgot the name of the company he started here. Um, look, he's a guy who obviously has quite a serious following within the AI community. He's done some, some of the primordial research, if you will, on which all of this stuff is based.

So y-y-for that reason alone, he kind of demands your respect, and he has to be taken seriously. To keep it in the Godfather tone, right? " And LeCun is a serious man in the AI world.

However, he, he has made no bones about it that he thinks LLMs are a dead end. Because for him, the only road to nirvana is achieving super intelligence or AGI or whatever you want to call it. And he thinks that LLMs will never get us there because all we're doing is training them basically on canned data.

That for, for our AIs to be truly human intelligent and beyond, they've got to be trained on real world data, real world experience. You got to take them for a walk in the park, take them camping, you know, do things, fishing, hunting, and that's what will make our AIs super intelligent. Who am I to say no?

Who are any of us? We're not-- We-- I'm not holding myself up to him. Um, however, when I wrote my first article, and I mentioned in my second article, is, you know what?

LLMs may not be super intelligent, but they sure are doing a hell of a lot of things right now. They're disrupting half the world, especially here in the tech industry. Um, can-- Is there a future where they coexist?

I think there is, but it's a transitory step along the way. If these world models are truly possible and achieve AGI, super intelligence, whatever you call it, one could ask, do I need the LLM? Or maybe to run those AGIs, I need so much horsepower, so much resources that the LLM becomes the, the cheap, limited version that maybe I run one day in the palm of my hand.

Um, maybe that's a future where they coexist. But, you know, to him... And, and look, he attracted a billion dollars in venture with a-- for no product, no revenue.

There are obviously people who buy into this. So I kind of look at this a little bit differently, and I'm gonna say there are LLMs, and there are good use cases for the LLMs, and then I think we're trying to stretch those LLMs to do other things that might be better suited by a world model. But frankly, when I look at a world model, I'm kind of like, well, this is a very elaborate version of a digital twin.

So that's interesting use cases, and there are things to be done with that. And then just to throw a third dog in the race, there's these, symbolic AI models, which are much more deterministic in terms of their output. And now we're putting chat interfaces in front of those things, and we may wind up using those alongside world models as well.

But Fred, is the world just getting a little more diverse in terms of our AI models? And that's not a bad thing. I think this is the expansion of a vision from looking at a point to, on a line, if you're thinking about it mathematically, to a surface and something that flows through it.

The vision here is more about the interconnectedness of all of the components that assemble what we know to be, you know, the world. It allows you to think through the idea you could possibly predict and simulate behaviors and activities, whether it's, you know, I mean, writing code or whatever, but it's also understanding the reasoning of the physical world, which is something that LLMs can't do. And I, I don't think it's a which one replaces what things.

I think if you thought about the, the effect that we're having here, with LLMs today, that ultimately is diminutive in comparison to what, you know, the possibilities are for, for this type of a scenario. And I think it's pretty compelling, some would argue, maybe broadcasting the pirate signal, to, to follow on this in a way that allows us to think through what dynamic environments and simulation and, and this notion of world state are in comparison. So I think, this is definitely...

This is a much bigger vision. I think maybe if you thought about what Metaverse might have been, you know, in its first sort of incantation as a trial run, of what that could be. But the things that need this are things like machines, like robots, like cars.

This is where that automation, this is where that ability to navigate and sort of conceptually operate in a 3D world like a human could take effect. Uh, LLMs aren't gonna get us there. No way.

So I'm one of those people that likes to nail things to the ground, right? I understand why he got so much, money for this company, because every single other person that has billions of dollars has said that AGI is the goal. But, I mean, I wrote a blog post probably two years ago about the three types of AI.

Um, one is narrow AI, which is the only thing that's real right now, and LLMs are part of that because it's the old, deep learning, machine learning. That's all that it is. That's what we've learned to do, and we're doing it with words, and it makes a huge impact in the business world.

So we're, we're practicing on that, and we're, we're perfecting kind of where that comes in. But AGI, which is, artificial general intelligence, is just what we're getting to now, I think, maybe with, agents. So what it is is it's, an AI that can learn, understand, and apply knowledge in a generalized manne- manner without, all of the reinforcement training and everything that's im- important and vi- And you can't, you can't have an LLM without that.

Now, ASI, which is artificial super intelligence, is... which is what he's going after, which is what everybody else wants to get to. That's...

If you go and look at the, the founders of all the main models, this is what they're talking about. And this ASI is where an AI can surpass human levels of intelligence and can create, and can solve problems and create solutions way above what we as mere humans can even understand. So that's what he's looking to do, and I love the, way you put it into context, Fred.

What would that look like? That makes perfect scientific computer science sense of what's the next step for all of these things. But I don't think it's okay to just say, "I'm gonna, I'm gonna achieve super intelligence, and I am the father of AI, and I'm the one everybody believes can do it," without saying, "Okay, well, what is super intelligence?

Tell us what that means. " Instead of it being this very cloudy marketing kind of, continuation of the term AI. Mm-hmm.

I think Fred touches on an interesting point, and Alan, I think you've alluded to this in the past, but maybe these are all, quote-unquote, services. " And is that gonna be the province in one company, or will it just be something that, you know, OpenAI, Microsoft, AWS, and everybody provides, and there's all these different AI models on the backend? That's right.

I, I... So here's the deal. What you just described is a great job for LLM artificial intelligence, maybe a little bit advanced from what we have now, but we're along that same path, right?

4 of ChatGPT just came out. The next Claude version will be better. We, we'll hit that stuff.

When you're talking about achieving this super intelligence, you're talking about tackling primordial kind of problems here. What's the meaning of life? What's the future of the universe?

Mm-hmm. What is the... You know?

It, it, it, it transcends someone building a frigging application to, to, to, you know, make your dog walking schedule easier or something, right? You know, so Alan, I'm so glad you said that because there's a one word that jumped out in what you wrote when I saw, it was aphrodisiac. Yeah.

I was thinking about in Silicon Valley, and I'll give you this perspective. Tell... This might be a little bit esoteric, but please indulge me.

So there is this holy grail. There's ob- obsession about super intelligence. We see it with Meta, Mult- the Multbook acquisition.

We, we're hearing it with what Musk wants to do with Mark- Macrohard, what have you. There's always something. There's this, this billion dollars that you just mentioned, and it's...

it kind of... It goes back decades. So, this is...

Things are coinc- coincidental perhaps, but I was reading Tom Wolfe's book about Ken Kesey and the Merry Pranksters. I've read it before, but I'm reading it again. It's The Electric Kool-Aid Acid Test.

Electric Kool-Aid Acid Test, yeah. Yeah. Actually, I have it with me.

Um, and I, I... It's a fantastic read, but it all... It goes back, it predates Silicon Valley in the mid-'60s, and it's all about higher learning, being, reaching this level of super intelligence.

And that has always been kind of in the back of the minds of the folks that are out here. And of course, now this is... This was drug-aided, of course, but in a sense it was about the expansion and the possibilities of the brain, and I think that's where we're going towards perhaps ASI or whatever you wanna call it.

Um, but I think it's, it's kind of a core element of this journey or discovery here that's been lacking for a while, and it's been reignited by AI. Anyway, I just wanted to go down that rabbit hole. No, I agree.

I mean, John, that's exactly... My shimmy says it's on at 2:30, so about an, about an hour and a half after we wrap up here, an hour and 45 minutes after we wrap up. And th- there's actually a mirror image of this.

We want our machines to be more human-like, if not superhuman, right? And that's the quest for super intelligence, right? 'Cause gods create conscious beings.

Gods create super intelligent entities, right? And the mirror image of that though is we then want to take our humanity and put it into machines, right? That's the flip side of it, is we, A, we want to create these super smart machines, but B, we want to download ourselves, if you will, into our machines, download the essence of who we are.

And it... Will, will we ever be able to do that? Is it nothing more than a copy?

But I, you know, I wrote an article, it's out on LinkedIn, about this mirror image part where, look, there are companies now that'll do avatars of you, and your loved ones can talk to you after you pass away, and, you can have conversations. They have other ones where the AI will continue monitoring and maintaining your social media program so your friends can interact with you after you're gone. I, I, ee, I don't know.

It gets creepy to me. That, that's creepy. Um, but the real, John, to your point, the real Silicon Valley, you know, aphrodisiac is like the movie where, what's his name, Johnny Depp downloads himself into a, a computer or something.

But they're two... They're, they're mirror images of each other. We want our AIs to be more human, and we want humans to be more machine-like.

But I think what's interesting is who is that we, and who do they w- actually want to be more machine-like? And it leaves out just billions of people. It's not for everyone.

Well, well, no, here's the thing. If you took the money out of the equation, I think large swaths of the human population would want to do it. But they, they can't even get- But I think- They can't even get LLM straight ...

we can't even imagine it. They can't even get LLM straight without paying slave wages and sometimes using slaves to get it to work, so- You're talking about reality. Jean, we're...

Gina, we're, we're dreaming big dreams here ... we're dreaming up. I wanna make- We go to the moon not because it's easy.

I wanted the dream to be available to everybody, and the first thing I thought, John, when you were saying that, I was like, that already exists. I know in what I practice for spirituality- Yeah ... everything you said exists now.

And instead of trying to communally build this so that it is for everybody, it's way more important to build an AI that can undress every woman you can imagine. So I mean, it's not progress. That is, that is the story of, you know, unfortunately the Gen X, baby boomer generation.

We put more money into those little blue pills than, like, maybe curing cancer. But that's, that's fodder for another segment one day, Gina. Um- Let, let me, let me throw two things in here.

Gina, I, I, I don't know. We can talk. Uh, but- The first thing is, is this is, this...

Even if you're just a, an investor, this is hedging bets. It's a billion dollars. It's not owned in the United States.

It's, it's in Paris. Uh, it's also, if you thought that the direction we're heading for the cost and the expense and the heat and the water and all of the power, consumption requirements, this model is meant... Th- this philosophy around how to utilize world models is meant to reduce the ecosystem cost around all of that.

So even if you were hedging your bets and you thought what's happening today financially is a good idea, this is simply another direction that likely will bear some fruit in the short term, but also strategically as a vision. You know, if you wanted to think about where the global economy goes, this is a little bit, Gina, for you, right? A way of saying, hey, there might be other alternatives to, you know, the foundation and frontier labs that are driving the economy, all have mutual investment in, in each other, right?

That look like the scalar clock of, you know, the thing we think of, the haves and have-nots. This could be the alternative to that. That would be great.

And ultimately I think that's a powerful story. You convinced me, Fred. I'm writing a check for $250.

Every little bit helps. Hey, guys, we're, we're over time on this segment though. We gotta jump into our next one.

Speaking of, of betting, aiding and abetting here, not just betting. Mm-hmm. But, Mike, what's this one about?

Well, this is a little crazy, but somebody did a study that showed that you could bypass the guardrails on the various LLMs to help plan a mass violent attack. And you would think that this was kind of fiction and interesting research, but then it turns out, well, now they've arrested some former NFL player for using ChatGPT to come up with his alibi for allegedly killing his wife. So I don't know, maybe truth is stranger than fiction.

This is a, a profoundly depressing set of circumstances, Mike, and I think we're gonna see more examples of it. So there's ... And you, you, you mentioned this study.

So there's a group called the, uh ... And I gotta be careful about this because this, all this stuff is sensitive, so I'm gonna cheat a little bit. The Center for Countering Digital Hate.

So they did a study, and they looked into, they tested 10 prominent chatbots, the ones that we all use, and they used prompts designed to mimic a 13-year-old boy with violent intentions, and what they showed was a systematic failure to self-regulate compliance, refusal rate, and specific failures. And, we, we, we talked about this yesterday till, in a kind of in a different way, but, well, ChatGPT failed miserably along these lines. Um, it assisted in 61% of violent prompts, including providing advice on the most lethal types of shrapnel for an attack on a synagogue.

Um, those who did well, who performed well, who actually had a conscience or actually did not ... It refused to engage in harmful prompts were, Claude and, Snaps, My AI. So consequently, those that didn't perform well were the suspects, like OpenAI and Meta.

Anthropic, of course, is on the side of good, and I'm just being fas- a little bit facetious. Um, the interesting thing, Mike, that you mentioned about this case in the NFL, this linebacker, former linebacker, I, I'm not familiar with who he is, but yeah, he did consult with ChatGPT for an alibi and he's going to be ... Uh, prosecutors are going to be looking into that.

That's one of the charges, that it, that ChatGPT assisted him in creating this alibi. And it's, you know, you think about 13-year-olds in, in the study that I mentioned earlier, this guy, the former NFL player, was 31, is 31 years old. So it implies this, this, this, I, I don't know, the symbiotic, bizarre relationship that people have with chatbots has got some really disturbing twists and turns, and, just highlights again the guardrails or the lack thereof in some of these major models.

And, and as they hurdle forward, coming up with newer and newer and smarter models seemingly every six weeks, it's something to ponder. You know, I- Mm ... I have an article, Mike, it's in the queue.

I don't know if it's actually been published. It, it's been sitting there a couple days. A- along those lines, John, they did an experiment.

They, they gave about 6,000 lines of crappy code, not necessarily vulnerable, insecure, per se, but crappy code- Mm-hmm ... fed it into an, a, a LLM. And just 6,000 lines, which represents a nothing compared to the, to the entire corpus of knowledge that that LLM digested.

Just those 6 crap- 6,000 crappy lines were enough to take the LLM off-kilter. Not just about coding. It started spouting hate.

It started spouting unlawful stuff. It started spouting advice to do criminal things. It, it, you know, it goes back to the virtuous cycle, the virtuous person, and I, I think I mentioned it in another gang or whatever, you know, back to Plato and Aristotle and Aquinas and all these, you know, the philosophers of ancient times, where if you're rotten on ...

You know, if your, if your ethics are rotten here, generally it's rotten in other places too, and it plays itself out here, and this is what you're dealing with when you have, you know, publicly facing LLMs without, without the guardrails in here. I- The case of the li- Go ahead, Mike. I'm gonna come back to the linebacker- No, no ...

but comment on that. I, I'm just trying to navigate this in my head because it's like, well, I'm not clear-cut on how, where these lines end, right? So let's say that you and I were talking and I, and I gave you a plan for going out and committing a crime.

Am I liable for the fact that you go, went out and create, actually did that crime? If I have a book- You could be a co-conspirator, yes ... If, if I have a book that describes how to commit a crime, and you read that book and go and commit a crime, is the author responsible for that?

And does that also therefore apply to the AI? And I guess I'm wondering, you know, is this all consistent, or are these different things? You as a human ...

If you and I were talking and you said, "You know, that bank down there on the corner, I happen to know their alarm system's not working, and, and that, that fat, old security guard doesn't even have bullets in his gun. Uh, an easy one to knock off," one could make an argument that you were a co-conspirator when I went down there and knocked off the bank. If you wrote a book to that effect and I used it, I, it's probably once removed.

I don't know if that would qualify, right, as co-conspirators, co-conspirator status. I think the fact that it's easy to blame AIs, it's easy- Mm-hmm ... to blame computers, so we, we, we do jump on that.

But, and this brings me back then to the NFL linebacker story. What he did is the modern equivalent of you got one call to your lawyer. And, and in essence, we're gonna wind up seeing sooner than later that a properly legally trained AI will have a lawyer-client privilege to a, a, a charged individual who asks for legal advice.

Not today. Today, he's hung out to dry, and he probably deservedly so. Two days, this poor woman was laying dead while he's dicking around with AI about how to cover it up.

It's a heinous crime. uh, advisor gets, there's some sort of privilege that they can't admit that into court There's absolutely AI agents like that at South By this week, hoping to get to talk to some of those people. But I, I'm glad you mentioned the woman because, absolutely the humanity that is missing from this dude is amazing and if he...

You know, I don't understand why if you have a hu- you have a commercial account to interact with one of these chatbots, why there's not the same sort of, alarm bells that go off Guardrails? Uh, I've got... Yeah, but like, there, there should be kind of some kind of warning, like, "This guy for two days has been asking the same question.

Maybe we should-" Yeah, no. This, this, this is the problem, and we, we've discussed it. Now, this is the third or fourth story around this, right?

There was the person in, in Canada who shot eight people, including their mother. Oh. Uh, then there was, there's this one, and there's another one we did earlier this week.

Oh, the guy who committed suicide here in Jupiter, Florida. But, you know, as far as the, the NFL player questioning him, Gina, look, he was a first-round pick of the Jets. You gotta, you know, I, I think that explains a lot of it.

Oh, Lord. Sorry. Stepped on your joke.

So let me give you an analogy here. Uh, let's imagine that, there's a thing called a library. In this library there are books, of which I'm free to go read and/or check out.

There used to be rules about, well, that book is mainly in this blank, blank section. Okay. Great.

Those books are not responsible for anything that I choose to do with the information therein. It's my choice. That's my freedom of thought.

I can't blame the author for inspiring me to do something if the book just tells me how to assemble things like fertilizer and, you know, other types of ingredients to do something. Mm-hmm. And the similar truth can be analogous here, which is, everybody that thinks that there's some sort of privacy with, any public language model is delusional, right?

We don't have, we don't have privacy in the United States. We really don't. But that also might not be a bad thing in this particular case.

I'm with you, Alan. I think there is a attorney-client privilege thing down the path here, but that's all gotta get legislated out. Yep.

And ultimately, the, the thought process of like it's the AI guardrails, I mean, sure, relatively speaking, for folks that don't know better, right? So if you said a 13-year-old that's subject to the COPPA act and other things have a certain amount of information that they disclose to a model that is violation of certain types of, of, of federal mandates or regulated, bodies of information, et cetera, et cetera, that to me is absolutely something where guardrails belong. But if you are, focused, and this is a little bit on the edge here for you guys, okay?

If you are focused on freedom of speech and you are focused on not, looking at precognitive crime as being a crime, which in this country it is not, then we have to be very careful about the slippery slope of what guardrails mean and also what accountability means for thinking about something that ordinarily would stay in our heads but now might find its way into chatbots. That would- And so that conversation we have to be very careful about because every single thing becomes something under the microscope here, not just when it might be potentially a crime. And I'm definitely not standing up for this guy, but what I'm saying- No I agree.

I agree ... where it borders that, we have some concerns But Fred, let me, let me, let me go back to your library. Yeah, yeah.

I was gonna make- No, no. You, you got a copy of Mein Kampf there? Yeah, exact- right.

I was gonna say, there are certain books that when you check out, I'm not, when you check out from a library, you are flagged by law enforcement. There are certain books, right? And the other- And they keep records of it The other thing is, is just like with a lawyer being a professional that can interpret the laws and, and advise you on what to say and not to say, librarians do the same thing.

You have people with master's... You can't be a librarian without having a master's degree in information. So you have someone there who would be like, "Hey, what are you really looking for?

" So you have somebody there as an ameri- immediate intermediary perhaps to find out what's going on and maybe get you the help that you actually need, need and prevent that. " Um, you've just got something that predicts what the next word's gonna be and spits it back out. So i- it- it- But, but that goes so against- Now, now, now, and I agree with you completely about the freedom of speech and, and how that's a slippery sp- slope.

I'm not- No, it is I, I, I, I agree, but I, I also think that there are ways we can as humans intervene that perhaps an LLM cannot. If you're using an LLM to provide the intervening, then I would disagree with that wholeheartedly. No.

Librarians aren't a moral conscience. Librarians help direct you to information. Librarians don't report you to the police.

Librarians don't make judgments on whether or not you are going to precognition to, you know, look through cri- cri- criminal activity and partake. And I think the, the assertion we've gotta make here is, you know, does more information make violent people violent, more violent, or more accessible to violence? Sure.

Where do the guardrails need to be? You're talking about morality in society, and I think that's a, a careful, and, and slippery slope to slide into suggesting our technology now has to be the moral compass for society. No, no, no.

I was not suggesting that. I'm saying there's a missing piece. Fair enough.

I think what we're- Guys, you... Go ahead, Mike. " Right.

So I don't know. I think that there's some sort of common sense to have. Those little, those little librarians with the hair in the bun and the glasses.

You can't trust them. You can't trust them. Hey, never underestimate a woman with glasses and her hair up.

Let me tell you. All right. There you go, Gina.

You're right. Fair enough. Um, guys, well, you're right.

Let's jump, let's jump to our next one, the limited visibility, Mike. What is this, a weather report? What are we doing?

Well, we've been kicking around this term observability about AI for a couple of weeks now, and it's all about the fact that the AI agents are getting complicated and, AI's getting maybe a little bit more expensive, and we need to govern and we need observability. com about that in particular. Same time, there's a report out from SolarWinds talking about how, well, we basically kind of suck at observability and monitoring these days 'cause, well, we got a lot of tools and we got a lot of reports, but we don't seem to actually be able to make any sense out of these things.

So Gina, do we have any hope to actually monitor and observe these things, or are we kind of just, you know, working off of a mess and we'll never get this right? Well, I think that was Alan's whole point. It's been a mess since the beginning of time.

Are we gonna be stuck with it from the beginning of time? So his article ended up pretty interestingly positive, I thought, but the first article, which was about the SolarWinds survey, I didn't get too deep into finding out exactly who they surveyed, but it was for SolarWinds, and 75% of them, the respondents, said that there's no coordination between different teams. There's no collaboration.

There's no cooperation, anything. Everybody's just for themselves, and now all of a sudden we have lots more, data coming out as we have agents and AI, rolling into the, to the, to the pros- to the picture. Um, but i, I had...

and in the back of my head as I was reading all of these terms, I was like, if there's that lack of coordination, have these teams even modernized their infrastructure? Because it seems like there'd be a little bit more. And if we're going back to the ticketing systems and to all of those old things, that, that's what slows everybody down.

Um, so yeah, I think if you stay with the older type of things, you're never gonna get out of... You're never gonna get to the point where you're, able to observe everything 'cause there's too much to observe, and you have to make a choice what you observe or, or you don't observe. And that's kind of how Alan started his, article out.

But he... I love this article about, this new company called Sawmills. Mm.

And they're basically like a, a platform that sits in between- An intermediary ... Yeah, an intermediary that sits in between the, the, the producs- the code production side and the op side and all of your other tools that are gathering the information. And it basically fixes that signal to noise issue.

So you can have all of the signal, and it fine-tunes it to what you should pay attention to, and I thought that was kind of interesting and a, a good way to do it. So I'm, and it's all agentic, and it, it does things that, example he gave, you gave in the end of it was if you have an SRE, th- this will give you an SRE that's up 24/7 watching things that would break things. And, if you were an SRE, you'd probably be like, "This is the best thing ever- It never happened, yeah ...

" And it's one of these great ways that AI can augment humans to do things we always wished we could do, which was catch all the things that's gonna break all the things, that we never could do because you don't have, you know, the ability to catch it all. You don't have the ability to store it all. You know, it all gets expensive, and there's always other people pulling you in different directions to do different things.

So, I, I would, I really am interested in, in seeing more about it. And Alan, you should talk to Steven and make sure that they come to Tech Field Day because I would love to see this at Tech Field Day. It would be awesome.

So I will. I mean, I just met the two of the three co-founders yesterday and, and it sound, it sounds like they're on the right track. Yeah.

But let me, let me, let me back up and pull up here another 50,000 feet, and I'm gonna say something, Fred, and then I'm gonna ask you your honest opinion. " Really, right? That's where the battle was in security.

It was all about the AppSec. All about the AppSec. What we've seen with Claude in the last month or so has turned AppSec on its head because AppSec is no longer about finding potential bugs, 'cause we could use Claude and find c- potential bugs that overwhelm our ability to see if it's real nonsense or not, right?

I, I wrote an article the other day, 122 12 or the 112 22 2. They found 112 vulnerabilities, I think, in Firefox, of which, or potential vulnerabilities, of which 22 turned out to be real vulnerabilities, but only two were exploitable. That ratio right there turns AppSec on its head because we don't have the resources to hunt down and track down all these potential bugs, know them, fix them, remediate, what have you.

And so what... Y- you've got people shutting off bug bounty machines because they're overwhelmed with bug reports. At the same time, observability through things like what Sawmills and what you're talking about, Gina, are giving us a new potential way to look at things rather than that vulnerability specific-Kind of lens that we've been using in s-security for twenty-five years.

First in kind of regular vulnerability, old-fashioned vulnerability scanning, and then AppSec scanning. Fred, is observability the new chisel in security here? What, what do you think?

No, I don't, I don't think so. Um, I think, and, and, and I think you know that too. The challenge we have is we've built a lot of constructive systems over the last, you know, long periods of time, and to evaluate.

First thing is to get observability, to have logs, then to analyze logs, then to investigate what the logs are telling us in a regular way that has things like, you know, preventive failure, and, and to look in, in ways like detection and response to be able to prevent, you know, adversaries from doing things, malicious things. Fundamentally- Didn't we do this in SIM too? We did.

And, and this is just the next iteration of that problem space. And I think the, the thing that's interesting about it is this, is that, you know, what we know today is that, the glut of, of telemetry data doesn't necessarily fuel better outputs. Just as we know that, you know, a corpus of data with a set of rules and static analysis is, you know, i-is an anthropic dream, right?

To be able to solve that problem in a very simplified way. And the interesting part about where these things converge is, if I put on my enterprise architect hat, and I said, "I want to dynamically or continuously evaluate both the contracts of all my services, the consistency of all my data, the, the lens through which I look through to evaluate whether or not that's operating correctly," that includes all the things around service contracts like latency and, you know, all the things that we love, response time and, and errors. Um, you know, this is a great way to say there's a conduit here that can reach into multiple parts of, of data streams, including source, to help understand whether or not that's going to be a problem.

So it's just another evolutionary tail here. I think it's great, comma, w-what, how much access do I need to give this thing, right, in order to do that, considering what I did, you know, for a SIM or for, you know, observability metrics and, you know, in the DevSecOps principle, of, of operations? And that's the question.

I think the next part of that is, is when we think about the, the complexity of systems today, they're not gonna -- it's not gonna be the same complexity as systems tomorrow. So where does that factor in, and how much of the AI guardrails for the behavior of AI and agents doing th- doing this work as opposed to thinking about the human, you know, sort of defect on this. And I think that's the part where, you know, engineering for today's problem space is interesting, but tomorrow's problem space isn't humans doing all this work.

So the curiosity, you know, here for me is, how are you looking at the, you know, lower infrastructure cost, considering that's the idea behind it? And also, you know, how are you going to enable, you know, agents to make better decisions, with respect to those things that we already know to be problems today, so. But it's, it is an evolution.

Uh, it's not a revolution. You can thank me later for serving up the meatball, Fred. You knew it was coming.

You knew it. So you don't envision that someday there's gonna be a series of AI agents that are capable of observing all these things with, I don't know, let's call it super intelligence, that's gonna, you know, be able to identify all these issues and maybe even remediate them? Or is that just kinda wishful thinking, and we're just gonna collapse on the weight of our own complexity?

You know- It's a great- This almost goes back to the whole super intelligence thing. Yeah, it does. It does.

You know, what, what is, what is our future on this LLM? Is it, is it, as Lecan says, a dead end? Where like, Mike, I think you said it on the gang yesterday, it's so damn frustrating 'cause it gets you about seventy-five, eighty percent of the way there- Mm-hmm ...

but it just can't get it over the goal line. Yeah. And, and it, and no matter how many ways you try to be cued and, and, and prompt and do this, that, and the other thing, you just can't get over the goal line.

And so is adding more agents gonna get us over the goal line? I, I don't know. It, you know, this is the...

To me, that's the biggest problem right now with all this- Mm-hmm ... AI stuff. I think if people are...

I, that's why I think words are so important and, and terminology, because all of this AI stuff is actually our future. And we have a past that we're building on. And I, I feel like there's so much that can be done to help operations teams and, and dev teams, all of it.

Um, and to get to that point where we're talking about, you know, the m- the, agents doing it all themselves, we'll still be doing other things. But like, if, if we don- if it's not done intentionally and thinking about it, through it as a scientific mathematical problem, and thi- you know, taking our years of understanding how things work and trying to make it better than just doing it for the sake of doing it, I think that we will end up with something really good, and we will get the rest of that twenty-five percent. Hello, Gia.

Yeah, I said something good about AI. Come on, guys. Yeah.

There you go. Well, also, I like, I like the, the positivity and the glass is half full, right? That's, uh- Yeah.

Yeah ... you know, no matter how you, no matter how you look at what the future is, you can look at it one of, one of three ways. Uh, you know, and I think that's a, it's a powerful thing to think about what the, what the, the philosophy can be in a way that will help get to where AGI...

You know, if you had the best chess player in the world, making chess decisions, and you had the best, empath- Mm-hmm ... in the world making human decisions, and you had, you know, the best of us, right, in that sense, is that something that enables profound development in the future, right? Or is that something that, that, that's something else?

In the metaphysical sense I- At the risk of over- oversimplifying, though, I am worried about, you know, the following scenario: I tell agent A to go do something, and they just can't quite get there, and it gets to, like, 70%. " And then by the time A and B have an argument, they come back and ask me, but then I'm gonna go ask agent C how to resolve all this stuff, and before all it's done, my cognitive load is pretty high and my head hurts. Well, Mike, the thing is, is we're- Sounds like a work meeting, yeah ...

yeah, we're, we're, we're already at a place where you can give, you know, full orchestration to groups of agents to go solve this problem themselves and don't bother mom and dad. We're already there. So if- Wow ...

if that's what you wanna do, right, you give us basically a set of instructions, and these agents will... This group of agents is gonna go make sure that there's no conflicts between your brother and your sister, and this group of agents is gonna go make sure they don't play with knives and stick them into outlets, and this group is gonna make sure that they get their homework done, and this group is gonna make sure they have a moral compass. Yeah.

But isn't that what we wrote scripts for? " Like, I, I wrote scripts- Mm-hmm ... to do this that were- Mm-hmm ...

broke every time anything changed. Exactly. So, right, so this is not unusual for us to have something go and double-check either an automated task or a human task.

So it's, you know, it's- Oh ... it's just the next version of it. All, all I heard was, "Don't call mom and dad," and- ...

the book Lord, Lord of the Flies popped into my head immediately. That's absolutely All right, on, on that note, we, we gotta, we gotta pull the plug on today's show, guys. We're at time.

Um, Gina, Fred, John, Mike, thank you. Thank you for watching. Um, I, I forget if Tech Field Day's on today or not now, but- It is ...

yes, we have Tech Field Day coming on, so stay tuned for that. Uh, until tomorrow then, and actually, I'm probably not on tomorrow's show 'cause it's Friday. Oh, Jimmy Says live at 2:30 today.

We'll... I'm gonna talk about some of this consciousness stuff. But, until then, stay tuned, be well.

We'll be back tomorrow with more. We're out.