Knowledge Engine Built for AI Agents

Hey everyone, welcome back to Techstrong TV. My next guest on Techstrong TV has actually been on Techstrong TV before. I think we interviewed Aaron, I think before he was with Pinecone, but welcome Aaron Kao to our show.

Aaron, it's great to see you again. I hope all is well. Hey, Alan.

Really glad to be back. All is well. Thanks for taking the time to chat today.

So tell me, when was the last time we had you on? I think it was a few years ago when I was still at Pulumi. And- Sure.

Joe Duffy and the Pulumi crew, right? That is right. Good people there.

So Aaron, we've already told people a little bit, you've been at Pulumi, but give people a sense of kind of how you wound up here at Pinecone. Yeah, sounds good. So I've been here at Pinecone for the last year and a half.

Prior to that, I was at Pulumi, and then as you may have remembered before that, I was at AWS for over seven years. Did all the product marketing around the serverless containers, that entire new compute edge, that era. And then I joined Pulumi because the rise of platform teams building out large-scale systems for entire enterprises, Pulumi was uniquely able to power a lot of that through infrastructures code.

So that was super exciting. And then, as the AI wave took off, joined Pinecone to lead its marketing efforts, and it's been a incredible ride since. Absolutely.

And just so we get it, we got your title on the lower third of the screen, but tell people kind of what your role is at Pinecone these days. Yeah. My title is VP of Marketing, but as with all startups, I do a bit more than that.

So obviously marketing includes comms, product marketing, events, dev rel, but also do a lot with partnerships and business development. And then helped launch one of our new services, Pinecone Nexus. Which we're going to talk about in a minute.

Which we'll talk about. Yep. You know Aaron, I kind of laugh a little bit.

You describe Pinecone as a startup, and it is a startup, right? But this isn't one of these companies that just emerged from stealth last year or something, right? Right.

Give folks a little bit of the Pinecone story, if you can. Yeah, sounds good. So Pinecone, we started back in 2019, and we pioneered the concept of vector databases and the concept of retrieval-augmented generation.

So the mission of Pinecone has always been make AI knowledgeable, and that's exactly what we do. We allow you to make your models and your LLMs, the AI systems that you build, knowledgeable. So this company's been running for the last seven years, and we've pioneered a system and technique, and we continue to build a platform around that.

Over a million developers now, 10,000 organizations, everything from mid-market teams to global enterprises. They're just running production AI workloads on Pinecone and using our system, the vector database, in order to empower their systems with knowledge. Got it.

So, and just for those way in the back, maybe who aren't as paying close attention, you guys started talking about vector database and RAG, you're saying in 2019, 2020. 0 is the one that really went, or maybe it was... 0, I think, was the one that really kind of exploded.

But that was late 2023, wasn't it? Right. So, you predated this AI explosion, if you will.

Yeah, absolutely. The concept of search has been around for some time, and using building vector indexes in order to search. So, prior to the whole transformer GPT technology explosion, you use vector databases to do power recommendation systems for e-commerce sites, or power semantic search for website search or entire company corpus document search and things like that.

So, and then when AI came around, Pinecone pioneered a technique of retrieval-augmented generation, which is like, "Hey, a LLM doesn't really know anything. " So how about all the internal knowledge that you have? How do we give that information to a LLM?

Right? And that's where retrieval-augmented generation comes in, where, hey, you search the vector database to gather information, that data, and then put it into LLM so it can answer with the most up-to-date and accurate information. Right?

So that's sort of the history of Pinecone, moving from a vector database for other types of things to now right in the center of the AI wave. Got it. Absolutely.

And of course, if we're talking about the center of the AI wave, it's all agentic now, right? All of a sudden, the whole generator thing that we did for a couple years is, it's all about agents now. That was then, this is now.

Yep. Totally. You mentioned just, oh.

Go ahead. You want to say something? Go ahead, Aaron.

Yeah, no. That actually leads into sort of where Pinecone has gone over the last six months or last year, really, which is, we started to see at the end of last year agents surpassing humans and the people using our API, right? And so everything is more and more agentic, and it's led to just different query patterns, different usage patterns, which led us to invent what we launched as Pinecone Nexus.

So, as an industry, we saw a bunch of different shifts that happened. One was people token maxed and then regretted token maxing. People gave all their stuff to AI, and then Satya Nadella had a post talking about the reverse information paradox, which is like, hey, if you keep giving everything to AI, they end up knowing everything and competing with you as a business because you're- Absolutely.

And you're paying twice. Right? You're paying by giving them your data, and then you're paying them to use your data, right?

And it's- Right. And that don't make sense either. Another industry shift that we've seen is just the rise of open-weight-like agents because people are like, "Hey, I want to be locked in.

I want to be able to tweak and tune this," all these things. So we see this agentic shift and the people going like, "Hey, how do I actually make use of all these agents, get a ROI on my AI investment without costing way too much, without being locked in, and without giving all away our data," right? " Right?

In the old world, AI, you use AI and humans in the loop to sort of interpret the results and to piece together context and knowledge and use that information. Now we have agents that act. It's multi-turn, it's autonomous.

They're chaining decisions together. And every retrieval feeds the next action. So it becomes really, really expensive when things get wrong and when it finds things that are different or are just not finding the right information, not retrieving that.

And that's sort of what led us to build Nexus, which is like, agents don't have a model problem. What they have is a knowledge problem, if that makes sense. Yeah, absolutely.

Let's jump into Nexus. How do you describe it? Yeah.

So we describe Nexus as a knowledge engine built for agents. So to back up a bit to what I was just connected to what I was just talking about, which is agents don't have a model problem, they have a knowledge problem. So what's happening here is there's a few ways that you can power your agents.

A very common way is I tell Claude Code or Claude Cowork or Codex or whatever to go perform a bunch of things, and to connect it to a bunch of different data sources. So it can go and sort of ping all these data sources, collect a bunch of data, and then stitch it together. And my coworker and all the other coworkers may be doing some variation of the same thing.

So that work is being redone on every single call, and that becomes incredibly expensive. And it takes a lot of time, it is expensive in terms of token costs, and it's repetitively expensive because many people are maybe doing the same thing. And then lastly, the problem of giving your knowledge and data away.

So what we did was we thought, and when I talk about knowledge, knowledge is something internal to your organization that it's how your business actually runs. The entities, how they connect, the judgment only your people have. That's stuff that you have in your company, and that's your competitive map.

So what Nexus does is, instead of at runtime going and retrieving all this data and stitching it together and then giving it to the agent to work with, what we're doing is we're taking all your thousands, tens of thousands, millions of data sources, and we are pre-compiling what we call knowledge artifacts. And these knowledge artifacts allow me to then, when you want to ask questions, when your agents want to retrieve data and knowledge, it's able to do that very quickly. Instead of computing it all on the fly, it's no, we pre-compute it and allow you to access that and speed everything up in terms of knowledge.

So we've seen a few things here. When people switch over to something like Nexus, we're able to save people up to 97% on token costs, up to 77% faster latency, and then up to 94% or so accuracy gains. So the results sort of speak for themselves, which is, hey, it's really incredibly expensive to...

LLMs are good at reasoning. It's good at thinking through and solving problems. But it is not a good search engine itself.

And having it search every single time just to do some basic or complex work is expensive. So Nexus provides that knowledge engine, that knowledge layer, that offloads all that work into a separate layer and then gives it to you so that as a company, as an organization, it's just going to be cheaper and faster and more accurate for you. And then lastly, the way Nexus, we built it is all the governance piece and the open weight piece, those trends that I just talked to you about, we deploy Nexus within your cloud.

And then you can use whatever models within that. So you're able to stay with whatever models that you need or want. You're not locked into anything.

And then Nexus allows you to own that data and the knowledge artifacts from there. So we don't touch it, so it allows you to maintain a good governance boundary and posture. Got it.

Aaron, from where I sit, I see the world break into two camps. One camp is, they're perfectly okay with using, let's say, frontier models, agents, whether it's Claude Code or Codex, Gemini, and it's off the shelf. COTS, what we used to call COTS software, commercial off-the-shelf.

And they tell the agent to go out and do the research and do what it needs to do to complete a task. Then there's another group of people who say, "Hey, that's expensive. " And these are the people who are setting up, they were the people traditionally setting up their own racks.

1 or," I forgot what the new ChatGPT that just came out is, "but I don't need that for 75% of what I do. I could use one of these open weight models and kind of roll my own," if you will. Right.

Would you say Nexus is clearly a help to the people who are rolling their own, but are the people who are just using the commercial off-the-shelf agents, are they using Nexus as well? I think Nexus is for, in a fullness time for everyone, but what we see is that enterprises, and Nexus is targeted towards enterprises building large scale agents, and the platform teams building agents and giving access to those to the business users. And for them, they need a layer like this.

They need something with greater control. I do think for people that want to just buy off the shelf, if they're doing anything at any scale, they'll need a layer like Nexus. They'll need their own knowledge engine.

" And it took forever, and it may or may not be blowing through token budgets. So it becomes even people with commercial off the shelf, they are going to run into this problem, and Nexus is uniquely positioned to just solve that problem. Got it.

Hey, Aaron, we're over time, but for people who want to get more information about Nexus or actually about anything Pinecone related, what's the best? io, and then you can also follow us on LinkedIn or X or YouTube. And yeah, we put out a lot of information about our products and just about generally what we think about this industry.

So yeah. Excellent, man. Hey, thanks for coming here on Techstrong TV, Aaron.

Don't be a stranger. Keep us posted, okay? Sounds good.

Thanks, Alan. Thanks for having me. Bye.

All right. We're going to take a break here on Techstrong TV. We'll be back in just a moment.