Nathan Cordeiro on Accelerating RAG Development with the Pinecone Assistant

This is Techstrong tv. Hey everyone, welcome back here to Techstrong tv. You know, we've been talking about AI on the show here now for, oh, going on two years at least.

I bet. And a, a company named that We Band Bandy about a lot is Pine Cone. You know, when you talk to people who are really living this AI thing, not just writing marketing pieces or that, but really kinda doing operationalizing AI as, as we call it, they all talk about, you know, the role of Pine cone vector databases, creating your own s SLMs, LLMs, et cetera.

But I don't know if we've ever actually had anyone from Pine Cone here on the show with us. So I'm really happy to introduce you to Nathan Co Cordero. Nathan is principal product manager for Gen ai, a Pine C first of all, Nathan, welcome to Text Trunk tv.

It's great to have you on here. Thanks for having me, Alan. You know, no pressure, but you're carrying the entire way to Pine Cone on your shoulders here, coming in here, you know, as, as the first person.

So you gotta blaze a trail that others will follow. Well, I'm really happy to be here. Appreciate you coming on.

Um, Nathan, as I mentioned, you're the principal product manager for Gen AI over at Picon. Um, you know, I, I would imagine you've gotta know a little something about databases for that. You gotta know a little something or a lot of something about Gen ai, but, you know, it's not something you went to school for necessarily.

They didn't have it in school when you went there, but, so give us an idea. How, how did you come to, to this role? What, what's kind of your journey been like?

Sure. Um, going back to the beginning, you know, I was, I was an engineer to begin with, so back in the day I started my career kind of coding first at startups and then at smaller consulting companies. And at some point I made the jump to product management where I am now.

Um, and I was at Google for about nine years where I worked in a variety of roles where a consumer or an enterprise, but I spent, um, you know, five years of my time at Google doing kind of deep research. So working with machine learning research on trying to adapt kind the latest and greatest research to all of Google's products across like search and ads and YouTube and all the rest of it. So I really kind of learned the fundamentals of like machine learning and how to apply it to like solving complex problems there.

From there I ended up joining, uh, Coinbase where I kinda led kind of data and ai and I kind of build out our machine learning strategy there, which was really kind of pushing the bleeding edge of how you can apply machine learning and that domain. And that's kind of how I ended up landing at Pine Cone, which, uh, you know, was kind of breaking ground in this new space, uh, bringing kind of machine learning to the broader community. So, uh, you know, I've been here about 18 months and I've been kind of leading gen AI for, for most of that time.

Excellent. So you went from engineer to Google product or uh, to product management to Google. Mm-hmm.

Then the Coinbase sort of a crypto kinda growth thing and, and now Pine Cone, you know, one of the darlings of, of the, uh, AI kind of world. Um, but making the jump from machine learning to gen ai mm-hmm. How, how big a jump was that for you?

Yeah, I think machine learning in general, uh, you can consider it is like the precursor or the superset of kind of gen ai. Um, it felt as if machine learning was really, uh, you know, a specialized tool that was being used perhaps inside of some, some big companies and people with a lot of resources or with really specific problems to solve. Whereas Gen ai, you know, we all saw what happened in November of 2022 with the launch of chat GPT suddenly, you know, every, everyone wanted in on the gen AI kind of rush.

And so what you saw was this transition from it being a specialist tool or a specific engineer's kind of, uh, you know, toolkit to being a much more generalized type of, uh, solution. And so you saw this explosion across like domains and different types of users in a way that you hadn't seen before. So the big shift was really trying to figure out how to communicate, uh, to a broader set of users on a broader set of use cases who might not be all machine learning engineers who might be platform engineers or, you know, just hobbyists or a whole, like other set of people just try to apply, uh, you know, machine learning to their problems now.

Absolutely. Sir Nathan Pine Cone, you know, for many Pine Cone will be forever linked with Gen ai Sure. And LLMs and that kind of stuff.

But, you know, there was, there was a Pine C before there was a chat GPT, right? Yes. And, uh, I don't know how many people realized that or, or kind of really understand kind of the Pine Cone story.

Sure. If you wouldn't mind, I realize you've only been there 18 months, but you, I'm sure you got some of it is rubbed off. Um, give us, give us the Pine Cone story.

How did they wind up in the cat bird sheet here? Yeah, I mean, so Pi Cone, as you correctly point out, is kind of over five years old and, you know, PI Cone is really meant to be the leading vector database to building performance AI systems, you know, at scale and in, in production. And I think a lot of the insight came from realizing that, um, even prior to Gen AI companies were sitting on huge hoards of all of this unstructured data, proprietary data, emails, conversations, images, contracts, you know, this is almost like 80% of the data that's out there and it's all like locked away, um, in this unstructured unaccessible format.

And so Picon, the idea was to be able to find ways to unlock the potential of this data to build all these new kind of products and capabilities. And so you needed a new mechanism to try to represent this information. And kind of the factor in the Vector database was like the absolute right one.

It's allowed you to take this unstructured data and turn it into a usable format and really effectively like search over it. So, you know, the first use case that Capco was kinda actually built for was things more like semantic search, right? Not non gen ai.

So recommendation systems, semantic search, anomaly detection. These were all like the use cases that PCO kind of built for earlier. It was only in the past few years that we saw kind of the explosion, which Gen ai and we still today see that, uh, that, you know, semantic search recommendations are still like some of the most powerful east cases that are built on Apple.

So Ri being the early, the early player was able to like build real infrastructure that can handle scale and can really provide, you know, men's service for this type of database in a way that, you know, no one else can really kind of do to that. I get it. Um, it it, it's interesting though, you know, just in the nick of time necessity, mother intervention is just fortuitous that here comes this gen ai, I, you know, a technology that, you know, ignites the world and, and how do I get information into my ai Well Vector database is a real easy, relatively speaking, real easy way of doing it.

Well, who's got a Vector database? Pine Cut. It sounds like something outta a Monty Python movie, right?

Pine Cone does. Oh. So Pine Cone must be good with Gen AI and that boom, right now Pine Cone and Gen AI are, are forever together.

Um, all right, so let's talk a little bit about, you know, the role of, of Vector is in building out LLMs and, you know, gathering the, the data that you need to train your AI in, right? I mean, of course last week was all about how did they train that AI in China, right? Deep seek and they just grab someone else's, you know, uh, data, but I mean, vector database, especially for, I mean, if you're not OpenAI and you're not grabbing the entire internet as your data set, right?

If you're using a smaller specialized dataset, vector databases is still the preferred method to use for this kind of thing. Correct? Yeah, I mean, so certainly you can use a Vector database to aid with training, uh, model to try to find specific examples to help train or fine tune a model.

But what we find vector databases are probably the most useful for on the gen AI landscape is that to build kind of knowledgeable systems under the paradigm, people usually call rag, right? So this is where you take your kind of proprietary information and you put it into a Vector database and then had query time in May, you can use that kind of a vector search to find the most relevant documents from your proprietary database and provide them to the LLM at the specific time of inference. So this is this idea of being able to augment the reasoning capabilities of an LLM with the real knowledge that your kind of particular business or domain might have.

Think of things like, you know, if you're in the legal domain, all specific contracts, or if you're in, um, you know, the finance domain, it could be kind of proprietary financial documents, things that the LLM would've never have seen in training. Um, now you can make those kind of available to an LLM for, for reasoning, uh, using pine cones vector database. I love it.

Um, all right, so Gen AI is so 2024, right? 2025. We're all about agentic AI and, and agents that are gonna help us and, and Pine Cone, you know, being Pine Cone and, and keeping their position in the market.

You guys have recently announced, I, it was interesting. I didn't think you used the word agent. You used the word assistant, didn't you?

That's right. Yeah, that's right. Is it okay if we called an agent or do you take umbridge to that?

Well, I mean, the definition of what an agent is seems to change depending on the hour of the week. So, you know, the product, we've called it Pine Code Assistant out in the market, but it certainly can fulfill Agent Ag agentic like functions. Tell us what it's, what is it?

What does it do? Yeah, So the assistant is an API service that allows developers to like really easily create knowledge based AI applications, particularly if you're trying to solve these chat use cases or these kind of agentic use cases. Assistant kind of takes your documents, um, like I was mentioning, like legal or other domains, and you can provide a way to quickly ingest them and turn them into a Python assistant, and then you can query that assistant to create grounded factual answers, uh, generated or non generated.

So really giving you the power of, uh, the vector database through kind of higher level APIs that allow you to really quickly, easily build high quality chat or agent applications. You know what's funny, Nathan, my oldest son, he's in his last year of law school up in mm-hmm. Boston, and, uh, he goes to Suffolk University Law School and he's in the legal technology lab there.

Yeah. And, um, they actually are doing just this, right? They're building Chad interfaces for people who, I don't wanna say indigent, but you know, when you go to like family law or a housing court or, you know, those kinds of courts, not criminal court, was it, or corporate.

Um, most people can't afford lawyers, right? And they, and they don't know how to navigate the system. And, and so they're building chat interfaces, chat bots to help people, you know, your landlord is, is trying to evict you, and you've got a good reason why you shouldn't be evicted.

How do you file a counterclaim? How do you file a response? You know?

And, and it, it's, it's exactly what you are talking about, right? But this is where rubber meets the road where real people are getting really helped, you know, a single mom trying to get child support or aid or what have you, right? How do they get into family court, file a claim, get things done, and, you know, this is, this is life changing.

I mean, you know, we talk about AI writing code and all the great things AI's gonna do in the tech world, but this is, this is where, you know, the nitty gritty is, and, and, um, it, it's, it, it's life changing doesn't even begin. It, it it's a game changing type of, of, of, uh, technology for these people. Yeah.

Yeah. You're really seeing, um, a democratization of these kind of frontier technologies in a way that are making them accessible to everybody. And, uh, you know, you see this with both things like lms, but with tools like the assisted, you know, really anyone can kind of build one of these things and need under an hour and get access to the same kind of cutting edge tools that you would have if you were kind of heavily resourced or, um, you know, how to, how to kind of build a company around them even.

Yeah, I mean, we talk about disruption and what role AI's gonna play in it. Mm-hmm. It's this kind of assistant technology and then combining that with the ability to import data.

Yeah. Right. Um, man, what a great, I mean, it, it really, you know, it's, it, I, I don't want to gush about it, but it, we shouldn't underestimate how powerful this is.

Um, so is this assistant available or is this only in the pro the pro version coming down later to other versions or what have you? One of those kinds of things? io today, anyone can log on on and kind of sign up for the free tier of the assistant where they can kind of try it.

Um, as a developer, we're, we're in general availability now. So you can try building an assistant either programmatically via our APIs or, um, within the kind of KU console itself, if you wanna just do a quick kind of proof of concept. Uh, the whole idea is, you know, your time to value should be like really quick.

Um, and you should be able to like, push something in production in a matter of hours. So, you know, that exists today for anyone who's interested in getting started with it, You know, I'm gonna call my study to make sure their, their lab knows. For all I know they're probably using Pine Cone, but I don't dive in that deep into his life.

Of course, you know of, but that, that's great stuff. So it's Pine Cone, do io. Mm-hmm.

Okay. And you could just follow the yellow brick road from there, I guess, or follow the dots? Hundred Percent.

Yeah. Our documentation's up and running there. We have a couple how to guides that'll show you exactly step by step how to build something, uh, or you can just kind of sign in for an account and it's pretty intuitive.

Very cool. Let's talk about going forward. You know, I, I said 2025 will be the year of Agen AI and all this.

Mm-hmm. Now we want to take it to the next step. I, I got my papers and, you know, the Pine Cone assistant helps me develop an app that allows me to draw up a, a response or a complaint.

Mm-hmm. Or so it's court document, but now I want it to go file it for me. To me, that's the next, then, you know, now I need an agent that goes, doesn't goes and does that.

Right. Is that something you see like a Pine Cone assistant version two or something, or, Yeah, I mean, so our perspective on the future of the agentic ecosystem is that it's gonna be hundreds of players. You know, pine Cone is only one of 'em.

So for every given problem, there's probably gonna be, you know, five different people trying to craft that. Right? And so we, we see ourselves as providing that fundamental knowledge, the power of the rest of the age I ecosystem.

So you can imagine that, you know, the Pine Cone assistant can develop all these age agent capabilities, but it'll mostly be focused on trying to solve this knowledge problem better. So if you imagine you ask a question and the pin code assistant will be able to tell whether it's answered your question or whether it needs to go search harder or search deeper, or go to different sources. Um, or you can imagine that the, uh, assistant will develop the capability to handle different modalities.

So if you have things in images or videos or a combination thereof, being able to understand all those different types of modalities. So we're trying to focus in on, you know, making, uh, the assistant more effective at like, understanding knowledge and being able to think about knowledge as part of like more compound AI systems and that, you know, people who are building these more domain specific solutions will go and figure out some of the nuts and bolts about how to file something with the government or how to kind of, uh, perform an action like, based on the dollars that we've provided it. I love it.

That's excellent stuff. Um, Nathan, I think we've covered just about everything. We're about outta time, but before we go, is there anything else you wanna let the audience, our audience know about what to look for a pine cone or something else?

Yeah, no, if you're, if you're trying to build, um, uh, any type of AI system, you know, semantic search recommendations, you name it, and you have kind of any tech scale of private data, you know, pine Code is probably a, you know, a really great way to kinda get started. You know, we have a, a free tier, uh, that is, can be easily accessible after a couple clicks, and you can build on top of our Core Rector database. You can also build on top of the Pine Code Assistant, which will give you kind of a really powerful knowledge based assistant and under an hour.

So if you're, if you're building in this space, you know, come and give us a shot. Excellent. All right.

Nathan Cordeiro, principal product manager for Gen AI over at Pine Cone. io. Mm-hmm.

Check them out. They've got this new assistant slash agent can really help you, as Nathan says, get up and running in just an hour or two even so it, you know, you want AI working in your business is the way to get it. We're gonna take a break here on Tech Drunk tv.

We'll be back in a bit. Thanks everyone.