Google Partnership for AI Assisted Code – Brandon Jung, Tabnine

This is Techstrong tv. What the pleasure of being joined by Brandon Jung. Brandon is VP of Ecosystem and Business Development at tab nine.

Welcome, Brandon. Mitch, thanks for having me so much. Appreciate it.

Uh, absolutely. Great to have you. A fellow Coloradan, by the way.

Absolutely. I have my background from Florida. That's not that, that what is deceiving.

The first time we met I was like, wait, you're coming from the East coast? And he's like, no, no, no. I should, I You could put the mountains there.

Let's put like Rocky Mountain National on the back for you. There you Go. And a buffalo standing in the back or something, right?

Yeah. There you go with the flatirons. Um, so tell, tell us about yourself and tell us about tab nine.

Sure, of course. So, uh, Brandon Young, I think historically, I, I started IBM for 10 years, learned a bunch. I was a, you know, basic Linux engineer and some sales, some m and a built a consulting practice.

Uh, went to Google when they first started Google Cloud and got to launch, uh, Kubernetes was on the board of the Links Foundation, helping get that out, which was an amazing run. Just to see something that is technologically underpins most and now what we sort of take for granted and containers and, and how those run. Um, and then went to GitLab for a little while and then at tab nine here, uh, about two and a half years and, uh, lead up helping us with AR and pr, with marketing, with sales, customer success.

So, uh, everything you need when a little company's getting started. So, uh, been a fun run for two years and not so little anymore. So we got lot, lot of work to do.

Mm-hmm. Yeah. Well that can be a good thing.

You know, we look at old days is the good old days, but it's nice to have growth and You need both. You need both. Sounds like you're the guy that add some oils to the gear, oil to the gears.

Keeps things smooth and keep you in front of hopefully. I mean, we'll see. Yeah.

I mean, you know, we can always check on that. Hopefully I don't grind too many gears while I'm at it. Uh uh.

But love building. I really enjoy building new teams Yeah. And problems that haven't been seen and that's just kinda, uh, took a while to figure that out.

But, uh, I think, uh, I think I found a spot that I enjoy and, uh, can bring some value and that's kinda what's life about. Right. Find the, some fun, fun place you can help people.

We share that too. Building teams and applying technologies. It took me about five years in early my career before I figured out like that's what it is.

That's what I Well, you're way faster than me. It took me to week I 20 to figure it out. Okay.

Oh, I like building teams when they're new and they're not well defined. Ah, Well that's a great, that's a great space to be in. Maybe not for everybody.

So, but speaking of great spaces to be in, I want to talk about tab nine. And I know you have some news around, uh, Google, but you know, this space is hot right, right now because of generative ai. It's Yes.

In Telecode with Microsoft has been around for a while. It's not brand, brand new, but mm-hmm. Um, you know, co-pilot and that's been, you've been around for a little while too, but those are things have surfaced up to the, you know, the visibility just because of the broad nature of how he Yeah.

I just now talked about cuz of chat, G p t generative ai. Yep. Well, you've been, you, you meaning tab nine have been at this for a while, have you not We, yes.

We actually, originally we started back in 2012, uh, well before there was anything known generative AI didn't, no one even had the name. Uh, we start actually mostly on really kind of more along the lines of an IntelliSense. So, so not a AI based, but, uh, semantic completion in the Java space.

Mm-hmm. And then 20 18, 20 19, uh, was when the first, when G P T two first arrived and our founders, uh, before I was there, so this is totally drawn Iran or two founders, they be the business and said, no, this is actually how we think code will evolve and took all the best practices around UIUX and all they've done for the job of space and then just change the backend. Just, just as probably, it's not quite that simple, but, uh, change out that backend to leverage the Little refactory.

Yeah. Just a small refactory. Yeah.

I mean, right. Exactly. That's, this is a little better way.

Okay. But here's Yeah, the magic refactory, we can go into how many conversations we have on that. 0 and it did it for him, I'm Sure just came out.

Yeah. So, um, yes, it was kind of, it was, it was, I mean we're now on G P T four kind of, and we're talking at least fifth kind of version on that. We played a, we used G P T two when it was open source and then that got closer.

So we, uh, really ended up jumping in and then we learned what and how to build own models, trainer models, a lot of other things we learned through the process. Um, and so taught us a lot over the last four years. And, uh, we've brought along, you know, uh, millions of devs on the, on that journey and they've been kind and very generous to us, uh, been the foundation for what we've learned and foundation for our growth.

And we've enjoyed that. And now it's really shifted with, I think you hit on it, I think it was chat G P T that we saw change it co-pilot was big, but it was really chat G P T that brought this mainstream to everyone saying, oh, I really need to solve this and I need to do it in a secure, scalable way. And I think that's honestly what we're wrestling through as a, as an industry is the trade off between convenience and ease and speed and quality security and, uh, repeatability.

So those are takeoffs welcome to life. Right? Yeah.

So are you still using, uh, the strategy or Well, the, the p PT large language model, yes. Okay. So you own, Well, we, we wanna on our own.

And I think the, when we look at sort of the market, I'm gonna go down a rabbit hole, but I think it's kind of interesting when, uh, I think it's easy because GP t's sort of been attached to generative ai. They've sort of become, in many cases, thought of as the same. That's just one family of large language bombs.

Exactly. Uh, obviously Google's had this for a long time with anything that was, uh, was burnt before. Now they've got palm and there's many, well, based on The transformer par, uh, It's all the transformer based.

Yep. That idea implemented in many ways. Yeah, Yeah.

All all really coming out of the, uh, a paper from Google called attention's all you need. And that's really kind of where these all develop from. Mm-hmm.

But, um, at this point, there's a huge number of open source models, um, out there. Most of the open source models are smaller than the very large ones that clouds are running. And there's, there's reasons for that.

There's a, there's a cost, there's utility, there's a, uh, fit for purpose question that each one of these we will wrestle through. But I think we've got very large models and even G P T four has said, Hey, I don't think we're gonna see G P T five. Uh, and I think what we've seen is sort of the normal sort of inflection of mm-hmm.

You know, one model, the idea initially was all the, the bigger the model, the more the data, the better the solution. And that's probably a true statement in a statement of where you're the a chat G P T I have no idea Where you're generalized Yeah. That's, you know, many, many domains, right?

Yeah, exactly. Yeah. A ton of domains.

And, you know, you need to write a, uh, a Cardi B song, uh, like Shakespeare or vice versa, whichever you prefer. Um, and, and at the same time, you want to have it write you a Python application for a, uh, data pipeline thing, doing the same area, the trade off on this. You can make a really big model.

Uh, it won't be as good as fit for purpose. And there's another implication, which I have not seen many people discuss. There's a massive cost difference in terms of how you do that.

So, uh, you can have a very large model, but if you're gonna run a very large model, the inference to run that model is very expensive. Well, even the data prep, the data, you know that as well into it for sure. That's massive.

So yeah. I mean that's gonna extend how long it takes to get the next rev of the model, right? So yeah, you, you, this is the targeted Yeah, exactly.

You have two piece, and you hit on both of 'em, which is there's the model training. So a single model training run at this point for a a, a very, very large model, like a G P D four is hundreds of millions of dollars in compute, or tens of millions at least. But we're talking very, very large, uh, to the degree that, you know, Microsoft or Open A was clear like, Hey, we built G P T four, we're not, we think we can do better, but we're not gonna do it cuz it's too expensive and uses up too much compute on the, the base level training.

And they may do it again later. That's one aspect. But then there's also the survey, right?

The bigger, the bigger the model. Uh, I mean, it's like a tree, right? You have to transverse the tree.

Mm-hmm. And the more branches you have to go through branches in a model equal equal number of compute cycles. We had this just in basic neural net back when I was doing ai, right.

The bigger that thing was, the longer it took to traverse the net, Right? Yeah. So that's why, you know, open ai, why Microsoft Open AI built a completely custom data center for opening AI with, uh, you know, you know, millions if not billions of dollars of investment.

Uh, well, hundreds of millions of billions of dollars into a single data center, maybe multiple. We we're not a hundred percent sure. That's a, that's an interesting solution.

Is that sustainable, is that it's one way of solving it, but it has other trade offs. And I'm not sure that for at least in our space, I think we're already seeing those very large models are, uh, less purpose fit. Yeah.

Mm-hmm. Yeah. So if I put my analyst hat on for a minute, um, sure.

And I'm not trying to do to do a commercial for you, but it seems No, no, no. Yeah. Pretty wise to me that you split off from, you know, the open AI path as somebody else's large language model and did your own so that now a few years later, you're not suddenly stuck in with everybody else who's also using that same model and then trying to figure out how they go create their own maybe love that may Be The rosey picture of it.

I'd love to say so. So it was absolutely fortuitous, but I I think it was fortuitous, not necessarily because, uh, we chose it. Um, when Microsoft made the investment in open AI and G P T three, uh, we weren't allowed to use it.

Yeah. And that was because they were rolling out co-pilot and hey, you know, when you put a billion dollars and 10 billion into a company, you do it for a reason. And that's a commercial reason.

That's fine. But I think it actually was one of those that we ended up, uh, it's a little bit like taking the training wheels off, uh, my kid's bike. And you're like, Hey, good luck with that.

Yeah. Kick you outta the nest. You kicked Outta the nest.

Right? So we had to go learn a lot about those models. We had to learn which ones are the best ones, be able to quickly train them, adjust them, pivot.

And so it became a muscle, right? We've now learned that muscle that says what are the models out there that we can leverage and which ones are best for code? And maybe some, and sometimes some are better for one code than another.

Uh, and those are all in a space that moves very quickly. That's a, that's a core skillset that's turned out to be really good for us. So mm-hmm.

Uh, I don't know if we would've, I don't think we would've intently chosen necessarily, and it was mm-hmm. But it was, uh, was sort of, uh, Forcing function. It Was a good forcing function.

Agreed. Yeah. Yeah.

Sometimes it's, well, you know, done startups too, and you should need absolutely great people and funding and lots of things happening for you, but Right. Plays the right time and luck, you know, the mm-hmm. Those are hard to beat, you know, the timing of things and sometimes things work to your favor.

So that's awesome. Uh, so talk about your announcement, um, with Google. Curious Oh, sure.

What's happening there? Yeah. So I mean, from this perspective, um, probably some framing is probably helpful.

So when we think, I sort of like to think of, and this is because I am, uh, not as deep tactically as, as many of our founders and many, many people's space by, I think that the easy way to sort of think of, uh, J of AI is it's got three primary components. It's got a UIUX component, which varies on function. So maybe a chat, just a simple chat is the best, uh, for code, the end of day developers live in an ide that's the indivi, uh, individual development environment.

That's what you need. That's where they live, that's where you support them. And that takes a lot of work to get them the right com com or the right suggestion, the right time, the right length, all that kind of stuff.

So it's kinda one piece. Then there's the code piece, and then there's the model part. And, you know, um, all these need to run somewhere.

And so from an efficiency and quality standpoint, we run our standard SaaS on G ccp. Uh, and that's because, uh, we rely on Kubernetes in a really good network to deliver that out to end developers. Both of which, you know, I did come from Google, but this was decided before I came to tab nine.

Mm-hmm. Great. Kubernetes, great infrastructure, very cost effective, and a great network to deliver that last mile to your, our developers, our million plus developers all over the world.

So that was kind of where it started. And then, uh, back to sort of, you actually already hit on it, the questions of like, Hey, fortuitously, we ended up learning how to build and, and develop our own models. Um, there's no question in the space that Google has both the talent, the data, the models, they have a lot of pieces there that are valuable.

And so, you know, we're not, uh, dogmatic about solving this. Our, our vision and our goal continues to be to double developers productivity. And I think we're early on that version.

And part of that is Google likely has a lot of good resources. And so, you know, they've got some APIs could be of interest. We're supporting them as they're launching those and we're using them in, uh, not in production, but we're testing them as they launch them and mm-hmm.

That could end up being something really interesting. Um, so, uh, they've got, think of them as like a amazing piece parts to what we do. Uh, some around the model and some around the data.

Um, not really u UIUX that hasn't been historically in the dev space that's with, except Android studio's amazing. But really, if we look at, uh, that's kind of in Microsoft's space, uh, and a lot of independent vendors and kind of Microsoft Yeah. The Japaneses and Visual Studios of the world.

Exactly. Yeah. Yeah.

And then they've got GitHub and they've got, you know, uh, which also was a, an acquisition for both data and ui ux. I'm not really a surprise, they kind of feed this. Yeah.

Uh, so yeah, so continue to work closely with Google and, um, will in the future. Uh, our intention is to, if there's something that they deliver, that's great, we'll get it to our customers and, and we're making the bet they probably will if there's someone in the market that's gonna have game-changing. Uh, we also share, uh, uh, an ethical view of AI that I think aligns best with Google, So, mm-hmm.

Interesting. Yeah, there's so many open questions. We could spend a whole in two hours on those things.

Sure. You know, that everybody starts asking about, oh, what about this code? Is it copywriter?

Can I use this? I know who owns it. La la la.

Sure. We'll talk about that another time. That's, you know, about 10 rabbit trails we will run down.

Um, I'm, so you do a script subscription model, right? And then it's a IDE plugin, is that correct? Mm-hmm.

Yes. So we've got a, uh, really kind of two options. We got a pro version and an enterprise version and a free version.

So the free version, uh, we way back some historic, when we first started with G P T, uh, two, way back when, it was a lot, quite a bit smaller. We spent a bunch of time figuring out how to run it on your CPUs so you can actually run it. The original version of tab nine ran completely locally on your laptop.

Oh, that's interesting. Uh, and still does, so still does today. So there's a free version there.

We have committed to that we'll always offer. We have a pro version that gives a ton of all this functionality, but also leverages at this point longer. Comp compute con com, um, uh, completions and stuff require GPUs in a cloud.

You just have to hammer through 'em at bigger models. And so we host those and those will mix. So when a pro user has, uh, all this securely running on their laptop, plus some really great suggestions that'll come from the cloud.

And the enterprise version is designed for enterprises that either need to build custom models or they wanna run, uh, tab nine in their own VPC OnPrem in their data center. Mm-hmm. But where their code is, you know, they're uncompromising on the security and, uh, performance of their code.

Uh, that's a place that tab nine addresses. So a little bit, we kind of hit the whole spectrum, I'd say users. Well that was actually where I was gonna go because you know, we all know enterprises, you know, get into government.

I don't know if you're there, but they have different requirements and mm-hmm. I was thinking you've, you've probably have already hit organizations, particularly larger ones that have their own, you know, uh, code or things that they wanna put in the model that are unique to them, that are making their developers, maybe it's even, you know, an IP advantage, right. They can build in and that way no, there are no other developers have access to it, you know, half the time it's not knowing we have that.

Right. We've got great stuff and no one knows about it. Yes.

Yeah. We have a library to do that. We already have, you know, microservice running.

Okay, great. Glad I know. Okay.

And co completed. Very nice. Um, great.

It's, this is, this is fantastic. I'd love to have you back and we have a, a lot of discussion around this almost every virtual conference that we do. Um, and we have a, a cloud native now coming up in, in July, um, that we'll be doing some conversation about this and the fact that you guys have been doing it for a while, um, and, and you don't have the kind of Google Nest or, or uh, um, GitHub ness or Microsoft ness of how you need to kind of control the message or whatever.

And I, I don't, I'm not dissing them, I just, I know they have a certain way they have to do things. Um, I think we're trying to have some free-spirited discussions about this, cuz these are a lot of questions people have. They just wanna know mm-hmm.

Am I setting my developers down the right path? Maybe they're already been going down there, you know, they've got their own subscription and it's cool. But I think just, uh, exposing some of those questions and having that dialogue helps accelerate the adoption and, you know, clarity around that.

So we'd love to have you back and Hey, would, would appreciate it. I think this is, uh, this is a space that we, we have the, the luxury on our side just of having had thousands of conversations with companies that have asked the questions. So you kind of are used to here's what, here's the pieces they need to address.

You know, my CISO says I need to x, Y, and Z and here's y uh, but look, as an industry, this is a question we're gonna wrestle through in a very deep way over the next six months. Six to who knows how long. I think this is gonna be a big pivot point.

Mm-hmm. Uh, and I've never seen anything take off the speed. And I say that sitting on the Port of Linns Foundation and launching Kubernetes mm-hmm.

This is two, three times faster. It's amazing. Yeah.

Uh, and that's, that's amazing. So I think, uh, I, I'm always a, uh, an optimist, but with eyes wide open and, and the more discussions we have, the more, uh, we wrestle through to a good answer. And that good answer is, uh, all I can assure you is the right answer is not what we know today.

Yes. And you don't want, you know, I I, I'm not against any, I I'm not for any of the bands. That's, so those are all overreactions.

Let's, let's talk about it, work it through and you know, if we banned every technology, we'd still be handy, uh, hanging, you know, magnetic tape and paper and printers and big data centers. Yep. That's so real tape.

So good talking with you Brandon. Uh, appreciate it much. com.

Like you said, there's a free version. Um, I haven't checked it out yet, but I'm definitely going to, cuz I've been working with these quite a bit. And, uh, look forward to hearing more from you and good things that are happening there.

Mitch, thanks so much for having us. You bet. Talk to you soon.

Cheers.