OverflowAI – Prashanth Chandrasekar, Stack Overflow

This is Textron tv. Hey everyone, welcome back here to Textron tv. Really happy to have our next guest.

He's, he's been on with us before, but it's always a pleasure to have him here. Let's welcome Prashant Chandra Sekar, and I hope I got it right, Prashant. Excellent.

Welcome. Prashant, if you don't know, is the, uh, c e o Chief Executive Officer over at Stack Overflow, one of the real go-to sites in our world of, of tech people where we go to get help, to learn to help others. Community's great site.

Great, great resource. Prashant, welcome. It's great to have you back on here.

How are you? I'm well, thank you, Alan. Uh, wonderful to be back and to see you again.

It's been, uh, a few months since we last spoke, and I was sort of teasing our announcement last time. Uh, but obviously our team's been hard work and very, very happy to, to share the great news here with you. Absolutely.

Well, let's not waste time. I don't, I don't think Stack Overflow needs an introduction to our audience. I think most of them are, have accounts there or using it already.

Um, what's news? Yeah. So, uh, just as a, as a frame, uh, I would say that, you know, just so everybody understands, stack is made up of two things.

One is our public platform, which has, you know, close to 58 million questions and answers in all things technology topics. Uh, and we have stack of, for Teams, that's our SaaS product that's used by over 15,000 organizations, uh, and enterprises across the world. Uh, and that's a private stack workflow for people to leverage inside the organizations for knowledge sharing.

So that's just as a level set. What we announced, uh, uh, this past week at the VR Developers Conference, uh, was our launch of, uh, overflow ai, which is our generative AI product capabilities that are now sort of now gonna be present on the public platform, as well as Stack overflow for teams. And this is the realization of the vision that, uh, we had laid out, uh, earlier in the year where we said, in this AI world, uh, what we really believe, uh, our, our sort of, uh, role is, is this confluence of community and ai and the power of those two things.

Uh, AI obviously brings all these great things like summarizations and Semantic Search and all the other things that we've been reading about. Um, and community brings some amazing attributes like, uh, creation of new knowledge, validation of new knowledge, connecting the dots to, you know, make infra, you know, to really sort of, uh, make judgment calls, et cetera. And all of that, uh, when you bring that together, produces what we believe is a tremendous option for Stack Overflow, uh, to really bring trusted, attributed, highly accurate, high quality feedback based and recognition based answers in a generative AI context, uh, to our users and customers.

Uh, and that's what overflow AI is all about. Uh, we announced five different things. I'm happy to get into each one of those five things underneath overflow ai, but that's the, the high level message.

Excellent. Uh, we, we were talking off camera, you know, similarly here at Textron, we've launched Textron ai, which just covers the AI space 'cause that's what we are, but there's a lot to be done, but there's a lot of ways to skin this, Kat. Right?

And, and I want to jump into that with you, which will probably be in the five ways you're doing it. But initially I wanted to ask you, is overflow AI sort of a separate offering within the public stack overflow, or is it just capability that's that's kind of, you know, joined into the, to the base, if you will? Mm mm Excellent question.

So in some cases, it is, uh, an extent. It's all gonna be, it's all about serving our developers and customers. So for the public platform in it is very much an extension of the current experience, but it's opening up the experience to a lot more users because, uh, and I'd go through each one of them, uh, but the, it's an extension of the current experience, right on the stack hold of a team side.

It's not only an extension of the current experience, but it's also adding new capabilities that we've never had, uh, on stack hold. And we are using this moment to sort of leapfrog into this new world, uh, for even to really accelerate a lot of things that we can do, uh, to drive value for, for, for our users and customers. So it's sort of a combination, Alan, just at a high level of both an extension of the ex existing experience, but leveraging generat generative AI to solve even, even even some of the friction points historically and trade offs you've had to make to have a high quality knowledge base, but also, uh, giving people now new ways in which they can get the right answer at the right time, powered by generative ai, uh, all in their flow of work and in places that we've never actually been before.

So this is, those are the new capabilities, which I can walk through again. Absolutely. I've got another just is not issue, but question.

So a lot of companies are just, you know, large language module emphasis on the large, right? They're dumping everything in everything into just a vector database. And, and so you've got this huge capital L L M and, and, you know, and the beautiful thing about this technology is if you have the horsepower, it, it, it will summarize, it'll do its thing, it, it'll pull and it'll get better.

You know, theoretically, the more information it has, the better. However, in talking to some AI experts, they're saying for these kinds of uses, kind of like what you're outlying, you might be better off with a smaller L l m, right? That's more targeted.

So if I wanna talk just about out, I don't know, automated testing, I don't need all of this non-automated testing, uh, data in my L L M I that just gives me hallucinations and poisons it, right? I, I just want real relevant data. And I think, you know, that's gonna be something we learn as we work more, not you or me, but as a world, as we work more in this generative AI space that sometimes less may be more if it's, if it's, if we're looking to fine, you know, fine tune rather than shotgun.

I wonder if that's something you guys have looked at? Yeah, absolutely. Uh, we, we've got, uh, uh, back in April, we carved out about 10% of the company just to focus on all things gender ai as we, uh, leading up to our launch this past week with overflow ai.

And as part of that work, we actually, uh, looked at our own data. You know, we have a tremendous wealth of information from 15 years of being, uh, of doing what we've done at Stack Overflow, serving the community, uh, and with, obviously with all of all work we've done with customers. And so what we did was, in that we've run several parallel experiments and, you know, to get our alpha, uh, launch out for overflow ai, we picked one of those.

5 turbo and, and G B T four, et cetera. Uh, we looked at an open source model and layered in our own data on that. And we've also, we are in the process of working with the other, uh, L L m uh, providers as well, like all the other ones that you can think of with the other, the hyperscalers and others.

Um, and so we are running parallel experiments. And what I would, what, to your point, the second example where we took our own data layered in on top of open source l l m models, was in a very lightweight way, uh, give us, gave us some very, really, really remarkable results. I think we, we were, we were quite surprised how good the, uh, the kind of the fidelity of the information, the quality of the information was, especially in this domain that we're operating in very specific domain, uh, and how cost efficient that can be, right?

So we are still, we have to, there's still some work to do to sort of determine how best to sort of, uh, choose between these various options. 'cause there are multiple, uh, trade offs there. Uh, uh, and so that's what we are in the process of doing for the Alphas.

We actually chose one of those, uh, more the commercial models, and we basically have launched that, uh, to get us going. But it is, you're right in that these models over time, uh, they're likely to be much more domain specific and hence, you know, more economical, uh, to run because you don't need the world's data on everything to sort of do something very specific in a domain. So, absolutely.

Yeah. Yeah. I, I think we'll all be learning as we go forward.

I will further refine these Prashant, let, let's dig into these five different kind of announcements, though, that came out of your conference, if you don't mind. Yes, please. Yeah.

Thank you, uh, for the opportunity. So rooted in our, these five announcements is really one statistic I wanna just sort of share, which is, uh, the fact that when we surveyed our developer community, which we do every year, this past year, and our 2023 developer survey, 90,000 people responded, and over 70% of them said they were using AI tools and or wa or wanted to use AI tools for software development, but only 40% of them trusted what was coming outta these AI tools. So that key statistic just totally got embedded in our thinking as we thought about at this point in time.

And it doesn't mean that's always gonna be the case, but at this point in time, the problem to solve is to make sure that our developer community has access to highly trusted, attributed high quality, accurate knowledge, and they're getting credit, by the way, for the work that they're doing, by contributing to the community, and there's feedback loops, et cetera. So that has been the, the focus of these five things I'll make, I'll announce here. So trust is an attribution.

So going into the, uh, into, into each one, the first one around Alpha AI is really, really powerful. Semantic search on our public platform of 58 million questions and answers. We have tens of millions of developers coming onto our public platform every week, uh, and for them having a really, uh, fantastic ability to write, to ask natural language questions on, on Stack Overflow public, getting a generative AI summarized answer.

But within the summary, it's gonna have embedded links and attribution to the quash questions and answers from Stack Overflow public platform from the 58 million questions and answers. So, you know that this answer that's coming through is a general answer in the moment where you don't have to have a high bar of asking it the right type of question, et cetera, which has always been one of the hurdles on Stack Overflow, because you, the bar is pretty high. Now you're asking a national language question, and it's gonna provide you with this answer even before you tap into the community.

And why that's so powerful is that you can refine your question by providing additional context. You can chat with overflow AI on the public platform. You can include your piece of code.

So it's taking all that context and providing more and more richer answers as part of that. So that's the search experience, uh, and really, really powerful search and, and and chatting experience that we have done. That's number one.

Uh, which we believe the, the impact of that is to really be a lot more welcoming, uh, to the millions of more developers who are going to enter this space. The youngsters who are trying to get into software development, who really wanna just learn very, very rapidly. And the bar for learning is quite high on stack because the quality expectation's very high.

So even asking a question can be quite intimidating sometimes historically. So we are solving that. So that's, that's number one.

Number two is we are in the context of Stack Overflow for teams and Public Stack Overflow. We are announcing for the first time ever, uh, the ability to have, uh, stack overflow content in your I D E or integrated development environment. So specifically Visual Studio Code.

So while you are writing code or while you are generating code, uh, you can, for example, ask again, overflow AI in your I D E, how do I do this in Kubernetes or whatever else. And it's gonna provide a generative AI answer with, again, links and attribution back to Stack Overflow content, public Stack overflow or Stack Overflow for teams. It, you can also highlight a piece of code, and you can ask for ex for an explanation.

And that explanation again, is gonna come from content from Stack Overflow, from a 58 million questions and answers and from Stack Overflow teams within that company, as an example. So that is the second announcement. We are bringing Stack Overflow into the I D E.

Uh, number three, uh, is the same generative AI experience that I just described in the I d E. The other place that people spend a lot of time is in their Slack or, or their chat ops, uh, uh, you know, windows. So we are now bringing overflow AI into the, into Slack where you can, again, have a conversational interface or conversa, uh, natural language conversation, asking very technical questions.

And the answers again, are gonna come back in agenda ai, uh, summarized answer, but it's gonna come from Public Stack overflow and the internet, and also stack overflow for teams within your own company. Quick question on that. Is that just a Teams feature or anybody could do that?

So we are launching both the, so I've gone up to three. We are launching the I D E and the Slack, uh, slack capability in Alpha for our team's users first. Okay?

Uh, but it doesn't mean that we are, we want to get some early testing and get some customer feedback and user feedback. And then we were, we're always open to obviously expanding. The goal is to provide this to as broadly as possible, because I, I was ready to sign up today, but okay, I'll wait for you to tell me.

It's okay. You can use, you can use it even now if you have it. You know, you can sign up for a teams account and basically use it.

That's the idea, right? So, so that's number three. Uh, Alan, which is the Slack, uh, integration number four and number five are a lot more deeper enterprise use cases.

So number four is specifically the ability to have really ingest existing content from within your organization into Stack Overflow for teams, uh, by using generative ai. So it's pointing to, say, 50 documents of 500 documents in, let's say Confluence or Google Drive or SharePoint or anything else. And basically it is gonna convert them into q and a automatically generate tags, automatically generate subject matter experts and give the admin immediate ability to ingest all of this into stack of teams to prevent any sort of cold start, uh, sort of, uh, elements when you start out with new software, like stack of teams within your company, and you bootstrapped effectively a knowledge base and, and a community within your company using this technology.

Uh, so we're really, really excited about this, especially for our enterprise customers that have been asking for this capability upfront as well as ongoing in their experience. So that's number four. And number five is similar to the first item that I mentioned, which is search on the public platform.

We have now really enhanced our search capability in within Stack Overflow teams to include enterprise features. What does that mean? It means the same.

You can ask a national language question, the answer's gonna be a generative AI answer with links to public and private stack overflow content. Again, from the tens of millions of, uh, content, we have pieces of content as well as links to other document repositories within your company. So it may return a, uh, a, a, a link to a Confluence document.

It may return a link to a GitHub read me file. It could. And so all so effectively are bringing, uh, all relevant information together, uh, by, by surfacing it at the right place at the right time through the Stack Overflow workflow, obviously.

Uh, but we believe search has been broken within companies for many, many years. And so we wanted to solve this problem as we've been hearing that from customers, uh, about this. So that is the fifth item enhanced, uh, search within organizations with stack of teams.

So to reiterate, search on the public platform, uh, number two i d e extension on Visual Studio Code number three, a Slack integration, number four, uh, content ingestion, uh, into stack over for teams. And number five, enhanced, uh, enterprise search capabilities for Stack over for teams. So that is what overflow AI is all about.

All of that is powered by generative ai. Uh, and, you know, there's a, there's a three minute clip that we posted that shows all these features in action. So we'd love to set up for your audience to take a look at that and, and also sign up for our Alpha program, uh, which is now kicking off here in August.

co/labs. Well, that was my next question. 'cause we gave people a lot to ingest, no pun intended.

Um, a lot to ingest here today. So if they want to go review this and dig in a little deeper, let, let's give it to 'em again. Stack Overflow, do co not C com slash laps A Bs.

Go check it out. There's a lot in there. There's a lot coming.

Last question. I'll let you go, Prashant, when do you anticipate moving from Alpha to better to Generalise with this? Yeah, our goal is, uh, to do that over the next several months.

Uh, you know, we, we, we generally believe by the, uh, no, no pun intended, again, that it'll be by the end of the calendar year, uh, is sort of the, the goal. Um, and so we wanna make sure that we are receptive to and, and responsive to user and customer feedback over the next couple months that'll determine, uh, perhaps which ones we get out the gates first. Uh, but it'll, given that we've announced five things, uh, we, we wanna sort really, and we, we are seeing immediate some early data to suggest, you know, for example, the I D e, uh, extremely well, uh, a lot of people interested in that one, along with the Slack integration, along with the, you know, our enterprises asking for the ingestion and search capabilities.

Uh, but, uh, in some cases things that are available on the, on the private stack flow teams capability, people want it on public platforms. So we're seeing all this feedback and that might determine, you know, where we go, uh, faster, uh, over the next couple months. Priorities.

Fantastic. Hey, man, I love it. Uh, as I told you off camera, watching what you guys did gave me confidence in what we're doing because, uh, you know, I, I I think we're thinking along similar lines.

Yeah. So congratulations. Amen.

And, um, keep us posted. You know, you could come here anytime and talk with us. We always love to have you on Prashant.

We appreciate it. Thank you, Alan. I appreciate the time.

Thank you. Prashant Chandra, seeker Ccar, C e o of, uh, stack Overflow here on Tech Drunk tv. We'll take a break.

We'll be right back.