Generative AI Platforms in App Development – Kin Lane, Postman

This is texturong TV. Hey guys. We're here with Ken Lane as Chief evangelist for Postman.

And we're talking about chat GPT and coding and application development in the future as we understand. It's evolving as we speak Ken. Welcome the show.

Thanks for having me. So it looks like these platforms can maybe write code or at least copy it more efficiently and perhaps our overall productivity can go way up. So what could possibly go wrong with all this kid?

Yeah, I'm there's a lot of lot of potential a lot of optimism out there right now and I think a lot of really interesting signs coming out of experiments and and folks playing with I know my team super excited in doing things but I think as you're alluding to the devils in the details there a little bit I think the the lack of provenance of where a lot of these models were trained on and and kind of you know that black box nature that Ai and ml can often be I think it's gonna continue to to nip at us here. I've seen some interesting experiments and discovering apis and then generating documentation and Collections and Integrations on the Fly and and even doing visualizations all just asking chat GPT. And then once you scratch at it, well those apis are deprecated and they're no longer in production so you can actually connect with it and other things that just aren't true.

Fires what it's needing to actually connect the dots but there's a lot of time saving potential in there. I think if we can really be honest about the provenance of what these models are lacking be honest about what their their strengths and weaknesses are I think we can make our way forward and definitely save ourselves some time and do some interesting things. It seems like a lot of the issues have to do with the fact that the model that was trained up until a certain point in time and apis may have been updated or deprecated or have simply gone away.

And so that's part of the issue. But also do we need to worry about the model is just kind of relaxing a better term hoovering everything. It finds and doesn't really distinguish between that which is maybe subpar and bad versus that which is good and exceptional and so we wind up with this Michigan stuff that we can't really trust and I'm not quite clear how that issue gets resolved.

Yeah, I mean garbage in garbage out bias in bias out, you know, it's being trained on the real world and we all know that the real world's pretty messy and I think we're looking for Tech to solve our problems and solve a lot of this mess and we perpetually don't want to do the work that's needed actually to fix those inputs those those illnesses and I just don't think there's ever gonna be it's we're humans and it's Earth and we're we're messy creatures and things are always gonna be complicated. So while we're able to generate models and augment ourselves, we got to do those incrementally we got to do with a provenance attracts where they were trained on each iteration for me. They're much like apis apis or something that's living you iterate on them you gather feedback from your consumers and you you iterate in the right direction.

Ml models are going to be no different. Gonna you're gonna need to iterate on them test them understand where they fail where they they can be better incrementally improve that and and move forward if we don't have visibility into that that's where the biggest problems are gonna come because people are over promising and then under delivering when it comes to this kind of value. Do we have some legal concerns here because a lot of code is subject to copyright and if we don't know where it came from, we don't know who owns it.

So is somebody gonna send you an email one day and say hey, I think you're using my code and here's my bill. Yeah, and and again, there's a lot of misconceptions about you know, what gets copyrighted what what licenses apply to code what applies to the interface of apis I worked on the Oracle V Google copy right case so I am very familiar with these issues. And yeah, we're starting to see those emerge when it comes to chat GPT and others open AI being sued for by Shutterstock about images and and derivatives of those images.

And then you see the initial challenges around GitHub co-pilot and and what they've done as far as as you said hoovering up just everything out there on GitHub and assuming that that's good that's high quality and the right way of doing it and intellectually you have access to and ownership over that property to be able to then reapply it as part of your ml models. Yeah. It's gonna be messy.

Is there any place right now that you think the developers should feel comfortable using these platforms? I mean am I using them for research or you know, they do provide some valuable? Where's that line?

Yeah research is I would say squarely where you want to do this prototyping research understanding the more control you have over. Training it on your data and your artifacts and having a saying that so, you know chat GPT and open API just one of many companies or open source Solutions out there API Solutions so shop around and look at ones that give you more control over how you train the models and and iterate on them and be able to see what they do and and that's gonna help I think. Give you more control give you more say and more understanding and it's not just this wild harvesting of everything on the net and the days of Watson level claims that there's this big brain and here's what Ai and it knows everything.

We should leave that in the past. Today's modern. Ml is very modular.

Very domain specific. Very incremental and isn't doesn't know everything it can know very precise things and augment humans and very interesting precise ways, but there's no single bullet or whatever. The analogy would be to to solve all of our problems.

Do you think as we get more processing power that we'll be able to train models using less data, and then that becomes a more interesting conversation because I can narrow the amount of code that I'm using. the Train the platform in a way that makes the whole thing still work, but I'm not necessarily creating all that level of noise around bad code because I'm kind of being very judicious about what I actually exposed the model to Yeah, I think that that that's back to that modularity in this in the in the Precision of models is narrow how it's trained on the data is trained on and then iterate on that and then the data generated from usage of the model is fed back into the model and you increment the models based upon how it's used. Not just so maybe you seed it on a certain data set a certain number of eight open apis or artifacts that describe apis and then use it to design develop do different things with apis.

And based upon that feedback loop and I've on my podcast breaking changes. I had interviews with like Ford and the way they're training models on delivery drivers is they'll they'll have a model about routing and and how packages are delivered and The driver of the vehicles involved in that feedback loop with that model. They know they're part of a training process.

They're more they're part of that feedback loop. And so they drive in a certain way to enhance how that model learns and the data that gets gathered then fed back in and created the next version of it. So, yeah, I think that's gonna be really key and how we we build meaningful modular usable models that are gonna reflect what we need in business.

As we go along do you think that we'll see an increase in developer productivity as these models come along? I mean ultimately what will be the benefit and you know why you're thinking about that what might be the impact on our devops workflows. Yeah, I mean I think initially here we're gonna we're gonna see decrease in productivity because we're getting distracted with all the different ways.

And I know my team is done a few projects that didn't and with any outcomes any stories any deliverables, but I still encourage that because I want them to understand it play with it, but for business value didn't quite get where we're going. Now. There are a couple that are really have massive business potential when it comes to like IDE like code completion when you're designing apis or securing your API.

So we have some really interesting ones that we've we've done so there's some value there but that balance I think is gonna balance out between successful and unsuccessful projects, but I think moving forward, you know, it's like any other period in time. I mean, I've I read a book one time on the the the mule carriers who used to move the barges up and down the Columbia river outside of Oregon Portland being upset when the the barges were were mechanized with steam power and their jobs went away and I think every wave of tech every way there's jobs that are displaced and then there's new jobs that are created and existing work that's augmented with these new technologies. So I think it's gonna take us a little bit longer to get there.

But I think it's gonna do away with some devops job some automation, but we're gonna be able to you know with a more platform. Feel a more platform Ops view. I think we're going to be able to augment some things that devops folks were doing kind of manually and stitching together based upon existing traffic and models derived from that and we're gonna be able to secure those apis make those apis more discoverable and less devops work needed.

But then there's gonna be new work created on how to how to do things at the platform level that that those models can't do. So, it's just gonna free us up. Give us some some new work ahead.

So Conversely. Well, what else may those devops processes more accessible to a wider range of companies? Because the today one of the big hurdles is the amount of engineering expertise you have to have on especially alone a lot of manual tasks.

Yeah. Yeah, I mean it's it'll be synonymous with low code kind of no code efforts, you know, as far as coding and building applications websites mobile apps, but for business automation platform automation, I'm seeing more product managers emerged as part of the API lifecycle and and working alongside API developers devops other release type people and so giving product managers the controls they need to make their apis more reliable and and higher quality governance that are back to buy little micro machine learning models that learn from traffic learn from the the history of how things work and then give knobs and dials and controls for those business stakeholders to to adjust. No, you know, no devops coordination needed.

I think there's a lot of potential there. Do we also need to worry about what the bad guys are up to because might they not find a way to write malware more efficiently or find those apis that are more easily exploitable. Yeah, it's where when you equip developers you're clipping both sides.

That's just how it works. And it's another reason not to keep them black box back to the what I said about provenance and you know, a lot of the AI That's out there when you read about openai and and these other rock stars. It's not always the the provenance of where these models came from and which university research program.

They came out of and which approaches. So there's a lot of history there that gets lost in the marketing Shuffle of venture back startups. That's part of the provenance.

We need to be more clear about and and kind of in that dust you can hide a lot of other incompetencies and other things and that's where the opportunity for bad guys exist is in that obfuscation black box-ness. They understand they know these models came from here to certain type of ml. It's a certain tensorflow approach and they'll be able to reverse engineer and understand how the models work almost even better than some people implementing them and that's a problem.

So I think we just need a lot more education literacy and transparency when it comes to you know, what these models do and what they don't do. So what's your recommendation? Do I just turn everybody loose to experiment or should I have a more control approach to how these experimentations occur?

Well, I mean, it's it's kind of like what I see with apis is we're coming out of the Wild West period the last 20 years has been very wild west with apis like just create whatever you can move forward fast create the Twitter's world the stripes the twilios, but that's stabilizing and and we're getting more governance. We're getting more standardization regulations Healthcare Finance when it comes to it. So that's what's gonna happen with with this machine learning wave and every future wave is yeah, there's some experimentation.

There's a lot of fun. There's a lot of cool things built that don't need structure, but if you're Putting it in front of a bunch of customers and we've seen this in like healthcare spaces. If you're actually going to be providing therapy for people using an ml model or chatbot.

Probably structured should probably have some scaffolding should probably have some oversight should probably have some architectural decision records kept and some provenance on how and what's going on. What's working? What's not and then some some oversight from external forces or Powers so that You're not doing some shady things in there or some unethical things.

And so that sort of transparency Scaffolding in framework. I wouldn't say you need them for all projects and all models being trained. But once you start widening who who's impacting if it's deciding my credit score if it's deciding whether I might end up back in jail or not.

If it's doing facial recognition on me at the airport, there should probably be some some oversight and some some sort of framework for how we move those things forward. All right, folks. I heard it here generative AI is cool.

But proceed with caution. Hey, Ken. Thanks being in the show.

Always, thank you. All right back to you guys in the studio.