How LLMs Are Transforming Business Processes with Arthur O’Connor

Hello, and welcome to the latest edition of the Techstrong AI video series. I'm your host, Mike Vizard. Today we're with Arthur O’Connor, who, academic director for data Science at the City University of New York, from the professional studies organization as I understand it.

And we are gonna be talking about, well, what's going on with all these AI models in particular, deep seek, which seems to have set everybody back, but no one's quite sure what's real and not real here. But Arthur, welcome the show. Thank you.

Thank you for having me. As I understand it, deep seek, at least the folks who, uh, are behind the model claim that they found a less expensive way to train that model. And, um, but also folks are saying that some of the guardrails were bypassed and some of the outputs are a little more, um, shaky than others.

So what's your assessment of what's really going on here? Well, the, I think the development is a kind of a useful reminder of, uh, to all of us that, that, you know, artificial intelligence, particularly generative artificial intelligence, is not just about the number of parameters the size of the training data set, and the, you know, how many GPUs, uh, are you using in a enormous server farm that's consuming all kinds of energy. It's really about how smart you are and how creative you are in designing, uh, the data.

And certainly a lot of the, what is called these distilled models that, uh, R one represents is really about not just the size of the training data set, but the quality and the relevance of the data set instead of scraping the whole internet. Yes, it's useful in learning the constructs of diction and language and how to form human-like sentences, but it's not particularly, the web is not particularly, uh, good at explaining, uh, complex logic or solving math equations. Um, and that's where if you start to focus on certain data sets or very high quality to use those design your parameters and your test timing through what's something called interference training, uh, in addition to the initial supervised fine training and, uh, reinforcement learning, that that really can improve the quality of the output.

So do you think for most organizations, they're gonna wind up focused more on these distilled models that are narrowly aimed at a particular use case, rather than everybody trying to make use of the largest language models in the world because, well, that's just a more expensive approach every time you invoke one of those directly, It's more expensive for the people developing the models. Uh, the, you know, the, the, there are literally hundreds and thousands of open source variants out there, which you can find on me, meite, such as HuggingFace, the real challenge for most organizations just to find out what's available, how they work, and most importantly, how they can be safely and effectively applied in their business process. And that remains a major challenge because, um, you know, unlike, for example, data science, which is almost universally, uh, can be applied to just about any kind of field.

So far, the current generation of, uh, generative AI tools have really been, you know, they perform some neat tricks and have been really good at certain things like visual creation or editing or writing code snippets, but it's been fairly limited to a, you know, maybe a dozen or so use cases or business models, what have you. So it's really gonna take, uh, organizations some, uh, getting up to speed on kinda what's out there, what did they do, how much they cost, and how do you mitigate the risks of using these tools. Things too that everybody's obsessing about is the GPUs that were used to train this model are maybe not the most costly GPUs in the world.

And we're also starting to see people talk about things other than GPUs for both training and inference. So do we need to be smarter about what classes of processors we're using to train various types of models? The success and the performance of R one, uh, that's the deep seek, uh, uh, model certainly confirms that, that it's, you know, not just about size, it's also about, you know, quality and creativity, how you use those things and how you architect that solution.

How much expertise do we have about those types of, um, more advanced uses? I, I would say, but it seems to me a lot of the data science teams that I talk to, you know, they kind of just run right to the most expensive GPUs and processors they can find. And they have maybe a particular model, and they're not really thinking through the implications of running this thing in a production environment.

So who's gonna be smart enough in these organizations to kind of sort all this out? Well, Michael, you've hit upon one of the more interesting structural and cultural implications of the whole solution. And that is that in most organizations, uh, are basically structured around the previous digital revolution.

And so you have data science expertise sequestered in these, uh, centralized in these IT organizations that kind of have their own mysterious language and processes that are removed from the main line, uh, and the main business units. Um, and this generative AI revolution's really gonna require data science expertise to be mopped far more diffused to not only maximize the benefit of using these tools, but also minimize the rather significant risks, uh, of, of the, of such tools. And you are correct in the Silicon Valley high tech mindset that, you know, bigger and better and faster and more powerful is, you know, necessarily what you're going for.

And in many cases, uh, it's not, there's lots of, uh, business people that, uh, are, uh, IT people who are overwhelmed by the business unit saying, I want a large language model to do this, to do that. And it turns out that the business case would be far easier, better, and more cheaply solved by, you know, for example, a collaborative fil filtering model for personalization. But everyone's lapped up in this gen AI large language model, uh, that those important distinctions and that important knowledge is not sufficiently diffused in the organization.

We need to rethink how those IT teams are organized then. 'cause historically we have yes, um, infrastructure and software developers, and now I'm throwing in a bunch of data scientists and there's security people and it takes a village to do anything. So, um, how should the village be organized?

Well, if it, it has to be, uh, sort of centralized and decentralized. So, uh, the expertise and the knowledge of understanding what data science techniques or what generative AI techniques works best for what use cases and at what cost and at what, you know, ROI, that needs to be more, that has to be ha decentralized. Uh, but what needs to be centralized is this policies, procedures, and the architectural standards.

Uh, because as, as mentioned before, you know, a lot can go wrong. Uh, you can some well-meaning individual trying to, you know, fine tune a model can wind up, uh, training the model, uh, on proprietary information that now is part of the model and part of the, you know, public, uh, knowledge base. So there's some important guardrails to be put in.

Uh, but as you probably know, most organizations still don't have policies and procedures. It's, it's, as I note in my book, it's, it's kind of wally world. And in that regard also, I don't think people really understand the degree to which maybe these models might drift over time and that which seems to be working perfectly fine, suddenly six months later is not.

And do we have a process for kind of observing that, monitoring that, and updating and replacing models when necessary? The short answer is no. Uh, on one hand, I think most users don't realize how powerful these models are.

They still use them for, you know, summarization or text generation. They don't realize that you can assign a role and you can ask it to figure out fairly complicated things with a fairly high degree of accuracy. But because they're not, you know, super users, they don't quite appreciate that.

And at the same time, you have this sort of naive a or ignorance on just what you should put in a model and what you really shouldn't. Or if you're gonna put in proprietary information, how do you use it in, for example, there's a concept called in context learning where you, uh, sequester that private information from the public domain. Well, maybe you have some insights, 'cause we've been talking about this in other interviews, but it seems like people are struggling a little bit with, um, a lot of the business processes that they want to use AI in are essentially deterministic.

They need to be done the same way every time. And gen AI does things differently almost every time. So that's a probabilistic outcome.

How do I insert something that is probabilistic into a business process that is deterministic? We're still trying to figure that out. Uh, all we know right now is that we have organizational structures and performance indicators for employees that are based on things like expertise and credentials in seniority.

Whereas this revolution is all about the democratization of expertise so that, you know, the junior, uh, uh, user or employee can potentially have the same amount of expertise, subject matter expertise as the senior, uh, person. And so this is really gonna upend the whole performance hierarchy in organizations at least has the potential to. And so we don't quite have the tools or structure in place to, to handle that.

And, uh, I would point out a recent, um, art study issued this month actually in February, 2025 by, um, Microsoft and Carnegie Mellon. And it did this study of these three hundreds or so knowledge workers and the results find that it, IM, you know, the ability to, they call it AI whisper, the ability to know how to quite prompt or interrogate these models is actually becoming just as important as actual subject matter expertise. And it also finding that, that the use of these models actually decrease the employee, uh, critical thinking skills.

Uh, so because of this phenomenon called cognitive offloading, meaning it's the risk of using GGI as is the calculator has done to our arithmetic skills as the smartphone has done to our memory of the phone numbers of our loved ones or what GPS tracking is doing to our sense of direction. I'm not entirely sure whether that's a good thing or a bad thing. 'cause I could probably interview.

Yeah. Um, so how do I kinda navigate this all with some reasonable expectations? 'cause you see, there seems to be a disconnect.

Every CEOI talk to is like, AI is going to be awesome, change the world, and we're gonna be more profitable than ever. And then when I get into the middle managers, they're kinda like, well, maybe, yeah, but it's sure not easy to operationalize this thing. Yeah, I I would not want to be a CEO or, or a CTO, uh, of a large organization right now because, uh, they're, they're overwhelmed.

They're, they're being, uh, you know, they're getting calls from the board of directors saying, well, why aren't we doing ai? And, um, from people whose, you know, enthusiasm, um, and interest in, in the business model is, is certainly commendable, but whose knowledge of data science may not be quite up to snuff. And so they're put in the un envious position of explaining all this stuff.

Uh, and we don't have an infrastructure, we don't have an organizational structure, and we don't have a performance employee performance metric, uh, to measure creativity and adaptability. Uh, and so we're just sort of foundering right now. So what's your best advice to folks then about how to do this?

Should I take everybody and put 'em on some sort of corporate retreat and say, this is what's real and not real? Or are we just gonna stumble our way through this? Uh, both, uh, uh, we, we, we need to certainly most employees seriously need to upskill, uh, and, and get smarter and more knowledgeable about what these models can do.

And I think for the average users, they'd be shocked at, at the level of sophistication, um, these models can achieve. I mean, it's, remember Michael, that this was, you know, this wasn't really expected when the first transformer models, uh, uh, arrived. You know, they called it an emergent capability, meaning that they had no idea it could do this stuff.

And so we're still on that path of discovery, and particularly when you start talking about, uh, these reasoning models, uh, and intelligent autonomous agents, um, it gets really interesting and it's going to take, um, a keen eye and a cool head to figure all this out. Uh, and it is going to have major impacts on the future of work. Um, and, uh, there, as you said, there are a few guidelines right now, but, uh, I would say stay flexible, get smart, and, um, slowly, uh, you know, don't forget the, while it's important to figure out and understand new technologies, don't forget what is permanent and what is universal.

I'm thinking of a quote from Jeff Bezos, uh, you know, who was talked about, people talked about, you know, Amazon and what the new technology was that enabled this business model. They said the important thing is to focus on what doesn't change. And what doesn't change is people want choice.

People want convenience. People want low cost. And it's very important to remember those things when you embark on these grand ideas of artificial intelligence.

You used the phrase knowledge worker earlier, and I'm scratching my head sometimes about that term because I wonder if we're evolving into not knowledge workers, but maybe knowledge supervisors, and we're gonna have all these AI agents that are kind of gonna be repositories of knowledge that we're gonna have to figure out how to orchestrate. Is that where we're headed? Well, that's certainly the finding of this, the Carnegie Mellon Microsoft study, uh, in, uh, this month was that, uh, the nature of work is changing.

So you're overseeing, uh, and curating a knowledge process rather than doing the knowledge yourself. And, um, you know, Michael, when you think of it, you know, a lot of what you and I and millions of other people do is we sit at desk and we respond to emails, and we've spawned to emails and we synthesize information. Uh, and that's exactly what these models do.

So, um, uh, anyone tell you that, oh, they're just word calculators that they, they can't possibly threaten what I do. Uh, they may be mistaken. Are there, you mentioned your book, what's the title and where do I find it again?

It's on Amazon. It's called Organi, seeing for the New Productivity Revolution. Um, uh, it was out, uh, in December of last year.

Uh, and it just goes through a lot of these steps in a lot of these tips on how you organize around this new technology and how you make the best of it, and how you get your data in line to optimize, uh, the outputs and minimize the risks. Right. Folks, you heard it here.

Hey, even in the AI h you should look before you leave and there's a whole book about it. Absolutely. Arthur, thanks for being on the show.

You're very welcome. Glad to have it. And thank you all for watching the latest episode of the Textron AI series.

You can find this episode and others on our website. Until then, we'll see you next day.