Rethinking IT Management with Synadia Communications’ Derek Collison

This is Techron tv. Hey guys, thanks for the throw. We're here with Derek Colson, who is CEO for CEDIA Communications, and we're talking about data streaming, open source nets and all kinds of interesting things related to multi-cloud computing.

Derrick, welcome to show. Nice to have, uh, nice to be here, Mike. Thank you.

You guys just raised $25 million in additional funding and it's interesting to me 'cause, well, a, it's difficult times for raising money anywhere, but you don't see a lot of open source projects in the networking space to begin with. So what's kind of driving your efforts here and how will you be applying that 25 million? Yeah, so it's definitely an an interesting macroeconomic climate, uh, for for sure.

And, you know, most folks know Zenadia as kind of, you know, intelligent messaging and connectivity, but I think what the investors saw was a, a grander vision around redefining how distributed systems are actually built and deployed that stretch, not only from, you know, multi-region and multi-cloud, but more importantly, out to edge. Edge is kind of one of the big inflection points that I think has, um, made people kind of sit up and, and take notice, if that makes sense. In some ways, I feel like this is a conversation about back to the future.

'cause for the last few, uh, years, maybe even a decade, we've been busily moving data into the cloud, and yet we have all this data at the edge, and it feels like we're going back to bringing the compute to the data instead of trying to move the data to the compute. And the whole thing feels like it's becoming a lot more federated. So to your point, what does the future of distributed architectures look like?

Well, that's a great, you know, um, observation and, and one that, that we, you know, made a bet on that, uh, the future distributor systems would look very, very differently. Um, so for example, the way we approached our system, which was very novel six years ago, was instead of concentrating on workloads and then data and then networks to kind of tie them together, whereas if you look at cloud infrastructure and for example, cloud native architectures, that's kind of the, the pecking order. And what we did was we said we really need to totally rethink connectivity first, data second, and then workloads third, and then move to, uh, specialized vertical solutions.

Uh, when we have those three pillars, and when we started, I think everyone kind of looked at us as swimming upstream or, or going against the current, but as covid hit, as the pandemic hit, as, um, things really got stretched out to remote work, and you could see that the motivation to reduce latency to access either a service or data kept increasing. And, and the metaphor that I use is kind of like, you know, if if you paid by, you know, credit card in a restaurant, it took an hour to get that, that payment to clear, right? That's what we're talking about, about latency.

It's, it's not that the payment gets processed and the, you know, it's great fraud detection in X, Y, Z whether it's another technology system or an end user or a partner. They're always trying to, you know, get more value out of accessing services and data, but reduce the latency and reducing the latency used to be, oh, pick the right region in, in a cloud provider, right? And, and make sure that we're there.

Um, but I think it's pretty obvious now that, um, at least from my perspective, you know, every company is, is a technology company. Every technology company is a global company, and global means latency and, you know, latency access to services and data that transcend, you know, what I call cloud providers or even traditional like edge providers like CDNs or the new, uh, folks on the block, like, uh, you know, Dino deploy Versa. Things that you might've heard about, even into what I call the far edge, which is, um, your factory, your store, your distribution center, your charging station, your electric vehicle, all of those endpoints need these access to services and data to be as fast as absolutely possible.

And it's not reasonable to say, let's forklift and pick everything from a cloud provider and try to force it into, let's say, an ECU on an electric vehicle or onto an ECU inside of a, uh, a charging station or into the international, you know, space station type stuff. And so what we were betting on as a company was, is that distributed systems would continue to grow in the number of moving pieces, but more importantly, they would be stretched out. They would become very, very sparse.

And that the rules of engagement as edge kind of took hold. And I made a bet about four years ago that Edge would dwarf cloud in terms of what we consider, for example, like mainframe. Mainframe still exists, but everyone feels that their interaction models are with clouds.

I think the same transition's gonna happen with Edge. All of our interaction models would be happening with Edge. And again, for, for us, we define edge as three buckets.

The cloud providers trying to say, um, no, me too. Right? You know, um, the traditional, um, edge providers like the Akamai and the fast leads and the cloud flares obviously, and then the new kids on the block, Pursel and things, and then the far edge, which is the edges, it's defined by you, your web browser, your phone again, your factory store, things like that.

Aren't we moving to more of a near real time application experience? And is that part of this conversation? Because historically, you know, we did everything in batch mode and, you know, if the applications kinda, you know, synced up in a 24 hour period, we were kind of happy, but feels like today, you know, people want everything to be in sync.

Um, the minute the data's created and analyzed, it's gotta be in sync with everything else around it. Yes, I, I agree with that and I was, I was very fortunate earlier on in my career to be in the financial industry, uh, you know, in the nineties and, and early two thousands where Wall Street was moving to, it used to be T plus three settlement to T plus one, to now everything is kind of just instant. And I think you see that the access to technology and information, uh, the, the, the desire to always have that in real time, again, decrease the latency to access the service or data, I don't think that's ever going to, to go away.

And I think even though CEDIA fits into what I call the connective economy, right? Where I think the, the, the way things are connected is driving the value and the innovation and velocity. The AI economy, which is the other big one, which you probably spend the majority of your time thinking about, is having a drag along effect of expectations that everything should be going faster, right?

You know, if we say, oh, we might have that next year, you know, a lot of times you might see a, a customer look at you and go, well, OpenAI just recreated, you know, um, video, uh, text to video in, you know, six months, you know, what are you guys gonna do for us? And so this notion of being able to move very quickly and reduce latency access in real time semantics, like you said, I think that's where CEDIA shines because if you try to take a cloud architecture and then stuff it into an edge, right? And, and again, around the goal of decreasing, uh, latency and, and moving it more to a real time experience for either an end system end user and partner, um, that is gonna, it's not impossible today, but it's gonna take you a very long time.

It's kind of like putting a square, you know, peg in a round hole. And with Zenadia technology, we're not fundamentally changing the what the way distributed systems work. Um, you can kind of think of it like going from a petrol car to an ev, you still got a steering wheel, still got a gas pedal, right?

Meaning you still have microservices and streaming, but the how radically changes in a way that what you do in a cloud in a different region, in a different cloud provider, in an edge provider all the way out into your factories and things like that is exactly the same. I think that's what allowed us to get the, the raise in this macroeconomic climate, especially around a B round B rounds are usually an inflection point. They're usually challenging for, for folks.

Mm-Hmm. Well, what impact will AI have on the edge? Because I think we're all obsessed with the training aspects of ai, but there's this whole inference engine that's gotta get deployed and updated, and a lot of those are gonna be running in each of these different edge computing systems.

So, um, how will we manage all that? Well, that's another amazing opportunity I think for Canadian. And the reason that I think that is, is that we saw the bifurcation from training to inference, you know, uh, sometime back.

Um, and we know that we have these large LLM models that have, you know, incredible capabilities, um, and they're moving at a breakneck pace, right? In terms of, you know, people's concern around energy I think is, is going to be, um, you know, squashed with the contracts with nuclear data centers that, that Microsoft and I think Amazon has now. Um, and so what people were originally thinking about inference was is how do I squeeze these things down to run on an iPhone?

Let's say, what I think we've seen in the last year is that that is not at all what inference is gonna look like. What inference is gonna look like is you present a raw prompt of I want to do this or I want you to generate this for me. And I think what we're seeing with technologies like RAG and RAG plus and prompt, you know, augmentation and things like, uh, technology that came out just the other day, um, Devon, which has kind of taken the world by storm, at least the software engineering world, because it's like Uhoh, is it gonna take all of our jobs?

Is you're going to see this tech stack that takes that, that initial raw prompt at the edge, let's say on your phone or on your, um, on your web browser or, or or in your vehicle, whatever that is. And it is going to immediately gather all kinds of real time information where they don't know the location of this, it could be local, it could be remote. 5, um, the Claude three, you know, um, Opus I think it's called, or the Gemini Ultra ones.

And so what CEDIA is seeing is, is that, and I wish I, I could have said that we foresaw this and we didn't, but, but we are definitely observing that there's going to be this kind of go-to tech stack for inference at the edge, and we want to play a really interesting role there because when you say things like, how do I gather real-time information as fast as I can and I don't know where it is, that's exactly what we're very, very good at. How do I utilize multiple LLMs? Some might be local, some might be remote, some might migrate from being remote to local, like, you know, from a cloud to like inside of a vehicle, right?

When I connect to wifi, my garage, it updates overnight this big huge LLM, which just 10 years ago, there's no way you're gonna, you know, do a 40 gig upload or download over wifi, but these days it's no big deal, right? And so you park your car in the garage, it updates the LLM that's running locally, but in my opinion, inference is not gonna be figuring out how to squeeze, uh, an LLM onto an edge location and have a prompt just go directly into it. There's gonna be a huge interface and and stack in between.

That's augmenting what essentially gets put into not only one LLM but multiple ones. And then again, it's gonna be gathering all the results and then filtering the results to, to give you the final experience. And so there's some amazing opportunities that I think are, are happening.

And what's interesting is, is that those innovations are happening even faster than the hardware and the big LLM, which are still going at an amazing clip that I don't think, at least I, I couldn't imagine it could go this fast. I knew I was probably wrong. Um, but watching how fast Nvidia went from a 100 to H 100 to B 100, which is coming out this year, the supply chain logistics stuff seemed to be, you know, mostly resolved.

And I think we have short term solutions for the energy stuff for the very, very big LLMs, mostly nuclear, but maybe there's some other, uh, options on the, the next decade horizon. And then of course with ization of LLMs where you don't lose loss of fidelity to squeeze them into an iPhone, right? And then to specialize chips that are gonna run iPhones and in your vehicle and in your house, I think it's gonna be an amazing world and I think you're gonna see everyone shift most of their focus to the inference side.

To your point, it feels like we're gonna be orchestrating all these different LLMs and I think we have a mindset that's kind of a legacy one where we think about processing and analyzing data on the platform, and then we get the aggregate and we ship it somewhere. But to your point around data streaming, it almost seems to me we're also gonna be analyzing the data in flight now as in addition to where it's located and one kind of nuances are changes in the way we think about it. Does that kind of bring about Yeah, that's a great question and, and what we're seeing with a bunch of our customers who have edge locations, especially ones that have what we call heterogeneous connectivity patterns, so meaning they might be going over multiple satellites or cell phone, uh, connections or asymmetric broadband, right?

Um, where upload is very still different than than download is the ability to send signals in at a rate that's acceptable to the cloud. That always represents real-time information. In other words, I'm not in the cloud going, oh, I'm getting information, you know, from, you know, Michael's system, but it might be two hours old because it's just a slow pipe that, you know, uh, the Canadian systems actually will only deliver when the cloud can receive it, meaning it's mostly the connectivity piece, the real time information, but they also are maintaining historical data that when you switch the wifi or you reconnect to the satellite and you've got a good connection, the cloud can get all of the data it wants to do the analysis, the updates, and then send the updates, whether it's an inference engine, um, you know, an automated data set for a microservice that wants to run locally.

We really see those patterns evolving at a rate that we were very surprised at, meaning we thought we would see this in some startups and some very forward looking, you know, um, global 2000 companies, we're now starting to see it in Fortune 100 companies that are trying to move at this type of, of pace. And they're starting to understand that what they want to achieve is not achievable, at least today with kind of what most people think of, oh, you know, you do an API gateway and you run something in the cloud and you've got, you know, um, you know, x, y, Z type stuff. And so we were very, uh, you know, elated but surprised that we have a mixture of both Fortune 100, a lot of Fortune 100 name brand companies that are trying to go as fast as the startups are.

And it's all around the, the things that you're talking about, right? This decreased latency enrich the experience, real-time semantics versus the batch for sure. Today we have all these IT teams that are somewhat, uh, isolated from each other.

There's a bunch of data engineers, there's a networking team, there's a DevOps team, there's an AI ops team. Um, is all that gonna flatten out at some point? I mean, or do we have to figure out a, some sort of way for all these folks to work in hand in glove?

Or are we looking at kind of a major restructuring of the way we think about managing it? That's a great question, and I think at least from our perspective, the rules of engagement of, you know, how I would interact with the microservice, let's say those are fairly well understood and they're going to evolve, but they're not going to, you know, be revolutionary, so to speak. Um, what I think is gonna change is two things.

One is I don't know where the micro service is. I don't know how many of them are around, but I trust it. So even if it's running right next to me, I never had to restart.

I'm just asking the question of the Michael service and I get the trusted answer back. And what the second piece of, of the answer is there is that I believe you're gonna see a massive disruption around what we've centralized around for multiple decades around what I call perimeter based security models. So printer based security models, firewalls, VPNs, you know, VPCs and in the cloud speak, so to speak.

Um, but if you think about your phone, yes, people have VPNs on their phone, but in general, if you have a SIM card and you paid your bill, your phone's live, no matter what cell tower it's connecting to, no matter where you are in the world generally these days, I think the edge is starting to challenge the perimeter security models. Not for the sake of just challenging it, but for the sake of saying, I don't know where the data is and I don't want to have to know, I don't want to have a DNS entry that points me only over to there. I want to simply say I wanna ask a question and get an answer as fast as possible.

Um, or I want access to data and I don't care where it is, but I want it as fast as possible and I want to be able to trust it. I know that I was talking to Michael or a clone of Michael, let's say, and I think those, and again, those roll up into those upper level goals of how do we decrease latency to access services and data. Um, those initiatives in terms of driving, you know, business value and innovation, I think are gonna put pressure on the older school primer security models to say, Hey, we need to start thinking differently about how this looks connectivity in our, our opinion.

ENA is, and that's the first pillar we started with, should be location independent, should be intelligent and should be should, you know, should be secure. Meaning that it's not an impediment, it just works. Um, and I think that whether it's anad technologies or other, you're gonna see a, a transition within the IT landscape of people moving more to something like that where the network and the connectivity, not necessarily the network, but the connectivity piece is the most important piece and it has to be an intelligent piece.

It can't be a DNS and IP address, kind of like we, what we've built on today. I mean those in my opinion, you know, are, are are throwback to the seventies and eighties where I had to know where you were, I had to know your phone number, you know, I had to hope that you were at your office that the phone number was attached to with the, you know, the desk phone or the wall phone. Believe it or not, you know, the majority of our technology that we think is so amazing and it is, it's still based on those models, it's still based on I have to know where you are.

I have to, you know, take a DNS and make an ip. And whenever you see something that doesn't look like it's doing that model, believe it or not, it's doing unnatural acts, right? And where you see, oh well we've got a load balancer and we've got, you know, you know, firewalls and global load balancers and all kinds of stuff to make that happen.

Um, and so I think you're gonna see a shift to saying, Hey, you know, connectivity and intelligent connectivity drives a different way of thinking about data, especially access and flow data can just move to where it needs to be and it can be securely accessed autonomously. You don't have to do anything there. And then the same thing for workloads, right?

Workloads can also move, not necessarily just let's say within a Kubernetes cluster, but any region, any cloud provider, any edge out to any device that again has a security model where if I have the right sim card, you know the right credentials, I can run that, that, that workload, whatever that is, whether it's an inference engine, a microservice, a stream processor, whatever that is. Alright folks, you heard it. Here it is never gonna be the same again.

And it all starts with connectivity. Hey Derek, thanks for being on the show. Thank you, sir.

I appreciate it. All right, back to you guys in the studio.