The Future of Platform Engineering: Roles, Security and AI in Practice – The Platform Engineering Show EP6

Hey everyone. Welcome back to the Platform Engineering Show, one of my favorite podcasts. It's one of my favorites 'cause my co-host is my friend Luca Galante.

Luca is in Sicily today. You know, part of doing this show is just to keep tabs on where Luca is in the world. Um, so it's good to see you, Luca.

How have you been, man? Yeah, it's good to see you too, Alan. I'm great.

Sicily. I'll probably, I'm probably gaining like a kilo a week, like solid. Oh boy.

Um, yeah, like, you know, like literally really yesterday I went to like, see this house that I'm checking out and, and like literally next door, there's this guy that's making fresh Rico ricotta cheese, and it's like 9:00 AM and he is like, no, no, you should try. You need to try. And it's like, it's still warm from the milk 'cause it's like milked the cow that morning and it's like, oh God, you need to try, you need to try it right now.

And so it just gives me like a spoon and I'm just like, 9:00 AM I'm eating like an entire, you cut it. And that's just like, it's like this, you know, constantly. Like people just come around.

I know it's always food. People just give you, just feed you all the time. It's the best food, you know, but it's like, it's hard to, uh, yes.

It's to, to stay strict on the diet. So I gotta tell you, when we went to LY and like you read Yeah, I did a lot of, I mean the street food in Palermo, just the, the Arancini, right? Yeah.

I mean you could, you could live on those. Yes. Um, but yet we came home and we didn't gain any weight.

Yeah. Maybe it's 'cause we were walking so much, but the food is so healthy there, especially you, you live in Europe, so you are used to that. In the US we don't, our food is not as, it's not all non GMO it's not all organic.

It's not all as natural. It's a lot more, it's more poisoned. A lot more poisoned.

Like I have, I have a lot of friends. I, it's not the first time I hear this, I have a lot of friends in the US every time they come right. They're like, they come too.

Same thing. Like you, it's like, man, I'm eating gelato all the time. I'm eating like all this pasta, which is all carbs, you know, like pizza.

Like how is it that I'm losing weight and it's just you, it's the truth. It's not only that I like, this isn't really anything to do with platform engineering, but I, I have friends who are like, here in the US they're, they're gluten intolerant. They don't have celiac, but they can't really pick candle gluten, but yet they go to Italy and they have, you know, the good Selina pasta there.

Yeah. They don't have any gluten problem with it. So it's something about the, the way the, the fertilizers, they're using the insecticides something entire supply chain.

Yeah, yeah, yeah. Exactly. Exactly.

It, it's true. Yeah. It's, it's very interesting.

Very interesting. Yeah. Mm-hmm.

Yep. Anyway, but we, we could talk about Italian food, but Stanley Tucci already did a great show on series on that know copy. He's, he so great.

And he's like, the way he does it is, yeah, absolutely. That's an amazing, my wife and I watched that over and over again. We make notes.

That's really good one when we go visit. But let, let, we wanna talk today about platform engineering, right? And it's, we're at sort of, uh, a little intersection here.

Yeah. We just finished Cloud native, which is a big platform engineering of, uh, you know, it's not all platform engineering, but Platform engineering was a major player this year. Yeah.

I think it was the largest, um, not zero day, they don't call it, but the, you know, the mini conference day with the, the like Tuesday of CubeCon week, they all have all those little conferences. I heard the platform engineering was the biggest one of, of the ones there. Um, we've got Platform Con coming up in just about two months.

Yep. Yeah. Right.

So, so much, so much going on in this space, but there's also a lot going on with the evolution of platform engineering, right. Uh, like everything else I think in life is things go from simple and they get a little more complex Right. As you layer in stuff.

And that's happening in platform engineering. It, you can't just be a platform engineer. Well, you can, but there's this kind of specializations that are subspecialists, you know, popping up.

Yes. Luca, you've got your thumb on the community. What are you seeing?

Yeah, exactly. I think it's, you, you framed it really, really well. It's, it's, I think we're at inflection point right now where, you know, platform engineering is moving past kind of this core platform engineer persona, which first defined themselves as kind of like different from, you know, DevOps, SRE infra cloud ops, right?

And it's like, well, you know, platform engineers, and we spoke about this before, it's a lot about this like product mindset and about, you know, treating your customer, your, your, um, your developers as customers, um, your know and so on. And so that was kind of like the first step of like building, I think this identity. I think the interesting thing that we're seeing in the community now is that it's that identities now start like growing and bifurcating into, into some sp sp more specialized identities, sub identities, if you will.

Um, especially as platform engineering is really becoming this new, DevOps is really becoming this new kind of like enterprise IT layer that's sort of like sucking in people from all sorts of walks of life, uh, from, you know, data engineers to observability people, security teams and so on. And everybody. And we see like one of the fastest growing segments in the community is data engineers.

Um, and so clearly like people are trying to figure, figure out, okay, what can this platform engineering thing do for me? And the other way around it's like, okay, we are seeing top performing engineering organizations are already thinking of this as one unified layer of, you know, data. You know?

'cause I think like in most organizations, it's still very siloed of you have data and ML stuff over here. And of course with all the AI craze that we talked about yesterday, it's, it's, you know, it's, it's growing very fast, but it's still kind of like in this bubble. And then there's all the, you know, traditional kind of like infrastructure DevOps side of things over here, right?

Uh, but of course, you know, I think what we see is top performing orgs already think of it as just like one unified layer because of course, why wouldn't you? Um, it's just like all converging into the same thing. And, um, and kind of just like one platform that serves different types of users from app devs to, you know, data scientists and MLIS users all the way to eventually more abstracted, you know, any business user really.

Right? Um, and so, um, and so I think as part of that, people are trying to figure out, okay, how does, you know, data engineering, for example, and play platform engineering combine? And, and, and, and what we're seeing is the emergence of this like sub-specializations around data platform engineering and security platform engineering and DevX platform engineering infrastructure, platform engineering, and so on.

Um, and, um, and I think that's very, very interesting. And in fact, we're, we're restructuring the way we, um, sort of like organize the content from a community standpoint as well along these lines of, um, of these different streams and these different personas. I I agree.

I, but as we were talking off, off camera, I saw a similar development in the DevOps space over the years, right? Mm-hmm. Where you used to have, you know, when it first started, everyone wanted to hire a DevOps engineer, even though no one really knew what, what was a DevOps engineer?

Is that someone who worked on chef and puppet? Is that someone who did Jenkins or, you know, later on was that someone doing GI ups? And then like, as it matures, you, you start realizing there are, it's very hard to be a jack of all trades, right?

Master of nut. There are specialties that you, you, you do, you mentioned security, you know what, so as a security person, it's always the same story. Sooner or later, security gets involved usually later than sooner.

Yes. Unfortunately. Yes.

Yes. But, but you think security's coming in now and, and making a a, an appearance in platform engineering. Yeah, a hundred percent.

You know, and we see people like check marks and, and, and, you know, like, I've been talking a lot to like chain guard and like a bunch of others that are like really looking at this and they're realizing, okay, platform teams are, um, you know, from a vendor standpoint are, you know, at the very, are, you know, in most cases becoming like a core persona. And, and you know, at, at the very least, they're like heavy influencers on the budget, right? Around this like security, security tooling.

Um, and I think, you know, it's interesting because I've always, I've always thought that, you know, to your point, security tends to get always involved, sometimes too late, often too late. Um, and often it has this like blocking role, right? Of like mm-hmm.

No, like actually we can't really do this. It's not secure, yada, yada. Which totally makes sense.

Um, I think the interesting thing that I've seen a lot of times when it comes to platform engineering is that when a platform is well designed, actually security teams not only don't block it, but actually they're, they kind of like quickly become like champions in favor of it because they're like, hold on a second, you're telling me that this, you know, again, if you, you know, we can get more into the details, but really if you design a platform well with like a proper backend really well thought out, sort of, you know, golden pads, effectively what you're doing is, you know, developers can move faster, which is what everyone else finds sexy without breaking things, which is what security teams find sexy. And, and so, um, and so, and not only like, I think not only that, if you, if you take that a step further, you realize that a well designed platform actually is a, um, sort of like automatic enforcing engine that drives security by design across your teams, across your workflows. Um, and it really, um, you know, and, and, and really benefits everybody that's evolved.

So, um, I think, I think that's, that's the interesting thing of like the intersection of security and platform engineering where really, um, I think from the get go, even like two, three years ago before this like security platform engineer was a thing, uh, or that's now starting to emerge. It already I think was a very natural fit. Um, you know, and, and now it's only getting more obvious, I think.

Yeah. You know, you mentioned chain guard. I don't know if you saw, they announced today a $356 million round today.

Today. Oh, wow. That's great.

The day we are recording this, uh, but I don't even remember what day did, is it 23rd of April? April, April 23rd. April 23rd, $356 million round d that's a huge amount of money in today's world.

Um, so clearly they're on, they're onto something there. Um, but you know, you mentioned that they're calling them security platform engineers, right? This has always been a problem that plagues security is that the security people, we, we, we security people deeply care about security, right?

And we tend to create tools and processes that if you're a security per, if you're a security person, it makes a lot of sense, right? We understand from a compliance point of view and from a best practices point of view, why you do this. Yes.

I think the problem we run into is whether we're talking about developers or DevOps people or ops people, or maybe even platform engineers, they look at it and say, why, why? You're just slowing things down. This makes no sense.

I don't seem to, I don't see the connection. Yeah. How are we bridging that in, in platform engineering, Luca?

Is it by having the security people as a platform engineer, is it by educating the platform engineers and they become security platform engineers, but the security people are, I know words, the security people aren't becoming platform engineers, right? The platform engineers are becoming more security aware. Yeah, exactly.

That, that's a great, that's a great question. And exactly like what we're seeing is like as platform engineering is maturing and platform teams are maturing in their practice, they go beyond the, okay, you know, everything I care about is, you know, building a delightful developer experience and making developer faster. But they start realizing, okay, what are the second, you know, derivatives of this, right?

Okay, I need to worry about, you know, all the security. And of course, it's not like that they weren't thinking at all about it before, but they're maturing in, in their approach to it. Um, and I think like a big part of that is also taking it step by step, right?

Like as you roll out a first platform to your developers, and you know, we've talked, we talked before about the importance of minimum and viable platforms and, you know, taking this like typical product management approach of like starting small, getting early wins, and then iterating from there in the early phase, security usually is less important. And it should be, because if you need to make an MVP that's like ultra secure and, you know, ready to cover all the possible edge cases in production, you're not doing an MVP. Um, and it's gonna take you forever, right?

So, and, and in general, like internal developer platforms at the beginning when they're nascent, they're really focused more on dev, dev and staging, right? As environments then as they mature, right, you get to production, and then of course you need to be, you know, uh, sort of like bulletproof and really solid, uh, on all security fronts, right? Um, and I think to your point, yes, one, I think I really see this, this, this like sort of like double path, right?

Like on the one hand is platform teams becoming more security aware and, and growing. And, and this is why we're now like, you know, we see this, this emergence of the security platform engineering as a, as a, as a sub practice. Um, you know, and by the way, this, this might seem like esoteric terms that we just come up with, but like if you look at, for example, infrastructure platform engineering, which is slightly more established, right?

It's been around for like six to 12 months top. Mm-hmm. Um, it's already, it's already in seven different hype cycles from Gartner, right?

Um, so these are things that like, seem, seem like small differences, but then they really become like established practices. And you're already starting to see a lot of people that are calling themselves devex platform engineers and infrastructure platform engineers, because that's like 12 months ago people started talking about it. Now we're starting to talk about reliability platform engineering and security platform engineer.

I can guarantee in 12 months from now, you'll see that on LinkedIn profiles, right? I just look at as a side note, but I think the other, to answer your question, the other, the other, I think the other way that you see this convergent is honestly just by following best practice, right? So beyond just the mature, the maturation and the evolution of the people working in platform teams is also, well, like, look, actually, if you design a well-designed platform, right?

Which is beyond just like, Hey, let me do like some front end, let me do now some LLM because it's the latest, you know, cool thing. Um, and you know, and, and just plug it on top of the existing workflows, but actually design the, the, the platform from the ground up, which means starting from the backend and really have a well designed backend with, you know, good business logic, good RBC role base access control and, and all these things built in. Then you realize that actually security is also is, is again, a second derivative, but it's second derivative that's just automatically enforced by the platform by default, right?

Um, across all these different workflows. And so I think those two things, like more awareness by the actual people doing platform engineering and then following best practice is ultimately how you see security and platform engineering converging as disciplines. Absolutely.

And it'll be interesting to see how, how that plays out observability. Mm-hmm. I don't know if you, you saw this, but we, some folks are starting to reach out to us now.

Observability is a thing, obviously, right? I mean, OpenTelemetry I think is the second biggest, uh, cn, uh, CNCF project. Uh, every, you know, there's a lot of buzz around observability.

So observability, you know, it, it's, it's really becoming like a platform engineering like DevOps, like SRE, it's becoming a singularity, if you will, right? Its own thing, not part of other things, but yet as you know, it's, it, it's, to me, it's a part of DevOps to me. It's a part of platform engineering.

To me, it's a part of security, right? Yeah. How do you think that plays out Luca specifically platform?

Yeah, it is super interesting, um, to me because for a few reasons. One, I totally agree with you. Like, I think it's, it's clearly still just like a subset of, you know, DevOps platform engineering.

And I think most people look at it that way. Yes, it's grown very, very fast. And, you know, you have massive companies like Datadog and so on, like, so, so clearly it's its own niche, but it's a very, very big niche.

You know, if you're go to a place like CubeCon, probably the, you know, the biggest, the biggest boots out there all like observability folks, right? Mm-hmm. So, um, and security.

Um, so, um, so clearly, you know, it has a very, it has a, a very valid kind of niche, but I think specifically when it comes to, um, platform engineering, it's very interesting because, you know, a huge part of platform engineering, what we discussed before has been this like front end thing, right? This idea of like, well, building a platform is just like building an application. You have a back and your front end, and we've said before, right?

Like a lot of platform teams, I think make the quick mistake of starting from the front end because it's a quick win. And then to figure out, okay, actually now I need to build all this thing. And what we just talked about with security, right?

Like, if you have like a, a well, well-designed platform, you start from the backend, that's where the all the, you know, security best practices go, all the configuration flows go on infrastructure side and so on. Um, but the observability is interesting because it, it's, it's merging very clearly with this like front-end layer of, uh, of the platform, right? And so if we're thinking like portals, right?

Like at the end of the day when we're talking about, um, fr you know, like platform front ends, we're really talking about portals, right? Things like Backstage, which still has 95% plus market penetration when it comes to portals, they're really the only game in town. Um, you know, but you know, what do you know, like Datadog has now launched a portal, right?

Um, and mm-hmm. And there's some other, um, observability vendors that are following suit very quickly. So, and, and like of course it makes sense actually, right?

Because again, you, you think of this like observability vendors that are like growing really fast, like we said, and they're like owning more and more things. Why wouldn't they own this sort of like portal layer on top of the platform, right? Um, and I think that's, that's another very, very interesting development that we're seeing that I think like we weren't really, you know, thinking about if you, if you, you know, go back like one or two years, I think the vast majority of people are still not thinking about it, but it's happening.

Um, and you know, that's kind of like the work that that, that, that we do in the community is trying to spot these trends and, and, and kind of like pinpoint them to people. And that's why, um, there's gonna be soon an update to the ref architectures that have been such a, you know, essential part of the, of the community, of the platform engineering space and how people think and talk about the platforms that they're building to one another. 0 I'd say, that came at Platform call 23, uh, two years ago.

0 version. Um, and I think, uh, there's gonna be some interesting updates, especially when it comes to the observability plane. So, um, I'll, I'll, I'll, uh, I'll, I'll, I'll, I'll, I'll keep it there.

Well, we, I think we dropped the teaser right there. Yeah. So we, we don't have a lot of time left, but I, I've got one observation on observability.

All right. Um, I think AI is going to change what we think of as observability and the players in observability and what we do in observability is going to get turned on its head with ai, with agentic AI already regenerative. Um, I think it's going to, it's really going to like have observability take off, right?

Because you're gonna be able to do so much more, so much easier. Yeah. And, and so I would start getting ready your ref architecture three oh AI version or something, right?

Oh, yeah. Oh yeah, yeah, yeah, for sure. Mean look, AI's gonna do all of these, but the thing is, is, is as, as you know with AI, for sure, we are seeing an acceleration across all these different things, right?

Security is changing there a lot is changing. The whole dex layer is changing, right? And again, like during me, because it's like, okay, you know, now a developer can not only sell serving for structure through an LLM, you know, and this type of stuff, but also can query the system.

And so you have this whole like, observability layer that's also built into this, the same interface. So yeah, I definitely see us just working harder and harder and harder just, just to keep up with how the landscape changes and changes and how best practices evolve according. Yeah.

Yeah. Yeah. It's a crazy time, man.

It's a crazy time. Yeah. Yeah.

Luca, we're about outta time. Um, how long are you gonna be in Sicily? Um, one more month, I think.

Oh, yeah, you hear there. So for our next show, you'll also be in Sicily. Enjoy it.

Don't eat. Yeah, I will be, I will be. Yeah, exactly.

Yeah, that be nice. The triangle ones. Yeah.

Yeah, yeah, exactly. And I did, I did mention, I did mention, um, platform com, by the way. The, um, I think by the time this drops, we're gonna have, um, the, um, the not so early bird, um, that's gonna be about to expire at the end of April.

com and, and text on community. So, um, also stay tuned. We'll have that out there.

And this will, this will air before the end of April. This, this will probably air, I'm gonna guess by the 24th, something like that. Okay.

Okay. Perfect. Well, no, the 25th maybe.

Mm-hmm. Something like, or, or the following Monday. But we'll get it out there.

Stay tuned for those codes, Luca and Joy, don't eat too much there for now. This is another episode of the Platform Engineering Show. Bye-bye.