Platform Engineering and DevOps – Dotan Horovits, Logz.io

This is Textron TV. Hey guys. Thanks for the throw.

We're here with don't ten Horowitz who's principal developer advocate for long Zio and we're talking about platform Engineering in the context of devops doten welcome the show. Hey Mike, thank you for having me pleasure being here. Okay, so about once a year or so a new buzzword comes along and everybody kind of nods their head.

And here we are with platform engineering but I'm scratching my head a little bit frankly because it seems like it's just the term that we're applying to a group of people that seem to be implementing devops and the fact that they're all in one group somewhere. We decided to use a shorthand for the team in column a bunch of platform Engineers, but it doesn't seem to me that that's entirely different discipline or am I missing something? I created the sort of an evolution in the maturity of devil.

So it's not entirely different. That's that's for sure. I think it's taking to the next level in terms of making it into a platform as the name suggests or reusable elements for soft self-service across the organization.

Obviously, it's much more relevant for larger organizations that have lots of product lines a lot of development groups where this reusability and consolidation and makes a significant impact. So it's almost about the level of maturity of the devops workflow because I now have a team that is creating a set of processes that are reusable across multiple development teams. Whereas if I went into a lot of organizations in the last few years, I might see I don't know eight nine 10 different devops platforms and workflows all across these different business units.

So is this an effort to kind of centralize devops workflows a little bit and therefore we're gonna have a team to manage that I think it's pretty balls down to pretty much what you said because essentially the evolution is that as you said the devops grows Bottoms Up from Individual product teams that need their own devops processes tools methodologies and each one solves the problems, but then at the organization grows suddenly realize that different teams actually solved the same problem just over and over again without being aware of the fact that the neighboring team and deal with the same problems away. You need, you know certificate rotation. You need some sort of ability to instrument and add this with the tracing or anything else just instead of this soil happening again and again and again, You can solve it once across the organization and obviously in a more aligned way that will be aligned with the organizations standards processes, maybe compliance requirements and others that not always are very clear to all the individual teams when it's it's different.

A lot of these issues seems to also come back to moments in time where five or six years ago. Somebody may have built a devops platform and workflow. But yeah, it was a lot of shall we say Do It Yourself work and they kind of stitched a lot of things together and supporting that over time becomes more problematic.

So as part of this whole effort are we starting to replace things that people may have built by hand with things that are more of a product that they buy and maintain or even a service that somebody else provides versus them trying to do every little thing from the ground up by themselves. So I would say one very important distinction is that it's not a product that you buy of the Shelf. It's not the third party like you choose your own monitoring monitoring tool of the shell or some automation tool or some others platform Engineers or Platforms in the context of platform engineering are things that are developed inside the organization in a way in just to Showcase why I put such emphasis on that and you probably remember that as well as I do being the veterans in the industry.

We remember less than a decade ago the bus around past platform as a service when we used to neatly organize things as IAS plan and sales and and back then we did the idea was that would we would be able to take off the shelf platform as a service that will then consolidate all the underlying infrastructure for us provide services? Apis, and then we'll be able to develop the size the software as a service on top of that obviously didn't work out. It hasn't worked out and we see what happened to her Roku and and the the web the cloud vendors equivalents for that stockings and others but have been stocks are and others but the thing is that the reason it didn't work.

Is that a third part never have the best fit for your organization a vendor never be able to tune that and this is where I think the real power of platform engineering comes into play because these are organic development devops in sitting within your organizations familiar with the constraints of the organization the culture the tool set of choice and and so on and so forth and then can be very flexible in adapting because not all teams would work with the same tool set with the same Technologies with the same. Those and we can enforce that the engineers are an opinionated the species. I know being an engineering background and and it is this flexibility and this is the power of platform engineering and it brings to the organization.

When do I ultimately decide to build it myself is that when I'm always going to do in a platform engineering context or would I bring in at third-party open source tool or some sort of tool that's already exists that does I don't know 90% of what I need and then customize it what's that? How do I strike a balance between what I'm going to allocate my limited resources to versus wheels that are already invented. So that's a very good point and it's important to say it doesn't mean that the platform engineering teams don't choose anything off the shelf and do it only homegrown.

So that's not it and usually actually they have the platform Engineers of the golden path of tool sets for let's say monitoring for the ability for management for security and devsecopes related the chores and tasks. So so that they don't use these party tools. The question is how you wrap it up and then externalize it as reusable Services internally within the organization.

So you have a team and that again as I said, let's say needs to have some instrumentation at least the tracing to the to it's and microservices and then you have another team faced with the same thing or they going to do the same and when someone had the centralized view across the organization and see this tasks, Being implemented again and again and can raise that flag and say no I will take it upon myself and I will expose a reusable service for you with the constraints and and the ways that they work each one of these teams in the required adaptations. This is where the platform platform engineering and the underlying infrastructure could be utilizing off the shelf services or sdks or similar. Will there be Specialists within the platform engineering team it will continue to see that evolve.

I mean it's kind of shorthand but it doesn't seem to me that every platform engineer has the same skills or the same job functions. So is there a hierarchy or platform Engineers or how do we need to think about that? So one thing that is important to understand is platform Engineers are engineers and oftentimes also developers.

So maybe unlike some organizations where the devops lean more towards the upside of of skill set here. The the development side of things is important. It is after all a product and this is something that I emphasize often treating platform engineering as the platform is a product and in total product but product facing so you need someone to function as a product owner.

It could be one of the engineers but ideally I I being a past product can also other life. I think that product mindset is very important you need to treat the users as customers to understand the use cases the pains the needs that arise from the different teams the conventions within the organization and only then derive their necessary functionality and the relevant apis, so I would say this is the product function then you need develop. It could be front-end related.

If it's API basically even as or back end to the front end even if you if you offer a UI, I don't know if you've heard about backstage by Spotify that has recently fairly recently been accepted to the cncf incubation the cloud native Computing foundations incubation program. This is a magnificent example. This is an internal tool that Spotify the developed internally to make the developer experience data, but by consolidating all the different components And available in the services and the capabilities around cloud and so on into one one screen to put it cleanly and now they're actually they've seen such a great value that they contributed to the open source, and now they're actually trying to monetize on that by even offering some paid plugins on top of that.

So this is a very interesting the power of platform in a nutshell and I think it's an excellent example. Hello, I strike a balance between what you might have described as maybe a benevolent dictatorship and all these developers out there that always want to kind of, you know rip and add new tools into their pipelines as they see fit and they're very independent-minded people. So this is exactly where I think platform as a service failed the decade ago and where platform engineering shows more adequate Direction This flexibility is very difficult to achieve with a third party of the Shelf.

But when you have the engineering team internally within your organization, and then they approach each and every one of these teams understand their needs and then derive elicit these requirements and and turn them into reusable components. Then you can do the other patients. So if a certain team likes is different flavor of monitoring or different Convention of dashboarding or different automation these like Jason and he's like yamo and things like that.

You can create the Adaptive layer on top of underlying reusable services so you can externalize maybe a different UI convert Jason to yaml or the other way things. Dad Dad so you give the developed developer experience is the developer productivity I'd say and developer experience are the value proposition of the platform engineering and you need that. You need the flexibility to actually provide this value proposition to different teams in different ways.

Is this all provided under the it organization or does it kind of sit apart from the it organization because developers and the centralized it folks and that always had the best relationship and it's very hard now for those two teams to have this conversation. So are we creating, you know, essentially a bridge using this platform engineering concept? It's also very interesting question because it's it's an emerging parting and you see different evolution in different incarnations in different organizations.

And by the way, even the name itself in some organizations indeed cold platform engineering team in others. It would be called so shared services so Cloud platform and other variants, but so where does it belong within your organization is also dependent I would say that should be organically part of the engineering organization Department because the DNA in the nature is engineering that's one pro for this approach and the other is to tighten the relationship between the platform engineering and the that's a product Engineers. Well their users so they need to treat platform as the product and these are their customers of users.

So they need to be very very close to them. To understand the needs the pains and be able to close the feedback loop very closely. This is the advantage of having in-house team.

So if you create this this Silo, it will be closer to a third party vendor with all the problematic side of that. So we should avoid that as much as possible. This would be advice.

I would give organization. All right. Do we need to write up a new social contract between developers and centralized platform engineering teams about what's expected from both parties and you know what rights am I giving up as a developer and return for some Central sense of less chaotic it environments.

I think the And as developers, we always have this tension between two contradicting needs or wishes that we have. We have the need that you brought up for flexibility. I want the control.

I want to be able to configure every bit and bite and every switch in the command line and thing like that and on the other hand, we don't we're lazy beasts if I may say so the engineer myself and we like Simplicity and we like someone to take care of things for us. So the challenge with platforms is always been through all the years and decades that I have been in the industry a balance between these two edges the flexibility and the Simplicity and where you strike this talents as I mentioned paths, for example took direction that is more towards Simplicity maybe too much platform engineering tries to bring it back more to flexibility and the contract the social contract that you mentioned is exactly about that we as platform Team provide you with Simplicity. This comes with certain conventions certain as a golden paths that you can do in many cases.

They don't actually enforce or it's not it's not mandatory, but then you'd say if you choose a different stack. Will be able to provide a higher level support and not in the finance granularity that we provide for the golden path type of tools tool train and tool set where we manage the contract where we iterate maybe with the contractor if we talk about third parties underneath the hood for for bug fixes and for for enhancements and things like that. We'll provide as much as we can but this is the sort of the even there.

It doesn't have to be All or Nothing be in off the shelf and golden path stack and some higher level support for other alternative. Well as they always say folks platform Engineers, maybe they should have to run for reelection every three or four years, but just remember democracy is messy. Hey don't thanks for being on the show.

Thank you very much for having Mike. All right back to you guys in the studio.