Martin Logan, Caesars Entertainment | DevOps Experience 2022

Hello everyone. Thanks for joining me on this session today. I'm going to talk.

I mean, you know about the title of the title is how thinking devops made Caesar's entertainment and online gaming Powerhouse. So Caesars Entertainment is now one of the big four in the online gaming space. We have 13% of the overall Market there have been you know tens if not hundreds of competitors.

Only four have double digit market share. When I came on board we had no platform, right? Well, we had a platform but it was built in the 1990s and written in Pascal.

I don't know if any of you remember the Pascal language, but it is Long dead, we had 20 developers when DraftKings and FanDuel had over 200 each and they also had a dead platform that supported their online fantasy betting operations. You know for I think it had been almost a decade at that point so many of the components necessary for doing this we had Pascal right? So we had no platform and no people So how do you become a player when you're that far behind right the mission was to become a player, but obviously we were you know.

Way behind along with hundreds of other small players that now have exited the market, you know, and haven't made it. So how did we go from being way behind to being one of the major players in the market? Well, that's what I'm going to talk about here today.

So in 2007 working as a Dev manager for orbits, I took on a team doing our real-time analytics infrastructure was basically like working in the knock. I started to notice the same pattern there. That we had with our centralized database team which was you know, essentially don't let anybody change anything because if you do things will break and then your life is going to be hellish trying to fix it while getting blamed, you know, non-stop for everything that's going wrong while you're trying to do that.

So shortly thereafter. I heard about this New Concept called devops and I it immediately captivated me and I got involved, you know, I linked up with Jess humble and thought Works was in Chicago. So, you know Jazz was there a number of other people that works with their thinking about devops linked up with the company called drw trading and we put on I think the first devops Meet Up group in the country.

com forward slash devops, you know, no need for a location hyphenation needed. It wasn't devops Chicago. com for slash devops.

We're that. Um, we also put on a Citywide devops conference way back then. And you know, we really just brought a lot of people to the community just doing the excitement.

com back then because I wanted to blog on it and help Advance the cause and of course, I pushed the concept at orbits as well. You know, my passion for the topic was was tremendous. And I think what you know why I focused on this so much or what what captured me the most was this one problem and one solution and I think it was really first articulated by the devops community and the problem was local optimization.

And the solution was breaking down walls in service of Greater verticality and ownership. And I mean, that's really what devops is. And so I I really knew at my core.

This was going to be a world changer. If this wasn't gonna be a world change, I didn't know what was gonna be. So it turns out it was an industry Game Changer, you know, I think history is born that out for sure.

You're at this conference. But it was also a game changer for my thinking right. This talk isn't really about devops and I know that's a funny thing to say devops conference, but this talk is really about where the devops mind can lead you when abstract it out.

I think it applies all over the place and I'm gonna give you a place that applies, you know one domain but Also, I want to say the onset I'm gonna present you something here that is going to sound neat and perfect the way everything does in talks. It ain't it is messy but it's also wonderful and it's been a huge part of the formula that's allowed us to make crazy amounts of progress, you know coming from way behind to where we are right now. So let's fast forward a decade now pretend it's 2019 and I'm faced with the problem of making a dent in an industry.

We're way behind it. So what I applied to the problem was the result of a seed that was painted. So strongly back in the office with this whole devops journey that I talked about.

When Dev was able to own their software from Dev right into production and feel the pain of outage as well as get the praise for stability the level of developer engagement. Shut up. That engagement led to better ideas to more stable and performance software and Innovation up and down the value chain.

We went from seeing half a developers brain applied to problems to somewhere approaching the whole brain and it really comes down to this this simple formula that I want to share with you guys. Engagement, you know how how Focus you are in a problem is the product of the belief your belief that this problem needs to be solved or this opportunity needs to be captured. Plus or sorry it's the product of that belief and the likelihood of the outcome.

You know, how how much you feel you can affect the outcome that you want. So let me put this into something concrete. Let's take politics.

For example, many of you may feel that there are big things that need to change. You know from your perspective big things need to change in your your passionate about these changes, but you are out there doing anything about it because you don't feel that you have a really strong likelihood of affecting the outcome, right? There's not a great deal that you feel that you can do.

So when you have wow, I really think this needs to change that's a 10 and I don't think I can actually make a dent in the problem. That's a zero, then you're engagement is zero IE, you are not politically active, right? So maybe some of you are politically active.

That's great. But hopefully this illustrates the point for you, right? The engagement is the product of the belief and need and you're feeling that you can affect the outcome that you want.

So let's just talk about some of the components in that right so and specifically, you know, you can see these in the devops for me. Right? So the belief in need, right we need to do something about this feel the pain of an efforts failure or feel the glory of an effort success.

So, you know, one of the big devops changes was the developers should And feel the bugs at three in the morning, right when the system crashes they should be there dealing with that, you know, same thing on the success when we have great stability. They should get some you know, some of the glory what it used to be right was. Developers would build things and then Ops would feel the pain right they get woken up at three in the morning Ops couldn't do much about that right because they're not the ones writing the code and so you know, what what happened is they put up lots of walls and obstacles, right?

We change that and we allowed the devs to feel the pain of their own poorly written software and we'll talk a bit about what you know how Ops job changed a bit. But you know, that was a major a major shift now the likelihood of outcome. The degree or or influence Direct Control, you know, you kind of you know, you go in these this sort of a spectrum right the best thing for likelihood of outcomes.

You have direct control over the thing you're trying to to change right to solve the problem. You know at the worst it's nothing or you know, perhaps a vague and indirect influence which you know, I think many people feel, you know, if you go back to the political the political analogy there. Another one is do you believe that you're you have enough ability and knowledge to solve the problem, right that would influence her ability, you know your your assessment of whether or not you could you could change the outcome.

Yeah, and then another one is funny one here. The solution is your idea. If the solution is your idea you're gonna tend to be a bit more ego invested in it and you're gonna inflate it's probability of changing things.

Therefore work harder if it's successful, you're gonna feel the the glory of that and if it's a failure you're gonna feel the pain of it. So it also having the solution be your ideal. It's right back to that belief in need it basically believe that you need to you need to make this happen so that you don't you don't feel the pain of the failure.

So these are some of the things that influence, you know, the two variables in this equation. And they'll come back in a second here. So Devops, you know really align folks toward creating devops really aligned Ops folks toward creating the right rails for Dev and influencing uptime and stability and that way right creating the pipelines that devs would use to get things into you know into production.

This was something they could control the monitoring the alerting there's so many things that they could control that we aligned Ops with creating that substrate or the environment that then Ops was that then Dev was going to use to pipe software into production a Cloud's a great example of this too. Ops folks, we're excited about this as they should because now they have more direct control over, you know, influencing the goals that they have. Devops also aligned devs with creating not just new software but working that new software all the way into production.

So this is something they now own and can control and they were pumped to tackle that problem right? Because they they could They can their modifications to software directly influenced how many times they were woken up at three in the morning. So it's a very exciting problem for them to solve and you know, these groups have tackled it look where we are 50.

If you've been in the industry, you know for for 20 years, you know, look back 15 years ago at the state of software development and in the state of how quickly we can get things into production with with how many bugs I mean it's night and day right things have changed tremendously in terms of our ability to deliver value to customers rapidly and with quality. So let's abstract that out. So what this says is that if you allow people to directly experience the problem I in this case the pain of production crashes and then you give them the freedom and the access to make decisions that directly affect the outcome of those things.

They will be engaged. They will be creative. They will solve the problems at hand.

So if we go back to our formulas here, you know, you kind of you know that the effect of Engagement is Is feeling the problem plus the freedom to directly act on it, right? And it's you know, actually probably could phrase that as the product as well. But you know, in this case, it's just that's what you're trying to put together you want people to feel the problem and you want to you want the freedom for them to directly act on it, right?

That's what you're that's what you're trying to. Trying to pull together here. That's what I was trying to pull together in shaping the organization to tackle.

The problem that I talked about earlier how to Caesars move faster than our competitors, right? We need greater engagement. And the way that we're going to get the greater engagement is we're going to use these devops principles, but now writ large in the management of a whole organization.

We're gonna have people feel the problem more and we're gonna give them the freedom to directly act upon it. But you know that there's still a problem with this right because that's we're still in the individual place. Right?

And I just said we had a heavy organization do this. So. We need to move past kind of this individual feels the problem and has the freedom to act on it to how we get an organization to be more engaged.

So the formula needs a few more parts. So let's add those parts right now. So it's feel the problem.

Feel the freedom to direct have the freedom directly act upon it but you also need the big picture. And because we're dealing with a large organization, we need accountability, right if you have a whole bunch of people and they can't see the big picture. It's like cats right?

We're gonna be all going in different directions. We're not going to solve a single problem because we don't know what the single problem is. So that's really important.

Right? It can't be just feel all these different problems, you know without a connection. We need to connect all of what people are feeling in service of something larger and then because we're dealing with lots of people, you know, we need some accountability around this right?

It's still you know, we're still humans and we have human nature, right? So we need some accountability to make sure that we are going in the right direction that big picture dictates. It's also hopefully hopefully that makes sense, but we can simplify this picture just a little bit.

And take it down this so the organizational engagement. There's three ingredients. It's the big picture.

It's the freedom to directly act on it and it's transparency. and so you'll see we kind of derive this from you know from from the formula that we that you know, we had earlier so the big picture still the big picture. The transparency is going to take the place of of feeling the problem as well as providing accountability.

So that's where that's how we're able to simplify and then the freedom to directly act. We'll talk a little bit about you know about how that changes in the organizational context. So let's start with big picture.

The first thing is the vision or the strategy the big picture everyone in an organization needs. If you're going to follow these devops inspired principles. Everyone should have access to the big picture because that's the problem too.

Right? It's the problem statement. It's where are we going, you know all of us, right.

So people need to have access to it. They also need a feel it, right? So you have to spend a lot of time going over this right?

So got to be attached to the big picture, you know, which is really the strategy another way to say it is the strategy. Everybody should be in touch with the strategy at the end of the day strategy and tactics are the same thing strategy is tactics are just the things you do to accomplish the strategy, but if you have another if they this is your strategy and these are your tactics then one layer down if you have some tactics here, then these tactics just became strategy, right? So it's kind of this this layer right put another way.

Your tactics are there to accomplish your strategy, even if the strategy represents tactics to accomplish yet a larger strategy. So you have an organization and there are you know, there's sort of the highest level and then you have an organization. We then are broken up into a number of domains, right so you can think about betting and you know marketing, you know, marketing technology and and player account management.

These are these individual domains. They also have their own strategy have their own strategy which essentially our tactics to accomplish the bigger strategy. And then you go down from there, right?

So we first outline a broad strategy on an annual basis. The different domains, you know in the organization also engage in a quarterly strategic spread. So they Define their strategy that aims to propel us to the larger goals that we set our annually.

We don't change the strategies mid quarter. We stick to them this is to say we're mostly strategic. So 70% of our work is against the strategy 30% is just the stuff that comes up.

And we spend it in order amount of time making sure everyone knows the strategy and then we say it over and over again until people can really feel it. We also bring people in so they can mix with the creation of the strategy rights or create this big annual strategy, but it's abstract. It's it's just high level.

There's a lot that needs to be filled in IE tactics those tactics represent the strategy each quarter for the different domains that we're broken up into and then as you go below that you get into the agile process, which is going to take that stuff and decompose it into Sprint by Sprint commitments, right? And we go over this and we go over it and we go over it and we're constantly talking about it. So that everyone is connected from the bottom to the top on that strategy.

We spend a crazy amount of time on this. so let's move on from that and it'll kind of makes sense why we spend so much time on it why I'm harping on this big picture thing so much when you listen to the next part, so the next thing is the freedom to act directly on a problem right the freedom to act directly. So we don't have a lot of people telling our folks what to do.

We hire adults and then we treat people like adults if you're capable of getting a mortgage and and taking care of children. Then I think you're capable of you know acting with a great degree of autonomy at work. If we provided we give you the right things starting with the big picture.

So, you know if you think about this, right if the guy next to you says, hey stay in your lane. You can debate it. And if your argument fits the strategy better, then you have a good chance of convincing it.

If somebody two layers above you in the five layer hierarchy says stay in your lane, then that person might as well be a be a deity. You know what they say is unchallible almost literally, let's take a look at this here. So this is this is what you get.

Also I want to there's another problem with that hierarchy, right? So one is it's unchallengeable. and it kind of relates to this which is At the top, you've got the person in a traditional hierarchy with all the autonomy, right and they can do what they need to do and they have all this Freedom.

So what are they there engaged? They see the properties be solved. They have the freedom to act on it.

So their whole brain is engaged below you have people that have to ask permission of that person above right but they're pretty engaged right because that's the guy next so they see them all the time right or see her all the time. And so there's a decent dialogue and provide this person at the Top's not a you know, autocratic dictator type personality, you're good you build the relationship. Most of the brain is engaged at that level, but take it down a level and just in three layers, you know, you see you see quite a lot going away.

It's most organizations aren't three layers three layers great, you take it down another level and like people are just you know, they're like, they're just not engaged right? Because if the person two layers above you in the hierarchy says something you can't tap them on the shoulder and Hey, I think that's a bad idea. You have no relationship with this person.

You don't know what they're like if you offended me my fire you so you keep your mouth. Shut right you just go along. So none of the ideas really are your ideas because you're just trying to constrain your ideas to fit the personality of people that you've never met or deal with the dictates that they've handed down to you that you can't challenge.

Right? So your ability to think and own ideas and work directly on a problem is so socially constrained that you you know, you just stop feeling like you can really directly affect anything. You don't have a lot of autonomy.

You probably aren't very well connected to the strategy because you know who needs strategy. Why do you have to tell everybody the strategy when you can just tell your next level managers and they'll tell their next level managers and they'll just tell everybody what to do. So you don't know what the strategy is and you don't feel you have a great ability to to act upon it.

So naturally you're rather disengaged. This is what you get at most organizations. We can't have this right because we're behind we have to move faster.

So we need to give people the freedom to act directly. So the way that we solve for this is simply to limit the amount of authority in the company. So for you know, 500 plus people, they're really seven people with authority seven managers, you know in a traditional sense.

The rest is all peer-based. So these managers can't possibly deliver enough direction to get things done in the old-fashioned way. So they have to rely on everybody knowing the vision and they have to concern themselves with this as you know, one of their primary activities.

So let's move on then and talk about the third part of this which is transparency. So transparency is the final part of the equation. We all need a nudge sometimes and since we don't have the old manager hierarchy to rely on right the my manager told me to do it.

So I had to do it. We don't have that. Right.

So we use transparency to provide, you know, that motivation. Everyone knows the strategy and each quarter every domain is responsible for saying what they're going to do to move the needle. Those commitments are similar to Sprint commitments simply make them on a quarterly basis and they're higher level those commitments are made.

very very transparently and then we report on them at the highest level and at the Domain level as well and then agile takes care of things that the team level so we have this incredible transparency all the way through and you know, if you succeed you're gonna succeed publicly and visibly and it's gonna be fantastic and if you fail you're gonna fail publicly and visibly and you know, that's Rather nerve-racking, right? Nobody wants that. So there's there's you know, there's a lot of motivation there's sort of terroristic baked into it, but there's no person who's personality you have to deal with telling you what to do.

It's on you to want to be successful because look it's all out in the open and we're all we're all sharing in this thing. So if you think getting it, you know, everyone clear on the vision takes a long time and a lot of effort this takes five times more this transparency takes an incredible amount of discipline, but it's absolutely critical. If you don't have, you know, a massive hierarchy to rely on then you have to rely on vision.

And a transparent demonstration of progress if you're ever going to get to delivering results, but the benefit of this whole thing is that you know, the tremendous amount of Engagement that you get right people understand what the problems are. And they have a much greater ability to act directly on them. And you know this this is you know, we've done some amazing things at Caesars, you know, we've gotten bigger in this insanely competitive space many companies have spent millions and millions of dollars and now have one percent of market share and have, you know, exited the industry and there were, you know tens if not hundreds of people trying to, you know, come into this space.

You know and and we've made it even you know, as a way come from behind kind of a story. I think we are going to build a multi-billion Dollar business. We've got the You know, we've got the momentum and we've done all of this in the way that I've described above.

You know, you can go back to Q4 of 2019 and our Wiki and see our strategy and you can see how well we executed on it and you can walk that forward every quarter up until now and just see our path to getting from having no platform to you know, what we have today. So this isn't theoretical it's a real world example of how devops thinking can can influence even Beyond devops and and do some great, you know, great amazing things. I'm tremendously thankful for having you know, that sort of fateful day.

I bumped into this concept because it's has changed the director the trajectory of my entire career. I hope this was interesting for all of you. I hope you've learned something.

You know, I think I think this kind of thinking. Hopefully is useful for you to to think about the type of organization that you're in. Or to judge the kind of organization that you ultimately want to be in.

I think this stuff is is a game changer and I think you'll see more of it as the as the years go by. So thanks for joining and have a good rest of your comforts.