Mary Thengvall – Holding a Non-traditional Retrospective

The next presenter in our track is Mary Thengvall. Mary is director of developer relations at Camunda. Mary's presentation is titled Holding a Nontraditional Retrospective.

I'm really excited that Mary's here to share some of her experiences of what is a retrospective for the open source community and how your community retrospective should be conducted. Hi, everyone, my name is Mary Thengvall and I am the director of developer relations at Camuda. We are an open source workflow and process automation company.

And if you aren't familiar with the term developer relations, basically our focus is enabling developers to have the best possible experience with our products as well as our community. And part of that enablement involves intentionally gathering feedback from our community members and then following up with them regarding the feedback that they so graciously and sometimes not always so graciously given us. Which leads me to my topic today, holding retrospectives, but perhaps in a slightly different manner than what you've considered previously.

We're going to cover three things today. First, what is a retrospective? Second, how should a community retrospective be conducted since it is slightly different than our normal standard retros?

And thirdly, when the retrospective is over, what happens next? So, first of all, let's define the terms. If you've ever heard me give a talk before, you know, I feel strongly about this because there's no better way to make sure that everyone's on the same page than to walk through keywords.

And likewise, there's no quicker way to sink a successful meeting than to not clarify terminology. So what do I mean when I say retrospective? I pulled this definition from the Article Five Things You Need to Know to facilitate a retrospective by José Alberto Zimmerman.

And he says a retrospective as a ceremony held at the end of each iteration in an agile project. The general purpose is to allow the team as a group to evaluate its past working cycle. In addition, it's an important moment to gather feedback on what went well and what did not.

And retrospectives are traditionally held to evaluate, as it says, how a launch went, determine what went well and what didn't. At a recent conference or maybe recap a recent outage, it's an opportunity to talk through frustrating situations as well as celebrate jobs. Well done.

And I'd say I want to emphasize don't forget that last part, because retros are so often held only when there's a problem. But establishing a regular cadence gives us an opportunity to highlight and celebrate good experiences as well. But what if we use this retrospective model with an open source community?

If we encourage a safe and open environment to share candid opinions and experiences, a retrospective can be a really effective way to solicit feedback beyond the questions asked on the forum. By using tools like common boards, GitLab issue boards, for instance, you can organize these Post-it notes, if you will, into columns of what's going well, what's going poorly, and you can have this truly open and honest conversation with your most engaged community members. And as I mentioned a moment ago, this is really an opportunity for your community members to surface frustrating situations as well as celebrate things that they're really excited about.

So what's a little bit different about this experience now that we're hosting a retro outside of the traditional company audience? The first thing that you need to keep in mind is that it's public, not private. Everything is held out in the open Right.

And while this might be typical for some retrospectives, I know at GitLab, that is, for instance, at most companies, there's often an internal retrospective followed by an external what we call incident report, which contains similar information but slightly different, maybe slightly cleaned up. And there's definitely advantages here, right? Leaving everything out in the open, leaving it public can lead to a greater authenticity and deeper relationships with your community.

But it can also lead to some sometimes very uncomfortable situations, which requires this mantra that I use for developer relations, maybe even more often than usual. And its mantra is to the company I represent the community to the community. I represent the company.

I must have both of their interests in mind at all times. And when you're holding these public conversations, you have to keep in mind that our priority is both the community and the company. Right.

So being able to represent the company, things that we're excited about, things that we're interested in, the things that we need to be pursuing, perhaps because of investors or because of the things that the board is asking us to do, but also keep in mind what the community needs and what their solutions are that they're looking for the problems that they're running into, because without their success, our products won't be successful because we'll lose the people who are using our products if they aren't happy and kept up to date on what's going on. The second thing that we need to keep in mind with this open source version or community version of a retrospective is that follow through is almost even more important than usual. These discussions are now being held publicly, as we just discussed, which means you need to both be careful of what you're promising.

Can you actually promise a solution to that problem? Do you internally? In your company, hold the authority to make that promise, is that within your control, or are you better off saying you're going to communicate that feedback and come back with more information?

We also need to be careful of whether or not we have the time to make those promises. Do we actually have the bandwidth to do that thing right now, either me personally or the company? We also need to be conscious about coming back and being public with any updates in the future, ideally by updating the issues that were created during the retrospective, which we're going to cover in a minute.

By documenting everything, it means that even if folks weren't able to make the retrospective, they're able to see the progress as it's happening later on, which frees up your time down the road and keeps the community actively informed rather than them having to keep coming back to you seeking the answers. Lastly, a code of conduct and expectations must be enforced. There's no exceptions here.

This has to be established early as well as be blatantly clear to everyone joining later on. And this does less because we expect or anticipate any issues to happen and more to help set the expectations for behavior. What do we expect of our participants and in turn, what can they expect from us?

In short, taking a retrospective outside of its traditional audience means that it's even more important than usual to do all of the things you would do normally, right, to make sure that things are transparent, to make sure that items are documented, to make sure that you're following through and making sure that it's the expectations and guidelines are set early and often, including what the end goal of the session is. So before we jump into the how do we actually do this section, let's chat a little bit about why you might want to run this kind of retrospective now that you understand a little bit more of some of perhaps the pros and cons about doing it and things you have to keep in mind. So what's the advantage here?

The first thing is it encourages open and honest feedback. I've mentioned this a couple of times, but holding this retrospective publicly lets your audience know, let your community know that you're willing to receive feedback. But it also actively encourages people to bring their feedback.

It puts action behind your words. So it's no longer you just saying, hey, let us know if you have any questions or comments, but saying, please come bring your questions and comments. We want to have a conversation about it.

The second thing is it gives you a tangible way to keep track of feedback. I had a fascinating discussion on Twitter a couple of weeks ago with folks around how they normally get feedback about their open source projects. And for most folks, they're just receiving it as a one off from a community member, someone who happens to come by to submit a request or happens to make a comment on Twitter or leave an issue.

And these things aren't consistent and it's not tangible, which makes it really difficult to track. And so it's holding this retrospective lets us move beyond these one off contributions. The comments on an issue of forum post, the comments that you happen to get in conversations at those things called conference booths that we used to have in a previous life, and maybe we'll have again someday.

But having this regular cadence of community retrospectives gives you a way to really establish this regular line of communication and keep this open and honest feedback going throughout the time that you have this open source projects. So let's dig in a little bit. What does this actually mean?

How is this actually connected? So as you're preparing for this retrospective, there's a couple of questions that I encourage you to ask out front. First of all, who do you invite?

And this is a question to consider internally with your coworkers, as well as regarding external community members. Do you want to open it up to everybody in your community and say, hey, anybody and everybody who wants to come give us feedback, answer questions, ask questions, listen in on what's going on? You're more than welcome to join.

Do you want to limit it maybe to only people who have contributed? Or maybe you have a core group of maintainers that you want to invite to really limit it to them? Or maybe you want to have multiple types of retrospectives throughout the year, one with your maintainers, one with your volunteer group, one that's broader to the whole community.

But figuring out who you want to invite really helps to limit the conversations and get a clear end goal out of it. Who you invite internally is also important because this could be you if you're responsible. And the main tech lead on this open source project, it could be your product managers, right?

It might be your head of product. You might want to invite your Devrel team or maybe the developer relations team is the one who's running the retrospective if they're trying to actively get feedback. So deciding on who you invite is kind of a first step.

The second step here is what expectations do you have? What are the rules you want to have in place? I call is a working agreement and we'll go through these cards in a minute here.

But this working agreement really covers everything from what's the timeline to what are the roles that we have? How do we make sure that we hear from everybody? How do we expect people to respond to each other?

What kind of attitude do we expect people to bring? And this goes back to setting those expectations that I was talking about earlier, making sure that people know upfront what's expected of them and also what they can expect from you. Next is this issue of roles that I referred to quickly here, who is officially responsible to run this retrospective?

Who is responsible to be the main face of the retro? Who's responsible to take notes? Who is responsible to make sure that the action items are followed up on who's maybe responsible to keep an eye on all of the community members and see if someone's making a lot of facial expressions but not speaking up.

They have an opportunity to actually say what they're thinking. And last question you want to ask is, do you have the bandwidth to follow up on action items? I know I already mentioned this before.

We'll talk about it more later as well. But this is super important because if the answer to this question is no, then I would strongly suggest you either reconsider holding this retrospective at all or establish up front. Look, we're gathering feedback.

It's going to be a while before we have a chance to do anything with this feedback, but we're simply gathering it so that we have a better idea of where we should be going, a potential road map for the future, what we should be looking for down the road. So let's actually dig in a little bit. So the first thing that I would encourage you to do during the retro is go through these working agreement cards that I mentioned earlier.

And you'll see I have a screenshot here from my board, which I'll switch over to in a minute. But I've actually put these on the actual retrospective issue board. And this is very intentional because that way, if someone joins later after the rector has already started, they can still click through and view what's expected of them and the working agreement that everyone there has, at least in theory, agreed to.

So let me switch over to that really quick so you can see here I have an open source retrospective template. This is all public and live on GitLab. You are more than welcome to come in here, play around with it, leave comments, create issues, duplicate it for your own needs as well.

So like I said, I have this working agreement cards and these are all just issues that have been tagged with working agreement. So they show up in this column and you can click through on each of them. There's a couple of different things that I highlight specifically.

One, of course, radical candor, which might be a concept that you're familiar with. I know it's gotten some flack in the past as well, but the idea of it is we must care deeply and personally. Excuse me.

We must care personally and challenge directly in order to give effective feedback. And it's this idea of making sure that people can give the effective feedback during the retrospective, but that they're not being a jerk about it, that they're giving constructive feedback and constructive criticism and that the person receiving it is able to see it that way as well. So there's a few other things I have listed in this working agreement.

You can feel free to go through them later. Most of them are fairly basic things, like trying to minimize distractions. Right.

Especially in a day and age where we're engaging with each other over zoom closing down your slack before you hop on a retrospective like this, closing down email, letting your family know, hey, you know what? I am unavailable for the next hour or so while we're in this discussion so that you can really be fully attentive and fully engaged in this session while you're talking to your community members. Another thing I highlighted here is when criticizing or saying something negative focus on actions, not people.

That goes back to the radical candor idea as well. It's easier to discuss a concept or a project or an action that happened rather than saying, hey, John did this thing and was responsible for this part. And it was a miserable experience.

Rather than saying, hey, you know, I really didn't enjoy this particular event or this particular experience or this request wasn't handled in the best way that it could have been. So the second thing that I mentioned earlier is identifying roles and again, figuring out who is going to cover each of those roles, who's going to lead, who's going to moderate, who's going to speak up for the community members when they think that people aren't saying enough or that some people are saying too much, perhaps. And so sorting out who is handling those roles early on is important.

And I would encourage you to not take too much on at once. So often we try to cover all of the roles of like, well, I can run it and take the notes and then it turns out we can't talk and type at the same time. Or maybe that's just a problem I run into.

But I know it's easier when you spread out those responsibilities across the team. Next up is confirming the timeline. I have a simple timeline here, you don't necessarily have to follow this, but having a specific timeline and time boxing the different areas here is important to make sure that you're keeping the discussion going and making sure that you're actually able to talk about as much as you possibly can during this time.

So I encourage you to figure out whatever your time boxed area is and then divided equally so that you have enough time to talk through things you will know. One quick thing that I have more time allocated to discuss the three top what's not going well issues. I only have 14 minutes allocated to discuss the what's going well.

If it's not clear, the main reason for this is simply that we spend more time or we need to spend more time talking about what is not going well because those are usually more difficult conversations. And so making sure that you've allocated enough time to discuss the things that you need to talk through is an important part of that. So let me walk you through a few of the cards here, as you can tell.

I have these two different columns set up. I've got the what's going well column as well as the what's going poorly column. I usually encourage people to try and leave these issues in here prior to the actual retrospective.

You can notice in the timeline that there's not really time allocated to add extra cards. So if people haven't added cards, I would say make sure you allot time for that as well. But this could be everything.

And the what's going on? A column from my feedback was accepted and I'm super excited to new community manager is doing an awesome job. And then you've got things that are in the works going poorly.

So maybe you need to increase diversity in your community or the new onboarding tutorial needs some help. There's gaps in the getting started docs, things like that. So as the person who is running this retro, once you take a look, you can start reading through things, start reading, start with the what's going well column and read through each of the issues there and then invite people to vote.

And specifically using GitLab issue boards, folks can actually just go in here and add a thumbs up to this particular card and you vote because you might not have the time to cover everything that's in here. So then the person who's running the retro can then go through and see how many votes does my feedback was accepted. Have it has one no one voted on anything else?

It's not going to happen that way for your right, of course. But you can go through and see at a glance which cards have the most people voted on them to determine which items you're going to have a chance to talk to. You also might see at times that you have things that could be combined into one topic.

So, for instance, there's gaps in the getting started docs and the new onboarding tutorial has some errors, right? I would say check with whoever submitted these cards. Hopefully they're in the meeting and you can ask them really quick.

Hey, Sarah and Casey, do you think these two can be combined? Do they relate closely enough that we can have this as one topic? If they say, yes, I would come in here and create a new issue, maybe, let's say getting started.

Info needs some love and I'd submit the issue. You'll see it automatically has the label of what's going poorly again, because I created it in this column. You can also go in here and associate cards with it.

So the other two linked issues are number 11 and number 12. I can type just number 11 in here and associate that. It can also issue number 12.

Those are now linked to this one card, which means that I can go back to the main board. And simply archive of these two other issues so that when we go through here and do voting, people can vote on this, getting started in some love as a generic topic, but then can be broken into some of the more specific areas. And you can see some of the more specific issues and you can go through and close out these two issues that are related.

All right. So what's the retrospective is over. Let's say you had a fantastic conversation.

Everything was great. You're done right? Everything's good.

Not quite so there are going to be action items that come out of this retrospective, right, as you're having these conversations, as you're going through the issues that were raised, there's going to be things like, hey, we need to follow up on the fact that we need a more diverse community. How do we do that? That's not a conversation that can be solved in five minutes, obviously.

So maybe there's feedback that needs to be communicated to the community manager. Maybe there's feedback that needs to be communicated to the maintainers that, hey, maybe we need to do some more training on how to encourage a more diverse audience or be more inclusive with the way that we're welcoming new members. In order to get more people in here.

Maybe you need to update the documentation, for instance, with the getting started guides, maybe things need to be more up to date. Maybe it needs to be more content created. Maybe there's a meeting that is scheduled to sit down and discuss how do we actually make this a better experience, maybe bring in some newer community members to have them walk through the getting started guide and kind of point out some of the errors.

So as you're discussing these things, as you're figuring out what the follow up items are for these issues that you've identified, you can tag them with the label action items and these will actually automatically be moved to their own board, which is a feature that I love about GitLab issues that I discovered recently. So as you can see, I'm going to go back to our getting started in finding some love example here. So if we're talking about this, we realize the onboarding issue needs to be updated.

It's just there are some spots where it's wrong. There's also some gaps. We can go in here and edit the labels and add action items as a label here.

And you'll notice it's added here, which is great. It's also added to the board here, which is fantastic. But if I actually navigate over to my action items issue board that I have, it now shows up automatically on here as well because it has the action items label there.

So then after the meeting I can go in and say, great, getting started, Infonet, some love. We need to collect feedback. We need to schedule a meeting.

We need to create new content or update new content, update the docs. There's a whole bunch of things we might need to do, right. So I can now open this issue on this separate action items board that might be only internal or it might be public as well.

I would encourage you to have a public so that the community can track what the status is. B, you can say, hey, cool, we're going to need to update the docs because they're incorrect. We're also going to need to create content and then we might need to schedule a meeting to get some feedback from the community.

Maybe you add a label for needs review or something else like that as well. So when you go back to your action board here, you can see this issue now shows up in all of the appropriate columns for all of the action items that need to happen. All righty.

So in summary. A public retrospective with your open source community gives you a tangible way to connect with your community to generate feedback in a manner that really lets them know that you're listening and gives you this insight into a community that you often won't have otherwise. Second up is transparency is key to communicating when the writer is taking place, how people can take part.

What happens after the retro? What's the status of the items that were discussed? Do you have time to address those?

All of those things are really, really important to be transparent and open about communicate early and often. Third, I would say set expectations. What do you expect from community members?

What can they in turn expect from you? And I'd say don't be afraid to hold yourself and your community to them, holding yourself to things like time, boxing, those those conversations. Right.

And things like that is difficult. But maybe you've got a timekeeper on the call who has their phone set with the timer and has it right next to their microphone. And it turns into almost something a little bit silly, right?

We're like, nope, we stop now. Doesn't matter if you're mid word midsentence. Things are documented.

We move on. But making sure you actually hold yourself to those expectations so that people know that they can trust you when you say them. And then lastly, follow through.

Don't make promises you can't keep. And again, this is perhaps the most important part of the retrospective process, making sure that the items that you've flagged as action items are taken care of ideally before the next retrospective, which might be a couple of months out. It might be quarterly, but it may at least be updated with some sort of progress report and then move to the archives or maybe a closed call them if they're actually done.

And the board cleared for the next retro so that folks actually know that there's momentum happening there, that you're taking action based on the feedback that they've given, which then in turn encourages them to continue giving more feedback. And with that, thank you so much for your time today. I've linked my Twitter handle, which is probably the easiest way for you to get a hold of me.

My DMS are open. I also links the GitLab repo, as you can see I mentioned earlier. Feel free to use that for a template for you.

Feel free to leave comments and suggestions for me. I'm always happy to chat about these topics and hear how I can help. Thanks so much.