An Agile SRE Meeting Plan
Engineers dislike meetings. What engineers really dislike are meetings for which they perceive no value. Below is described a meeting plan developed, iterated upon, and used over many years at multiple companies that has proven very effective to both maximize meeting value and minimize unnecessary time in meetings, so that engineers may do what many enjoy most, building things. This may not be the perfect plan for your organization, but will hopefully inspire conversation and discussion about how to structure the time of your SRE team.
The Structure
Over the years, I’ve experienced many different Agile implementations. Scrum is considered to be a pretty poor match for interrupt driven teams like Site Reliability Engineers (SREs), but how to get the agile benefits of Kanban and still retain many of the advantages of Scrum? How to have a schedule that is relatively light on meetings, but still keep the maximum amount of communication and transparency? How to continue to be agile, instead of just doing agile, especially with distributed teams?
In the plan outlined herein, we try to balance many of those things. We lay them out as Monday to Friday, but they could certainly be Tuesday to Tuesday, or whatever fits best to line up with the development team’s sprints. (Protip: line up as best you can with the development team sprints) When we first embarked on this path, we were on two week iterations, to match what the dev teams were doing. Over time, we discovered that we lacked the responsiveness we wanted to provide to those teams, and thus switched over to one week iterations, in order to maximize (internal) customer satisfaction.
The iteration starts off a bit meeting heavy to emphasize alignment, and then allows for plenty of time for our standard SRE work (elimination of toil, etc.), and finally closes the iteration with time to reflect and improve.
The Meetings
Iteration Planning
Purpose
The iteration planning meeting is, well, just what it sounds like, planning the iteration. Because SRE teams can be interrupt driven and use Kanban, iteration planning is not for committing to delivering specific work in a time box (like Scrum). Instead, it’s for making sure that the entire team is on the same page in terms of priorities, needs of the business, project work assignment (e.g. Anne really wants to work on the VPC network project), dependencies on other teams, urgency of different tasks, and looking forward toward the next few weeks and what work may be taken on.
This is really a time for discussion, and for identifying things that will require more in depth discussion. It is not a time for going deep on any particular task, but to make sure that everyone on the team is aligned for the next batch of work. Oftentimes, work from the previous iteration will simply continue in the current one, but this is also a good time to check in that the work is being delivered to expectations especially as a result of the demos (which we’ll get to later), that closed out the previous iteration.
Because we want to be agile, instead of simply doing Agile, this doesn’t mean that once work is agreed to that it’s set in stone for a week. It just means that we don’t want to oscillate wildly from day to day, or even week to week, and the iteration planning meeting is the opportunity to ensure the team is moving in the same direction simultaneously. Because we are “walking the wall” while negotiating tasks, this is also a great time to recognize blocked work and any of the “time thieves” Dominca Degrandis (@dominicad) describes in her book Making Work Visible.
At the end of this meeting, the team leader should have worked out with the team a balance of work being requested by other parts of the business and work proposed by the team itself. Some iterations, other team’s requests will occupy more time, some iterations the needs of the team will take priority. A successful manager will navigate the balance between the two ensuring that the needs of the business are being met while simultaneously allowing the team to reduce toil, perform chaos engineering experiments, collaborate with other teams, etc.
Mechanics
The iteration planning meeting should begin with an already prioritized Kanban board. The team can negotiate changes to the priorities during the meeting, but this is not a time for debating what it is that the business values. Depending on the size of the team, and the amount of discussion needed of specific tasks, this meeting should take no more than one hour. Past this point, engineers will lose focus and interest and “just want to start getting things done”.
- Holdover discussions from previous iteration
- Explanation of the top priorities from the business
- Explanation of the top priorities from the team
- Identification of merged prioritization
- Identification of resources working or interested in tasks
- Assignment of work if no resource volunteering for necessary work item
- Parking lot
Daily Standup
Purpose
The daily standup has the same purpose as it does in Scrum. To keep the team in close alignment with respect to deliverables and identifying any items that require assistance from other team members or management in order to keep the team operating at its highest velocity.
This plan only has standup during the middle of the iteration because the beginning and end already have time for team discussion with the iteration planning and iteration review meetings.
The daily standup should be restricted to the work at hand and not devolve into in-depth discussions on specific tasks which extend the meeting and hold the rest of the team hostage for the duration of the discussion.
Mechanics
- What did I do yesterday, what am I working on currently, identification of blockers. (for each team member)
- Parking lot
If working with distributed teams, one might want to allow the standup to extend out to a full 30 minutes to allow the team to socialize with one another and therefore build some of the bonds you would otherwise get by colocation. In that case, the standup should conclude as soon as parking lot is complete.
Inter-team Sync
Purpose
If good intra-team communication can be difficult to do well, then good inter-team communication can be even more difficult. Couple this with dependencies between teams, and one can easily see the need to set aside an agenda specifically for this purpose.
The Inter-team Sync is to ensure close coordination and transparency between the SRE team and their primary customer. We don’t want to fill each iteration with sync meetings between the SRE team and any customer they may have as that number may be very large and frequent context switching is a major impediment to delivery of work. But, the team that works most closely with the SRE team should have a short meeting to discuss work in progress, dependencies, upcoming projects, etc. In this way, we attempt to ensure that both teams are working at the maximum safe velocity and minimize misunderstandings and the conflicting priorities and unknown dependencies time thieves (see Degrandis above).
The Inter-team Sync is how DevOps is done!
Mechanics
The Inter-team sync meeting should be no longer than 30 minutes. Any discussions of deep architectural questions should be put on the agenda for the Architecture Meeting. There should be an agenda (we like Kanban boards for this purpose) created and maintained by the team leads or managers, that is widely available, which anyone can contribute to, that tracks all the work items shared between the two teams, especially dependencies.
The person who runs the meeting simply “walks the wall” until there are no more items to sync upon and then ends the meeting.
Architecture “Arch” Meeting
Purpose
If the iteration planning meeting and the daily standup are not the place for in-depth discussion, then the weekly arch meeting is exactly the place for such discussion. This is the forum for any deep technical discussions on the SRE team. This is also a forum where members from other teams can either be invited or be regular attendees to give guidance, ask questions, provide clarification, etc. of work with which the SRE team is tasked. In other words, DevOps!
The outcomes or inputs to the arch meeting are often technical specifications, diagrams, documentation, requirements documents, and experiments. This can be a time for senior staff to give feedback on proposals to other, both senior and junior, members of the team. This can be a time to solicit opinions from the group on a new or existing technology or to review past postmortems. This can be a time for helping to figure out how to navigate toward a long term goal. The opportunities are very wide open (by design), but the goal should be that by the end of each arch meeting, the entire team should have taken a step forward toward achieving the goals of the team and of the business.
The number of times I’ve heard the phrase “let’s table this and add it to the agenda for the arch meeting” over the years, are far too numerous to be counted. This is another opportunity for the team to ensure they are highly aligned as they move into the meat of the iteration.
Mechanics
The agenda for the architecture meeting tends to build itself over the course of the previous iterations. We always use a simple kanban board or Google Doc for keeping track of proposed topics. The person running the meeting can cover each topic in turn or it can be run Lean Coffee style or if someone has an especially important topic for discussion, that can be moved to the beginning of the meeting or the end (to allow more time for open ended discussion). It is really up to the attendees to determine which best suits the style of the teams involved.
Demos/Retros
Purpose
Students of Gene Kim’s Three Ways of DevOps know that the 2nd way is all about feedback. In order for this to be successful, we need to set aside time in our week (in the form of a meeting) to specifically enable that feedback to occur. The Demo/Retro meeting has two purposes:
- Have the team demo the work they accomplished (not necessarily completed) during the iteration.
- Have a retrospective to discuss how to improve the team in a psychologically safe environment (see re:Work)
The demo allows the team to get fast feedback on work they have completed or is already in progress. There is a saying in Agile, “maximize the work not done”, which reminds us to spend our time on work that is critical to our success. If someone is delivering a project that does not meet our needs, we’d like to give them that feedback before they finish the project so they can adjust course, not after all that work is complete. The bar for a demo is extremely low; unit tests, working demos, command line tools, a single API call are all acceptable demos. The point isn’t to dazzle, the point is to demonstrate working code.
The retrospective (retro) gives the team the space to improve in the kaizen fashion. We follow the traditional retrospective format (what went well, what did not go well, what could be better) with some modifications. The goal is for each team to be higher performing at the end of every year than they were at its start. By setting aside a safe space for the team to talk about how the iteration went for them, and how the team can improve, we are creating an environment that fosters and encourages that improvement.
Mechanics
The demo part of the meeting should be open to all. Any stakeholder that wishes to participate should be able to attend. Borrowing from a technique I developed with Greg Oehman at Salesforce, we always record the demos and post them somewhere afterwards (wiki, Google Drive, etc.) so that anyone who was not able to attend will be able to see the demos. This is critical if you have a globally distributed organization where time zones make attendance for all a challenge. However, the feedback from those folks can be invaluable to making sure we deliver the right work on time. Again, we’re trying to create an environment that maximizes transparency.
In the retrospective part of the meeting, only the team members should participate (in Agile terms, only the pigs). There should be no executives or project managers attending this part of the meeting. It is strictly for those who need a psychologically safe space to have open and honest conversation in order to move the team forward, or discuss problems without any fear of retribution or interference. Team members fill out a shared document or perhaps their own document with their thoughts about the iteration (what went well, what did not go well, what could be better) . Then each member in turn has an opportunity to read their contribution and explain in greater detail so that they know that they have been heard. During this section of the meeting, clarifying questions can be asked, Arch meeting agenda items can be added, etc.
When holding the demo/retro at the end of a week, we like to have the team spend the rest of the day working on documentation, testing in staging, development, etc. Basically anything that does not touch production before a weekend.
Conclusion
Finding a cadence upon which to work as an engineer can be difficult. As engineers are generally averse to meetings, oftentimes we wind up with sporadic meetings and a lot of people who are unclear on their priorities and goals. On the other side, we can find ourselves in environments that are extremely meeting heavy, and engineers often left wondering when there will be time to actually do the work they believed they were hired to do. The establishment of only necessary meetings, at specifically defined times, allows engineers to plan their time to minimize context switching, and and to maximize the time invested in their meetings with one another.
This plan is certainly not a one-size-fits-all solution, but is deliberately broad and flexible to allow modification to fit into your organization, while being prescriptive enough about the purpose of each interaction to allow for different implementations that accomplish the same goals, namely: transparency, collaboration, agility, and effectiveness.
I hope you are able to use it to advance the capabilities and success of your SRE teams.
Thanks
The thinking demonstrated in this plan has evolved and will continue to evolve over the years. There is no way it would have been possible without specific input from many trusted friends, and co-workers. I’m in debt to Evan Wiley (@absoludicrous), Peter Haggery, Peter Norton, the SFDC ISD team, Jeff Frasca, SWI Cloud SREs, Jathan McCullum, Mauricette Forzano, Eric Rapin, Stuart McCulla, Kelly Courier, and John Irwin.