May 4, 2023 4 min read oncall

On-Call Feels Survey

drawing of a girl/lit taper candle melting into a puddle

Need to get a pulse check on the on-call experience?

I mentioned an "on-call feels" survey in Avoiding Alert Bankruptcy and Burnout, my #WTFisSRE talk.

My preferred method is to conduct check-ins and retros with every team (ideally with time for 1:1 convos as needed) however a survey is a totally scalable option to tap into the lived experience of on-call in your org. This "anec-data" is powerful when synthesized with the raw numerical data you can gather from your pager, monitoring/observability, and ticketing systems.

I can hear you thinking

"...a survey...no one ever answers those"

to which I say

"Tell me your normal survey process. I'll wait!"

You cannot just slap together a Google Form/Polly, announce it one time in a single Slack channel and lament at the shitty response rate, k? (even if you leverage the incredibly well-crafted questions below 😜)

Tenents of successful surveys I've run are

Announce & Publicize the why

Treat this like any other program or initiative - send an email to @eng, drop that @here in the #oncall Slack channel, announce at an all-hands, pop by stand-ups and remind folks not only when the survey closes but why you're running the survey and how you plan to use the results

(the why should be something along the lines of "we're using this survey combined with on-call metrics to understand the priority of issues to tackle. bringing data to leadership will help them understand the risks of not acting" or something lol business it up!)

Anonymous

Being pragmatic I understand that if you're at the point of alert fatigue and burnout that has been dragging on forever and leadership is "aware" but not acting - on-call emotions can rightfully run hot. This is why I set the default for form responses to be anonymous. Would it be helpful to slice and dice by team, direct manager, role, etc? Yes of course.

Ask yourself if you think there would be a self-censoring to the candor in responses and if management would act punitively towards folks who answered in spicy ways. It is a risk that I personally don't think is worth taking. Besides if you are concerned about a particular team or individual...reach out to them and book a 1:1!

The Open Text Box

Every question with a text box should be optional. In my opinion the goal here is to get as many responses and perspectives as possible and there are some people who will get 75% of the way through a survey and bounce when they get to that text box. There are also a special set of people that will craft an entire novel in the small text box. That's good! I mean clearly they have some strong feelings and getting to express them is good! (obvi it is not good if folks release a diatribe and are rude and unprofessional in their response but I find by openly sharing survey responses people correct themselves quick)

You Are (Probably) Not a Sociologist...So Just Use the Likert Scale

The Likert Scale is widely used in sociology and involves a 5 point answer system to specify the level to which a respondent agrees or disagrees with a statement.

You could spend days and days digging into the most effective ways to interview and survey a population but you've got other things to do so it's smart to go with what works here.

The Final Question

This is a freebie you can use in any survey or when seeking feedback! Leave a long-form text box and prompt engineers with "Was there anything you wish was asked? Please share."

For the longest time I would excitedly point people to a spreadsheet or the raw survey results only to be met with "oh interesting" or radio silence.

What I eventually realized is that not everybody reads data the same way I do or uncovers the same insights.

To bridge this I'd run through the high level insights I gleaned and what conclusions/findings I was drawing from them first with my SRE team. This opened the door for teammates to either agree, point out a different angle to consider, or find interesting data that I had totally missed. The point was it opened the door to a rich discussion in a way that simply pointing people to the data wasn't doing.

After a team run-through I'd summarize the main takeaways, make some pretty and informative charts, and do a quick preso in an engineering forum or eng all-hands.

Something I didn't do but would also find interesting is presenting this to EM's (engineering managers) and Directors and discuss.

^ you want to do this within 1-2w of closing the survey, if you wait too long people will wonder why they bothered to take the survey and feel like it was a waste of their time and lose some faith in the initiative.