5 tips for designing highly effective mobile SLOs

Service-level objectives (SLOs) are a familiar concept to DevOps professionals and Site Reliability Engineers (SREs), as they are crucial for monitoring system health and sounding alarms when something is wrong. While SLOs have traditionally been the domain of backend engineering, their value is clear in helping mobile teams ensure high-performance apps and prioritize decisions between functionality and reliability.

However, for many organizations, mobile SLOs are a new and sometimes intimidating endeavor. But that doesn’t have to be the case. Any team can effectively implement SLOs as part of their mobile observability strategy by following a few best practices. Here are five tips for designing highly effective mobile SLOs.

1. Think in terms of end-to-end user experiences

For those with a DevOps background, it can be tempting to translate familiar concepts about endpoint availability, latency, etc. directly to mobile. But when building SLOs for mobile, you’ll need to change your mindset to look at end-user experiences as a whole. You want to build your SLOs around an end-to-end flow or activity that you are trying to optimize, such as a login or search process, rather than based on the individual technical components that enable the flow, such as screen displays. , API calls, etc. The technical actions are events within your SLO that, with the right tools, can be isolated when it’s time to troubleshoot a problem, but they shouldn’t be the ultimate focus.

2. Measure user impact metrics, not just incidents

Events happening on mobile devices can have an unexpected impact on your user base, both above and below what you might expect. That’s because mobile data is largely shaped by the concept of unique users and unique sessions, while backend data is not. For example, if you notice 1,000 instances of a certain type of error, how do you know how these instances are distributed among your users? Did 1,000 unique users each experience the error once, or did one unlucky user experience the error a thousand times?

If you only measure the number of incidents, it is impossible to know.

As a result, you may sound the alarm for SLO violations too strictly or too loosely. To really understand how to prioritize your response to SLO violations, you need to think in terms of both user numbers and event numbers.

3. Identify the user flows that have the greatest impact

Ultimately, the purpose of SLOs is to prioritize and direct technical work so that it benefits your business. Therefore, when considering which SLOs to develop for mobile, it is critical to identify the user flows that will have the greatest impact on your business so that you clearly understand why an SLO violation will force your team to prioritize that problem over other work.

Start with the most direct, obvious indicators of business impact. For example, if your customers cannot successfully checkout via the app, this will directly impact your sales. An issue with push notifications, on the other hand, can contribute to a gradual decline in sales, but it’s further down the sales funnel that a disruption to this functionality doesn’t have to mean your engineers have to drop everything to fix the problem.

4. Avoid sampling

One of the big challenges with mobile data is that there is a lot of it. You may be used to collecting data that is fed into backend SLOs to reduce data processing and storage costs. This makes sense: after all, you are dealing with a predictable environment consisting of a limited number of device types and other fairly stable variables.

But when it comes to mobile, these assumptions don’t hold. There are virtually endless permutations of device types, operating systems, app versions, network conditions, local infrastructure, etc. This means that sampling the data you input into SLOs almost guarantees that you’ll miss important visibility.

5. Define the population you really care about

How do you analyze a mountain of data with high cardinality if you don’t sample? And what does that mean in practical terms for your mobile SLOs? This quagmire can largely be solved by focusing too heavily on the populations you really care about, and in doing so you touch on the suggestion we made in Tip 2 regarding business objectives.

Consider which groups of all the people using your app are responsible for the majority of revenue. Depending on your business model, this could mean paying customers versus free trial users. Or they could be people using the latest version of your app, rather than laggards. Or it could even be those living in certain geographic markets that generate 80% of purchases in your app.

The point is that it’s impossible to always strive for a perfect experience for all users: you’d spend all your time on reliability and none of it on innovation. However, if you can isolate certain mission-critical audiences, you can fine-tune your mobile SLOs and resulting failure protocols to limit disruption to other important engineering work when reliability becomes an issue.

Continuously repeat and learn

There’s a lot more to it than just executing your SLO strategy, because every app is unique when it comes to its user base, product goals, and revenue structure. The above tips are applicable in almost all cases, but you should always consider your unique customer and business needs and map measurements accordingly.

One of the great things about SLOs is the ability to iterate, especially on mobile, where there aren’t yet “universal standards” or strict expectations. Don’t be afraid to measure and iterate again and again as you better understand what your app’s performance benchmarks are and what levels of failure your users are realistically willing to tolerate.

Maintaining strong app performance is a long-term effort, so consider SLOs as a tool to guide you along the way.

If you want an in-depth exploration of mobile SLOs, including more detailed best practices, examples and templates, download Embrace’s free Mobile SLO Guide.