Outline

First, I'm going to make sure I understand the metric change in more detail and more specifically. Second, I'll define a framework that covers all the potential categories of factors influencing the change. Third, I'll generate hypotheses within those categories of what could have caused the change, fourth, I'll talk about how to test some of those hypotheses, and finally, fifth, I'll give the overall findings that I would present. Does that make sense?

1. Understanding

Okay so I'll get started with the first step, which is making sure that I fully understand the metric change by asking a few questions.

• What is the exact definition of the metric that we're talking about? (e.g. what does "engagement" mean here? # of sessions or likes + comments + shares?)
• Over what time period did the metric change? A week, month, quarter, year? When did it start and how long did it last?
  • Was it gradual or sudden?
• Did the metric impact all segments equally? Or was a certain geography, device type, size of user, etc. impacted?
• How important is this metric? Is it something that we actually care about?

Segments: Geography (country, city, neighborhood, rural vs. urban), device, user age (bad), account age, account activity level, language, education,

Notes

2. Define Framework

Now I'll move on to step 2, defining a framework. I think the best way to go about finding the source of this change is to look at it in terms of internal and external factors: the cause MUST be within external or internal factors, since they cover everything that affects Facebook. Does this make sense for going forward?