Big data tools are now ubiquitous and, compared to years ago, cheap. But while these tools give you the \u201cwhat\u201d \u2014 fire hydrants of data \u2014 they don\u2019t provide the \u201chow\u201d of using it.<\/p>\n\n\n\n
A gaming app developer, for example, can set up easy, out of the box tools to collect millions of data points daily on users\u2019 gameplay behaviors, viral activities, ad engagements and so on. But it is up to the developer to figure out which of this data is good, bad, or not useful to study in the first place.<\/p>\n\n\n\n
Below we walk through a useful decision flow, using our fictitious game developer as an example.<\/p>\n\n\n\n
Start with the right questions<\/strong> For our example, let\u2019s say our developer starts with asking: \u201cWhy don\u2019t my users come back frequently?\u201d<\/p>\n\n\n\n After you know where to start, dive deeper<\/strong> Developer: \u201cWhy do my users open the app for long 5-10 minute sessions and engage in game play, but still don\u2019t come back frequently?\u201d<\/p>\n\n\n\n Formulate hypothesis statements that can answer your questions<\/strong> Developer\u2019s starting hypothesis: \u201cUsers of my game don\u2019t come back frequently because they have few connections on the app.\u201d<\/p>\n\n\n\n Then attach an action for each outcome (true, false or other) to give yourself an immediate next step after validation.<\/p>\n\n\n\n Developer\u2019s final hypothesis: \u201cPlayers of my game don\u2019t come back frequently because they don\u2019t have enough active connections. If this is true, then we need a strategy to bring over their friends to the game. If my hypothesis is false, then I look at other possible reasons like poorly timed notifications.\u201d<\/p>\n\n\n\n Segment your users when analyzing data<\/strong> Developer: \u201cIt will be a lot easier to create a referral campaign for 18-25 year old iOS users living in the US, vs just targeting all iOS users.\u201d<\/p>\n\n\n\n Establish success metrics<\/strong> Developer: How do you differentiate well-engaged users from the rest? Is it when they come back ten times a week? Should you measure iOS and Android users against the same metric?<\/p>\n\n\n\n
Before any data is ever gathered, you can start asking questions about business performance. These questions act as blinders to help you focus on what\u2019s important. While it\u2019s also possible to decide what you want \u2014 50 percent day 1 retention, for instance \u2014 and try to find clues toward getting there in the data, this approach lacks the focus of a question and risks confirmation bias.<\/p>\n\n\n\n
For best results, be as specific with your questions as possible. \u201cWhy don\u2019t my users come back frequently\u201d offers too many potential directions for investigation. A specific question should use basic insights a developer already knows about.<\/p>\n\n\n\n
After finalizing your questions, form assumptions that can be proven or disproven.<\/p>\n\n\n\n
The smaller your cohort, the easier it is to formulate a plan of action based on their behaviour. Slice your data by signup region, age, or device type and pair with factors like number of connections or behaviour toward notifications.<\/p>\n\n\n\n
Without success metrics to measure data against, it\u2019s hard to decide if you should continue a strategy, or if it\u2019s time to recalibrate. It\u2019s also difficult to pinpoint which users fall under the cohort that you want to examine.<\/p>\n\n\n\n