Having seen a couple of data driven disasters, I’ve gotten extremely suspicious of data driven business. I’ve ranted about this in some earlier comments, but it seems to very often be the case that the business “intelligence” people producing the data not only apply statistics wrong (things like taking averages of averages when the sets have very different sizes, or not taking into account how skewed the distribution of some variable is etc etc), but also tend to define “KPIs” so that they don’t necessarily measure what they believe they’re measuring in the first place. On top of all that, there’s also the fact that using “engagement” (however the hell they measure or mismeasure it in each particular case) as a key metric that you try to optimize most likely won’t lead you to build a better product as much as it’ll be a product that takes up more of the users’ time.