5 Things that Make Me Suspicious of Morgan Stanley’s Report on Apps vs. Mobile Web

This all started as a 20-minute task to dig into a report on mobile web usage to summarise it for our monthly client report. It turned into a couple of hours of digging and a tweetstream.

This VentureBeat article was shared widely in the digital marketing space at the end of last week. It’s based on this Morgan Stanley report [PDF] which in turn is largely based on comScore data. The core claim is that, while apps consume ~87% of time spent on mobile devices, more people use mobile web properties than their associated apps, and the gap is widening as the number of unique visitors to mobile websites is growing faster than app usage. (The whole report is completely US-centric).

Unfortunately, I have issues with the ComScore data and the Morgan Stanley analysis. At this point I should probably clarify my position by saying that I want this to be true. I believe in the future of the web (let’s stop calling it the mobile web), and I believe there is a good chance that the web is still dominating discovery but I think it’s especially important that we look critically at things we want to believe.

I’d like more details on: 

Things that don’t pass a sniff test

I like to sense-check claims against a common-sense “is there any way that is true” filter. A couple of things failed to pass that filter:

1. Snapchat mobile web usage is almost half its app usage

Snapchat usage inconsistencies

A likely mistake in the data.

VentureBeat highlighted this as “frankly shocking”. I think it’s literally unbelievable considering that (unlike Instagram) there is nothing you can do on the Snapchat website apart from download the app. There’s no logging in, no profile pages, nothing. After digging a little, I believe this is a simple mistake in the Morgan Stanley data - probably a copy-paste error as the appendix shows that 16,978 is the value for Yahoo Mail while Snapchat should be 2,362:

appendix.png

The correct Snapchat data as shown in a different table.

Mistakes happen, but I’d like to see a bit more sense-checking. Incidentally, I still don’t believe Snapchat mobile web traffic at ~7% of active app users.

2. The analysis of music apps is strange

The Morgan Stanley report has only two music apps in the top 50 - Pandora and iTunes Music (for clarity - this is the native iOS music app - not just the Apple streaming service of the same name). I’m surprised by the lack of other services (Spotify?) - and also surprised that Google Play is missing considering that it’s present in the ComScore data.

(It’s possible that ComScore’s reference to “Google Play” is to the equivalent of the iTunes app store rather than to the equivalent of Apple Music / iTunes - in which case why isn’t iTunes / the app store listed? And why doesn’t Morgan Stanley include either of them?).

This also highlights a weirdness in the analysis of time spent in apps - what is the “time spent” in a music app? Is it the time with the app open and the screen unlocked? The time with music playing and the screen unlocked? All the time that music is playing even if the phone is in your pocket? I suspect it’s the latter if Pandora is the 4th biggest app by share of time, but that makes it weird that no other music app makes the top 20 - especially when Pandora could well be losing out to other platforms.

Some things are just confusing or unclear

3. There is no clarity on in-app browsing of the mobile web

When some reports have “in-app Safari” as the fastest-growing browser, passing 10% browser market share, it’s clearly important to know how Morgan Stanley is counting a Buzzfeed visit performed in the Facebook in-app browser - especially when you consider that Morgan Stanley is claiming a little under 60m unique visitors to Buzzfeed when the (directly measured) Quantcast data reports >93m uniques.

In the other direction, there’s no clarity on whether a click on a (web) URL that gets redirected or handled by an app deep-link counts as a mobile web visit or not.

4. Some of the statistics are confusing when you put them together

We are to believe that:

  • Time on mobile web is up 53% since 2013

  • Time in apps is up 90% in the same period

  • Yet "usage" (i.e. unique visitors) of the top mobile web properties is now 2.1x app usage (when it was 1.8x in 2013)

I think that the only conclusion we can draw from this is that app engagement is deepening - i.e. each individual user is spending around 1.45x as much time in each app as they used to - a conclusion that seems at odds with the main thrust of the argument.

5. There are skews in the way that the properties are selected

As far as I can tell from the appendix, the authors chose the properties to analyse by adding together the monthly unique users of the app and the monthly unique visitors to the mobile website, then taking the top 50 (though it’s not entirely clear as this table is ordered by “browser reach advantage”):

full-appendix.png

The top 50 US mobile properties with some unlikely omissions.

This has a couple of problems:

  1. A single user could be a “unique visitor” to a mobile website and a “unique user” of the associated app - which will tend to inflate the numbers for properties that are commonly used in both (like Google search)

  2. The long tail is way longer for the mobile web than for mobile apps, so while this may capture 90+% of total app usage across the whole market, it will capture a much lower proportion of total mobile web usage - with the next 20-30 biggest mobile websites additionally penalised by their lack of overlap with an app (see point #1)

Things we believe are probably true

As I said above, I am bullish on the web, and there are elements of all these studies that ring true. So here’s my subjective assessment of things that are probably true:

  1. Most users spend the vast majority of mobile time in apps

  2. There is much less of a long-tail in app usage than there is on the web - people visit more websites than the number of apps they use, and they probably visit more websites regularly than the number of apps they use regularly as well

  3. Nothing is yet beating the mobile web for discovery - people are overwhelmingly discovering mobile websites (sometimes in other apps) and then (perhaps) installing the app rather than getting to the app first

i.e. these stats all seem pretty believable:

In summary, I am positive about Google’s prospects (though I don’t hold any stock), and I believe in the future of the web, but I’m concerned by some of the details of the data, of the Morgan Stanley analysis, and the reporting thereof. It can get pretty confusing when we’re looking at time vs. unique users vs. unique visitors etc. and so I’d like to see some deeper analysis that can be presented in a clearer and more intuitive way.

I have emailed the authors of the Morgan Stanley report and the VentureBeat article. I’ll update this post if I hear back from them.

Get blog posts via email

About the author
Will Critchlow

Will Critchlow

Will founded Distilled with Duncan in 2005. Since then, he has consulted with some of the world’s largest organisations and most famous websites, spoken at most major industry events and regularly appeared in local and national press. For the...   read more