Discrepancies: Why doesn't data always match up?
It’s an issue that arises on every single analytics platform - in one dashboard you have 5005 installs, on another, 7246. What’s the deal?
This issue is known as a ‘discrepancy’, and refers to the numerical differences between a metric on one platform to another.
Each dashboard will always have specific reasons why data does not always match up. Since ours is no different, this blog will provide you with the specifics on why discrepancies occur between us and everyone else. At first, we’ll look at common discrepancies that happen across platforms, before zoning in on issues only found on certain vendors, including Google, Apple and Facebook.
Below you’ll find common scenarios that lead to discrepancies. These are seen across all platforms, and are mainly due to the different ways we operate versus other systems.
1. Downloads vs. installs
There’s a fundamental difference between a download and an install, but often they’re compared, which can lead some to think that there are discrepancies between Adjust’s numbers and those on other platforms. Let’s look at what makes them incomparable.
Downloads represent when a user downloads an app from a store. An install is the event after a download, constituting the first time a user opens an app.
Adjust tracks installs, whereas store owners (Apple and Google) track both downloads and installs. We only look at installs for a couple of reasons. First, because we don’t have access to Google and Apple’s download data. As such, we gauge installs because this is information we measure ourselves.
Second, because of the way an SDK functions, it only starts on the first app open (essentially, it needs to be activated by the user). Once the SDK is fired for the first time, it signals our servers, letting us know that a new install has occurred. So, we only know when an app exists on a new device when the SDK tells us.
Discrepancies can occur from apps only being downloaded, but not opened. It also can affect the time of install, as Adjust records time of install on open, whereas Apple and Google will record the time of download.
With this issue, it’s a case of remembering that downloads on any platform do not equal installs on Adjust’s. Make sure you’re comparing install-to-install, not downloads-to-install.
2. User-based versus device-based installs
Both Apple and Google count their installs based on the store account of the user, whereas Adjust bases installs on individual advertising ID.
Examples of discrepancies that can occur are, for instance, when a user owns both an iPhone and an iPad and installs the same app on both. Here, we would count two installs because we are receiving data from two advertising IDs. Apple would count one, as the user has the same store account on each device.
3. Timezones and geolocation
The way Adjust works with the location of a user (and, as such, their timezone) is often different to the way several other platforms function.
With the location of users, Apple and Google base their data on the geolocation of the user's store account, while Adjust looks at the IP address of the user at the time of install.
So, if a user has a UK app store account but is in Germany when they install an app, then Apple and Google record the user's downloads/installs to the UK, while Adjust will attribute the install to Germany. As such, you’ll sometimes see differences between the two platforms on time of install and location.
When it comes to time zones, Adjust measures according to Coordinated Universal Time (known as UTC). Other platforms may differ by timezone (for instance, Google Adwords works on PST).
You can either change the timezone within the Adjust dashboard, or change the time zones of other platforms to match UTC. When it comes to geolocation of users, it depends what you want to look for. Whether it’s the location of the user’s store account, or the location of install, is up to what you’re wanting to discover from the metric.
4. App update effects
This discrepancy often affects newer clients who have had apps available to install for some time.
If an app did not first launch in the stores with our SDK, but added it later, all “old" users who update the app will be tracked as new users by Adjust. Apple and Google will merely note an update, but such users won’t be new to them.
This leads to a spike in the first couple of weeks of the update (potentially months depending on the age/popularity of the app) with a sharp decline towards more realistic numbers. If your app has a large user base prior to using Adjust, it will likely take longer for the effect to balance out.
If you have a dedicated Account Manager as part of your package, they will keep you aware of this when you first install the SDK, but it is something to keep in mind.
5. Third-party store installs
If you distribute your app (an APK with Adjust’s SDK integrated) beyond either the Google Play Store or App Store (for example, in a third party store) Adjust will count those installs. Apple and Google will not.
This often has a bigger effect on creating discrepancies for Android applications - where there are several competing stores beyond the Play Store. Ultimately, Adjust’s data creates a more holistic picture of all activity in this instance.
6. Comparing events
Google Adwords, Facebook and Adjust all use different attribution approaches for post-install events, which make the numbers on both sides not quite comparable:
- Google Adwords assigns events to the source of click, and has a 30-day event attribution window by default.
- Facebook assigns events to the source of click, and has a 28-day event attribution window by default.
- Adjust assigns events to the source of install or reattribution. We also don’t have an event attribution window. Instead, events are attributed to the user's source of install indefinitely (or up until the point the user may be reattributed.) From the point of reattribution, any subsequent events triggered by the user will be assigned to the source of reattribution.
Let’s look at an example - with Facebook, if a user installs via "Facebook Campaign A" and then clicks on an ad from "Facebook Campaign B", triggering an event, on Adjust all events will be attributed to "Facebook Campaign A" as its the source of install.
However, on Facebook, because they saw a click more recently from that user from "Facebook Campaign B" they would assign the event to "Facebook Campaign B". This can create discrepancies between campaigns on each platform.
This example applies in the same way to Adwords too.
When fraud causes discrepancies
Mobile fraud is a big focus of discrepancies - as it’s often the cause of them. Below you’ll find a summary of a couple of the biggest perpetrators of spoiled datasets.
1. Fake user accounts and jailbroken devices
Both official app stores filter out download and install activity from fake user accounts and jailbroken devices. On the Adjust side, we don’t have the same insight into what Apple and Google consider a fake account - though we do cover some of this with our Fraud Prevention Suite.
As we do device-based attribution, installs coming from potentially jailbroken devices (and from fake accounts) will be tracked by Adjust.
2. Incentivized campaigns: Resetting device IDs
Specifically on Android, if you’re running campaigns with incentivized traffic sources, certain users will attempt fraudulent behavior in order to avoid paying for in-app-purchases.
These users know the techniques to receive multiple credits/incentives - mainly by refreshing advertising IDs multiple times while re-installing and deleting the app over and over again. As a new advertising ID is produced, Adjust will count these as installs, though they are not recorded by Google.
3. In-app purchase fraud
If you’re with Adjust and tracking in-app purchases without any form of purchase receipt validation, you might occasionally get a surprise when comparing the revenue recorded on Adjust with the App Store/Play Store revenue. Fraudsters can use hacking applications that allow them to make free in-app purchases - to receive for example gems, coins, rewards etc without paying for them.
Utilizing a malicious piece of code installed within a jailbroken device, it’s possible to make free in-app purchases without purchase requests ever reaching the Google Play Store or Apple server.
Adjust offers clients to integrate our very own Purchase Verification SDK which verifies the purchases with the App Store/Play store in real time - fully reducing revenue discrepancies of this type.
We’ve conquered discrepancies that affect everyone - so it’s time to look at more platform specific issues. Facebook, Google and Apple all have different ways that they work with us, so we can go into more depth and look at why the numbers sometimes appear unequal.
The types of discrepancy that appear on Facebook can be broken into three sections: whether Facebook’s data shows more, less, and how re-engagement is calculated.
When Facebook shows more
Per default, Facebook measures on a 28-day post-click, 24-hour post-view basis. I know that’s a difficult sentence, but essentially these attribution settings can and should be modified in the Facebook interface to reach a better basis for comparison between the platforms. Per our default, we provide 7-day last click only for Facebook, not 28-day. This means the numbers include an extra three weeks of data by comparison.
To help solve this discrepancy, take note of the attribution window you have set on Adjust. For example, if you’re on business pro (or above) package, you can change to a maximum 30-day attribution window. Then, on Facebook’s reporting, you can choose which window you want to view. The data should refresh according to that window, and align more accurately with the Adjust data.
When Facebook shows less
Only a quick point here: on Facebook you’ll know that each ad account has a separate dashboard while Adjust aggregates the data for all ad accounts.
If your numbers are lower on Facebook, make sure that you check your reporting from every ad account, and combine them into one to see if there is a difference.
Facebook, Adjust and re-engagement
Facebook approaches re-engagement differently, at least to the way we do.
If you’re using Adjust to track re-engagement, you’ll know we operate with a user-based model. That means, one reattribution = one user. We count one reattribution when an existing user engages with an ad and re-opens (or is deep linked into) the app after being inactive for a specific period of time. By default, this is a seven day period.
So if an existing user is inactive for seven days, but then after this they click on a re-engagement ad in Facebook to re-open the app, Adjust will count one reattribution. If later this same user engages with the same ad again (and re-opens the app once more) this will not count as another reattribution, but instead will count as another session.
With Facebook, an engagement is counted based on event. If a user clicks on a re-engagement ad and engages in the app within 28 days (the event attribution window of Facebook), this will be counted as an engagement for this ad. If, later, the same user clicks on the same ad and engages in the app multiple times, Facebook will count multiple engagements.
Apple Search Ads
Apple visibility into last-click attribution
Apple doesn't know if a user clicked on an ad served by another partner in-between a click on an Apple Search Ad and an install.
In addition, a user could click on an ad from Apple and install an app but not open it. Then, later on, click a different ad from a different source and open the app afterward. In this case, we'd attribute the install to the last click, whereas Apple would claim the install happened from their ad.
Last click attribution windows
Adjust’s standard last-click attribution window is 7 Days, whereas Apple Search is fixed at 30.
Apple Search Ads reporting timezone is based on your account’s location. Whereas, as mentioned, Adjust reports in UTC.
If an existing user has uninstalled an app, but then later clicks on an Apple Search ad and re-installs the app, Apple will count this as a new install, whereas Adjust only counts this as a session.
At Adjust, we have a solution in place which utilizes an Adjust internal ID. This is to avoid counting a new install when users who have LAT switched on delete and re-install an app. We can instead count the re-install as a session. It is advisable to keep this in mind when comparing Adjust data with a platform that may not have such a method of device recognition in place, and instead counts re-installs from users with LAT switched on as new installs, leading to a discrepancy.
Limit ad tracking
When a user has enabled Limit Ad Tracking (LAT) on their device, we do not receive a response from Apple’s Attribution API, and so we’ll attribute the user as organic or to another source that has registered a click. However, Apple still claims these users and their statistics will reflect these installs.
It’s estimated that at least 12 percent of all iOS users have LAT enabled on their devices.
Data authorization (iOS)
iTunes Connect only reports the number of times that your app has been installed on an iOS device with iOS 8 or later, under the condition that the user has agreed to share their data with Apple. This means Apple will only take into account “opt-in only” user installs.
Typically we see that 20-30 percent of users do not wish to share that info with Apple, so Adjust would track 20-30 percent "more" installs than iTunes Connect, accounting for the difference.
Installs on Google Play Store versus installs on Adjust
If you’re looking at the "Installs" metric on the Google Play Store, remember that they include many variations on install metrics, none of which directly relate to Adjust. If you contact us we can work with you to find a way to directly compare the two, as this often needs a little more support.
Different attribution windows
Google Adwords works on a 30-day attribution window. Adjust’s attribution window is 7 days by default, though this can be changed depending on your type of account.
Comparing remarketing results of Adjust and Adwords is difficult, and something that we don’t recommend.
Adjust counts reattributions after a user (who has come through an Adwords remarketing campaign) has been brought back into an app via deep link reattribution. Meanwhile, Adwords uses events-based reattribution - just as technical, but different to the way we calculate.
As with remarketing, we don’t recommend comparing events either.
This is because Adwords has a 30-day event attribution window. As such, if they see a click on an ad within 30-days prior to the event, then they assign the event to the source of the click.
Specifically with events, Adjust assigns events to the source of the install (or reattribution) for the lifetime of the user.
Here, just keep in mind that we’re using different methods of event attribution that produce different results, and so it’s up to what you prefer to look at.
That covers the major causes of discrepancies between platforms. If discrepancies occur often - or if you have any other questions - please get in contact with our support team who’ll be able to help with your specific issues.