Introduction
The purpose of this page is to clarify how user identity tracking is done across sites and browsers/devices, what the different values and metrics returned from our system mean, how this knowledge can be used to explain the discrepancies from other analytics/identity tracking products, and eventually to cover the common pitfalls and misconceptions. Some of this is already covered by the existing documentation, in which case we will only provide a reference and a brief and convenient explanation. See also the existing page on retrieving user data.
By cross-site tracking we mean that we can say that a user visiting a site http://foo.com is the same user that visits another site http://bar.com . By cross-device tracking we mean that we can say that a person that uses physical device A (e.g., a mobile phone) is the same person that uses another physical device B (e.g., a laptop PC). Different browser instances on the same physical device will be perceived as being different devices.
Representing identities
A given user may have multiple identities assigned to him/her, by the same customer or by different customers. To disambiguate, an identity is therefore represented as a pair:
-
A type. This acts as a namespace for the actual value, to avoid collisions. For example, two users might both be referred to as user number 5 but in two different customer systems A and B, and the type helps us differentiate between these two cases. A Piano-assigned customer prefix is used to realize such namespaces for customers. The special type cx is used to refer to identity values issued by Piano.
-
A value. This is the handle that is typically used to refer to a user, but really needs the type to be properly disambiguated.
User privacy and identity types
Piano does not knowingly collect or store any personally identifiable information, and we assume all the identities and identity types described in the following are anonymous. (Customers that have custom identifiers that are not anonymous, e.g., email addresses, can apply a one-way hashing function to make them anonymous.)
There are three different types of identifiers:
-
1st and 3rd party cookies – set and used by our system to obtain a cross-site id.
-
Internal cross-site identities – generated by our system, map multiple cookies to one cross-site id.
-
External cross-device identities – supplied by a customer, map multiple cross-site ids to one cross-device id.
For simplicity, we may use "Audience user id", "global user id", "global id", "internal id" or just "user id" when talking about the internal cross-site identities, and "external user id", "external id", or "customer-specific user id" when talking about external cross-device identities.
Cross-site user identities
Our system operates with several 1st and 3rd party cookies, which in combination used to obtain an anonymous user identifier. More information on the different cookie types and their semantics can be found on this page, and Safari-specific workaround for domain isolation in Local Storage is described here. The important part is that matching even only some of these values allows us to track users across different sites, but not across different browsers and devices. A different device or a different browser installation used by the same physical person will mismatch on all of the previously known values, and therefore result in a different id. Thus, a Piano user id is a device or browser id and metrics generated from these count different devices or browsers rather than physical individuals.
Obtaining and using cross-site user ids
Piano Audience-specific ids can be returned from several of our API, for example /traffic/data and /dmp/traffic/data (the userId field). However, the value returned by cX.getUserId() from cx.js is rather a site-specific cookie id, which can be automatically resolved to a cross-site user id on lookup. For this reason, we allow both to be used under a common type 'cx' and leave the rest to be figured out by our system. Identities of this type are accepted by /profile/user, /profile/user/segment and Traffic API's user identifier filters as id under the type 'cx', and by /profile/user/external/link, /profile/user/external/link/update as cxid.
Cross-device user identities
In order to track users across browsers and devices, we allow our customers to define their own identity mappings (usually associated with a prefix associated with the customer). Identities can be mapped via cX.addExternalId() from cx.js or through /profile/user/external/link/update and the mappings can be read by /profile/user/external/link (it accepts a Piano-specific identity and returns an external identity for the given type). External ids can also be returned by several APIs such as /traffic/data, /dmp/traffic/data (externalUserIds) and /profile/user (identities), and used by Traffic API's user identifier filters. Moreover, for practical reasons, several APIs such as /traffic/user, /profile/user/external/read, /profile/user/external/update and /profile/user/external/delete only accept external user id types.
Counting users across sites, browsers and devices
While most of the Traffic and DMP APIs (e.g. /traffic and /dmp/traffic) support returning uniqueUsers, the returned value tells the number of distinct cross-site identities that satisfy the filters (including sites, period, etc.) defined for the request, rather than the number of physical users.
At the moment, there is no way to count external ids other than using /traffic/user (limits the number of external ids that can be returned) or /traffic/data and count the unique strings in a separate step. When using /traffic/data approach, implodeUsers filter can be used to remove most of the duplicate external ids (leaving only one per cross-site id).
Furthermore, discrepancies between our systems values and other vendors are explained here.
Common pitfalls
In the remainder we walk through several common pitfalls and misconceptions:
Unique external ids vs uniqueUsers
Because external identities map multiple cross-device ids to one, the uniqueUsers count returned to a /traffic request is often higher than the number of external ids returned by a corresponding /traffic/user request. In fact, the uniqueUsers values in the /traffic/user response tell exactly how many cross-site ids were mapped to each external id. Moreover, when the number of external ids is less than the cutoff limit, the sum of all uniqueUsers values from the /traffic/user response should be equal to the value returned by /traffic.
False positives due to identity mapping
Another problem may arise when /traffic/user is used to get external ids for users matching a negative filter, for example:
|
An examination of the returned ids, for example using /traffic/event with"groups":"city" and the user identity filter, may reveal that some of the corresponding users have nevertheless been to Barcelona. The reason for this is that one of the cross-site id the external id is mapped to matches the predicate while others not (for example, the user has traveled with his tablet while his laptop stayed at home), but the result from /traffic/user contains the common identity for both. In this example, if we carefully examine the output of a corresponding request to /traffic/data, we can see that although the externalUserIds will be the same, the userId of such "incorrect" events (i.e., the false positives) will be the one associated with the laptop.
User filters, explodeUsers and implodeUsers
At the moment, external identity mappings are not considered by explodeUsers and implodeUsers filters (i.e., we get all events or only first/last event associated with the same browser/device without consideration to external identities), while the user filter considers only one external identity type (i.e., we cannot specify user with one external identity type and then expand it by another).