Clickstream data: The authoritative guide for data buyers and sellers

There are big bucks in clicks.

Clickstream data provides insight into web traffic and an individual’s online user journey which can be used to help understand consumer behaviour.

Consumer habits, conversion rates, search activity… Even the likelihood of a company going bust can be assessed using clickstream data.

Its range of applications means clickstream is now one of the biggest areas of alternative data. It has become a vital tool for users ranging from professional investors like hedge funds and other alternative asset managers to traditional investors, consultancy firms and corporations.

With a global research team dedicated to finding novel sources of clickstream data and a platform hosting products from multiple vendors, Neudata is in a unique position to help institutional investors, corporate clients and leading global organisations.

Read on to learn about the different types of clickstream data, which types of vendors supply it, the shape of the market, how the data is used and possible issues to bear in mind.

Key takeaways include:

• What is clickstream data?
• How is clickstream data collected?
• How is it used for competitive intelligence and deciding investments?
• Are there any compliance considerations to be aware of when using clickstream data?
• What is the shape of the market?
• How can Neudata help?

What is clickstream data?

Clickstream data is derived from online activity across mobile and desktop devices. It is commonly divided into two types: aggregated and unaggregated.

Aggregated data, also referred to as web traffic data, provides insight into the overall visits to a domain or sub-domain at a given period. Unaggregated clickstream data shows the click path of an individual user.

Here are a couple of very basic use cases showing the different ways the two types of clickstream data could be used.

Aggregated: The web traffic of competing businesses could be compared to assess which website is attracting more visitors over a specific time-frame.

Unaggregated: The online journey of a user on a specific website could be compared across different websites, showing how they navigate sites, which pages they visit before making buying decisions and other findings.

How is it collected?

Larger sellers of clickstream data partner with website publishers directly to collect their internal domain traffic statistics.

Most of the market, however, gathers the majority of this data from ‘consumer panels’ — a phrase with a wide range of meanings, from the use of third-party cookies to agreements with mobile network operators, to browser extensions installed in exchange for a fee.

There are several drawbacks to panel-derived consumer data. First and foremost, clickstream dataset panel sizes may be small when the vendor doesn't offer a value proposition that's compelling enough to convince a user to share its data.

If a dataset is too small, it will be little use to end-users.

Churn rate, panel skew and stability are other potential problems. Not familiar with the jargon? Neudata’s analyst team of subject matter experts can help you understand the pros and cons.

How is it used?

Competitive intelligence

Many companies with their own websites use clickstream data to understand the behaviour of their own consumers. But it can be even more valuable to use similar data to understand the performance of a competitor’s site.

Vendors on the Neudata platform can help companies with questions like:

•        How do certain products and brands perform across different online merchants?
•        What was the overall volume of sales for a competitor’s ecommerce site?
•        What are the top searched keywords on Amazon on Black Friday?

Investment decisions

Hedge funds and other investors will use clickstream data to understand and compare the performance of companies in different markets.

Investments with millions of dollars at stake are made as a result — the high stakes mean there are big research budgets for this form of alternative data.

Here are some examples of investment use cases:

• Clickstream data can provide signals on opportunities and risks facing companies. In an example from 2019, one data provider used clickstream data to spot that a growing number of individuals at Tesla facilities were searching for “the WARN Act”, giving a clue in advance about potential layoffs. More than 1,000 staff members were let go, meaning the data had provided a strong signal.
• One data provider uses clickstream data to help it gauge the insolvency probability of thousands of global companies. Hedge fund managers specialising in distressed investing are one set of potential users for this data.
• Black Friday sales often serve as a barometer for retailers’ performance throughout the rest of the holiday season. Investors can use clickstream datasets relating to spending and retail strategy — from page visits and search activity to, in some cases, the specific items they purchase — to derive leading indicators for a wide range of retail stocks.

Are there problems with clickstream data?

As highlighted above, users should note possible shortcomings to clickstream data derived from ‘consumer panels’.

Clickstream is also a data type which has a reputation for being expensive, which could impact its uptake by some buyers.

The biggest potential problems surround compliance.

Clickstream data is an area which may face further scrutiny from regulators, particularly the Securities and Exchange Commission, due to risks around consents and valid collection methods, especially where data is collected from many sources.

The first SEC enforcement action focused on alternative data was against a data vendor collecting mobile app and clickstream data. App Annie (now data.ai) and its founder were charged with securities fraud over how it used data in one part of its business.

The case seemed a one-off and wasn't a commentary on the clickstream industry at large, but certainly acted as a reminder that regulators are looking at this area.

Neudata Sentry, an alternative data-focused legal and compliance intelligence tool, has highlighted that the barriers to entry are low for smaller clickstream vendors, opening the door for loose compliance practices — even if clients of these vendors have sophisticated internal controls.

The risks are hypothetical but important to bear in mind — and there have been several examples of compliance shortcomings in this market.

The Jumpshot episode

One of the leading companies in the clickstream market, Jumpshot provided data split at the user level.

It was closed down in 2020, months after its granular user-level clickstream was found to include personal identifiable information (PII) collected through web browser extensions.

The Washington Post published an article entitled “I found your data. It’s for sale” and the clickstream market faced a new era of scrutiny.

As we have seen, there may be more regulatory focus ahead, meaning it is important for end-users to have strong compliance and internal controls in place.

Quant implications

For reasons of compliance and methodology, the market leaders in this space tend to be providers of aggregated datasets.

But it should be noted that aggregated web traffic is not very quant-friendly — the data does not typically directly exhibit a strong correlation with financial returns, which makes the data difficult to use in systematic investment use cases.

When it comes to investment decisions, the end-users are likely to be discretionary — i.e. human traders — rather than quant or machine-driven strategies which trade using algorithms.

How big is the market?

As of March 2023, Neudata has identified more than 60 clickstream datasets from more than 50 unique vendors. The numbers almost double when you search for “web traffic” though not all of these vendors will solely utilise a clickstream approach.

The largest provider in the space provides aggregated web traffic data for 100m domains and 2.5m apps — giving a sense of the potential size of a vendor’s offering.

Providers are split between those providing aggregated and unaggregated data, and approaches differ widely. It can be a complicated field for the first-time user of clickstream data to navigate.

In addition, it should be noted that most specialists do not offer an “oven-ready” off-the-shelf dataset.

In our experience, vendors that can offer aggregated data at the domain/subdomain level for desktop and mobile devices tend to perform best with data buyers.

How can Neudata help?

Neudata is an alternative data-focused research platform that specialises in the objective and neutral assessment of data vendors and datasets. We connect institutional investors, corporations and leading global organisations to the most relevant alternative data sources that are used in their internal data ingestion processes.

When it comes to clickstream data, Neudata offers a traffic light rating which measures the likelihood that readers can leverage the vendor’s offering, based on our analysis of methodology, compliance protocols, data structure and use case flexibility.

Our platform is the global authoritative source for unbiased, independent alternative data intelligence.

We don’t buy or sell data, or require data providers to pay us a revenue-share or commission in exchange for recommending their products to data buyers. That means you get unbiased intelligence that’s tailored to your specific research goals and strategies.

Neudata users leverage insights from our research experts based in London, New York and Shanghai. We can also advise on data scouting.

Since 2016, we have helped our clients understand the landscape of available datasets, increasing the efficiency of their data spending budgets. Neudata’s data buyer clients represent 60-70% of industry-wide spending on alternative data.

If you have clickstream data and want advice on monetising it, or if you're looking to buy this type of data, please reach out to info@neudata.co to learn more about how Neudata can help.