Composer 1x FAQ

How many scripts do we need?

You only need one script. After the script is integrated into the appropriate element of your website, Piano is activated on every subsequent page and subpage on your domain.

What does the script do?

The script calls all three Piano libraries and sends data to all Piano products configured for your account.

What is content crawling?

Content crawling is an automated, continuous process that parses page content and performs semantic analysis to create or update content profiles.

How does the crawler work?

The bot visits a page, downloads its text content, parses it, identifies key values according to the taxonomy, and creates or updates the corresponding content profile.

For new pages, crawling starts after three page views within six hours, which often happens within seconds of a new article being published. For known pages, the crawler visits again when a page has been modified and contains specific meta tags.

Are there any limitations to what can be crawled?

If you have whitelisted the crawler, there are no limitations. Otherwise, the crawler cannot access pages behind your paywall. If you experience issues, contact us so we can review the crawler logs.

How many predefined segments does Composer 1X ship with?

The predefined segment menu is split into 11 categories, with 87 segments in total.

What license do I need to use Composer 1X?

The required license depends on the Composer 1X feature you want to use:

*Segments are created only when at least one user fits the criteria. They are also available to clients using Linked Terms. Empty segments are not created, so clients who do not meet certain conditions, such as having no paid trial users, will not see those segments until relevant data is available.

Functionality marked as available with an implementation project requires paid services. Contact your Account Manager for details.

Can I use LtS and LtC without a Management + Billing license?

Yes. Contact your Account Manager. Using LtS or LtC without a Management + Billing license requires an additional scope of work and an implementation cost.

What segments are supported for Management + Billing clients using custom terms?

Piano does not provide out-of-the-box subscription segments based on custom terms. You still have access to 29 machine-learning-based User Interest segments, five User Frequency segments, and five User Recency segments, because these do not depend on Management + Billing data.

What is the Composer 1X crawler?

The Composer 1X crawler is a web crawler, or bot, that reads the content on your website and indexes it for later use. It automatically searches for new pages so they can be indexed as quickly as possible, and it periodically checks your website for changes to existing pages.

Indexed content powers the segmentation engine. Without it, you cannot target specific pages, topics, dates, or authors.

When implementing Composer 1X, whitelist the crawler so it can crawl pages behind a paywall. Indexing every article builds stronger segments and helps you create more robust experiences.

Is Composer 1X segmentation and functionality supported on Mobile Experiences using the Piano SDK?

Yes. Mobile Experiences support Composer 1X segmentation and functionality, including recommendations.

Are updates to consent-aware tags needed for legacy clients starting to use Composer 1X?

Yes. If you are a legacy Cxense client participating in the Composer 1X rollout, you need to update the consent-aware tags in your integration code. For example, if you use code like this:

var cX = window.cX || { options: { consent: true }};
cX.callQueue = cX.callQueue || [];
cX.callQueue.push(['setSiteId', '<site_ID>']);
cX.callQueue.push(['sendPageViewEvent']);
cX.callQueue.push(['invoke', requestConsent])

Remove these two lines:

cX.callQueue.push(['setSiteId', '<site_ID>']);
cX.callQueue.push(['sendPageViewEvent']);

Both methods are called directly after Composer 1X execution.

More information about Piano Audience, Piano Insight, and Piano CCE consents is available here.

Can the Cxense and Composer JS scripts coexist?

Yes. However, if you have a Composer 1X integration, the Cxense script should already be integrated automatically.

Should events already sent with the cX object also be included in the Composer tp object?

No. This is not necessary.

What IP addresses are requests sent from?

Requests are sent from the following IP addresses:

• 5.9.190.101

• 5.9.190.106

• 5.9.190.107

• 5.9.190.127

For simplicity, you can also use the network specifications 5.9.190.96/27 or 5.9.190.96, with netmask 255.255.255.224. The content processing cluster is hosted by Piano infrastructure in Germany. Ensure that EU-based IP addresses are not blocked.

What is the request type and protocol?

The request type and protocol are HTTP:*.

How many concurrent or maximum requests/connections will be initiated?

There is no formal limit, but our current technology limits concurrent requests across all clients. We do not expect more than five to ten concurrent sessions on a specific site at any one time.

What is the frequency of crawling, connections, or requests?

Crawling is triggered when:

• A specific URL generates three or more page views after an article is first published or updated.

• Documents are manually pushed before a URL is published on a page, using the /document/update API.

When the site is crawled, each URL is checked against the Piano system to determine whether:

• A profile for the URL already exists.

• The existing profile is less than 30 days old.

If both conditions are true, no further action is taken. In practice, most pages are crawled once when they are new and recrawled only if they are still being viewed more than 30 days after publication.

There is no option to limit total concurrent connections established to a single IP address.

Are IF-MODIFIED-SINCE queries issued, and do they conform to 304 Not Modified?

Pages are fetched by a regular web browser instance, so each page is processed as it would be for a real user. For example, all resources are loaded, scripts are executed, and HTTP headers that are meaningful to the browser are confirmed.

Do you respect and conform to robots.txt deny/allow entries?

No. We do not respect or conform to robots.txt because we need to load the page as a real user would.

We also do not respect crawl-delay from robots.txt.

Do you respect Cache-Control response headers?

Caching is technically supported but not guaranteed. Pages are fetched by a regular web browser instance.

Do you respect and follow the Retry-After response header?

No. A general failure route is implemented instead.

If a domain has more than 10% failures across 100 samples, or fails more than 50% of the times it is accessed, we consider the domain unhealthy and temporarily disable crawling for 30 minutes. If the domain has not recovered after that, crawling is disabled for one hour. Page views received while the domain is disabled do not trigger crawling.

What is the User-Agent field for the connections?

The User-Agent field is:

Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US) AppleWebKit/537.36 (KHTML, like Gecko) cXensebot/2.0; +http://www.cxense.com/bot.html Safari/533.3

Is the application using BloxCMS web services?

No.

How long will this script need access to scrape?

This is a continuous process. Crawling is essential for the Content module and Segmentation to work for any client.

Last updated: August 14, 2026