We strive to improve our service every day. To provide the best search experience, we need to collect data. We use web crawlers to gather information from the internet and index it for our search engine.

Liner uses two bots, each with its own user agent, robots.txt token and signing key.

LinerBot

LinerBot builds and refreshes Liner's search and reference index: paper metadata, citations and DOI resolution.

You can identify LinerBot by its user agent

User agent token: LinerBot
Full user agent: User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; LinerBot/1.0; +https://liner-bot.liner.com/docs/liner-bot)

LinerBot's requests carry the header Signature-Agent: "https://liner-bot.liner.com", and its public key is listed in the key directory at https://liner-bot.liner.com/.well-known/http-message-signatures-directory.

Liner-User

Liner-User fetches a page in real time when a Liner user asks a question. The answer cites the page as its source.

You can identify Liner-User by its user agent

User agent token: Liner-User
Full user agent: User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Liner-User/1.0; +https://liner-user.liner.com/docs/liner-user)

Liner-User's requests carry the header Signature-Agent: "https://liner-user.liner.com", and its public key is listed in the key directory at https://liner-user.liner.com/.well-known/http-message-signatures-directory.

Customizing access

Each bot follows the robots.txt group that names its token, LinerBot or Liner-User, and the * group when no group names it. Rules for LinerBot do not apply to Liner-User.

Disallow LinerBot

To prevent LinerBot from accessing your site data, add a record to your site's robots.txt

User-Agent: LinerBot
Disallow: /

Disallow Liner-User

To prevent Liner-User from accessing your site data, add a record to your site's robots.txt

User-Agent: Liner-User
Disallow: /

Custom access rules

You can also customize access, disallowing data retrieval only from specific paths.

User-Agent: LinerBot
Allow: /public/
Disallow: /private/

Verification

Both bots sign their requests with HTTP Message Signatures (Web Bot Auth), each with its own key. The Signature-Agent header of a request names the bot's key directory origin, listed above for each bot.

IP address ranges

We do not currently publish IP ranges for LinerBot or Liner-User, so our IP range list is intentionally empty.

Contact

For questions about LinerBot or Liner-User, or to report a problem, submit a request (support.liner.com/hc/en-us/requests/new).