LLMs & AI Crawler Access Policy
This page explains how Crane Hub Global provides access instructions to large language models (LLMs) and AI crawlers, and how we manage the use of our publicly available content for indexing and training purposes.
Our approach balances open information sharing with protection of restricted, private, or sensitive site areas helping AI systems understand which content they may access and which they may not while respecting our site’s structure, privacy, and intellectual property.
Policy Overview
Crane Hub Global publishes industry news, editorial content, and educational resources intended for public viewing. To support transparency and ethical AI usage, we provide a machine-readable llms.txt file that communicates our preferences regarding how AI systems may access this content.
These instructions are designed to:
-
Allow indexing and reference of public editorial content
-
Prevent access to private, account-based, transactional, or sensitive areas
-
Protect user privacy, platform integrity, and intellectual property
How AI Systems May Use Our Content
AI crawlers and large language models may access and index publicly available sections of CraneHubGlobal.com, including news articles, blog content, categories, and general informational pages, provided they respect the access rules defined below.
Use of Crane Hub Global content for:
-
commercial redistribution
-
proprietary model training
-
data resale
-
content replication at scale
requires explicit written permission from Crane Hub Global.
Protected & Restricted Areas
AI systems are instructed not to access:
-
Admin, login, and account areas
-
Checkout, cart, and transactional pages
-
User-specific or private data directories
-
Internal scripts or system files
These restrictions exist to protect:
-
user privacy
-
platform security
-
proprietary systems
AI Crawler Instructions (llms.txt)
Below is the current crawler policy for CraneHubGlobal.com. This file is intended for automated systems and AI crawlers that support the llms.txt standard.
# llms.txt for CraneHubGlobal.com
# This file provides access instructions for AI crawlers (Large Language Models)
# ========== ALLOWED CONTENT ==========
Allow: /news/
Allow: /blog/
Allow: /categories/
Allow: /crane-types/
Allow: /images/
Allow: /media/
Allow: /static/
Allow: /pages/
Allow: /cranes/
Allow: /manufacturers/
# ========== DISALLOWED CONTENT ==========
Disallow: /admin/
Disallow: /account/
Disallow: /login/
Disallow: /checkout/
Disallow: /cart/
Disallow: /scripts/
Disallow: /user-data/
Disallow: /private/
# ========== CRAWLER INSTRUCTIONS ==========
User-agent: *
Crawl-delay: 1
# ========== CONTACT ==========
Contact: info@cranehubglobal.com
Policy: https://www.cranehubglobal.com/privacy-policy
# ========== METADATA ==========
LLMSpec-Version: 0.1
Permissions & Contact
If you are an AI developer, platform provider, publisher, or organization seeking permission to use Crane Hub Global content beyond standard indexing or summarization, please contact us at: info@cranehubglobal.com
For details on how we handle user privacy and data, see our Privacy Policy.
What Is llms.txt?
llms.txt is a community-emerging standard similar to robots.txt that allows site owners to declare how AI models and crawlers may index or use website content. It is designed to improve transparency and protect content creators while facilitating ethical AI interactions.
Crane Hub Global supports ethical AI innovation while protecting the integrity of industry journalism, community trust, and creator rights.