…and how it helps protect your content in the age of AI
Looking out for your business in a changing digital world
You’ve probably heard of a robots.txt file, the little text file that helps search engines like Google understand which parts of your website they can or can’t access. But there’s a new file that’s worth knowing about: llms.txt.
If you’re running a business or community organisation, this might sound like yet another bit of tech jargon you don’t have time for. We get it. You care more about outcomes than you do about file names or web protocols.
So let me try and simplify it quickly for you.
The problem: AI tools are reading your website, often without you knowing
In the background of the internet, all sorts of bots and crawlers visit websites every day. Some are good and necessary, like Google indexing your pages so people can find you in search results.
But now, many large AI tools (like ChatGPT, Google Gemini, Claude, and others) also crawl websites. They’re not just looking to index your content. They are absorbing large amounts of content to help the AI become “smarter.” That includes blog posts, service pages, and any other public-facing content.
For small businesses and local organisations, this raises a few important questions:
- Do I want my website content to be used to train AI tools?
- What if I’ve paid a copywriter or spent hours crafting this content myself?
- Could my words or ideas be reworded or reused by an AI tool and presented somewhere else without credit?
Unless you’ve taken specific steps, your content may already be part of these systems without your knowledge or permission.
Why it matters: Content control and brand protection
You’ve worked hard on your website. Even if you didn’t write every word yourself, the content on your site reflects your tone, your knowledge, and your values. It’s part of your brand.
Now imagine a future where:
- An AI chatbot starts offering answers that sound a lot like your content
- Competitors use AI tools to summarise or repackage information from your site
- Your advice, product descriptions, or client resources end up training a system that offers similar content to other people for free
This is no longer hypothetical. It’s already happening.
Even if you’re not worried about AI right now, taking small steps today can help you stay in control down the track. That’s where llms.txt comes in.
The solution: Add an llms.txt file to your website
llms.txt stands for “Large Language Model Services.” It’s a new type of file that works a bit like robots.txt. It gives instructions to AI tools about what they can and can’t do with your site.
What does it actually do?
It can serve two main purposes:
- Suggest limits or rules to AI tools about what content they can or can’t access
- Highlight key content you want AI tools to focus on, rather than treating your whole site the same
It’s a way of communicating, not enforcing. Think of it more as a signpost than a security gate.
But it means if you add the file, you can begin to set the terms. You regain a bit of control.
But can it actually block AI tools?
This is where some confusion has crept in.
It’s important to know that llms.txt is not a technical blocker. There’s nothing stopping an AI tool from ignoring it. In fact, many probably will.
However, the major AI players, like OpenAI and Google, are beginning to support it and may choose to respect your preferences.
So while it’s not a lock on the door, it’s a clear and visible way of saying, “This is how I want my content treated.”
Just like with robots.txt, responsible bots may listen. Bad actors won’t. But having the file in place still shows you’ve taken steps to protect your content, and that matters.
What it looks like in practice
The file itself is just a plain text document that lives at the root of your website.
Here’s a basic example:
User-agent: OpenAI Disallow: / User-agent: GoogleAI Allow: /
In plain English, this means:
- ChatGPT (OpenAI) is being asked not to crawl your site
- Google’s AI tools are allowed
You can tailor this depending on your preferences. You might block all AI tools, allow specific ones, or revisit your settings over time. The point is, you have a choice.
Highlighting content for AI tools
Beyond blocking, llms.txt can also help AI models better understand which parts of your site are most relevant or authoritative.
Think of it like a content map. You can guide AI models toward the pages that reflect your brand’s knowledge or services and steer them away from outdated or lower-quality pages.
This is particularly helpful if you:
- Publish expert articles or community resources
- Want to establish your site as a credible voice in your niche
- Are comfortable with your content being used, but only in a specific way
Example format (in Markdown):
# llms.txt ## Recommended content - https://yourwebsite.com/our-approach - https://yourwebsite.com/advice-centre/top-10-questions ## Do not use - https://yourwebsite.com/test-page - https://yourwebsite.com/old-newsletter-archive
This gives responsible AI tools a clearer sense of what to use and what to avoid. Again, it’s voluntary, but helpful.
What we recommend at Regional Business Toolkit
We’re now encouraging all our Website Care Plan clients to add an llms.txt file as a best-practice security and brand protection measure.
We see it as:
- A proactive step in managing how your content is treated online
- A way to signal your intent to AI platforms
- A small action today that could prevent bigger problems tomorrow
Whether you want to block AI tools, guide them toward your best work, or simply keep your options open—we can help you get it set up quickly and confidently.
We can assess your website and help you decide what settings make sense for your business or organisation.
Do I need one?
If you’re a small business, community group, or regional organisation with original content on your site, we think it’s worth having.
You might want to add an llms.txt file if:
- You’ve invested time or money into your website content
- You’d like to control what content is prioritised by AI tools
- You’re concerned about your information being used without consent or credit
- You want to show that yo’re taking digital content use seriously
It’s especially useful for businesses that publish educational content, professional advice, or community resources, or anything that might be considered valuable by AI tools.
How to get started
If you’re on a Website Care Plan with us, we can implement the llms.txt file for you. No stress, or technical headaches needed on your part.
Not on a plan? That’s no problem. We can still advise you, create a tailored llms.txt file, and publish it to your site.
It’s a low-effort, high value action that keeps you ahead of the curve.
Final thoughts: It’s your content. Let’s help you protect it.
Technology keeps changing. That’s a fact of life in today’s world. But just because things are moving fast doesn’t mean small businesses and regional organisations should be left behind, or left unprotected.
At Regional Business Toolkit, our role is to keep an eye on these shifts and give you practical, clear guidance that protects your time, your reputation, and your digital assets.
An llms.txt file might sound like a small thing. And it is. But sometimes, the smallest changes are the ones that quietly safeguard what you’ve worked hard to build.
Need a hand?
Whether you’re already a client or just looking for a bit of advice, we’re happy to chat.
- Want to check if your website is being used by AI tools?
- Not sure what to include in your llms.txt file?
- Curious about other simple ways to protect your content?
Get in touch with us today. We’ll keep it simple, clear, and useful… and always with the intent of helping your grow your impact.
