From 043bb8a723e87fbd0eb71b3c40dbc33b72d00214 Mon Sep 17 00:00:00 2001 From: Marvin Sanders Date: Mon, 6 Jan 2025 16:02:09 +0100 Subject: [PATCH 1/3] Added how do i configure a robots.txt file in shopware 6 on hypernode --- ...ots-txt-file-in-shopware-6-on-hypernode.md | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 docs/ecommerce-applications/shopware-6/how-do-i-configure-a-robots-txt-file-in-shopware-6-on-hypernode.md diff --git a/docs/ecommerce-applications/shopware-6/how-do-i-configure-a-robots-txt-file-in-shopware-6-on-hypernode.md b/docs/ecommerce-applications/shopware-6/how-do-i-configure-a-robots-txt-file-in-shopware-6-on-hypernode.md new file mode 100644 index 00000000..3f145556 --- /dev/null +++ b/docs/ecommerce-applications/shopware-6/how-do-i-configure-a-robots-txt-file-in-shopware-6-on-hypernode.md @@ -0,0 +1,82 @@ +--- +myst: + html_meta: + description: Learn how to configure a robots.txt file in Shopware 6 on Hypernode to optimize search engines. + title: 'How do I configure a robots.txt file in Shopware 6 on Hypernode? ' +--- + + + +# How do I configure a robots.txt file in Shopware 6 on Hypernode + +In this article, we explain how to set up a `robots.txt` file in Shopware 6 on Hypernode to optimize the indexing of your webshop by search engines. + +A `robots.txt` file allows you to instruct search engine crawlers on which parts of your website they are allowed or not allowed to index. This helps prevent duplicate content, saves additional resources, and results in more efficient indexing and better SEO performance. + +## Setup for a single robots.txt file without multistore + +Create a new text file named `robots.txt` and place this file in the `public` directory of your Shopware 6 installation. +You can insert the example text below into this file: + +```text +User-agent: * +Allow: / +Disallow: */? +Disallow: */account/ +Disallow: */checkout/ +Disallow: */widgets/ +Disallow: */navigation/ +Disallow: */bundles/ + +Disallow: */imprint$ +Disallow: */privacy$ +Disallow: */gtc$ + +Sitemap: https://YOUR_DOMAIN/sitemap.xml +``` + +Adjust the `Sitemap` rule with the exact URL to your sitemap. + +Use the [robots.txt tester](https://support.google.com/webmasters/answer/6062598) in Google Search Console to verify if the configuration is correct and the desired pages are being blocked or allowed. + +## Setup for a Multistore Robots.txt shopware 6 Webshop + +In the root of your Shopware 6 installation, navigate to the `public` directory. Create a new folder called `robots`: + +```bash +mkdir public/robots +``` + +Within the newly created `robots` directory, create a separate `robots.txt` file for each domain. For example: + +- `public/robots/www.example.com.txt` +- `public/robots/shop.example.com.txt` + +Each file should contain the rules specific to the corresponding domain. Below is an example for `www.example.com`: + +```text +User-agent: * +Allow: / +Disallow: */? +Disallow: */account/ +Disallow: */checkout/ +Disallow: */widgets/ +Disallow: */navigation/ +Disallow: */bundles/ + +Disallow: */imprint$ +Disallow: */privacy$ +Disallow: */gtc$ + +Sitemap: https://www.example.com/sitemap.xml +``` + +Adjust the `Sitemap` URL and other rules according to the specific requirements of each domain. + +Edit or create a new NGINX configuration file at `/data/web/nginx/server.robots` and add the following rewrite rule: + +```nginx +rewrite ^/robots\.txt$ /robots/$host.txt; +``` + +This rule ensures that requests to `/robots.txt` are dynamically redirected to the correct file based on the domain making the request. From 2e7cbdfbdf77ef6d7da2abd3232c614bfc3527ac Mon Sep 17 00:00:00 2001 From: Marvin Sanders Date: Tue, 7 Jan 2025 10:09:17 +0100 Subject: [PATCH 2/3] fix mdformat --- ...e-a-robots-txt-file-in-shopware-6-on-hypernode.md | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/docs/ecommerce-applications/shopware-6/how-do-i-configure-a-robots-txt-file-in-shopware-6-on-hypernode.md b/docs/ecommerce-applications/shopware-6/how-do-i-configure-a-robots-txt-file-in-shopware-6-on-hypernode.md index 3f145556..0cc734b5 100644 --- a/docs/ecommerce-applications/shopware-6/how-do-i-configure-a-robots-txt-file-in-shopware-6-on-hypernode.md +++ b/docs/ecommerce-applications/shopware-6/how-do-i-configure-a-robots-txt-file-in-shopware-6-on-hypernode.md @@ -1,9 +1,6 @@ ---- -myst: - html_meta: - description: Learn how to configure a robots.txt file in Shopware 6 on Hypernode to optimize search engines. - title: 'How do I configure a robots.txt file in Shopware 6 on Hypernode? ' ---- +______________________________________________________________________ + +## myst: html_meta: description: Learn how to configure a robots.txt file in Shopware 6 on Hypernode to optimize search engines. title: How do I configure a robots.txt file in Shopware 6 on Hypernode? @@ -15,8 +12,7 @@ A `robots.txt` file allows you to instruct search engine crawlers on which parts ## Setup for a single robots.txt file without multistore -Create a new text file named `robots.txt` and place this file in the `public` directory of your Shopware 6 installation. -You can insert the example text below into this file: +Create a new text file named `robots.txt` and place this file in the `public` directory of your Shopware 6 installation. You can insert the example text below into this file: ```text User-agent: * From eda60f2cafac7910605c08167db314f918a7849b Mon Sep 17 00:00:00 2001 From: Marvin Sanders Date: Tue, 7 Jan 2025 10:16:02 +0100 Subject: [PATCH 3/3] mdformat command remove the myst --- ...ure-a-robots-txt-file-in-shopware-6-on-hypernode.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/docs/ecommerce-applications/shopware-6/how-do-i-configure-a-robots-txt-file-in-shopware-6-on-hypernode.md b/docs/ecommerce-applications/shopware-6/how-do-i-configure-a-robots-txt-file-in-shopware-6-on-hypernode.md index 0cc734b5..9f767ab5 100644 --- a/docs/ecommerce-applications/shopware-6/how-do-i-configure-a-robots-txt-file-in-shopware-6-on-hypernode.md +++ b/docs/ecommerce-applications/shopware-6/how-do-i-configure-a-robots-txt-file-in-shopware-6-on-hypernode.md @@ -1,6 +1,10 @@ -______________________________________________________________________ - -## myst: html_meta: description: Learn how to configure a robots.txt file in Shopware 6 on Hypernode to optimize search engines. title: How do I configure a robots.txt file in Shopware 6 on Hypernode? +--- +myst: + html_meta: + description: Learn how to configure a robots.txt file in Shopware 6 on + Hypernode to optimize search engines. + title: How do I configure a robots.txt file in Shopware 6 on Hypernode? +---