Merge pull request #8205 from magento/sync-2.4.2

Sync 2.4.2-develop

Author: dshevtsov

src/_data/toc/cloud-guide.yml

@@ -215,14 +215,15 @@ pages:
               versionless: true
 
         - label: Launch Docker
-          children: 
-            - label: Developer mode
-              url: /cloud/docker/docker-mode-developer.html
-              versionless: true
 
+          children: 
             - label: Production mode
               url: /cloud/docker/docker-mode-production.html
               versionless: true
+ 
+            - label: Developer mode
+              url: /cloud/docker/docker-mode-developer.html
+              versionless: true
 
         - label: Functional Testing
           children:

src/_data/var.yml

@@ -9,7 +9,7 @@
 # Enterprise Cloud Edition (ECE)  ---> Commerce Cloud
 # Enterprise Edition for B2B      ---> Commerce for B2B
 
-#NOTE: The new product name variables use the old product name abbreviations (e.g., ce, ee, ece, b2b). 
+#NOTE: The new product name variables use the old product name abbreviations (e.g., ce, ee, ece, b2b).
 
 # Product Name variables
 
@@ -21,17 +21,17 @@ mbi: Magento Business Intelligence
 # Cloud product name variables
 
 ece: Magento Commerce Cloud
-ece-release-date: August 5, 2020
+ece-release-date: November 9, 2020
 csuite: Magento Commerce Cloud Suite
 ct: ece-tools
 ct-repo: magento/ece-tools
-ct-release: 2002.1.2
+ct-release: 2002.1.3
 mcp-prod: Magento Cloud Patches
 mcp-package: magento/magento-cloud-patches
 mcp-release: 1.0.8
 mcd-package: magento/magento-cloud-docker
 mcd-prod: Magento Cloud Docker
-mcd-release: 1.1.2
+mcd-release: 1.2.0
 mcc-prod: Magento Cloud Components
 mcc-package: magento/magento-cloud-components
 mcc-release: 1.0.7

src/_includes/cloud/cloud-docker-config-generator-cmds.md

@@ -2,10 +2,11 @@ You use the following commands to generate the Docker configuration files and bu
 
 Action | Command
 :----- | :------
-Builds the Docker environment in [production mode][mode] by default and verifies configured service versions. | `ece-docker build:compose`
-Builds the docker environment in [developer mode][mode]. | `ece-docker build:compose --mode="developer"`
-Builds the docker environment in [production mode][mode]. |`ece-docker build:compose --mode="production"`
-Convert PHP configuration files to Docker ENV files. | `ece-docker image:generate:php`
+Builds the Docker environment in [production mode][mode] by default and verifies configured service versions. | `./vendor/bin/ece-docker build:compose`
+Builds the docker environment in [developer mode][mode]. | `./vendor/bin/ece-docker build:compose --mode="developer"`
+Builds the docker environment in [production mode][mode]. |`.vendor/bin/ece-docker build:compose --mode="production"`
+Convert PHP configuration files to Docker ENV files. | `.vendor/bin/ece-docker image:generate:php`
+Builds a custom `docker-compose.yaml` file | `./vendor/bin/ece-docker build:custom:compose`
 
 For example, the following command starts the Docker configuration generator in developer mode and specifies PHP version 7.2:
 

src/_includes/cloud/error-codes.md

@@ -1,4 +1,4 @@
-<!-Note: The error code tables in this file are auto-generated from source code. To request changes to error code descriptions or suggestions, submit a GitHub issue to the magento/ece-tools repository.->
+<!--Note: This topic is auto-generated from the https://github.com/magento/ece-tools/blob/e3b13ca023afe324cd93d0caa2fd62d5683b60b9/config/schema.error.yaml source code. To request changes to error code descriptions or suggestions, submit a GitHub issue to the magento/ece-tools repository.-->
 
 ## Critical Errors
 
@@ -71,9 +71,10 @@ Critical errors indicate a problem with the {{ site.data.var.ece }} project conf
 | 129 | install-update: reset-password | Unable to read reset password template |  |
 | 130 | install-update: cache_type | Command failed: `php ./bin/magento cache:enable` | Command `php ./bin/magento cache:enable` runs only when Magento was installed but `./app/etc/env.php` file was absent or empty at the beginning of the deployment. Check the `cloud.log` for more information. Add `VERBOSE_COMMANDS: '-vvv'` into `.magento.env.yaml` for more detailed command output. |
 | 131 | install-update | The `crypt/key`  key value does not exist in the `./app/etc/env.php` file or the `CRYPT_KEY` cloud environment variable | This error occurs if the `./app/etc/env.php` file is not present when Magento deployment begins, or if the `crypt/key` value is undefined. If you migrated the database from another environment, retrieve the crypt key value from that environment. Then, add the value to the [CRYPT_KEY](https://devdocs.magento.com/cloud/env/variables-deploy.html#crypt_key) cloud environment variable in your current environment. See [Add the Magento encryption key](https://devdocs.magento.com/cloud/setup/first-time-setup-import-import.html#encryption-key). If you accidentally removed the `./app/etc/env.php` file, use the following command to restore it from the backup files created from a previous deployment: `./vendor/bin/ece-tools backup:restore` CLI command ." |
-| 132 |  | Can not connect to the Elasticsearch service | Check that credentials for elasticsearch are correct and service is running |
+| 132 |  | Can not connect to the Elasticsearch service | Check for valid Elasticsearch credentials and verify that the  service is running |
 | 133 | validate-config | Remove Magento Braintree module configuration which is no longer supported in Magento 2.4 and later versions. | Support for the Braintree module is no longer included with Magento 2.4.0 and later. Remove the CONFIG__STORES__DEFAULT__PAYMENT__BRAINTREE__CHANNEL variable from the variables section of the .magento.app.yaml file. For Braintree support, use an official Braintree Payments extension from the Magento Marketplace instead. |
 | 134 | validate-config | Magento 2.4.0 requires Elasticsearch service to be installed | Install Elasticsearch service |
+| 135 | validate-config | The search engine must be set to Elasticsearch for Magento >= 2.4.0 | Check the SEARCH_CONFIGURATION variable for the `engine` option. If it is configured, remove the option, or set the value to "elasticsearch". |
 
 ### Post-deploy stage
 

src/cloud/docker/docker-config.md

@@ -134,6 +134,20 @@ The default {{ site.data.var.mcd-prod }} configuration includes the [MailHog] se
 
 When the MailHog service is installed, go to the following URL to open the service and view outgoing emails: `http://magento2.docker:8025`
 
+By default, MailHog listens on port 1025 for SMTP and port 8025 for the frontend dashboard and API. You can change the default ports using the `--mailhog-http-port` and `--mailhog-smtp-port` options.
+
+```bash
+./vendor/bin/ece-docker build:compose --mailhog-smtp-port=1026 --mailhog-http-port=8026
+```
+
+After updating the configuration and restarting the Docker environment, you can connect to the MailHog service from `http://magento2.docker:8026`, and use port 1026 for SMTP communication.
+
+If you do not need the [MailHog] service, use the `--no-mailhog` option to generate the Docker compose configuration:
+
+```bash
+./vendor/bin/ece-docker build:compose --no-mailhog
+```
+
 [php]: https://www.php.net/manual/en/install.php
 [Composer]: https://getcomposer.org
 [Docker]: https://www.docker.com/get-started

src/cloud/docker/docker-containers-service.md

@@ -75,6 +75,20 @@ See [Important Elasticsearch configuration][] in the Elasticsearch documentation
 {:.bs-callout-info}
 If your Cloud project uses Magento version 2.3.5 or earlier with MySQL search, add the `--no-es` option to skip the Elasticsearch container configuration when you generate the Docker Compose configuration file: `ece-docker build:compose --no-es`.
 
+### Elasticsearch plugins
+
+The `analysis-icu` and `analysis-phonetic` plugins are installed by default and can not be skipped.
+If you use Elasticsearch 6.5 and later, the default Elasticsearch plugins are installed automatically along with any custom plugins added to the `services.yaml`file. When you generate the `docker-compose.yaml` file, you can add additional custom plugins to the Docker environment using the `ES_PLUGINS` environment configuration option.
+
+The following example adds the `analysis-stempel` and `analysis-nori` plugins to the Docker environment.
+
+```yaml
+services:
+    elasticsearch:
+        environment:
+          - 'ES_PLUGINS=analysis-stempel analysis-nori'
+```
+
 ### Troubleshooting
 
 On some Linux systems, when you launch the Docker environment, the Elasticsearch service fails to start and the following error displays:
@@ -163,15 +177,23 @@ docker-compose run --rm redis redis-cli -h redis
 ## Selenium container
 
 **Container name**: selenium<br/>
-**Docker base image**: [selenium/standalone-chrome/](https://hub.docker.com/r/selenium/standalone-chrome<br/>
+**Docker base image**: [selenium/standalone-chrome/](https://hub.docker.com/r/selenium/standalone-chrome), based on the [selenium/standalone-chrome/](https://hub.docker.com/r/selenium/standalone-chrome/h) Docker image<br/>
 **Ports exposed**: `4444`<br/>
 
-The Selenium container, based on the [selenium/standalone-chrome/](https://hub.docker.com/r/selenium/standalone-chrome/h), enables the [Magento Functional Testing Framework (MFTF)](https://devdocs.magento.com/mftf/docs/introduction.html) for Magento application testing in the Cloud Docker environment. See [Magento application testing]({{site.baseurl}}/cloud/docker/docker-test-app-mftf.html).
+The Selenium container enables the [Magento Functional Testing Framework (MFTF)](https://devdocs.magento.com/mftf/docs/introduction.html) for Magento application testing in the Cloud Docker environment. See [Magento application testing]({{site.baseurl}}/cloud/docker/docker-test-app-mftf.html).
+
+## Test container
+
+**Container name**: test<br/>
+**Docker base image**: [magento/magento-cloud-docker-php][php-cloud], based on the [magento/magento-cloud-docker-php][php-cloud] Docker image<br/>
+**Ports exposed**: None<br/>
+
+The Test container, based on the [magento/magento-cloud-docker-php][php-cloud] Docker image, has a writable file system and is used for Magento application testing in the Cloud Docker environment. See [Magento application testing]({{site.baseurl}}/cloud/docker/docker-test-app-mftf.html).
 
 ## TLS container
 
 **Container name**: tls<br/>
-**Docker base image**: [magento/magento-cloud-docker-tls][tls], which is based on the [debian:jessie](https://hub.docker.com/_/debian) Docker image<br/>
+**Docker base image**: [magento/magento-cloud-docker-tls][tls], based on the [debian:jessie](https://hub.docker.com/_/debian) Docker image<br/>
 **Ports exposed**: `443`</br>
 
 The TLS termination proxy container facilitates the Varnish SSL termination over HTTPS.
@@ -187,7 +209,7 @@ To increase the timeout on this container, add the following code to the  `docke
 ## Varnish container
 
 **Container name**: varnish<br/>
-**Docker base image**: [magento/magento-cloud-docker-varnish][varnish], which is based on the [centos]<br>
+**Docker base image**: [magento/magento-cloud-docker-varnish][varnish], based on the [centos] Docker image<br>
 **Ports exposed**: `80`<br/>
 
 The Varnish container simulates Fastly and is useful for testing VCL snippets.
@@ -211,7 +233,7 @@ docker-compose exec varnish varnishadm ban req.url '~' '.'
 ## Web container
 
 **Container name**: web<br/>
-**Docker base image**: [magento/magento-cloud-docker-nginx][nginx], which is based on the [centos] Docker image<br/>
+**Docker base image**: [magento/magento-cloud-docker-nginx][nginx], based on the [centos] Docker image<br/>
 **Ports exposed**: None<br/>
 
 The Web container uses NGINX to handle web requests after TLS and Varnish. This container passes all requests to the FPM container to serve the PHP code. See [Request flow]({{site.baseurl}}/cloud/docker/docker-containers.html#request-flow).

src/cloud/docker/docker-containers.md

@@ -43,17 +43,18 @@ The following table shows the options to customize service container configurati
 | Name       | Service   | Key & options  | Available Versions | Notes
 | ------------- | ---------- | ---------- | ------------------ |------------------
 | [db] | MariaDB or MySQL<br>     | `--db`, `--db-image` (MySQL)<br>`--expose-db-port`<br>`--db-increment`<br>`--db-offset`<br>`--with-entrypoint`<br>`--with-mariadb-config`|10.0, 10.1, 10.2<br>5.6, 5.7 | Use the increment and offset options to customize the [auto-increment settings][Using AUTO_INCREMENT] for replication.<br><br>Use the `--with-entrypoint` and `--with-mariadb-config` options to automatically configure database directories in the Docker environment<br><br>*Example build commands:*<br>`ece-docker build:compose --db <mariadb-version>`<br>`ece-docker build:compose --db <mysql-version> --db-image`
-| [elasticsearch] | Elasticsearch | `--es`<br>`--es-env-var`<br>`--no-es` | 1.7, 2.4, 5.2, 6.5, 6.8, 7.5, 7.6 | Use the options to specify the Elasticsearch version,  customize Elasticsearch configuration options, or to build a Docker environment without Elasticsearch.
+| [elasticsearch] | Elasticsearch | `--es`<br>`--es-env-var`<br>`--no-es` | 1.7, 2.4, 5.2, 6.5, 6.8, 7.5, 7.6, 7.7 | Use the options to specify the Elasticsearch version,  customize Elasticsearch configuration options, or to build a Docker environment without Elasticsearch.
 | [FPM][fpm-container] | PHP FPM | `--php`<br>`--with-xdebug` | 7.0, 7.1, 7.2, 7.3, 7.4 |  Used for all incoming requests. Optionally, add Xdebug configuration to debug PHP code in the Docker environment.
 | [node][node-container] | Node | `--node` | 6, 8, 10, 11 |  Used gulp or other NPM based commands
 | [rabbitmq][rabbitmq-container]| RabbitMQ | `--rmq` | 3.5, 3.7, 3.8 |
 | [redis][redis-container] | Redis     | `--redis` | 3.2, 4.0, 5.0 |   Standard redis container
 | [selenium][selenium-container]| Selenium | `--with-selenium`<br>`--selenium-version`<br>`--selenium-image`| Any | Enables Magento application testing using the Magento Functional Testing Framework (MFTF)
+| [test][test-container]| PHP CLI | `--with-test`| Any | Container with a writable file system for running tests
 | [tls][tls-container] | SSL Endpoint |  |   |  Terminates SSL, can be configured to pass to varnish or nginx
 | [varnish][varnish-container] | Varnish | `--no-varnish` | 4, 6.2 | Varnish is provisioned by default. Use the `--no-varnish` option to skip Varnish service installation
 | [web][web-container] | NGINX | `--nginx` | 1.9, latest |
 
-Use the following command to view the available options for the `ece-docker build:compose` command:
+Use the following command to view all available options for the `ece-docker build:compose` command:
 
 ```bash
 ./vendor/bin/ece-docker build:compose --help
@@ -157,6 +158,7 @@ Now you can see all requests that are passing through the TLS container and chec
 [fpm-container]: {{site.baseurl}}/cloud/docker/docker-containers-service.html#fpm-container
 [redis-container]: {{site.baseurl}}/cloud/docker/docker-containers-service.html#redis-container
 [selenium-container]: {{site.baseurl}}/cloud/docker/docker-containers-service.html#selenium-container
+[test-container]: {{site.baseurl}}/cloud/docker/docker-containers-service.html#test-container
 [tls-container]: {{site.baseurl}}/cloud/docker/docker-containers-service.html#tls-container
 [varnish-container]: {{site.baseurl}}/cloud/docker/docker-containers-service.html#varnish-container
 [web-container]: {{site.baseurl}}/cloud/docker/docker-containers-service.html#web-container
@@ -171,4 +173,4 @@ Now you can see all requests that are passing through the TLS container and chec
 <style>
 table.docker-service-versions-table td:nth-child(3) {
   width: 200px;
-}
+}

src/cloud/docker/docker-development-debug.md

@@ -26,6 +26,12 @@ If you use Microsoft Windows, take the following steps before continuing:
    vendor/bin/ece-docker build:compose --mode="developer" --sync-engine="mutagen" --with-xdebug
    ```
 
+   For Linux systems, you must use the `--set-docker-host` option to add the `host.docker.internal` entry to the `/etc/hosts` file for the `fpm_xdebug` container.
+
+      ```bash
+      vendor/bin/ece-docker build:compose --mode="developer" --with-xdebug --set-docker-host
+      ```
+
    This command adds the Xdebug configuration to your `docker-compose.yml` file.
 
    ```yaml

src/cloud/docker/docker-mode-developer.md

@@ -26,13 +26,15 @@ To launch the Docker environment in developer mode:
 1. Install the template dependencies, and add the default hostname to your `/etc/hosts` file.
 
    ```bash
-   curl https://raw.githubusercontent.com/magento/magento-cloud-docker/1.1.0/bin/init-docker.sh | bash
+   curl https://raw.githubusercontent.com/magento/magento-cloud-docker/<magento-cloud-docker-package-version>/bin/init-docker.sh | bash
    ```
 
-   If required, you can add options to the `init-docker.sh` initialization script to customize your Docker environment. Run the following command to see the available options:
+   For `<package-version>`, use the [latest release of the {{site.data.var.mcd-package}}].
+
+   You can customize the options for the `init-docker.sh` initialization script your Docker environment. For example, you can specify the PHP version (default is 7.2) and the [Docker image version] (default 1.1). We recommend using the latest version of the Magento Cloud Docker images. Run the following command to see the available options:
 
    ```bash
-   curl https://raw.githubusercontent.com/magento/magento-cloud-docker/1.1.0/bin/init-docker.sh | bash -s -- --help
+   curl https://raw.githubusercontent.com/magento/magento-cloud-docker/<magento-cloud-package-version>/bin/init-docker.sh | bash -s -- --help
    ```
 
 1. On macOS or Windows hosts, install the selected file synchronization tool:
@@ -88,8 +90,18 @@ To launch the Docker environment in developer mode:
 
 1. Install Magento in your Docker environment.
 
+   -  For Magento version 2.4 and 2.4.1 only, run the following command to apply patches before you deploy.
+
+      ```bash
+      docker-compose run deploy php ./vendor/bin/ece-patches apply
+      ```
+
    -  Deploy Magento in the Docker container.
 
+      For Magento version 2.4 and 2.4.1, run the following command to apply patches before you deploy.
+
+      ```
+
       ```bash
       docker-compose run --rm deploy cloud-deploy
       ```
@@ -129,11 +141,15 @@ To launch the Docker environment in developer mode:
 
    -  `https://magento2.docker`
 
-[Synchronizing data in Docker]: {{site.baseurl}}/cloud/docker/docker-syncing-data.html
+<!--Link definitioons-->
+
 [cloud-repo]: https://github.com/magento/magento-cloud
+[Docker image version]: https://hub.docker.com/r/magento/magento-cloud-docker-php/tags
+[dsync-install]: https://docker-sync.readthedocs.io/en/latest/getting-started/installation.html
+[latest release of the {{site.data.var.mcd-package}}]: https://github.com/magento/magento-cloud-docker/releases
 [magento-creds]: {{site.baseurl}}/guides/v2.3/install-gde/prereq/connect-auth.html
+[mutagen-install]: https://mutagen.io/documentation/introduction/installation/
 [services]: {{site.baseurl}}/cloud/docker/docker-containers.html#service-containers
-[xdebug]: {{site.baseurl}}/cloud/docker/docker-development-debug.html#configure-xdebug]
 [service keys]: {{site.baseurl}}/cloud/docker/docker-containers.html#service-configuration-options
-[dsync-install]: https://docker-sync.readthedocs.io/en/latest/getting-started/installation.html
-[mutagen-install]: https://mutagen.io/documentation/introduction/installation/
+[Synchronizing data in Docker]: {{site.baseurl}}/cloud/docker/docker-syncing-data.html
+[xdebug]: {{site.baseurl}}/cloud/docker/docker-development-debug.html#configure-xdebug]