Skip to content

Magento Test Framework giving the failure results for Magento 2.1.0 ce #6227

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shahankit783 opened this issue Aug 18, 2016 · 21 comments
Closed

Comments

@shahankit783
Copy link

When I tried to run the default testcases of magento 2.1.0 ce using MTF , only few of the test cases are passing rest of all are failing. I had also changed the xml file configurations for the same

Preconditions

  1. Windows 8
  2. Magento Version: 2.1.0 ce
  3. SQL Version: 5.7.11
  4. PHP Version: 5.6.19

Test Run Steps:

1)Go to Magento Root Directory:
cd magento2.1\dev\tests\functional

  1. Now run the test case execution by following command:
    vendor\bin\phpunit

  2. I am also running the specific group by following command:
    **vendor\bin\phpunit --group "Bundle_Product_(CS)"

Expected result

Most of the test cases should "PASS" as per the Dev-docs.

Actual result

Failure Error:
Though there are many failure reasons but the very common Exception it shows is : "Configuration settings are not applied! Url: http://localhost:8080/magento2.1.0ce/admin_iaa328/admin/system_config/save/section/admin"

Please find the attached zip file in which i have added 3 configuration files (modified for running test cases)

Config_files.zip

Please provide the solution or the steps for configuration (if I missed any)

Thanks!!!!

@lanken
Copy link
Contributor

lanken commented Aug 19, 2016

Hi @shahankit783,

Could you, please, change your hostname from 'localhost' to something like 'localhost.com', cURL isn't not working correctly in this case.

Also, could you tell what is the version of your web browser and selenium server?

One more item, pay attention to the app_backend_url, it must be exact the same as the url after login to the Magento admin, if it contains index.php in it, it must be also specified in the phpunit.xml.

@shahankit783
Copy link
Author

Hello,
I have changed host name from localhost to localhost.com and it is running on port 8080.

Chrome Browser: Version 52.0.2743.116 m (64-bit)
Selenium jar :- selenium-server-standalone-3.0.0-beta2.jar

app_backend_url :- http://localhost.com:8080/magento2.1.0ce/admin_iaa328/

@lanken
Copy link
Contributor

lanken commented Aug 22, 2016

Could you, please, use stable selenium server releases: http://selenium-release.storage.googleapis.com/index.html (for example, 2.53)
What is your URL after you have logged in to the admin manually?

@shahankit783
Copy link
Author

shahankit783 commented Aug 22, 2016

I have tried to run it with selenium-server-standalone-2.53.1.jar with chrome browser.

The most common error in all failure test cases is "Exception: Configuration settings are not applied! Url: http://localhost.com:8080/magento2.1.0ce/admin_iaa328/admin/system_config/save/section/admin"

Manually URL which i fired in browser :-
BACK END URL :- http://localhost:8080/magento2.1.0ce/admin_iaa328/admin

after login it will redirected to below URL:
http://localhost:8080/magento2.1.0ce/admin_iaa328/admin/dashboard/

It might be possible that i am missing some of the configuration settings in magento 2.1.ce.

I have followed the configuration and settings from following URL only:
http://devdocs.magento.com/guides/v2.1/mtf/mtf_introduction.html

If magento needs to be configured manually for different modules, then can you please suggest which are the settings that needs to be apply before running MTF.

PS: I am novice in magento and i am working with MTF to verify the magento provided testcases.

@lanken
Copy link
Contributor

lanken commented Aug 22, 2016

Add '/' at the end of the backend url
http://localhost:8080/magento2.1.0ce/admin_iaa328/admin/

I guess, it's mistake, but the host name according to your words should be localhost.con, not localhost.

Also, have you set your credentials to log in to the admin?
http://devdocs.magento.com/guides/v2.1/mtf/mtf_quickstart/mtf_quickstart_config.html#mtf_quickstart_config_configxml

Could you, please, attach the curl response error. It should be in the folder dev/tests/functional/var/log/magento /module/test_case/variation/curl-response

@shahankit783
Copy link
Author

shahankit783 commented Aug 22, 2016

Yes in phpunit.xml it is the same as you have mentioned .

app_backend_url ="http://localhost.com:8080/magento2.1.0ce/admin_iaa328/"
app_frontend_url ="http://localhost.com:8080/magento2.1.0ce/"

@lanken
Copy link
Contributor

lanken commented Aug 22, 2016

What is about curl response?
Also, please, check http://devdocs.magento.com/guides/v2.1/mtf/mtf_quickstart/mtf_quickstart_magento.html especially Admin Account Sharing

@shahankit783
Copy link
Author

Yes, I have applied the same settings as mentioned in that guide.
When i try to run the specific group like "Cross-sells_(MX)" it does open the browser but it won't open any URL inside browser.

The following is stack of error:
D:\wamp64\www\magento2.1.0ce\dev\tests\functional>vendor\bin\phpunit --group Cross-sells_(MX)

PHPUnit 4.1.0 by Sebastian Bergmann.

Configuration read from D:\wamp64\www\magento2.1.0ce\dev\tests\functional\phpunit.xml

Time: 52.57 seconds, Memory: 55.75MB

There was 1 error:

  1. Magento\Catalog\Test\TestCase\Product\AddToCartCrossSellTest::test with data set "AddToCartCrossSellTestVariation1"
    AddToCartCrossSellTestVariation1)
    Exception: Configuration settings are not applied! Url: http://localhost.com:8080/magento2.1.0ce/admin_iaa328/admin/sysem_config/save/section/admin

D:\wamp64\www\magento2.1.0ce\dev\tests\functional\tests\app\Magento\Config\Test\Handler\ConfigData\Curl.php:126
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\tests\app\Magento\Config\Test\Handler\ConfigData\Curl.php:42
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\Fixture\InjectableFixture.php:257
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\lib\Magento\Mtf\Util\Protocol\CurlTransport\WebapiDecorator.php:125
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\lib\Magento\Mtf\Util\Protocol\CurlTransport\WebapiDecorator.php:102
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\lib\Magento\Mtf\Util\Protocol\CurlTransport\WebapiDecorator.php:89
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\ObjectManager\Factory.php:245
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\ObjectManager\ObjectManager.php:53
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\ObjectManager\Factory.php:156
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\ObjectManager\Factory.php:236
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\ObjectManager\ObjectManager.php:53
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\Factory\AbstractFactory.php:48
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\Fixture\InjectableFixture.php:257
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\tests\app\Magento\Catalog\Test\Fixture\Product\CategoryIds.php:60
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\ObjectManager\Factory.php:245
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\ObjectManager\ObjectManager.php:53
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\Factory\AbstractFactory.php:48
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\Fixture\InjectableFixture.php:225
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\Fixture\InjectableFixture.php:162
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\ObjectManager\Factory.php:245
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\ObjectManager\ObjectManager.php:53
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\Factory\AbstractFactory.php:48
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\Fixture\FixtureFactory.php:51
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\tests\app\Magento\Catalog\Test\TestCase\Product\AbstractProductPromot
dProductsTest.php:107
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\tests\app\Magento\Catalog\Test\TestCase\Product\AddToCartCrossSellTes
.php:67
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\TestCase\Injectable.php:217
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\TestCase\Functional.php:288
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\TestCase\Functional.php:202
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\TestCase\Injectable.php:201
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\magento\mtf\Magento\Mtf\TestCase\Injectable.php:158
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\phpunit\phpunit\src\TextUI\Command.php:179
D:\wamp64\www\magento2.1.0ce\dev\tests\functional\vendor\phpunit\phpunit\src\TextUI\Command.php:132
?[37;41m ?[0m
?[37;41mFAILURES! ?[0m
?[37;41mTests: 1, Assertions: 0, Errors: 1.?[0m

@lanken
Copy link
Contributor

lanken commented Aug 22, 2016

It's not a curl response, it's stack trace response.
It should be in the folder like
dev/tests/functional/var/log/magento /Catalog/AddToCartCrossSellTest/AddToCartCrossSellTestVariation1/curl-response

@shahankit783
Copy link
Author

shahankit783 commented Aug 22, 2016

Please see the attached file which shows Curl Response.
Curl_Response.txt

From the Curl Response as you can see it is trying to go to the "http://localhost:8080/magento2.1.0ce/pub/static/frontend/Magento/luma/en_US" url , but as it is not found it gives 404 error.

@lanken
Copy link
Contributor

lanken commented Aug 22, 2016

According to response to send request to "http://localhost:8080/magento2.1.0ce/" not to "http://localhost.com:8080/magento2.1.0ce/"

@shahankit783
Copy link
Author

When i manually run the both URLs ("http://localhost:8080/magento2.1.0ce/" or "http://localhost.com:8080/magento2.1.0ce/") in browser it works same and fine.

But when you read the Curl_Response file it shows the base URL is"http://localhost:8080/magento2.1.0ce/pub/static/frontend/Magento/luma/en_US". so Until pub URL is working. but after that the URL is not working.

@lanken
Copy link
Contributor

lanken commented Aug 22, 2016

Have you reinstalled Magento after changing host name or updated Magento base URLs?

@shahankit783
Copy link
Author

when i tried to change the URLs , I observed that localhost:8080 works same as localhost.com:8080 from WAMP server. So I have just changed the URLs in configuration file only

@lanken
Copy link
Contributor

lanken commented Aug 22, 2016

Please, check whether the localhost.com is specified as a Magento backend url, if not change it and flush cache.

@shahankit783
Copy link
Author

shahankit783 commented Aug 22, 2016

Yes, it is localhost.com only. Please see the below file configurations:

phpunit.xml :
env name="app_frontend_url" value="http://localhost.com:8080/magento2.1.0ce/"
env name="app_backend_url" value="http://localhost.com:8080/magento2.1.0ce/admin_iaa328/"
env name="testsuite_rule" value="basic"
env name="testsuite_rule_path" value="Magento/Mtf/TestSuite/InjectableTests"
env name="log_directory" value="var/log"
env name="events_preset" value="base"
env name="module_whitelist" value="Magento_Install,Magento_Upgrade"
env name="basedir" value="var/log" />
env name="credentials_file_path" value="./credentials.xml"
env name="mage_mode" value="developer"

config.xml :
appBackendUrl=http://localhost.com:8080/magento2.1.0ce/admin_iaa328/
backendLoginUrl=admin/auth/login

And I haven't made any changes in credentials.xml .

@lanken
Copy link
Contributor

lanken commented Aug 22, 2016

I meant in Magento application, not in functional test configuration.

@shahankit783
Copy link
Author

shahankit783 commented Aug 22, 2016

Yes , I have made the changes in Base URLs under Stores->Configuration->General->Web.
Now Base URL is : http://localhost.com:8080/magento2.1.0ce/

But it gives the same error as it was giving earlier.

@lanken
Copy link
Contributor

lanken commented Aug 22, 2016

The base URL should be localhost.com, like in phpunit.xml (Or it was previously localhost.com?)
Also don't forget to clear cache after changing URL.

@shahankit783
Copy link
Author

Yes it is same as phpunit.xml

In Configuration file:
env name="app_frontend_url" value="http://localhost.com:8080/magento2.1.0ce/"
env name="app_backend_url" value="http://localhost.com:8080/magento2.1.0ce/admin_iaa328/"

In Magento application:
http://localhost.com:8080/magento2.1.0ce/

@shahankit783
Copy link
Author

Thank you so much for spending your valuable time to fix this issue and helping me out from this problem.

It started executing test cases.

PS: Though it is failing , at least it started running.

Thank you again Dmytro!!! :)

magento-engcom-team pushed a commit that referenced this issue Oct 10, 2020
[Arrows] Fixes for 2.4 (pr95) (2.4-develop)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants