[Migration\Exception] Integrity Check failed

[Jeffery718]

bin/magento migrate:data -r vendor/magento/data-migration-tool/etc/ce-to-ce/1.7.0.1/config.xml
Reset the current position of migration to start from the beginning

[2015-12-07 06:11:57][INFO][mode: data][stage: integrity check][step: EAV Step]: started
100% [============================] Remaining Time: 1 sec
[2015-12-07 06:11:58][INFO][mode: data][stage: integrity check][step: Customer Attributes Step]: started
100% [============================] Remaining Time: 1 sec
[2015-12-07 06:11:58][INFO][mode: data][stage: integrity check][step: Map Step]: started
100% [============================] Remaining Time: 1 sec
[2015-12-07 06:12:05][INFO][mode: data][stage: integrity check][step: Url Rewrite Step]: started
100% [============================] Remaining Time: 1 sec
[2015-12-07 06:12:05][INFO][mode: data][stage: integrity check][step: Log Step]: started
100% [============================] Remaining Time: 1 sec
[2015-12-07 06:12:05][INFO][mode: data][stage: integrity check][step: Ratings Step]: started
100% [============================] Remaining Time: 1 sec
[2015-12-07 06:12:05][INFO][mode: data][stage: integrity check][step: ConfigurablePrices step]: started
100% [============================] Remaining Time: 1 sec
[2015-12-07 06:12:05][INFO][mode: data][stage: integrity check][step: OrderGrids Step]: started
100% [============================] Remaining Time: 1 sec
[2015-12-07 06:12:05][INFO][mode: data][stage: integrity check][step: Tier Price Step]: started
100% [============================] Remaining Time: 1 sec
[2015-12-07 06:12:05][INFO][mode: data][stage: integrity check][step: SalesIncrement Step]: started
100% [============================] Remaining Time: 1 sec

[Migration\Exception]
Integrity Check failed

[Vincenkt]

Hi Jeffery-Li,

Please check /var/migration.log file and let me know your detail issue.

[holozaen]

Same Problem here (trying to migrate magento ce 1.9.0.1-shop. Processing breaks at SalesIncrement Step. Unfortunately migration.log does not help, as it only shows as last line

[INFO][mode: data][stage: integrity check][step: SalesIncrement Step]: started

There is no further info regarding the error visible in the log file...

[bh-ref]

I have the same problem. I am trying to do a ce 1.9.2.2 -> ce 2.0.0 migration.

I used the map.xml and map-eav.xml until I got no more error, but the exception is still thrown. I even re-setup the Magento 2 installation, and retried the migration, but no success:

cat var/migration.log shows:

if it is of any value for you, here is the content of var/migration-tool-progress.lock:
a:14:{s:33:"Migration\Step\Settings\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:31:"Migration\Step\Stores\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:28:"Migration\Step\Settings\Data";a:1:{s:14:"data migration";a:1:{s:6:"result";b:1;}}s:26:"Migration\Step\Stores\Data";a:1:{s:14:"data migration";a:1:{s:6:"result";b:1;}}s:28:"Migration\Step\Stores\Volume";a:1:{s:12:"volume check";a:1:{s:6:"result";b:1;}}s:28:"Migration\Step\Eav\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:33:"Migration\Step\Customer\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:28:"Migration\Step\Map\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:28:"Migration\Step\Log\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:32:"Migration\Step\Ratings\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:43:"Migration\Step\ConfigurablePrices\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:35:"Migration\Step\OrderGrids\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:34:"Migration\Step\TierPrice\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:39:"Migration\Step\SalesIncrement\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}}

[bh-ref]

I tracked down the following (problem? bug?):

In the step [mode: data][stage: integrity check][step: Url Rewrite Step] happens the following:

In vendor/magento/data-migration-tool/src/Migration/Step/UrlRewrite/Version191to2000.php, the perfom() method is executed. There is this check which returns false:

array_keys($this->source->getStructure(self::SOURCE)->getFields()) == $this->structure[MapInterface::TYPE_SOURCE][self::SOURCE]. But that is not true, the arrays are just in different order:

var_dump(array_keys($this->source->getStructure(self::SOURCE)->getFields())):

array(10) {
  [0]=>
  string(14) "url_rewrite_id"
  [1]=>
  string(8) "store_id"
  [2]=>
  string(11) "category_id"
  [3]=>
  string(10) "product_id"
  [4]=>
  string(7) "id_path"
  [5]=>
  string(12) "request_path"
  [6]=>
  string(11) "target_path"
  [7]=>
  string(9) "is_system"
  [8]=>
  string(7) "options"
  [9]=>
  string(11) "description"
}

var_dump($this->structure[MapInterface::TYPE_SOURCE][self::SOURCE]):

array(10) {
  [0]=>
  string(14) "url_rewrite_id"
  [1]=>
  string(8) "store_id"
  [2]=>
  string(7) "id_path"
  [3]=>
  string(12) "request_path"
  [4]=>
  string(11) "target_path"
  [5]=>
  string(9) "is_system"
  [6]=>
  string(7) "options"
  [7]=>
  string(11) "description"
  [8]=>
  string(11) "category_id"
  [9]=>
  string(10) "product_id"
}

You should probably optimize that comparison, because it eventually raises the [Migration\Exception] in the runIntegrity method of vendor/magento/data-migration-tool/src/Migration/Mode/Data.php.

[bh-ref]

You could e.g. use array_diff_key for that comparison.

[bh-ref]

I created a pull request:
#25

[quangdog]

I'm getting the same problem, but I'm upgrading from 1.7.0.2CE to 2.0.0.CE, so I don't think the array mismatch as described above applies. Here are some relevant bits:

Here are the contents of my migration-tool-progress.lock:
a:9:{s:28:"Migration\Step\Eav\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:33:"Migration\Step\Customer\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:28:"Migration\Step\Map\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:28:"Migration\Step\Log\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:32:"Migration\Step\Ratings\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:43:"Migration\Step\ConfigurablePrices\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:35:"Migration\Step\OrderGrids\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:34:"Migration\Step\TierPrice\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}s:39:"Migration\Step\SalesIncrement\Integrity";a:1:{s:15:"integrity check";a:1:{s:6:"result";b:1;}}}

[quangdog]

I stand corrected. The perform() method in Version191to2000.php is being called when I try to upgrade 1.7.0.2CE -> 2.0.0 CE.

[andidhouse]

same problem here from 1.7.0.2 to 2.0.
Is there any solution so far for this?

Thanks!

[bh-ref]

@andidhouse you could add these changes to src/Migration/Step/UrlRewrite/Version191to2000.php: https://github.com/magento/data-migration-tool-ce/pull/25/files

If that alone does not solve the problem, you may use this code to further determine which step exactly is generating the error: #19 (comment)

[ilol]

Hey @JJJeffery, @andidhouse !
If you got a solution please close the issue. Thanks

[Reveren]

I can confirm that the solution from @bh-ref worked for me. I went from 1.8.1 to 2.0.0

I patched using this: https://github.com/magento/data-migration-tool-ce/pull/25/files