Cron not Reindexing Index #5836

ghost · 2016-07-27T12:50:01Z

The indexes are not being reindexed with the standard cron:run cron job. This is particularly noticeable when making a change to a category and it not being updated on the front end. I can add a static block or description to a category and save it in the backend and the updates will not appear on the frontend until the indexers have been manually reindexed. This could possibly be related to #2855.

Preconditions

Magento 2.1 CE
CentOS6
PHP 5.6.24
MySQL 5.6
Shared hosting with Plesk control panel
Using flat categories and products

Steps to reproduce

Make a change to a categories description or add a static content block
Save changes
Go to category on frontend to see result

Expected result

Category should be updated with new content

Actual result

Category is not updated with new content

I have noticed that the debug.log has been filling up quite fast with lines like the following:

[2016-07-27 11:33:01] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_design_config_grid_flat_1"},"is_exception":false} []
[2016-07-27 11:33:01] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_design_config_grid_flat_2"},"is_exception":false} []
[2016-07-27 11:33:01] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_design_config_grid_flat_3"},"is_exception":false} []
[2016-07-27 11:33:01] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_design_config_grid_flat_4"},"is_exception":false} []
[2016-07-27 11:33:01] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_design_config_grid_flat_1"},"is_exception":false} []
[2016-07-27 11:33:01] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_design_config_grid_flat_2"},"is_exception":false} []
[2016-07-27 11:33:01] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_design_config_grid_flat_3"},"is_exception":false} []
[2016-07-27 11:33:01] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_design_config_grid_flat_4"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_customer_grid_flat_1"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_customer_grid_flat_2"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_customer_grid_flat_3"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_customer_grid_flat_4"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_customer_grid_flat_1"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_customer_grid_flat_2"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_customer_grid_flat_3"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_customer_grid_flat_4"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_tmp_indexer_1"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_tmp_indexer_2"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_tmp_indexer_3"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_tmp_indexer_4"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_decimal_tmp_indexer_1"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_decimal_tmp_indexer_2"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_decimal_tmp_indexer_3"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_decimal_tmp_indexer_4"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_text_tmp_indexer_1"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_text_tmp_indexer_2"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_text_tmp_indexer_3"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_text_tmp_indexer_4"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_varchar_tmp_indexer_1"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_varchar_tmp_indexer_2"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_varchar_tmp_indexer_3"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_varchar_tmp_indexer_4"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_int_tmp_indexer_1"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_int_tmp_indexer_2"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_int_tmp_indexer_3"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_int_tmp_indexer_4"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_int_tmp_indexer_value_1"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_int_tmp_indexer_value_2"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_int_tmp_indexer_value_3"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_int_tmp_indexer_value_4"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_datetime_tmp_indexer_1"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_datetime_tmp_indexer_2"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_datetime_tmp_indexer_3"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_entity_datetime_tmp_indexer_4"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_flat_1_tmp_indexer_1"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_flat_1_tmp_indexer_2"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_flat_1_tmp_indexer_3"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_flat_1_tmp_indexer_4"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_flat_1_tmp_indexer_1"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_flat_1_tmp_indexer_2"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_flat_1_tmp_indexer_3"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_flat_1_tmp_indexer_4"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_flat_1_drop_indexer_1"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_flat_1_drop_indexer_2"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_flat_1_drop_indexer_3"},"is_exception":false} []
[2016-07-27 11:33:02] main.DEBUG: cache_invalidate:  {"method":"GET","url":"http:/","invalidateInfo":{"identifier":"DB_PDO_MYSQL_DDL_catalog_product_flat_1_drop_indexer_4"},"is_exception":false} []

I have setup a cron job just for reindexing as a work around for now and it seems to be working. My cron jobs are setup as follows:

* * * * * /opt/plesk/php/5.6/bin/php -d memory_limit=1G -c /opt/plesk/php/5.6/etc/php.ini /magento2/bin/magento cron:run >> /magento2/var/log/magento.cron.log
* * * * * /opt/plesk/php/5.6/bin/php -d memory_limit=1G -c /opt/plesk/php/5.6/etc/php.ini /magento2/update/cron.php >> /magento2/var/log/update.cron.log
* * * * * /opt/plesk/php/5.6/bin/php -d memory_limit=1G -c /opt/plesk/php/5.6/etc/php.ini /magento2/bin/magento setup:cron:run >> /magento2/var/log/setup.cron.log
* * * * * /opt/plesk/php/5.6/bin/php -d memory_limit=1G -c /opt/plesk/php/5.6/etc/php.ini /magento2/bin/magento indexer:reindex

The text was updated successfully, but these errors were encountered:

hostep · 2016-07-27T18:41:26Z

We also see this using CE 2.1.0 - PHP 7.0.8 - Mysql 5.7.13

When adding products, they don't appear on the frontend, until we manually run a bin/magento indexer:reindex every time.

Cronjobs are setup correctly and indexers are set to 'Update by Schedule'.

Sounds like a pretty important bug.

grasdaggel · 2016-07-28T08:29:17Z

We are experiencing the same problem on lower Mage2 versions (2.0.4 and 2.0.6). So far we haven't found a solution for that.

nmallepally · 2016-07-29T08:30:49Z

Yes, it is a important bug. I am experiencing the same issue as well, Magento 2.1 v, The cron is setup properly, but does not show products until it is re indexed.

boldhedgehog · 2016-07-29T12:47:32Z

Works for me if I set 'Use Separate Process' to 'No' for the index CRON group.

hostep · 2016-07-29T13:12:09Z

@boldhedgehog: tnx, but unfortunately it doesn't seem to fix our issue :(

ghost · 2016-07-29T13:19:30Z

Setting 'Use Separate Process' to 'No' for the index CRON group does not work for me either.

boldhedgehog · 2016-07-29T13:44:48Z

@hostep I had to wait a little and after 5-10 runs of cron:run Magento had added indexer tasks to cron_schedule:

schedule_id	job_code	status	messages	created_at	scheduled_at	executed_at	finished_at
1106	indexer_update_all_views	success	NULL	2016-07-29 15:34:45	2016-07-29 15:34:00	2016-07-29 15:34:47	2016-07-29 15:34:47
1105	indexer_reindex_all_invalid	success	NULL	2016-07-29 15:34:45	2016-07-29 15:34:00	2016-07-29 15:34:47	2016-07-29 15:34:47

hostep · 2016-07-29T13:57:40Z

@boldhedgehog: yes, I see them too, even with 'Use separate process' on 'Yes'. So this means the indexers are correctly executed through the cron, but something in the execution itself goes wrong I guess...

If I find some time in the coming days, I'll debug and try figure out what exactly goes wrong.

Tnx for the hints!

ghost · 2016-07-29T14:58:53Z

This is the same behaviour for me as well. The cron_schedule table says success for the status but the frontend does not update.

boldhedgehog · 2016-07-29T15:01:40Z

It has rebuilt my Elasticsearch index. This can be an index/caching issue, not a CRON/index issue you have now, in addition.

ojhaujjwal · 2016-08-10T16:07:12Z

I have the same problem in 2.1.0. I have setup the cron job nicely. I even tried to run cron:run manually more than once but the indexing was not done. However, running indexer:reindex did all the reindexing. So, I think that means a bug in cron:run which does not do the job of indexing.

ojhaujjwal · 2016-08-13T10:47:04Z

This seems a critical issue. Is there any workaround for fixing this until an official fix is done?

ghost · 2016-08-13T13:22:44Z

@ojhaujjwal I setup an additional cron job to run the indexer:reindex command for now. It fills up the debug.log file quite quickly when it's run though.

ojhaujjwal · 2016-08-13T14:59:56Z

@crtsl I noticed that if I add magento indexer:reindex to cron, the cache is invalidated everytime the command is run and I have to flush the page cache. This creates even more problems.

Ctucker9233 · 2016-08-16T21:00:48Z

I have noticed this behavior as well. I have to run indexer:reindex and it makes my debug.log incredibly large. It's a pain but I clear out information that is more than a month old. Magento 1 had an automated function to clear the logs. Wish they would bring that back.

ojhaujjwal · 2016-08-18T01:45:07Z

Can anybody tell me the status of this issue? Is there any internal-ticket for the issue? Is there any commits that may fix the issue?

gman-1986 · 2016-08-26T14:57:34Z

Has anyone found a solution to this problem yet?

ghost · 2016-08-26T17:28:05Z

@gman-1986 running php bin/magento indexer:reindex works to reindex but that's a manual approach, far from a fix.

I don't see an internal ticket reference for this issue though which is concerning considering it is an important bug.

patrickpeng2091 · 2016-08-29T07:31:36Z

I have the same problem in 2.0.7 ,PHP7.0.8 ,mysql 5.6.30

running bin/magento indexer:reindex ,
Customer Grid index has been rebuilt successfully in 00:00:00
Category Products index has been rebuilt successfully in 00:00:00
Product Categories index has been rebuilt successfully in 00:00:00
Product Price index has been rebuilt successfully in 00:02:17
SQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try restarting transaction, query was: DELETE FROM catalog_product_index_eav_idx
Stock index has been rebuilt successfully in 00:00:30
Catalog Rule Product index has been rebuilt successfully in 00:00:00
Catalog Product Rule index has been rebuilt successfully in 00:00:00
Catalog Search index has been rebuilt successfully in 00:00:00
Cleaned cache types:

gman-1986 · 2016-08-31T18:51:30Z

Does anyone know if this issue is fixed in 2.1.1.

ghost · 2016-09-02T14:13:14Z

The issue still appears to be there after upgrading to 2.1.1.

wildekin · 2016-09-10T10:46:59Z

+1 this, so annoying having to manually reindex via ssh every time I make a change to the site!

ghost · 2016-09-12T11:54:29Z

@oserediuk is there any update to this issue? Is there an internet ticket?

mbrehme · 2016-09-14T09:44:59Z

+1 This issue is critical for us

serzhik · 2016-09-14T10:26:51Z

I use temporary solution.
Create pub/reindex.php:

<?php
header('Content-Encoding: none;');

set_time_limit(0);

$handle = popen("php /var/www/html/bin/magento indexer:reindex", "r");

if (ob_get_level() == 0)
    ob_start();

while(!feof($handle)) {

    $buffer = fgets($handle);
    $buffer = trim(htmlspecialchars($buffer));

    echo $buffer . "<br />";
    echo str_pad('', 4096);

    ob_flush();
    flush();
    sleep(1);
}

pclose($handle);
ob_end_flush();

Run http://example.com/pub/reindex.php to reindex.

Still waiting for solving of this issue.

pitangaweb · 2016-09-29T21:32:42Z

I have the same issue with Magento 2.1.1 on cPanel.
Cron jobs run everytime without errors
Nothing in log files or php error log
indexer_reindex_all_invalid shows in cron_schedule table

But new products won't show if I do not manually reindex - php bin/magento indexer:reindex

gman-1986 · 2016-10-13T07:01:33Z

Does anyone know if the latest version 2.1.2 has fixed the problem?

ghost · 2016-10-13T17:42:40Z

@gman-1986 I just upgraded from 2.1.1 to 2.1.2 and this issue is not fixed.

romeof1980 · 2017-09-15T13:32:24Z

hi there!
we can confirm we were able to reproduce the issue on 2.1.8 too (after upgrading from 2.1.5).
truncating the table cron_schedule, setting the indexer on "update on save" and re-setting them on "update on schedule" and finally clearing page_cache fixed the issue once again for us (at the first try).
For completeness' sake: we still run with the following settings:
1 Stores -> Configuration -> Advanced -> System -> Cron (Scheduled Tasks) -> Cron configuration options for group:index -> Use Separate Process -> SET TO "YES"
2 Stores -> Configuration -> Advanced -> System -> Cron (Scheduled Tasks) -> Cron configuration options for group:default -> Use Separate Process -> SET TO "NO")

cheers!

karlsminton · 2017-09-21T13:12:30Z

Can anyone confirm whether or not this is only affecting the indexers? Do any other scheduled tasks work normally?

Edit: From further research it seems to just be affecting indexers

robbanl · 2017-09-21T17:29:11Z

How is it possible that this issue have existed since summer 2016 and not have been fixed by Magento? Following...

patadeloso · 2017-10-12T18:14:09Z

If the indexers work from CLI, but not from cron. Check the default system ulimits. I've found that to be an issue on some systems.

Nix-id · 2017-11-07T13:18:19Z

I have problem on M2.2.0 with cron job "indexer_update_all_views"
it's stuck in "running" status
also exists process
20404 ? Ss 0:00 /bin/sh -c /usr/bin/php /home/mag/www//bin/magento cron:run | grep -v "Ran jobs by schedule" >> /home/mag/www//var/log/magento.cron.log
20437 ? S 0:05 /usr/bin/php7.0 /home/mag/www/*/bin/magento cron:run --group=index --bootstrap=standaloneProcessStarted=1

i have some configurable products with many simple products assined. ~5k simple products.

I don't see any errors in log files

magento-engcom-team · 2017-11-23T12:30:47Z

@crtsl, thank you for your report.
The issue is already fixed in 2.2.1

dnadle · 2017-11-24T02:32:05Z

Will the fix be backported to 2.1.x?

jeffmalterre · 2017-12-06T20:17:49Z

Not sure if this is related or not... I'm having trouble running cron (2.2.1) I can reindex myself but when I try to run cron it just hangs. I checked php/magento logs and there was nothing in them, however... when I looked into mysql logs im getting (Got an error writing communication packets) timestamped at the exact time I'm trying to run cron.

Thanks,
Jeff

chlignell · 2017-12-07T21:29:00Z

Hi everybody!

I had the same problem with indexes (my case product indexes) not being reindexed with cron:run. This caused new products to not being shown on frontend without a manual index:reindex.

The problem was that version_id in mview_state table had a value much higher than the autoincrement value in catalog_product_category_cl table. Everytime magento 2 cron runs it checks whether the changelogs tables have higher values than mview_state indexes (version_id). In my case I was able to resolve my issue by setting version_id in mview_state to 0 for the faulty indexes.

This command solved my problem:
UPDATE mview_state SET version_id = '0' WHERE view_id = 'catalog_product_category';

I hope this can help someone else out there, I hade to spend several frustrating hours on this issue.

/Chris

gman-1986 · 2018-01-19T07:07:46Z

I still have the issue on 2.2.1 does anyone else?

rahimalizada · 2018-01-30T13:16:07Z

In my case column 'status' was stuck in 'suspended' for 'catalog_product_category' view in mview_state. Resetting the value to 'idle' fixed the issue. Additionally i set version_id to 0 and allowed the cron to reindex.
@chlignell Thanks for the tip!

toddbc · 2018-02-03T19:48:34Z

Yes - this can happen sometimes when the cron process either crashes or is terminated by oom-killer. For that scenario, you can also use magento indexer:reset instead of manually updating the database.

Note that before doing this, it's best to validate that no cron process is currently running.

unicoder88 · 2018-02-09T11:33:23Z

@magento-engcom-team , is it possible to see some references to commit or task number to understand what can cause this?
Currently I'm having very strange behaviour that sometimes AUTO_INCREMENT numbers are reset back to 1 for some indexes (_cl tables) and update by schedule no longer works for them (M2.1.7).

toddbc · 2018-02-09T14:19:24Z

@unicoder88 I've actually just opened #13577 to fix exactly that issue. You can see an explanation of why it happens there.

franckgarnier21 · 2018-03-08T12:08:41Z

You can check if the status of mview_state is not working after a script crash.

ericvhileman · 2018-03-17T16:52:33Z

We wrote an extension to fix these bugs, speed up performance, and control the execution of tasks: https://github.com/magemojo/m2-ce-cron

somnath12 · 2018-04-24T12:10:29Z

No need to run re indexing. to get instant help please contact here [email protected]

addinm · 2018-11-27T10:03:06Z

@crtsl Please see my bug report and the solution here: #19417

rksh1997 · 2019-03-27T09:11:55Z

I still have this issue on Magento 2.3.0

Crons are setup correctly and my custom indexer is set to update on schedule.

I have to run index:reindex my_custom_indexer everytime I want to run my custom indexer, the indexer is not triggered on save too although I have correct subscriptions in my mview.xml

It only works fine calling it through the command line.

JaromirMullerQuanti · 2019-06-12T12:49:45Z

I have same issue and found that magento cannot run separated process to execute index job. I had to switch settings to Use Separate Process = No and then scheduled indexes executes fine.

I also noticed that indexer_update_all_views has status run.

Magento 2.3.1

malwis · 2019-06-21T08:52:56Z

Hello, i need your Help:
Magento 2.3.1
php /var/www/webroot/ROOT/bin/magento indexer:reindex

Results:
Design Config Grid index has been rebuilt successfully in 00:00:00
Customer Grid index has been rebuilt successfully in 00:00:00

I can't reindex following Position:
catalog_category_product
catalog_product_category

Please tell my what i can to reindex this 2 Position. Thanks

AdeelMurtaza · 2019-08-20T20:39:51Z

We are facing similar issue where after a customer purchases items on our store, items gets disappeared on our store front from category pages. Only solution is to manually reindex which is not a solution but strangely it is only happening to one of our stores.

Magneto support yet to come up with a positive answer. We are on Magneto cloud 2.3.1 and yet facing such issue. I was hopeful after Adobe acquisition but no luck.

[TSG] MC-35316: Pagination products not work in admin create order page

oserediuk added the bug report label Jul 29, 2016

ghost closed this as completed Oct 13, 2016

magento-engcom-team added the Issue: Clear Description Gate 2 Passed. Manual verification of the issue description passed label Oct 3, 2017

peterjaap mentioned this issue Nov 16, 2017

Mview changelog tables are gone when they shouldn't be #12104

Closed

magento-engcom-team added Fixed in 2.2.x The issue has been fixed in 2.2 release line Issue: Confirmed Gate 3 Passed. Manual verification of the issue completed. Issue is confirmed labels Nov 23, 2017

magento-engcom-team closed this as completed Nov 23, 2017

ryan-readingroom mentioned this issue Apr 23, 2018

Magento 2.2.2 product price indexer not updating even though cron runs successfully #14817

Closed

HannesJa mentioned this issue Mar 13, 2019

"Index on Schedule" some products are not visible Smile-SA/elasticsuite#1301

Closed

pdohogne-magento pushed a commit that referenced this issue Jul 28, 2020

Merge pull request #5836 from magento-tsg/MC-35316

ab5120a

[TSG] MC-35316: Pagination products not work in admin create order page

Cron not Reindexing Index #5836

Cron not Reindexing Index #5836

Comments

ghost commented Jul 27, 2016

Preconditions

Steps to reproduce

Expected result

Actual result

hostep commented Jul 27, 2016

grasdaggel commented Jul 28, 2016

nmallepally commented Jul 29, 2016

boldhedgehog commented Jul 29, 2016 • edited Loading

hostep commented Jul 29, 2016

ghost commented Jul 29, 2016

boldhedgehog commented Jul 29, 2016

hostep commented Jul 29, 2016 • edited Loading

ghost commented Jul 29, 2016

boldhedgehog commented Jul 29, 2016 • edited Loading

ojhaujjwal commented Aug 10, 2016 • edited Loading

ojhaujjwal commented Aug 13, 2016

ghost commented Aug 13, 2016

ojhaujjwal commented Aug 13, 2016

Ctucker9233 commented Aug 16, 2016

ojhaujjwal commented Aug 18, 2016

gman-1986 commented Aug 26, 2016

ghost commented Aug 26, 2016

patrickpeng2091 commented Aug 29, 2016

gman-1986 commented Aug 31, 2016

ghost commented Sep 2, 2016 • edited by ghost Loading

wildekin commented Sep 10, 2016 • edited Loading

ghost commented Sep 12, 2016

mbrehme commented Sep 14, 2016

serzhik commented Sep 14, 2016 • edited Loading

pitangaweb commented Sep 29, 2016 • edited Loading

gman-1986 commented Oct 13, 2016

ghost commented Oct 13, 2016

romeof1980 commented Sep 15, 2017

karlsminton commented Sep 21, 2017 • edited Loading

robbanl commented Sep 21, 2017

patadeloso commented Oct 12, 2017

Nix-id commented Nov 7, 2017

magento-engcom-team commented Nov 23, 2017

dnadle commented Nov 24, 2017

jeffmalterre commented Dec 6, 2017

chlignell commented Dec 7, 2017

gman-1986 commented Jan 19, 2018

rahimalizada commented Jan 30, 2018

toddbc commented Feb 3, 2018

unicoder88 commented Feb 9, 2018 • edited Loading

toddbc commented Feb 9, 2018 • edited Loading

franckgarnier21 commented Mar 8, 2018

ericvhileman commented Mar 17, 2018

somnath12 commented Apr 24, 2018

addinm commented Nov 27, 2018 • edited Loading

rksh1997 commented Mar 27, 2019 • edited Loading

JaromirMullerQuanti commented Jun 12, 2019 • edited Loading

malwis commented Jun 21, 2019

AdeelMurtaza commented Aug 20, 2019

boldhedgehog commented Jul 29, 2016 •

edited

Loading

hostep commented Jul 29, 2016 •

edited

Loading

boldhedgehog commented Jul 29, 2016 •

edited

Loading

ojhaujjwal commented Aug 10, 2016 •

edited

Loading

ghost commented Sep 2, 2016 •

edited by ghost

Loading

wildekin commented Sep 10, 2016 •

edited

Loading

serzhik commented Sep 14, 2016 •

edited

Loading

pitangaweb commented Sep 29, 2016 •

edited

Loading

karlsminton commented Sep 21, 2017 •

edited

Loading

unicoder88 commented Feb 9, 2018 •

edited

Loading

toddbc commented Feb 9, 2018 •

edited

Loading

addinm commented Nov 27, 2018 •

edited

Loading

rksh1997 commented Mar 27, 2019 •

edited

Loading

JaromirMullerQuanti commented Jun 12, 2019 •

edited

Loading