Running Magento cron does not reindex indexer

[OgreMHDW]

I am very confused about a new experience in Magento 2. I have set up the Magento cron job to run every 1 minute using the command "magento cron:run". After the command is run by cron it does return "Ran jobs by schedule." After adding several products the indexer is not running to add those products to the site.

Per the page http://devdocs.magento.com/guides/v2.0/config-guide/cli/config-cli-subcommands-cron.html it states "The first command (magento cron:run) reindexes indexers" but that is not happening.

I am able to run the command, from http://devdocs.magento.com/guides/v2.0/config-guide/cli/config-cli-subcommands-index.html, magento indexer:reindex to have the products indexed and added to the store.

Is this a bug or is should I be setting up an additional cron job just to index the site?

[markshust]

Possibly related to #2672

[katmoon]

Hi,

Do you still experience the issue?

If yes, could you

• verify the indexers statuses after the cron job is run twice:

php bin/magento indexer:status

• check jobs statuses in cron_schedule table
• check error logs

Thanks,
Kateryna

[daim2k5]

@OgreMHDW i close the ticket for now because you didn't answered @katmoon . If you have still the problem please reopen the issue.

[bondimedical3]

I am finding the same problem. If you add a new product it does not show. If you run cron by SSH it's supposed to reindex data when it is set to update on schedule and make the product show. It's only when you reindex by SSH the product will show.

[pynej]

Still not working on 2.0.2. The cron job runs without error but the Catalog Rule Product never rebuilds. Manually running indexer:reindex solves the problem.

[katmoon]

Please note that You must run cron twice: the first time to discover tasks to run and the second time to run the tasks themselves.

I could not reproduce the issue with catalog rules on the freshest Magento version available on GitHub today.

Could you check jobs statuses in cron_schedule table and let me know if there's any error, please?

[bondimedical3]

Running cron by SSH twice works and reindexes data.
I have setup cron to run every 1 minute in cPanel which seems to be working as sales emails are being sent.
However, this does not reindex data and new products do not appear?

[katmoon]

This is a bug. To fix it, we must understand when it happens.
An error message could help. It can be in php error_log, in Magento logs or in cron_schedule table.

[bondimedical3]

As far as I can see there are no errors. All entries for indexer in cron_schedule table have status "success"

[pynej]

There are some errors though I'm mostly at a loss to troubleshoot them.

2079    update_last_visit_at    pending NULL    2016-03-25 05:55:00 2016-03-25 05:53:34 NULL    NULL
2780    aggregate_sales_report_coupons_data running Class Magento\SalesRule\Crone\AggregateSalesReportCouponsData does not exist    2016-03-25 20:00:00 2016-03-25 19:52:03 NULL    NULL
3902    aggregate_sales_report_coupons_data running Class Magento\SalesRule\Crone\AggregateSalesReportCouponsData does not exist    2016-03-26 20:00:00 2016-03-26 19:50:03 NULL    NULL
4996    catalog_index_refresh_price error   Bad value was supplied. 2016-03-27 20:00:00 2016-03-27 19:45:03 2016-03-27 20:00:03 NULL
5013    aggregate_sales_report_coupons_data running Class Magento\SalesRule\Crone\AggregateSalesReportCouponsData does not exist    2016-03-27 20:00:00 2016-03-27 19:45:03 NULL    NULL
5415    newsletter_send_all pending NULL    2016-03-28 04:30:00 2016-03-28 04:26:02 NULL    NULL
5416    captcha_delete_old_attempts pending NULL    2016-03-28 04:30:00 2016-03-28 04:26:02 NULL    NULL
5417    captcha_delete_expired_images   pending NULL    2016-03-28 04:30:00 2016-03-28 04:26:02 NULL    NULL

[katmoon]

@pynej thank you for posting the error.
Reindex of catalog_index_refresh_price failed and this blocks category products from being reindexed.
I created internal ticket MAGETWO-51405 for further investigation.

Thanks,
Kateryna.

[rganin]

@bondimedical3 I've noticed you mentioned cPanel. Is it a shared hosting? If yes, does it allow triggers in mysql database? Scheduled indexer mode is based on triggers usage that fill changelog with changed entity ids.

[rganin]

@pynej Class Magento\SalesRule\Crone\AggregateSalesReportCouponsData does not exist error was fixed in both develop and 2.0 branches. Please update your codebase (do not forget to cleanup config cache and var/generation folder).

[bondimedical3]

@rganin I am running my sites on a VPS

[rganin]

@bondimedical3 , still can't reproduce the issue you've described. To verify it's an indexer fault and not some rights-related issue could you please check that "indexer"-related cron jobs in cron_schedule table are generated and have been executed (see the execution time), query something like selectjob_code,status,executed_atfrom cron_schedule;

[bondimedical3]

@rganin , all indexer jobs in the cron_schedule table have status "success". Maybe it's easier if I give you admin access and you can have a look at it?

[ntzz]

@bondimedical3 @rganin Could it be part of my problem with launch correctly a cron to see the log?
http://magento.stackexchange.com/questions/113972/cron-magento2-didnt-work

[katmoon]

@ntzz It seems to be a different problem. You can submit a new issue here on github for your problem if it doesn't exist yet.

[antboiko]

@bondimedical3 We couldn't reproduce your issue. Looks like it's on your side.

Please try post your question on stack exchange or Magento community forums.

Best,
Anton.

[ntzz]

@antboiko i had already create an issue on stackoverflow a few comments before, could you check if you could find anything wrong there?

[bondimedical3]

@antboiko You need to investigate this issue further. Stack Exchange is already flooded with the same question from many people. You haven't even bothered to ask for my Admin details.

[pynej]

Ok, doing a composer update has fixed that issue.

I am still being a similar/related issue where when i add a new product and wait ann the con jobs run and all by index's claim they are valid, but the new product wont show up unless i manually run the reindex. I will try to get a more detailed explication next week.

[pynej]

Ok, I have identified a second issue with my cron configuration that was causing the jobs to fail but not produce any appropriate error messages.

I'm suing the following cron script:

*/1 * * * * php /usr/local/www/store.scope1.com/bin/magento cron:run >> /usr/local/www/store.scope1.com/var/log/cron.log
0 * * * * /usr/local/bin/php  -c /usr/local/etc/php.ini /usr/local/www/store.scope1.com/update/cron.php >> /usr/local/www/store.scope1.com/var/log/cron.log
*/1 * * * * php /usr/local/www/store.scope1.com/bin/magento setup:cron:run >> /usr/local/www/store.scope1.com/var/log/cron.log

As you can see the php is installed at /usr/local/bin/php. The default settings for cron when running jobs runs them in a limited environment witch isn't exactly the same as running a program in an interactive console. As such even though I was specifying the full path to php and the job would run, some internal code/commands would try to run php I assume and wouldn't find it in $PATH. This was because the cron environment only had a $PATH of /bin:/usr/bin.

I was able to correct my issue by updating my path in the crontab file.

PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin

I get that my system my be a bit oddball and this is not a common problem, but it was breaking, and took a long time to track down. The major issue being when bin/magento cron:run ran it reported back Ran jobs by schedule incorrectly when it should have been generating an error indicating that some commands were not found. It was incorrectly returning a success message and trying to track down exactly what was failing was very difficult.

[rganin]

@bondimedical3, seems your issue is not related to cron but to search indexer. Internal ticket MAGETWO-52583, will be delivered soon.

[vzabaznov]

Hi @OgreMHDW, some issue with indexer was fixed and delivered, commit - 76cee92, thank you for reporting, please feel free to write here, or to open new issue if doesn't help.

@bondimedical3 @ntzz @pynej if you still have issue, please create another one issues.

[bondimedical3]

Where is the internal ticket?

[rganin]

@bondimedical3, MAGETWO-52583 is Magento's internal bugtracker ticket.
Ticket is resolved and delivered under 377228f commit, fix is available in develop branch.

[bondimedical3]

I have updated to Magento 2.0.6 which I assume would include all these fixes but the issue is still present?

[katmoon]

@bondimedical3 2.0.6 is a patch version that includes only some individual fixes.
The fix is available in develop branch. It will be included into new 2.1 release. You can also apply it manually on your server.

[bondimedical3]

I am getting a new message every day that "Catalog Rule Product" requires reindexing. Can you at least put a button in the Admin which allows people to manually do this?

[katmoon]

@bondimedical3 if you setup a cron job to run in background, you don't need to do it manually http://devdocs.magento.com/guides/v2.0/config-guide/cli/config-cli-subcommands-cron.html#config-cli-cron-bkg

You can do this to run cron from browser http://devdocs.magento.com/guides/v2.0/config-guide/secy/secy-cron.html

[dv8nate]

Hi team,

I'm getting the same issues, Im on a hosted server running Cpanel and CentOS 6.6, PHP7 I have tried the below. In the Error log it shows 2016-05-24 13:23:02 Ran jobs by schedule, However, no indexing occurs.

/usr/local/bin/php /home/downupne/public_html/bin/magento cron:run | awk '{ print strftime("%Y-%m-%d %H:%M:%S"), $0; fflush(); }' 2>&1 >> ~/magento_cron.log

/usr/local/bin/php /home/downupne/public_html/update/cron.php | awk '{ print strftime("%Y-%m-%d %H:%M:%S"), $0; fflush(); }' 2>&1 >> ~/magento_cron.log

/usr/local/bin/php /home/downupne/public_html/bin/magento setup:cron:run | awk '{ print strftime("%Y-%m-%d %H:%M:%S"), $0; fflush(); }' 2>&1 >> ~/magento_cron.log

My hosting provider has tried to run via SSH to get the below error message,

Well, I've executed your crons manually, it doesn't seem if you've correct coding.

This is the error :-
/usr/local/cpanel/bin/jailshell: -c: line 0: unexpected EOF while looking for matching `''
/usr/local/cpanel/bin/jailshell: -c: line 1: syntax error: unexpected end of file

You would need to check the syntax of your files.

I'm utterly lost, I need to be up and running soon for the launch of a new product.

Any help would be greatly appreciated.

[karthigesh]

how would i find the cron is running or not

[PierAlex]

I'm experiencing the same problem of bondimedical3.
My installation is on a VPS, MySql 5.6 PHP 7.08 Ubuntu 14.04 Nginx
Magento CE 2.1

Via SSH or Cron job, i have the same result: New products does not appear in the catalog.

/opt/plesk/php/7.0/bin/php /var/www/vhosts/myhost/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /var/www/vhosts/myhost/log/magento.cron.log

No messages in the log file.
No messages in the SSH console. (/opt/plesk/php/7.0/bin/php /var/www/vhosts/myhost/bin/magento cron:run | grep -v "Ran jobs by schedule")

in the table cron_shedule, after running twice, I can see that the job "indexer_reindex_all_invalid" reports the status "success".

But my new product doesn't appear in the catalog.

The new product is online only after reindex with:
/opt/plesk/php/7.0/bin/php -c /opt/plesk/php/7.0/etc/php.ini /var/www/vhosts/myhost/bin/magento indexer:reindex

It appears that indexers are not invalidated after adding a new product.

In fact, changing manually in the database the value "valid" to "invalid" for the "indexer_id" = "catalog_category_product" in the table "indexer_state" and running the cron job, my new product appears online!

Is there a fix for this?

Thanks,
PierAle

[bondimedical3]

Make sure you have the 3 cron jobs setup correctly

[PierAlex]

@bondimedical3 did you solve this problem?
I think my jobs are setted up correctly. I receive mail for change passwotrd, orders, newsletters... and I can run upgrades via web setup (Readiness check it's ok for cron jobs).

[bondimedical3]

Yes. They fixed the issue with 2.1

[PierAlex]

@bondimedical3 i'm running 2.1 and i've checked also 2.1.2 without success.

Does anybody know how to fix this problem?

[bondimedical3]

I would say its your cron jobs. The ones in the official documentation don't work. Try the ones in the screenshot - "extende" is your username

[PierAlex]

@bondimedical3 Thanks,
this is my sheduled job:
/opt/plesk/php/7.0/bin/php -c /opt/plesk/php/7.0/etc/php.ini /var/www/vhosts/mydomain/bin/magento cron:run >> /var/www/vhosts/mydomain/log/magento.cron.log
It is the same of yours, except for the log output.

Setup and update jobs, same way.

I think that it is working because it is also responsible for password exchange email, send and receive emails for orders, invoice, shipment, send newsletters and all of those tasks are correctly running.

[bondimedical3]

If the cron jobs are not setup correctly then everything will work like order emails, newsletters etc but products will not appear in the frontend. You will need to look at issue in more depth or have your Magento Developer look at the problem.

[pantaoran]

I think I have the same problem as @PierAlex that I can add new products but they don't appear in the frontend. Magento CE 2.1.

It seems that it's not invalidating the index correctly, since "indexer:status" shows all indexes as "ready", which means that the cron job will skip them.

When I force a reindexing of every index with "indexer:reindex" the new product appears on the frontend.

Edit: seems there are many people having this problem in #5836

[bondimedical3]

In the Magento Admin do you guys have Index Management mode set to "Update By Schedule" ?

[ghost]

Yes. I have tried both update by schedule and update on save. Neither works correctly. The new products do not show until the indexers are reindexed manually through CLI.

[pantaoran]

Yes, update by schedule, cron running every minute.

[romeof1980]

hi there, possible solution for not fine-tuned cron jobs (expecially for shared hostings) can be found here: #5836