Poor performance on sales order update - string to integer

[asim-vax]

When running a sales order update, the where clause entity_id is a string. It is then comparing a string to an integer and the index is not used.
same issue as #20969 just for a different query.
eg. WHERE (entity_id='99015')
query below:

UPDATE `sales_order` 
SET `state` = 'processing', `status` = 'processing', `coupon_code` = NULL, `protect_code` = '623debed81d2068bf3e910a02e860066', `shipping_description` = 'Shipping Option - FREE Nominated Delivery', `is_virtual` = '0', `store_id` = '3', `customer_id` = NULL, `base_discount_amount` = '0', `base_discount_canceled` = NULL, `base_discount_invoiced` = '0', `base_discount_refunded` = NULL, `base_grand_total` = '179.96', `base_shipping_amount` = '0', `base_shipping_canceled` = NULL, `base_shipping_invoiced` = '0', `base_shipping_refunded` = NULL, `base_shipping_tax_amount` = '0',
`base_shipping_tax_refunded` = NULL, `base_subtotal` = '149.97', `base_subtotal_canceled` = NULL, `base_subtotal_invoiced` = '149.97', `base_subtotal_refunded` = NULL, `base_tax_amount` = '29.99', `base_tax_canceled` = NULL, `base_tax_invoiced` = '29.99', `base_tax_refunded` = NULL, `base_to_global_rate` = '1', `base_to_order_rate` = '1', `base_total_canceled` = NULL, `base_total_invoiced` = '179.96', `base_total_invoiced_cost` = '0', `base_total_offline_refunded` = NULL, `base_total_online_refunded` = NULL, `base_total_paid` = '179.96', `base_total_qty_ordered` = NULL, `base_total_refunded` = NULL, 
`discount_amount` = '0', `discount_canceled` = NULL, `discount_invoiced` = '0', `discount_refunded` = NULL, `grand_total` = '179.96', `shipping_amount` = '0', `shipping_canceled` = NULL, `shipping_invoiced` = '0', `shipping_refunded` = NULL, `shipping_tax_amount` = '0', `shipping_tax_refunded` = NULL, `store_to_base_rate` = '0', `store_to_order_rate` = '0', `subtotal` = '149.97', `subtotal_canceled` = NULL, `subtotal_invoiced` = '149.97', `subtotal_refunded` = NULL, `tax_amount` = '29.99', `tax_canceled` = NULL, `tax_invoiced` = '29.99', `tax_refunded` = NULL, `total_canceled` = NULL, `total_invoiced` = '179.96',
 `total_offline_refunded` = NULL, `total_online_refunded` = NULL, `total_paid` = '179.96', `total_qty_ordered` = '1', `total_refunded` = NULL, `can_ship_partially` = NULL, `can_ship_partially_item` = NULL, `customer_is_guest` = '1', `customer_note_notify` = '1', `billing_address_id` = '182939', `customer_group_id` = '0', `edit_increment` = NULL, `email_sent` = '1', `send_email` = '1', `forced_shipment_with_invoice` = NULL, `payment_auth_expiration` = NULL, `quote_address_id` = NULL, `quote_id` = '168962', `shipping_address_id` = '182938', `adjustment_negative` = NULL, `adjustment_positive` = NULL,
 `base_adjustment_negative` = NULL, `base_adjustment_positive` = NULL, `base_shipping_discount_amount` = '0', `base_subtotal_incl_tax` = '179.96', `base_total_due` = '0', `payment_authorization_amount` = NULL, `shipping_discount_amount` = '0', `subtotal_incl_tax` = '179.96', `total_due` = '0', `weight` = '3', `customer_dob` = NULL, `increment_id` = '', `applied_rule_ids` = NULL, `base_currency_code` = 'GBP', `customer_email` = '', `customer_firstname` = NULL, `customer_lastname` = NULL, `customer_middlename` = NULL, `customer_prefix` = NULL, `customer_suffix` = NULL, `customer_taxvat` = NULL, 
`discount_description` = NULL, `ext_customer_id` = NULL, `ext_order_id` = NULL, `global_currency_code` = 'GBP', `hold_before_state` = NULL, `hold_before_status` = NULL, `order_currency_code` = 'GBP', `original_increment_id` = NULL, `relation_child_id` = NULL, `relation_child_real_id` = NULL, `relation_parent_id` = NULL, `relation_parent_real_id` = NULL, `remote_ip` = '', `shipping_method` = 'productmatrix_Free_Nominated_Delivery', `store_currency_code` = 'GBP', `store_name` = 'Vax UK\nVax UK \nVax UK Store view', `x_forwarded_for` = '', `customer_note` = NULL, `created_at` = '2019-08-28 16:46:51', 
`total_item_count` = '1', `customer_gender` = NULL, `discount_tax_compensation_amount` = '0', `base_discount_tax_compensation_amount` = '0', `shipping_discount_tax_compensation_amount` = '0', `base_shipping_discount_tax_compensation_amnt` = '0', `discount_tax_compensation_invoiced` = '0', `base_discount_tax_compensation_invoiced` = '0', `discount_tax_compensation_refunded` = NULL, `base_discount_tax_compensation_refunded` = NULL, `shipping_incl_tax` = '0', `base_shipping_incl_tax` = '0', `coupon_rule_name` = NULL, `base_customer_balance_amount` = NULL,
 `customer_balance_amount` = NULL, `base_customer_balance_invoiced` = NULL, `customer_balance_invoiced` = NULL, `base_customer_balance_refunded` = NULL, `customer_balance_refunded` = NULL, `bs_customer_bal_total_refunded` = NULL, `customer_bal_total_refunded` = NULL, `gift_cards` = '[]', `base_gift_cards_amount` = '0', `gift_cards_amount` = '0', `base_gift_cards_invoiced` = NULL, `gift_cards_invoiced` = NULL, `base_gift_cards_refunded` = NULL, `gift_cards_refunded` = NULL, `gift_message_id` = NULL, 
`gw_id` = NULL, `gw_allow_gift_receipt` = NULL, `gw_add_card` = NULL, `gw_base_price` = '0', `gw_price` = '0', `gw_items_base_price` = '0', `gw_items_price` = '0', `gw_card_base_price` = '0', `gw_card_price` = '0', `gw_base_tax_amount` = NULL, `gw_tax_amount` = NULL, `gw_items_base_tax_amount` = NULL, `gw_items_tax_amount` = NULL, `gw_card_base_tax_amount` = NULL, `gw_card_tax_amount` = NULL, `gw_base_price_incl_tax` = NULL, `gw_price_incl_tax` = NULL, `gw_items_base_price_incl_tax` = NULL, `gw_items_price_incl_tax` = NULL, `gw_card_base_price_incl_tax` = NULL,
 `gw_card_price_incl_tax` = NULL, `gw_base_price_invoiced` = NULL, `gw_price_invoiced` = NULL, `gw_items_base_price_invoiced` = NULL, `gw_items_price_invoiced` = NULL, `gw_card_base_price_invoiced` = NULL, `gw_card_price_invoiced` = NULL, `gw_base_tax_amount_invoiced` = NULL, `gw_tax_amount_invoiced` = NULL, `gw_items_base_tax_invoiced` = NULL, `gw_items_tax_invoiced` = NULL, `gw_card_base_tax_invoiced` = NULL, `gw_card_tax_invoiced` = NULL, `gw_base_price_refunded` = NULL, `gw_price_refunded` = NULL, 
`gw_items_base_price_refunded` = NULL, `gw_items_price_refunded` = NULL, `gw_card_base_price_refunded` = NULL, `gw_card_price_refunded` = NULL, `gw_base_tax_amount_refunded` = NULL, `gw_tax_amount_refunded` = NULL, `gw_items_base_tax_refunded` = NULL, `gw_items_tax_refunded` = NULL, `gw_card_base_tax_refunded` = NULL, `gw_card_tax_refunded` = NULL, `reward_points_balance` = NULL, `base_reward_currency_amount` = NULL, `reward_currency_amount` = NULL, `base_rwrd_crrncy_amt_invoiced` = NULL, 
`rwrd_currency_amount_invoiced` = NULL, `base_rwrd_crrncy_amnt_refnded` = NULL, `rwrd_crrncy_amnt_refunded` = NULL, `reward_points_balance_refund` = NULL, `paypal_ipn_customer_notified` = '0', `account_id` = NULL, `case_id` = NULL, `agent_id` = NULL, `agent_name` = NULL, `nvm_agent_id` = NULL, `is_drtv` = NULL, `foc_code` = NULL, `foc_reason` = NULL, `sf_agent_id` = NULL, `call_product_sku` = NULL, `is_custom_upgrade` = NULL, `tcs_agreement_date` = NULL, `exported_to_gp` = '1', `gp_order_id` = '', `pentagon_order_id` = '', `pentagon_processed_order` = '0', `sent_in_bv_postpurchase_feed` = '0' 
WHERE (entity_id='99015');

should finish

WHERE (entity_id = 99015);

Column	Type	Default Value	Nullable	Character Set	Collation	Privileges	Extra	Comments
entity_id	int(10) unsigned		NO			select	auto_increment	Entity Id

Preconditions (*)

1. magento 2.3.2
2. ubuntu 18.04
3. php 7.2

Steps to reproduce (*)

1. turn on query logging, or view in mysql queries being ran
2. update an order
3. look at query
4. entity_id is wrapped in quotes - making it a string

Expected result (*)

1. entity_id is an integer
2. no performance loss

Actual result (*)

1. entity_id is a string
2. slow performance

[m2-assistant]

Hi @asim-vax. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.3-develop instance - upcoming 2.3.x release

For more details, please, review the Magento Contributor Assistant documentation.

@asim-vax do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

• yes
• no

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[irving0mar]

@magento give me 2.6-develop instance

[magento-engcom-team]

Hi @irving0mar. Thank you for your request. I'm working on Magento 2.6-develop instance for you

[irving0mar]

@magento give me 2.2.6-develop instance

[magento-engcom-team]

Hi @irving0mar. Thank you for your request.
Unfortunately, I can only deploy instances for 2.3 release line

[csdougliss]

This issue is on Magento 2.3.2

[m2-assistant]

Hi @engcom-Charlie. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Add label Issue: Confirmed once verification is complete.

• 6. Make sure that automatic system confirms that report has been added to the backlog.

[magento-engcom-team]

✅ Confirmed by @engcom-Charlie
Thank you for verifying the issue. Based on the provided information internal tickets MC-19791 were created

Issue Available: @engcom-Charlie, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

[csdougliss]

Further examples of this:

SELECT `sales_order`.`entity_id` FROM `sales_order` WHERE (entity_id = '105465') FOR UPDATE

and:

UPDATE `quote` SET `store_id` = ?, `created_at` = '2019-09-02 17:04:24', `converted_at` = ?, `is_active` = ?, `is_virtual` = ?, `is_multi_shipping` = ?, `items_count` = ?, `items_qty` = ?, `orig_order_id` = ?, `store_to_base_rate` = ?, `store_to_quote_rate` = ?, `base_currency_code` = ?, `store_currency_code` = ?, `quote_currency_code` = ?, `grand_total` = ?, `base_grand_total` = ?, `checkout_method` = ?, `customer_id` = ?, `customer_tax_class_id` = ?, `customer_group_id` = ?, `customer_email` = ?, `customer_prefix` = ?, `customer_firstname` = ?, `customer_middlename` = ?, `customer_lastname` = ?, `customer_suffix` = ?, `customer_dob` = ?, `customer_note` = ?, `customer_note_notify` = ?, `customer_is_guest` = ?, `remote_ip` = ?, `applied_rule_ids` = ?, `reserved_order_id` = ?, `password_hash` = ?, `coupon_code` = ?, `global_currency_code` = ?, `base_to_global_rate` = ?, `base_to_quote_rate` = ?, `customer_taxvat` = ?, `customer_gender` = ?, `subtotal` = ?, `base_subtotal` = ?, `subtotal_with_discount` = ?, `base_subtotal_with_discount` = ?, `is_changed` = ?, `trigger_recollect` = ?, `ext_shipping_info` = ?, `customer_balance_amount_used` = ?, `base_customer_bal_amount_used` = ?, `use_customer_balance` = ?, `gift_cards` = ?, `gift_cards_amount` = ?, `base_gift_cards_amount` = ?, `gift_cards_amount_used` = ?, `base_gift_cards_amount_used` = ?, `gift_message_id` = ?, `gw_id` = ?, `gw_allow_gift_receipt` = ?, `gw_add_card` = ?, `gw_base_price` = ?, `gw_price` = ?, `gw_items_base_price` = ?, `gw_items_price` = ?, `gw_card_base_price` = ?, `gw_card_price` = ?, `gw_base_tax_amount` = ?, `gw_tax_amount` = ?, `gw_items_base_tax_amount` = ?, `gw_items_tax_amount` = ?, `gw_card_base_tax_amount` = ?, `gw_card_tax_amount` = ?, `gw_base_price_incl_tax` = ?, `gw_price_incl_tax` = ?, `gw_items_base_price_incl_tax` = ?, `gw_items_price_incl_tax` = ?, `gw_card_base_price_incl_tax` = ?, `gw_card_price_incl_tax` = ?, `use_reward_points` = ?, `reward_points_balance` = ?, `base_reward_currency_amount` = ?, `reward_currency_amount` = ?, `is_persistent` = ?, `account_id` = ?, `case_id` = ?, `agent_name` = ?, `nvm_agent_id` = ?, `is_drtv` = ?, `foc_code` = ?, `foc_reason` = ?, `sf_agent_id` = ?, `call_product_sku` = ?, `is_custom_upgrade` = ?, `tcs_agreement_date` = ?, `delivery_date` = '2019-09-04 00:00:00', `marketing_opt_in` = ?, `marketing_post_opt_in` = ?, `instalments_first_date` = ? WHERE (entity_id='178265')

entity_id should not be enclosed in quotes.

[pierzakp]

"#mm19pl"

[m2-assistant]

Hi @pierzakp. Thank you for working on this issue.
Looks like this issue is already verified and confirmed. But if you want to validate it one more time, please, go though the following instruction:

• 1. Add/Edit Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 2. Verify that the issue is reproducible on 2.3-develop branch

  Details

  - Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 3. If the issue is not relevant or is not reproducible any more, feel free to close it.

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[magento-engcom-team]

Hi @asim-vax, @pierzakp.

Thank you for your report and collaboration!

The issue was fixed by Magento team.

The fix will be available with the upcoming 2.3.4 release.

[csdougliss]

@magento-engcom-team do you have a link to the commit or PR?

[o-iegorov]

This issue was fixed by internal team and PR currently in delivery queue

[pierzakp]

It's sad that there was no label / information that core team is handling it, waste of my time.

[o-iegorov]

Already merged into mainline

[o-iegorov]

@pierzakp there was information about internal ticket created, but i am agree, probably for such cases should be information about progress of internal ticket

[hostep]

Looks like this is the merge commit (for the people interested): 0a390bb

[pemann]

There are lots of places in magento code where string is used instead of int when querying by id-fields. You can seen them by first enabling db query log php bin/magento dev:query-log:enable and then browse some time your magento store. After a while check db log with: grep "_id = '" var/debug/db.log and grep "_id IN ('" var/debug/db.log"

[csdougliss]

@pemann Is that with this commit applied or before?

[pemann]

@craigcarnell before. My comment was just a remark that there are similar problems all-over magento. Someone should look into this with wider perspective and try to fix them all.

[pierzakp]

@pemann my PR was fixing it in all cases have confirmed that after reviewing var/debug/db.log, for update and select statements.

Commit mentioned above is fixing it (still think that my PR was a cleaner solution) only for update statements, what about selects??

[o-iegorov]

@pierzakp The issue opened for update query, if you have proved measurement that for select queries this will increase performance please add new issue

[hostep]

@o-iegorov: it looks like a new issue was opened for select queries some days ago: #25199