Description
Hi,
When the cron job Klarna\Core\Cron\UpdateApiLog is run it creates a collection from the table klarna_logs and sets the result to class variable, this loads the entire table into memory, as it contains a lot of JSON, the result is it consumes a lot of RAM.
Klarna\Core\Cron\UpdateApiLog
private function updateIncrementId(array $incrementIds)
{
$this->searchCriteriaBuilder->addFilter('increment_id', null, 'null');
$searchCriteria = $this->searchCriteriaBuilder->create();
$logs = $this->logRepository->getList($searchCriteria);
foreach ($logs->getItems() as $item) {
if (!isset($incrementIds[$item->getKlarnaId()])) {
continue;
}
$item->setIncrementId($incrementIds[$item->getKlarnaId()]);
$this->logRepository->save($item);
}
}From method getList in
Klarna\Core\Model\LogRepository
$collection = $this->getCollection($searchCriteria);
$searchResults->setItems($collection->getItems());In our case this routine was using well over 4 GB of memory each time it ran the cron job, this caused memory related issues.
We solved it by adding a few addAttributeToSelect()'s to the collection, so that it no longer loads data it doesn't even use.
Consumtion from that job is now at less than 10% what it was (and that is including bootstrapping Magento).
For what it's worth you could optimize it further by doing a DISTINCT select for the incrementIDs.
Or even better, not even doing collections and selects just perform the task in SQL by doing an update from join.
UPDATE klarna_logs k
INNER JOIN klarna_logs kb ON k.klarna_id = kb.klarna_id
AND kb.increment_id IS NOT NULL
SET k.increment_id = kb.increment_id
WHERE k.klarna_id = kb.klarna_idThere are similar issues in the Klarna\Core\Cron\CleanLogs
ps.
Also note that the use of private methods made it impossible to do a simple interceptor or just overriding the problematic method.
ds.
Preconditions
1, use Magento open source edition 2.4.3
2, have klarna checkout installed and configured
Steps to reproduce
- Receive a lot of klarna payments in a relative short amount of time
- have cron jobs enabled
Expected result
No issues with unnecessary memory consumtion.
Actual result
Crashing cron jobs due to memory issues