Skip to content

[2.2] - Add command to view mview state and queue #12122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Dec 7, 2017
Merged

[2.2] - Add command to view mview state and queue #12122

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
9021561
Add command to view mview state and queue
convenient Nov 5, 2017
1e34fde
Make indexer status mview 5.5 compatible
convenient Nov 5, 2017
6d3bffa
Use factories/interfaces correctly
convenient Nov 6, 2017
dede2d1
Update code style
convenient Nov 7, 2017
fa1b310
Remove the copyright year from file headers
convenient Nov 8, 2017
f4857ec
Fix extends phpunit class
convenient Nov 11, 2017
c80710a
Add mview getListSize command
convenient Nov 11, 2017
63ec3f7
Remove indexer:status:mview command
convenient Dec 3, 2017
d3d300b
Update indexer:status to display schedule backlog
convenient Dec 3, 2017
c63330f
Remove status-mview from di.xml
convenient Dec 3, 2017
709f88a
Update method visibility
convenient Dec 3, 2017
831000b
Correctly assert table output
convenient Dec 3, 2017
62e67e1
Code style fixes
convenient Dec 3, 2017
4af04b1
Add ChangelogCounterInterface
convenient Dec 3, 2017
700fe22
Add command to view mview state and queue
ihor-sviziev Dec 4, 2017
ca50c9d
Remove backwards breaking ChangelogCounterInterface
convenient Dec 4, 2017
10df2ce
Replace getListSize with getList
convenient Dec 4, 2017
3d34bc8
Remove changes to Changelog
convenient Dec 4, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 77 additions & 12 deletions app/code/Magento/Indexer/Console/Command/IndexerStatusCommand.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@

use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Magento\Framework\Indexer;
use Magento\Framework\Mview;

/**
* Command for displaying status of indexers.
Expand All @@ -30,21 +32,84 @@ protected function configure()
*/
protected function execute(InputInterface $input, OutputInterface $output)
{
$table = $this->getHelperSet()->get('table');
$table->setHeaders(['Title', 'Status', 'Update On', 'Schedule Status', 'Schedule Updated']);

$rows = [];

$indexers = $this->getIndexers($input);
foreach ($indexers as $indexer) {
$status = 'unknown';
switch ($indexer->getStatus()) {
case \Magento\Framework\Indexer\StateInterface::STATUS_VALID:
$status = 'Ready';
break;
case \Magento\Framework\Indexer\StateInterface::STATUS_INVALID:
$status = 'Reindex required';
break;
case \Magento\Framework\Indexer\StateInterface::STATUS_WORKING:
$status = 'Processing';
break;
$view = $indexer->getView();

$rowData = [
'Title' => $indexer->getTitle(),
'Status' => $this->getStatus($indexer),
'Update On' => $indexer->isScheduled() ? 'Schedule' : 'Save',
'Schedule Status' => '',
'Updated' => '',
];

if ($indexer->isScheduled()) {
$state = $view->getState();
$rowData['Schedule Status'] = "{$state->getStatus()} ({$this->getPendingCount($view)} in backlog)";
$rowData['Updated'] = $state->getUpdated();
}
$output->writeln(sprintf('%-50s ', $indexer->getTitle() . ':') . $status);

$rows[] = $rowData;
}

usort($rows, function ($comp1, $comp2) {
return strcmp($comp1['Title'], $comp2['Title']);
});

$table->addRows($rows);
$table->render($output);
}

/**
* @param Indexer\IndexerInterface $indexer
* @return string
*/
private function getStatus(Indexer\IndexerInterface $indexer)
{
$status = 'unknown';
switch ($indexer->getStatus()) {
case \Magento\Framework\Indexer\StateInterface::STATUS_VALID:
$status = 'Ready';
break;
case \Magento\Framework\Indexer\StateInterface::STATUS_INVALID:
$status = 'Reindex required';
break;
case \Magento\Framework\Indexer\StateInterface::STATUS_WORKING:
$status = 'Processing';
break;
}
return $status;
}

/**
* @param Mview\ViewInterface $view
* @return string
*/
private function getPendingCount(Mview\ViewInterface $view)
{
$changelog = $view->getChangelog();

try {
$currentVersionId = $changelog->getVersion();
} catch (Mview\View\ChangelogTableNotExistsException $e) {
return '';
}

$state = $view->getState();

$pendingCount = $changelog->getListSize($state->getVersionId(), $currentVersionId);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

\Magento\Framework\Mview\ViewInterface::getChangelog returns \Magento\Framework\Mview\View\ChangelogInterface so method getListSize cannot be used here as it is declared in ChangelogCounterInterface and is unkown in this code.

Seems there is no clean and backward compatible solution to implement counting without list load. I propose in the scope of this PR implement counting as was done in the initial version with count($changelog->getList()) and process this PR to deliver this feature to M2. Once PR will be merged new PR may be created with a refactoring of changelog and performance optimization.

@convenient, @ihor-sviziev, @kandy are you agree with this plan?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @vkublytskyi I was really struggling to balance out clean/backwards-compatability as you noted.

If @kandy agrees my plan of action will be

  1. Roll back to count($changelog->getList()) with associated tests
  2. Update description/title to properly highlight what is occurring in this PR
  3. Wait for approval from all parties (or potentially for merge?)
  4. Backport changes to [2.1] - Add command to view mview state and queue #12050

Sound good?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, looks good for me


$pendingString = "<error>$pendingCount</error>";
if ($pendingCount <= 0) {
$pendingString = "<info>$pendingCount</info>";
}

return $pendingString;
}
}
179 changes: 159 additions & 20 deletions app/code/Magento/Indexer/Test/Unit/Console/Command/IndexerStatusCommandTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
use Magento\Framework\Indexer\StateInterface;
use Magento\Indexer\Console\Command\IndexerStatusCommand;
use Symfony\Component\Console\Tester\CommandTester;
use Symfony\Component\Console\Helper\HelperSet;
use Symfony\Component\Console\Helper\TableHelper;

class IndexerStatusCommandTest extends AbstractIndexerCommandCommonSetup
{
Expand All @@ -18,35 +20,134 @@ class IndexerStatusCommandTest extends AbstractIndexerCommandCommonSetup
*/
private $command;

/**
* @param \PHPUnit_Framework_MockObject_MockObject $indexerMock
* @param array $data
* @return mixed
*/
private function attachViewToIndexerMock($indexerMock, array $data)
{
/** @var \Magento\Framework\Mview\View\Changelog|\PHPUnit_Framework_MockObject_MockObject $stub */
$changelog = $this->getMockBuilder(\Magento\Framework\Mview\View\Changelog::class)
->disableOriginalConstructor()
->getMock();

$changelog->expects($this->any())
->method('getListSize')
->willReturn($data['view']['changelog']['list_size']);

/** @var \Magento\Indexer\Model\Mview\View\State|\PHPUnit_Framework_MockObject_MockObject $stateMock */
$stateMock = $this->getMockBuilder(\Magento\Indexer\Model\Mview\View\State::class)
->disableOriginalConstructor()
->setMethods(null)
->getMock();

$stateMock->addData($data['view']['state']);

/** @var \Magento\Framework\Mview\View|\PHPUnit_Framework_MockObject_MockObject $viewMock */
$viewMock = $this->getMockBuilder(\Magento\Framework\Mview\View::class)
->disableOriginalConstructor()
->setMethods(['getChangelog', 'getState'])
->getMock();

$viewMock->expects($this->any())
->method('getState')
->willReturn($stateMock);
$viewMock->expects($this->any())
->method('getChangelog')
->willReturn($changelog);

$indexerMock->method('getView')
->willReturn($viewMock);

return $indexerMock;
}

/**
* @param array $indexers
* @param array $statuses
*
* @dataProvider executeAllDataProvider
*/
public function testExecuteAll(array $indexers, array $statuses)
public function testExecuteAll(array $indexers)
{
$this->configureAdminArea();
$indexerMocks = [];
foreach ($indexers as $indexerData) {
$indexerMock = $this->getIndexerMock(
['getStatus'],
['getStatus', 'isScheduled', 'getState', 'getView'],
$indexerData
);

$indexerMock->method('getStatus')
->willReturn($statuses[$indexerData['indexer_id']]);
->willReturn($indexerData['status']);
$indexerMock->method('isScheduled')
->willReturn($indexerData['is_scheduled']);

if ($indexerData['is_scheduled']) {
$this->attachViewToIndexerMock($indexerMock, $indexerData);
}

$indexerMocks[] = $indexerMock;
}

$this->initIndexerCollectionByItems($indexerMocks);
$this->command = new IndexerStatusCommand($this->objectManagerFactory);

$objectManager = new \Magento\Framework\TestFramework\Unit\Helper\ObjectManager($this);

$this->command->setHelperSet(
$objectManager->getObject(
HelperSet::class,
['helpers' => [$objectManager->getObject(TableHelper::class)]]
)
);

$commandTester = new CommandTester($this->command);
$commandTester->execute([]);
$actualValue = $commandTester->getDisplay();
$expectedValue = sprintf('%-50s ', 'Title_indexerOne' . ':') . 'Ready' . PHP_EOL
. sprintf('%-50s ', 'Title_indexerTwo' . ':') . 'Reindex required' . PHP_EOL
. sprintf('%-50s ', 'Title_indexerThree' . ':') . 'Processing' . PHP_EOL
. sprintf('%-50s ', 'Title_indexerFour' . ':') . 'unknown' . PHP_EOL;

$this->assertStringStartsWith($expectedValue, $actualValue);
$linesOutput = array_filter(explode(PHP_EOL, $commandTester->getDisplay()));

$spacer = '+----------------+------------------+-----------+-------------------------+---------------------+';

$this->assertCount(8, $linesOutput, 'There should be 8 lines output. 3 Spacers, 1 header, 4 content.');
$this->assertEquals($linesOutput[0], $spacer, "Lines 0, 2, 7 should be spacer lines");
$this->assertEquals($linesOutput[2], $spacer, "Lines 0, 2, 7 should be spacer lines");
$this->assertEquals($linesOutput[7], $spacer, "Lines 0, 2, 7 should be spacer lines");

$headerValues = array_values(array_filter(explode('|', $linesOutput[1])));
$this->assertEquals('Title', trim($headerValues[0]));
$this->assertEquals('Status', trim($headerValues[1]));
$this->assertEquals('Update On', trim($headerValues[2]));
$this->assertEquals('Schedule Status', trim($headerValues[3]));
$this->assertEquals('Schedule Updated', trim($headerValues[4]));

$indexer1 = array_values(array_filter(explode('|', $linesOutput[3])));
$this->assertEquals('Title_indexer1', trim($indexer1[0]));
$this->assertEquals('Ready', trim($indexer1[1]));
$this->assertEquals('Schedule', trim($indexer1[2]));
$this->assertEquals('idle (10 in backlog)', trim($indexer1[3]));
$this->assertEquals('2017-01-01 11:11:11', trim($indexer1[4]));

$indexer2 = array_values(array_filter(explode('|', $linesOutput[4])));
$this->assertEquals('Title_indexer2', trim($indexer2[0]));
$this->assertEquals('Reindex required', trim($indexer2[1]));
$this->assertEquals('Save', trim($indexer2[2]));
$this->assertEquals('', trim($indexer2[3]));
$this->assertEquals('', trim($indexer2[4]));

$indexer3 = array_values(array_filter(explode('|', $linesOutput[5])));
$this->assertEquals('Title_indexer3', trim($indexer3[0]));
$this->assertEquals('Processing', trim($indexer3[1]));
$this->assertEquals('Schedule', trim($indexer3[2]));
$this->assertEquals('idle (100 in backlog)', trim($indexer3[3]));
$this->assertEquals('2017-01-01 11:11:11', trim($indexer3[4]));

$indexer4 = array_values(array_filter(explode('|', $linesOutput[6])));
$this->assertEquals('Title_indexer4', trim($indexer4[0]));
$this->assertEquals('unknown', trim($indexer4[1]));
$this->assertEquals('Schedule', trim($indexer4[2]));
$this->assertEquals('running (20 in backlog)', trim($indexer4[3]));
$this->assertEquals('2017-01-01 11:11:11', trim($indexer4[4]));
}

/**
Expand All @@ -59,27 +160,65 @@ public function executeAllDataProvider()
'indexers' => [
'indexer_1' => [
'indexer_id' => 'indexer_1',
'title' => 'Title_indexerOne'
'title' => 'Title_indexer1',
'status' => StateInterface::STATUS_VALID,
'is_scheduled' => true,
'view' => [
'state' => [
'status' => 'idle',
'updated' => '2017-01-01 11:11:11',
],
'changelog' => [
'list_size' => 10
]
]
],
'indexer_2' => [
'indexer_id' => 'indexer_2',
'title' => 'Title_indexerTwo'
'title' => 'Title_indexer2',
'status' => StateInterface::STATUS_INVALID,
'is_scheduled' => false,
'view' => [
'state' => [
'status' => 'idle',
'updated' => '2017-01-01 11:11:11',
],
'changelog' => [
'list_size' => 99999999
]
]
],
'indexer_3' => [
'indexer_id' => 'indexer_3',
'title' => 'Title_indexerThree'
'title' => 'Title_indexer3',
'status' => StateInterface::STATUS_WORKING,
'is_scheduled' => true,
'view' => [
'state' => [
'status' => 'idle',
'updated' => '2017-01-01 11:11:11',
],
'changelog' => [
'list_size' => 100
]
]
],
'indexer_4' => [
'indexer_id' => 'indexer_4',
'title' => 'Title_indexerFour'
'title' => 'Title_indexer4',
'status' => null,
'is_scheduled' => true,
'view' => [
'state' => [
'status' => 'running',
'updated' => '2017-01-01 11:11:11',
],
'changelog' => [
'list_size' => 20
]
]
],
],
'Statuses' => [
'indexer_1' => StateInterface::STATUS_VALID,
'indexer_2' => StateInterface::STATUS_INVALID,
'indexer_3' => StateInterface::STATUS_WORKING,
'indexer_4' => null,
]
],
];
}
Expand Down
Loading