No output when testing with VIM_VSPEC=nvim for certain tests on some environments

[rbong]

I am assuming Neovim is partially supported because this issue was resolved, let me know if that's incorrect.

I have the following tests:

runtime! plugin/fugitive.vim

describe 'vspec'
  it 'test'
    edit README.md
    Git status
  end
end

I'm running the tests with

bundle exec vim-flavor install
~/.vim-flavor/repos/kana_vim-vspec/bin/vspec ~/.vim-flavor/repos/tpope_vim-fugitive t/flog.vim

I get:

Checking versions...
  Use kana/vim-vspec ... v1.9.2
  Use tpope/vim-fugitive ... v3.4
Deploying plugins...
  kana/vim-vspec v1.9.2 ... done
  tpope/vim-fugitive v3.4 ... done
Completed.

That's the end of output.

If I use a different test, I do get output:

describe 'vspec'
  it 'test'
    Expect 1 == 1
  end
end

...
Completed.
ok 1 - vspec test
1..1

Since this gives no output, I also get Parse errors: No plan found in TAP output when running vim-flavor test. I have seen this related issue, but the suggested fix of building nvim/vim from source does not resolve the issue.

Environment:

• I only get the error running tests using GitHub Actions
• I don't get the error:
  • Running tests using vim with GitHub Actions
  • Running nvim tests on my local environment
  • Running tests in a similar Docker container to GitHub Actions (ubuntu:bionic)
  • Using :!git instead of :Git
• Versions:
  • ruby 3.0 (set up using ruby/setup-ruby@v1 in GitHub Actions)
  • Gemfile
    • vim-flavor 4.0.0
    • rake 13.0
    • made with bundler 2.2.25
  • Flavorfile
    • vim-vspec v1.9.2
    • vim-fugitive v3.4

I have already put in significant time narrowing down this issue as much as I can, I am stuck determining what environmental factors are causing this and what in :Git is causing it. Any assistance narrowing this down further is greatly appreciated, I'm just not sure about my debugging options from here.

[rbong]

I just saw the :Debug command, I thought I would use it to provide more insight. Apparently the error does not occur when using fugitive#Command() directly:

runtime! plugin/fugitive.vim

describe 'vspec'
  it 'test'
    edit README.md
    Debug fugitive#Command(1, 1, +"0", 0, "", "status")
  end
end

#
 call fugitive#Resume()|silent checktime
ok 1 - vspec test
1..1

So maybe the error has to do with command definitions. I will keep trying to narrow this down further so this issue is not quite as gargantuan.

Edit: Debug execute('Git status') results in:

...
Completed.
#

No further output.

[rbong]

I am working around this for now with a combination of using different commands and using SKIP. The error probably has something to do with nvim job control, but fugitive job control is complex so reducing this issue further is difficult.

[kana]

I am assuming Neovim is partially supported because this issue was resolved, let me know if that's incorrect.

It should work unless there are tricky stuffs within your plugin or its dependency.

• I only get the error running tests using GitHub Actions
• I don't get the error:
  • [...]
  • Running nvim tests on my local environment
  • [...]

That's strange. I think the same error should happen on your local environment.

I'm running the tests with

bundle exec vim-flavor install
~/.vim-flavor/repos/kana_vim-vspec/bin/vspec ~/.vim-flavor/repos/tpope_vim-fugitive t/flog.vim

That is not the right way to run tests.
That command replaces ~/.vim/pack/flavors, so that it affects further (non-test) Vim invocations.

You should use the following command instead:

bundle exec vim-flavor test

This will install dependencies into ./.vim-flavor and they will be used only for tests.

If you don't get the error on your local environment even if you use the above command, the problem might be caused by environments rather than Vim plugins.

If you want to investigate raw output from a test, use the following command:

./.vim-flavor/pack/flavors/start/kana_vim-vspec/bin/vspec . ./.vim-flavor/pack/flavors/start/tpope_vim-fugitive t/flog.vim

[rbong]

That is not the right way to run tests.
bundle exec vim-flavor test

Running bundle exec vim-flavor test is how I normally execute tests:

Since this gives no output, I also get Parse errors: No plan found in TAP output when running vim-flavor test

In the related issue you also asked for the output of vspec t/test.vim so I just wanted to give you the output of both.

If you don't get the error on your local environment even if you use the above command, the problem might be caused by environments rather than Vim plugins.

I don't get the error locally when using bundle exec vim-flavor test. I do still get it in the other environment.

I figure it's environmental at this point, but I am at a loss as to how to debug further since the output is empty.

If you want to investigate raw output from a test, use the following command:

Is this command different than the one I'm using? There is no .vim-flavor directory when I install using bundle exe vim-flavor install in this environment.

[rbong]

It should work unless there are tricky stuffs within your plugin or its dependency.

Does "tricky stuff" include job control? Since Fugitive uses a lot of job control.

[kana]

Does "tricky stuff" include job control? Since Fugitive uses a lot of job control.

Perhaps. The problem might be caused by that job control, and only on specific environments. But I'm not sure.

I'll try look into the problem later.

[kana]

Is this command different than the one I'm using?

It depends on the context. If you always run the two commands:

bundle exec vim-flavor install
~/.vim-flavor/repos/kana_vim-vspec/bin/vspec ~/.vim-flavor/repos/tpope_vim-fugitive t/flog.vim

Fortunately, there is no difference.

But ~/.vim-flavor is not intended to be directly used. It's a directory to cache local clones of plugins, and its contents might be changed whenever vim-flavor is executed in different context.

There is no .vim-flavor directory when I install using bundle exe vim-flavor install in this environment.

Ah, sorry. I assumed that you did run bundle exec vim-flavor test at least once.

[rbong]

This issue is fixed for me now.

I was installing neovim previously with the following steps in GitHub Actions:

    - name: "add neovim repository"
      shell: "bash"
      run: |
        sudo add-apt-repository -y ppa:neovim-ppa/stable
    - name: "install neovim"
      shell: "bash"
      run: |
        sudo apt-get -y update
        sudo apt-get -y install neovim

I have switched to thinca/action-setup-vim and now nvim tests work fine.

I don't know what environmental factors could have been causing this issue but they seem to be resolved now.

Apologies for not following the CI guide, I was migrating from an old CI setup and not from scratch.

[kana]

I tried reproducing the problem on my local environment, but I couldn't. I also tried narrowing down the problem by deleting unrelated code from the plugin and running the test on GitHub Actions based on your setup, but it's hard to do so because of the large amount of job control code. Therefore I failed to take a good guess about what code causes the problem.

Anyway, I'm glad to hear that the problem doesn't happen by using thinca/action-setup-vim. In addition to resolving the problem, it's faster than the old setup because its result is cached by default.