We'd like to ensure that voice commands don't break. Probably the most comprehensive way to do this would be to convert all of our cursorless VSCode recorded tests into voice command tests. We'd basically take the spokenForm field of the test, call mimic on it, and ensure that its output matches the command in the test. We'd need to do some bulk processing to fix the spoken forms that didn't come from knausj
This effort would require talonvoice/talon#375 in order to run in CI, but will start by getting some tests running locally to function as a starting point for a conversation around our needs for a talon CI runner
See also knausj test setup, esp:
Also maybe worth having a look at contract / hypothesis testing?
Approach
We'd like to ensure that voice commands don't break. Probably the most comprehensive way to do this would be to convert all of our cursorless VSCode recorded tests into voice command tests. We'd basically take the
spokenFormfield of the test, callmimicon it, and ensure that its output matches thecommandin the test. We'd need to do some bulk processing to fix the spoken forms that didn't come from knausjThis effort would require talonvoice/talon#375 in order to run in CI, but will start by getting some tests running locally to function as a starting point for a conversation around our needs for a talon CI runner
See also knausj test setup, esp:
Also maybe worth having a look at contract / hypothesis testing?
Approach
cursorless.generate_spoken_form_teststhat iterates recorded test cases and populates a tmp directory with spoken form tests, returning the path to that directory. Might look atcursorless/packages/cursorless-engine/src/core/Cheatsheet.ts
Line 58 in 037389f
user.cursorless_default_vocabularythat switches all spoken forms to their normal forms (eg stock knausj alphabet, no modifier overrides)cursorless.generate_spoken_form_testsVSCode actionuser.cursorless_default_vocabularymimicon the spoken form, setting up the payload checker with the payload. Possibly we should run 3) once per test casecursorless-talon-developernormalizeSpokenFormsand check it in to version control so we can use it if necessary in the future if spoken forms change for some reason"cursorless record"enables theuser.cursorless_default_vocabularytag so that we don't end up recording tests with bad spoken forms in the future