Non-printables..

[willwade]

Non-printable characters..

I was thinking of creating a lookup table of aliases e.g:

key_aliases = {
    '\r': 0x24,
    '\t': 0x30,
    '\n': 0x24,
    'return' : 0x24,
    'tab' : 0x30,
    'space' : 0x31,
    'delete' : 0x33,
    'escape' : 0x35,
    'command' : 0x37,
    'shift' : 0x38,
    'capslock' : 0x39,
    'option' : 0x3A,
    'alternate' : 0x3A,
    'control' : 0x3B,
    'rightshift' : 0x3C,
    'rightoption' : 0x3D,
    'rightcontrol' : 0x3E,
    'function' : 0x3F,
    'home': 0x73,

which could then be used in tostring..
my thought though is whether this would work across different keyboard layouts.i.e. is it only the main letter keys that are fixed? Do you know at all?

Also - why is it pykeycode always gives 65535 for (some?) shifted forms e.g:

>>> keycode.tokeycode('!')
65535
>>> keycode.tokeycode('^')
65535

thanks again
will

[willwade]

Re: "different key codes for different layouts" see: http://stackoverflow.com/a/16125341/1123094 - note that there is a enum of codes that do change and some that don't.. not sure if this helps - it looks old.

[willwade]

correction above...
"pykeycode always gives 65535 for ALL shifted forms.."

I get it - but how do we get a shifted_codedict with a list of all shifted forms to. Possible?

[abarnert]

First, I'm not sure what you mean by "used in tostring". That function has to return a single name for whatever keycode you gave it; if you define multiple aliases for each keycode (like '\r' and 'return'), how is it supposed to decide which one to return?

As for keycodes for different layouts: classic Mac guaranteed that the special keys (return, F1, left arrow, etc.) were the same on all keyboards, but the normal keys (including things like KP -) weren't. As far as I know, OS X doesn't offer any such guarantees anywhere in the documentation, but the Carbon.h headers have the exact same list, with the same comment, so presumably you can still count on that as long as the Carbon framework exists (so at least up to 10.10). This means that you can build a table (in both directions) for the special keys and use that, and only fall back to the TIS functions for normal keys. (Notice that you still can't distinguish between, e.g., 0 and Keypad0 this way.)

Finally, for "shifted forms", that 65535 is actually an error. (Notice that the newer ctypes-based version raises an exception; I was just too lazy with the original C implementation.) There is no ! key on most keyboards, so there is no keycode for !.

If you can explain what you're trying to do, I might be able to help more. But I can try to give you some background on the basic concepts.

When you hit a physical key, the keyboard driver emits a keycode. That keycode has no meaning other than what physical key you happened to press. If you're on a US MacBook Pro, and you hit the key labeled S, whether you're using English—US or English—Dvorak or something else, you're going to get the same keycode, 1.

Meanwhile, there's always a current keyboard layout (I think these are called keyboard input sources nowadays… I learned this stuff on System 6, so you'll have to forgive me…). A keyboard layout is a mapping from modifier states (and dead key states) to keyboard tables. A keyboard table is a mapping from keycodes to characters or a few other things. So, the US keyboard layout's default table maps keycode 1 to character s, while its shift table maps the same keycode to S, the option table to ß, the shift-option table to Í, the control table to \x13, the shift-control table is the same as the control table, the control-option table is empty, etc. Meanwhile, the Dvorak keyboard's default table maps it to o, its shift table to O, and so on.

If you're interested in the "few other things", my knowledge is probably way too out of date to be accurate, but I can at least give you a vague idea. First, when I said "characters", that was cheating a bit; it's really "strings of UniChars". A UniChar is a UTF-16 code unit, and some Unicode code points take two UTF-16 code units, but they still represent a single character, and can still be mapped to a single key. Also, sometimes Apple likes to use decomposed normalized Unicode, so what looks like ä is actually an a followed by a composing umlaut. And in the old days, when you could edit keyboard layouts easily in ResEdit, some people set up simple macros, so ctrl+shift+i could be int and ctrl+shift+f for and so on. Keycodes can also map to modifier state transitions—that's how the input manager knows that the left shift key means to go from the option table to the shift-option table (actually, I think that stopped being true in OS 9). And they can map to dead key prefixes (which I won't explain, but try hitting option-E then A if you don't know what dead keys are).

Anyway, if you look at the function I'm calling, UCKeyTranslate, it actually takes a keycode and a modifier state (and a dead key state). I'm passing 0 for the modifier state. If you wanted to, you could write a wrapper that takes a keycode and a collection of modifier flags and looks up the character. And then you could look up all the keycodes under all the modifier combinations to build a bigger map, instead of just character to keycode, use character to a tuple of keycode and modifiers. The tricky bit is to detect when you can collapse certain modifier states (which I don't know how to do). In theory, there are 256 possible combinations of modifiers; in practice, there are only around 6 tables for each layout, and you don't want to spend 42x the setup time and memory for nothing.

Also, notice that there can often be multiple ways to type the same character. Most layouts have (or at least they did, in the old days) separate tables for shift and capslock, but most of the entries were the same, so shift+a and capslock+a were both A. On a German keyboard, ä has its own key, but you can also type it with the opt-u dead key. And so on. So, asking "what keycode (and modifiers and dead keys) produces this character?" is an ambiguous question.

Now that I've explained way too much and probably confused you even more, maybe if you explain what you're actually trying to do and why you think pykeycode is the answer, I can either help you move forward, or explain why you're doing the wrong thing, or admit that I'm as lost as you. :)

[willwade]

Sorry for being a little vague! I'm way out of my depth..!

OK - here's the end plan:

• I'm working on pyUserInput and in particular some methods in pykeyboard) (If you are really interested the rabbit hole goes a bit deeper..)
• It has several methods such as press_key (pass it a human readable form of the key and it looks up the correct keycode then types it), release_key (stop the hold down of the key), tap_key (press and release) and type_string (i.e. pass it a whole string such as "Hello World!" and it types it all out). Its a neat idea. The mac code has some of the ObjC embedded to do the pressing (better than the applescript approach IMHO as its faster..)
• The pykeyboard - mac code works by having a lookup table of a US keyboard layout of codes and the keyboard, which as you realise (!) is short sighted for anyone on a non-US/standard qwerty layout. In its current form it (possibly quite rightly) struggles to do a press_key('H') because it has no idea about shifted characters. You would have to do a press_key('shift') press_key('h') to do what you want.. which makes sense.. Because of the lookup table there are some aliases like 'return', 'f1' etc which do press the correct keycode for 'non-printable' which is handy.
• There is a function 'type_string' (see base.py) which tries its best to look up a shifted form of a character within a string. Its a neat idea but this uses a US only keyboard layout so £ (shift 3 on my UK layout) won't work.
• I really wanted to solve some of the above problems and figured that pykeycode could help some of this and so forked it with the compile-free version embedded in it as a solution. You can see this differs in that there is no longer a lookup table of all key characters - just the fixed ones for speed. So this now somewhat fixes the keyboard layout problem but leads to other problems, largely because a) it has no way of converting a string such as 'return' into an actual non-printable return key and b) shifted characters are still a pain. To explain a little more:
  • The example of type_string("Hello World!") no longer works because pykeycode doesn't have a way of working out what the code is for !. Now I could try and do what type_string tries to do by having a shifted and non_shifted table that it looks between then presses the shift when it encounters a shifted char but that leads to a problem again due to the keyboard layouts being different (you would need a lookup table for all different keyboard layouts which is nuts.. Or maybe as you say, it could do with a " a wrapper that takes a keycode and a collection of modifier flags and looks up the character". Agh.

In essence what would make my life super great was if pykeycode (or something else):

• Had a method that gave a keycode and any modifier states required for any character - including 'aliased' non-printable characters
• Supplied the character (or any non-printable alias) for any keycode

Maybe this is just too tricky! Thanks for the explanation of how this all works.. Its actually fairly straightforward - but the solution doesn't look like it is! I hope this helps clarify the situation - thanks for your time!

[abarnert]

The good news is, changing our UCKeyTranslate wrapper to take a modifier state is trivial.

The bad news is that the naive way of building the dict to go in the opposite direction is completely infeasible, because there are 2^32 possible modifier states for each of the 128 key codes.

So, the right way to solve this is to use the UCKeyboardLayout that we've gotten, parse it with ctypes, and use its modifier state to table mapping. See the docs (https://developer.apple.com/library/mac/documentation/Carbon/reference/Unicode_Utilities_Ref/Reference/reference.html#//apple_ref/c/func/UCKeyTranslate); it's all straightforward, but tedious and painful.

But there's a nice shortcut that I think will work for you. I'm pretty sure the only modifier combinations that have ever produced printable characters in any keyboard layout provided by Apple are none, shift, option, shift+option, and control. And I'm pretty sure the modifiers 0x2, 0x8, and 0x10 have been shift, option, and control on every Apple keyboard forever. So, all you have to do is handle modifier states 0x0, 0x2, 0x8, 0xa, and 0x10. So I checked in code that does this. (I left out control, because I didn't think you wanted it, but it's trivial to add it in.)

This is of course relying on undocumented behavior, but if I'm right and it's behavior that's remained unchanged from System 6 through OS X 10.10, I think it's safe.

The new keyCodeForChar returns a tuple of keycode and modifier state. So, your type_string can look something like this:

def type_string(s):
    for keycode, modifier in [keyCodeForChar(ch) for ch in s]:
        if modifiers & 2: emit_shift()
        if modifiers & 8: emit_option()
        emit_keycode(keycode)

(I used a list comprehension to try to map the whole string in advance, so type_string('Hääää') on a US keyboard will raise a KeyError immediately, instead of after typing the H.)

If you want this to work for dead keys as well (after all, ä is typable on a US keyboard, you just type the option-U dead key and then type the a), I don't think there's a similar quick&dirty solution; that really would require parsing the keyboard layout.

Getting back to your original point: As you can see, all we're doing is building up a dict of 512 Unicode characters to keycode-and-modifier-state pairs. If you want to add aliases for non-printables, and add names for non-character special keys (which, as you discovered, are fixed across all keyboards), that's trivial:

aliases = {
    u'space': u' ',
    u'tab': u'\t',
    # ...
}
for alias, c in aliases.items():
    codedict[alias] = codedict[c]

specials = {
    u'shift': 0x38,
    # ...
}
codedict.update(specials)

And now:

>>> keyCodeForChar('space')
(49, 0)

[abarnert]

Just for fun, I decided to see how hard it would be to parse the structures directly. And it's not that bad. See layout.py. It obviously needs a lot of cleanup if you want to use it for real, but it handles modifiers, and even simple dead-key states (like those on the US keyboard), and extending it to handle range-based dead-key states wouldn't be too hard.

I didn't bother with the functions; just look at the dicts directly. mapping[key] is either a (keycode, modifier state) pair, or a tuple of such pairs in the case of dead key sequences. (You'd want to fix that to be consistent—maybe always return a tuple of pairs, even if it's usually just one.) revmapping[keycode, modifier] is a string representing the character, or something printable if there is no character. (You get something like "<deadkey #4: ¨>" for option-U, and obviously option-U doesn't send that sequence of characters. But it's nice for debugging.)

In the case of characters that can be typed multiple ways, it prefers the earlier modifier tables to later, and it prefers non-dead-key sequences (e.g., on a US keyboard, a bare umlaut is shift-option-u, not option-u dead-key followed by space).

Notice that I switched from ctypes to struct for parsing this stuff, because ctypes is no good at dealing with C libraries that use the struct hack and/or treat structures as byte buffers, both of which are all over the Unicode Utilities types. In real-life code you'd want to add some comments explaining the formats rather than just unpacking 'IIHHH' into five cryptically-named variables, but I think you should be able to figure this out. In fact, it's a lot simpler; I'm just using ctypes to call a handful of functions and then never touching it again.

[willwade]

thats ruddy amazing. Thanks a million. I have to admit I've only briefly looked at it (this is a hobby project!) but I'll try and find some time to properly digest all that.. I think we may have worked out a better system for the whole of pyKeyCode actually.. thanks @abarnert :)

[willwade]

two (I'm sure stupid) questions:

   python mac_keycode.py ^
   u'^' ('^'): keycode 34, mod 8

should that not be mod 2 (its shift 6 on my layout) - not mod 8?

Can I just confirm as well:
2: Shift
8: Option (Alt)
10: Command
0: No modifier...??

?
thanks again.. I have to admit - until I get my head around some objc this is still making my head spin. Its really appreciated :)

[abarnert]

The first one is odd. I'll take a look and see what happens on my Mac when I get home. Does the same thing happen with the newer layout.py (the one that parses the keyboard layout structures), or only the older keycode.py version (the one that asks the Unicode services to do it for you)? Anyway, my first guess would be that on your keyboard, opt-6 is the caret dead-key prefix (which at least my older code would treat the same way as the caret itself), and therefore shift-opt 6 is probably a normal caret as well as shift-6 (in the same way that opt-E is the acute dead-key prefix, and shift-opt-E is an acute accent). If there are multiple ways to type the same key, it has to pick one of them. IIRC, layout.py iterates the modifier maps in the order you probably want (no mod, shift, opt, shift-opt, then everything else in numerical order), but the older code may have done so in whatever order the tables were defined or something stupid. Anyway, if the code is still treating dead-key-caret the same as plain-caret, and picking it even when plain-caret exists elsewhere, I should fix that.

For the second one: I think I forgot to explain that mod is a bitmap, so 00000001 (decimal 1) is… control or fn or command or something, 00000010 (2) is shift, 00000100 (4) is caps-lock, 00001000 (8) is option, 00010000 (16) is command or control. If two or more modifiers are active, you | them together, so 00001010 (decimal 10) is shift and option together. And 0 means none of them are active.

Finally, I don't think there's any Objective-C in either the original version (the C extension) or the latest version (the one that parses the layouts manually), just plain old C. The one in between did use PyObjC to let me use Python strings instead of ctypes strings, and make Python automatically release some objects instead of manually calling CFRelease, but I later decided it wasn't worth dragging PyObjC into it just for that.

[willwade]

Thanks @abarnert

Re: the bucky bit... Just to confirm - this is what I thought it was (from Win)

MOD_SHIFT = 0b00000001 # == 1
MOD_CONTROL = 0b00000010 # == 2
MOD_ALT = 0b00000100 # == 4
MOD_WIN = 0b00001000 # == 8

But its different on the mac?:

MOD_CONTROL = 0b00000001 # == 1
MOD_SHIFT = 0b00000010 # == 2
MOD_CAPS = 0b00000100 # == 4
MOD_ALT = 0b00001000 # == 8

Apologies - I totally forgot layout.py (pah! Sorry after all your hard work!) So I'm firing it up but getting a Unicode error on layout.py :

Traceback (most recent call last):
  File "layout.py", line 172, in <module>
    mapping, revmapping, modmapping = getlayout()
  File "layout.py", line 168, in getlayout
    ret = parselayout(layout_buf, ktype)
  File "layout.py", line 136, in parselayout
    revmapping[j, mod] = '<deadkey #{}: {}>'.format(nextstate, basekey)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xb4' in position 0: ordinal not in range(128)

https://github.com/abarnert/pykeycode/blob/ctypes/layout.py#L136

[abarnert]

Oops, I was carefully writing everything to work with both Python 2.x and 3.x, but apparently I only tested the last set of changes in 3.x… Yeah, you don't want to format a Unicode string into a str like that, because in 2.x it means trying to encode it to ASCII. Plus, revmapping is supposed to be a mapping from keycode/mod pairs to unicode anyway. Also, 2.7 didn't have str.isprintable (or unicode.isprintable). Try it now.

Meanwhile, control is 16, not 1; I'm pretty sure command is 1. But the others are right. They're not the same on Windows and OS X because each has a long chain of backward compatibility running back to the late 70s/early 80s, and Apple ][ keyboards and IBM PC keyboards weren't even remotely similar (different set of keys in a different layout, different electronics and a different way of talking to the computer, etc.).

[willwade]

I've done some tweaks to the layout.py code now - really just some wrapper functions to make it a bit more accessible.

In real-life code you'd want to add some comments explaining the formats rather than just unpacking > 'IIHHH' into five cryptically-named variables, but I think you should be able to figure this out.

I have to admit (embarrased to admit) that whole chunk is like black magic to me! I've tried looking at it drunk and its not helped..

Any tips on bringing in a aliases lookup table at all? I can't quite figure out how that should work..

One thing too.. I'm slowly getting my head around unicode - its making a bit more sense now. Does isprintable ever return true? I can't see how it would so everything gets returned hexified right?

[panathea]

Hey @abarnert,

Not sure if you still care too much about this project, but I'm looking for some guidance as I'm planning on using it as a base for a fix on a project called Plover. This issue might not be the best place to have this discussion, but here you get very close to what I need.

So, I'll give you the brief. I'm working on Plover, stenography software. Part of the stenography process is emulating a keyboard, so that given a string, like "Katz are awesome!" , we'll get the simulated keypresses no matter the user's layout. Right now, we just send key code 0, and set the Unicode string. This proves to be problematic on software that ignores the Unicode string, like VNC.

What I need is the ability to read in the user's layout, then be able to process a string, and if the character is available to write on the layout, then output it (I'm assuming using only key codes, no Unicode string), and if the letter is not on the current layout (other locales, or special symbols like 🐱), then fall back to the Unicode string method.

I'm all right with doing the work, just wondering if there's a place I can start. I realize you guys had a lot of issues with the extended key presses, I'm trying to consider at what point to cut off and just fall back to Unicode.

Any guidance would be thoroughly appreciated, cheers,
Ted

[panathea]

Hm, actually looking into your code on PyUserInput, I think I could work off of that branch. The only bit that concerns me is the use of Carbon APIs instead of Cocoa. That, and I guess I'd try to port it over to PyObjc in the process.