Named parameters are not highlighted for built-in functions like dict

[FichteFoll]

In the following snippet named parameters are not highlighted for built-in functions like they are for any other function. This works correctly in the standard Python definition.

The reason is likely that you are consuming the dict keyword and the construct is not recognized as a function but a tuple (or actually nothing at all because you don't match tuples).

dict(param="")
other_func(param="")

[MattDMo]

I've looked briefly at the source of the two different language definitions, but I haven't found the source of the change yet. I have a bunch of free time coming up this week, so I'll try and nail it down then.

[facelessuser]

This is because the current python syntax file defines dict as a type but not also as a builtin function. What is more, the syntax file does not properly treat builtin functions like all other functions. I have fixed this in my Python language. @MattDMo, you should pick up the change in these two: facelessuser/sublime-languages@95bf5b6 and facelessuser/sublime-languages@d48e0bd

[facelessuser]

@MattDMo, you can probably just diff mine against yours to see the changes more clearly. I don't pull in everything you do (I am not always sure of the reasoning behind some of the things you add), but for the most part, mine is pretty close to yours as I sync up useful stuff on occasions.

[FichteFoll]

I would have tried to match a generic function signature with lookaheads and then match for built in functions as keywords. I'd need access to my debug environment (home) to test this if your patch has the same effects though.

[facelessuser]

Meh...It doesn't have to be implemented like mine. It's a personal branch, so sometimes I do things quick and dirty and clean up later...sometimes I'm lazy and don't clean up. I did minimal work to make this happen.

The point is that the solution works whether you pretty it up or not. However you decide to do it, the language has to capture those builtin types as functions when they are followed by (). So you need to make sure the types don't preceed (). And then something needs to captures them as a functions later. MattDMo likes to color everything, so I imagine this would allow him to highlight builtin functions differently if he desired in his neon theme (I really don't know what he colors or not). But letting them get captured by the generic function would probably work to as long as there is no builtin rule that captures them first. But print is special in Python 2 and must be specially captured to work as a keyword function , but can be avoided if using futures version of print or Python 3.

[FichteFoll]

Sure, the print keyword from py2 is special, but as far as I can see we have no way to differentiate between the py2 print keyword followed by a tuple or a function call anyway. So we only need to match that separately.

[facelessuser]

Sure, the print keyword from py2 is special, but as far as I can see we have no way to differentiate between the py2 print keyword followed by a tuple or a function call anyway.

I am differentiating it in my patch...
It first tries to match print("my string stuff") if that fails, it matches it as print "my string stuff"

Again, I'm just presenting what I did to offer a solution, I leave it the PythonImproved to improve upon the final solution. What is better is also subjective based on the desired end result. I am not suggesting my solution has to be used in its entirety. Just a working starting point.

Do all built in functions need to be captured and scoped as builtin functions? No, but I did in this case. The code was already there, all I did was flesh out the body of it to handle params and such and move types that are also builtin functions to it. The builtin functions pattern array could surely be moved to its own named pattern and used by the generic function pattern object as well. If no one wants builtin functions to be categorized by their own scope, then it can be generalized even further. print must be a special case no matter what.

[facelessuser]

I see there is some confusion about what I am doing, so here is the final snippet of code:

        <key>builtin_functions</key>
        <dict>
            <key>patterns</key>
            <array>
                <dict>
                    <key>begin</key>
                    <string>(?&lt;!\.)(__import__|ascii|abs|all|any|apply|bin|bool|buffer|bytearray|bytes|callable|chr|classmethod|cmp|coerce|compile|complex|copyright|credits|del|delattr|dict|dir|divmod|enumerate|eval|exec|execfile|exit|file|filter|float|format|frozenset|getattr|globals|hasattr|hash|help|hex|id|input|int|intern|isinstance|issubclass|iter|len|license|list|locals|long|map|max|memoryview|min|next|object|oct|open|ord|pow|print|property|quit|range|raw_input|reduce|reload|repr|reversed|round|set|setattr|slice|sorted|staticmethod|str|sum|super|tuple|type|unicode|unichr|vars|xrange|zip)\s*(?=\()</string>
                    <key>beginCaptures</key>
                    <dict>
                        <key>1</key>
                        <dict>
                            <key>name</key>
                            <string>support.function.builtin.python</string>
                        </dict>
                    </dict>
                    <key>end</key>
                    <string>(\))</string>
                    <key>endCaptures</key>
                    <dict>
                        <key>1</key>
                        <dict>
                            <key>name</key>
                            <string>punctuation.definition.arguments.end.python</string>
                        </dict>
                    </dict>
                    <key>name</key>
                    <string>meta.function-call.python</string>
                    <key>patterns</key>
                    <array>
                        <dict>
                            <key>begin</key>
                            <string>(?=[A-Za-z_][A-Za-z0-9_]*(?:\.[A-Za-z_][A-Za-z0-9_]*)*\s*\()</string>
                            <key>end</key>
                            <string>(?=\s*\()</string>
                            <key>patterns</key>
                            <array>
                                <dict>
                                    <key>include</key>
                                    <string>#dotted_name</string>
                                </dict>
                            </array>
                        </dict>
                        <dict>
                            <key>begin</key>
                            <string>(\()</string>
                            <key>beginCaptures</key>
                            <dict>
                                <key>1</key>
                                <dict>
                                    <key>name</key>
                                    <string>punctuation.definition.arguments.begin.python</string>
                                </dict>
                            </dict>
                            <key>contentName</key>
                            <string>meta.function-call.arguments.python</string>
                            <key>end</key>
                            <string>(?=\))</string>
                            <key>patterns</key>
                            <array>
                                <dict>
                                    <key>include</key>
                                    <string>#keyword_arguments</string>
                                </dict>
                                <dict>
                                    <key>include</key>
                                    <string>$self</string>
                                </dict>
                            </array>
                        </dict>
                    </array>
                </dict>
                <dict>
                    <!-- Non function style print -->
                    <key>match</key>
                    <string>(?&lt;!\.)\b(print)\b</string>
                    <key>name</key>
                    <string>support.function.builtin.python</string>
                </dict>
            </array>
        </dict>

[FichteFoll]

It seems that you're rather misunderstanding what I meant to express, though I have to agree I probably didn't convey the message very well. I'll try to draft out and test what I was thinking of but there's something more important right now and I'm a bit alienated from working with plists, but I'm sure you can read YAML well enough.

Edit: The old print is not a function, it's a keyword (for the print statement). Sadly you can also use the new print function as an identifier just like any other so it'd be hard to differentiate between these (e.g. as callback).

dbg = []
def register_debug(func):
    dbg.append(func)

def debug(*args, **kwargs):
    return [func(*args, **kwargs) for func in dbg]


register_debug(print)
from functools import partial
register_debug(partial(print, end='\n---\n'))  # file="out.log"

debug("test")

[facelessuser]

Yes old print I'd not really function, so you could scope that case differently. I guess you could just scope it as a keyword or type if it doent have brackets.

[facelessuser]

You could probably check if it's followed by a comma or a ). But personally it doesn't bother me too much. Most syntax highlighters don't do much better.

[facelessuser]

FYI, there will have to be exceptions for more than just print. del is another that is used as a keyword and there are probably more. It will take me some time to come up with a full list, but once sorted out, enough info should be available to properly fix this.

[FichteFoll]

To my knowledge (and some quick googling) del is always a keyword and not a function.

[facelessuser]

To my knowledge (and some quick googling) del is always a keyword and not a function.

I know, it was one of the things I didn't do though, I didn't go through and evaluate the ones that exclusively should not be functions. I thought of print of the top of my head, but there are more.

[MattDMo]

@facelessuser sorry, I haven't been following along as closely as I should have been. So basically, the take-home change for right now would be to modify the current contents of builtin_functions:

        <key>builtin_functions</key>
        <dict>
            <key>match</key>
            <string>(?&lt;!\.)\b(__import__|abs|all|any|apply|callable|chr|cmp|coerce|compile|copyright|credits|del|delattr|dir|divmod|enumerate|eval|exec|execfile|exit|filter|format|getattr|globals|hasattr|hash|help|id|input|intern|isinstance|issubclass|iter|len|license|locals|map|max|memoryview|min|next|open|ord|pow|print|property|quit|range|raw_input|reduce|reload|repr|reversed|round|setattr|sorted|sum|unichr|vars|zip)\b</string>
            <key>name</key>
            <string>support.function.builtin.python</string>
        </dict>

with what you posted above in your huge XML dump? I just want to be sure I'm on the right page before I convert it to YAML and merge it with some other (admittedly much more minor) changes I've been working on recently. I also have really gotten tired of trying to interpret regexes surrounded by plist foolishness - YAML just makes so much more sense, even more so than JSON.

---

Never mind, I just went ahead and did it...

[MattDMo]

So if you look at the Simple Statements part of the Python 2 docs, then in my mind it seems to imply that print and del on their own are no different than other statements like yield or return or raise. As of right now \b(elif|else|except|finally|for|if|try|while|with|break|continue|pass|raise|return|yield)\b are classified as keyword.control.flow.python, which doesn't seem to fit in with print and del. Do we keep them as support.function.builtin.python, or come up with some other category for them?

[facelessuser]

Good questions. That is really up to you. I have them currently as builtin function exceptions (no parenthesis), but I may end up removing it and letting it fall to types right below it...don't know yet.

[MattDMo]

Changes incorporated into release 1.3.0. Thank you, everyone!