Skip to content
This repository was archived by the owner on Apr 14, 2022. It is now read-only.

Cache function eval result per argument set #988

Merged
merged 42 commits into from
Apr 19, 2019
Merged
Changes from 39 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
8bca56b
Fix #668 (partial)
Mar 1, 2019
a731c2a
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 1, 2019
069910b
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 5, 2019
7ffc9db
Tests
Mar 5, 2019
6303c45
Revert "Tests"
Mar 5, 2019
945451c
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 5, 2019
034e437
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 5, 2019
f997d68
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 6, 2019
0e4ff95
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 11, 2019
878c8c5
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 13, 2019
35e1033
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 13, 2019
1073711
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 14, 2019
86028da
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 14, 2019
4d4bca7
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 18, 2019
ac04f76
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 18, 2019
bab03da
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 20, 2019
85ec685
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 27, 2019
be7466d
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 28, 2019
01781c0
Exp
Mar 27, 2019
f544f8a
Limit concurrency
Mar 28, 2019
01bd719
Concurrency limit
Mar 28, 2019
2f8e564
Drop cache after analysis
Mar 28, 2019
9a933bd
Merge branch 'master' of https://github.com/Microsoft/python-language…
Mar 28, 2019
4ad56bb
Fix regression
Mar 28, 2019
c280c55
Merge branch 'master' of https://github.com/Microsoft/python-language…
Apr 3, 2019
b21b8b8
Merge branch 'master' of https://github.com/Microsoft/python-language…
Apr 7, 2019
babaee4
Merge branch 'master' of https://github.com/Microsoft/python-language…
Apr 8, 2019
092efd6
Merge branch 'master' of https://github.com/Microsoft/python-language…
Apr 8, 2019
b97594d
Merge branch 'master' of https://github.com/Microsoft/python-language…
Apr 10, 2019
2a50eb9
Fix test
Apr 10, 2019
9c7f58d
Merge branch 'master' of https://github.com/Microsoft/python-language…
Apr 10, 2019
00faa5f
Merge branch 'master' of https://github.com/Microsoft/python-language…
Apr 12, 2019
4c8aa92
Merge branch 'master' of https://github.com/Microsoft/python-language…
Apr 12, 2019
1a67bc8
Merge branch 'master' of https://github.com/Microsoft/python-language…
Apr 12, 2019
7a13215
Merge branch 'master' of https://github.com/Microsoft/python-language…
Apr 16, 2019
3d907d8
Merge branch 'master' of https://github.com/Microsoft/python-language…
Apr 16, 2019
e5a225f
Merge branch 'master' of https://github.com/Microsoft/python-language…
Apr 17, 2019
d5ba0db
Cache argument eval
Apr 17, 2019
cd67de7
Fix hash calculation
Apr 17, 2019
87b0d33
Merge branch 'master' of https://github.com/Microsoft/python-language…
Apr 18, 2019
ab6da4d
Fix exception when there is a space before triple quote
Apr 18, 2019
5747ba4
Fix exception
Apr 18, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,7 @@ public IMember GetValueFromFunctionType(IPythonFunctionType fn, IPythonInstance
// make sense to evaluate stubs since they already should be annotated.
if (fn.DeclaringModule is IDocument doc && fd?.Ast == doc.GetAnyAst()) {
// Stubs are coming from another module.
return TryEvaluateWithArguments(fn.DeclaringModule, fd, args);
return TryEvaluateWithArguments(fn, args);
}
return UnknownType;
}
Expand All @@ -207,9 +207,26 @@ public IMember GetValueFromProperty(IPythonPropertyType p, IPythonInstance insta
return instance.Call(p.Name, ArgumentSet.Empty);
}

private IMember TryEvaluateWithArguments(IPythonModule module, FunctionDefinition fd, IArgumentSet args) {

private readonly Dictionary<int, IMember> _argEvalCache = new Dictionary<int, IMember>();

private IMember TryEvaluateWithArguments(IPythonFunctionType fn, IArgumentSet args) {
var name = fn.DeclaringType != null ? $"{fn.DeclaringModule.Name}.{fn.Name}" : fn.Name;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be affected by #979 where the DeclaringModule of a function I had was null, so all functions of that style with the same name would get cached together.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but it is a bug. I'll work on it right after this. All types should have declaring module, there are many many places where we would crash otherwise.

var argHash = args
.Arguments
.Select(a => a.Name.GetHashCode() ^ 397 * (a.Value?.GetHashCode() ?? 0))
.Aggregate(0, (current, d) => 31 * current ^ d);
var key = fn.DeclaringModule.Name.GetHashCode() ^ name.GetHashCode() ^ (397 * argHash);

if (_argEvalCache.TryGetValue(key, out var result)) {
return result;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't hash collisions here make some unrelated functions have the same evaluation?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functions have fully qualified names like module.class.method which should have unique hashes

}

var fd = fn.FunctionDefinition;
var module = fn.DeclaringModule;

// Attempt to evaluate with specific arguments but prevent recursion.
IMember result = UnknownType;
result = UnknownType;
if (fd != null && !_callEvalStack.Contains(fd)) {
using (OpenScope(module, fd.Parent, out _)) {
_callEvalStack.Push(fd);
Expand All @@ -222,6 +239,8 @@ private IMember TryEvaluateWithArguments(IPythonModule module, FunctionDefinitio
}
}
}

_argEvalCache[key] = result;
return result;
}

Expand Down