Apply rules to annotate candidates (in addition to the ML part of the filter-rank module)

[Riruk]

We could use the hard-coded rules to identify some commits relevant to security fixes.

For example: if a commit message says that this commit is a fix for a particular CVE, then it is a very strong candidate for the commit to be related to the vulnerability fix.

I will create a PR for the implementation part and I propose to have this issue to discuss possible hard-coded rules that we could create.

[Riruk]

The initial work is done in the context of the following pull request: #149

[copernico]

Another rule could be something like:

• the advisory mentions a file (or class) and a given candidate is the only one (or one of the few..) that touches that file (class)

[copernico]

Another obvious one:

• the Advisory mentions the commit at hand

NOTE: this is not conclusive per-se (some advisories point to commits that contain changelog changes, not the actual code fixes...)

[copernico]

As for the naming: I guess we could settle on calling these just "rules" (implying "manually-defined", or "handcrafted"),
unless the context might cause ambiguity.

[copernico]

Hi @Riruk, any progress on this issue?

[Riruk]

Hi @copernico, I was a bit busy last week with a paper. It's submitted now, so I will come back to working on the prospector tool from tomorrow

[Riruk]

What about adding a rule "contains text in commit message"?

[copernico]

What about adding a rule "contains text in commit message"?

The "text" would be a keyword extracted from the advisory? If so, yes, definitely a useful rule.

[copernico]

Work continues in #161

[copernico]

I think we need to elaborate on the representation of the results to be shown to the user before proceeding.
I would propose that, after the candidates are obtained from Git and they are processed to compute their features, we apply one or more "analysis" steps that produce "annotations" to be attached to the candidates. The user will then be able to inspect the results by seeing which "annotations" are attached to each candidate.

Examples:

commit X1 from repository R

• Reason: token "user" in advisory matches the path "src/main/whatever/User.java" changed in the commit
• Reason: the commit is in tag "v1.2.0" but not in the subsequent "v1.2.1"

Note: the reasons are human readable, but are constructed automatically based on the candidate features and annotations.

[copernico]

One point I'm not quite sure how to handle is the conceptual distinction between the above annotations and the features computed with the extract_* functions. In some sense, they look the same. Maybe, the annotations are what the user will see...?

[Riruk]

Work continues in #175