add XML templated matcher

[ri-tatsu]

XML Templated Matcher

Motivation (Why we made it)

When mocking HTTP request with XML body, a matcher support partial matching is needed if we want to ignore some data like timestamp, or to match an auto generated string with a fixed pattern.

Design

We add a matcher called xmlTemplated, who can define the expected XML data with 3 kinds of expressions for the value in a node; that is, in XML node <tag>expression</tag>, expression can be one of the following three.

Expression	Description
string	match exactly equal to given string
{{ignore}}	match arbitrary value
{{regex:RegularExpression}}	match string with given regular expression

※ The reserved keyword ignore and regex are case-sensitive.

{
    "matcher" : "xmlpartial",
    "value" : "<xml><item>123<item><item>{{regex:^[A-Z]\\d{5}$}}</item><item>{{ignore}}</item></xml>"
}

If more than one kind of expression occurs in an list node of XML data, the matching process will loop on all nodes in the matcher, and in each round, pick one matched item from the actual data in HTTP request and mark it as consumed which can not be picked in the next rounds. If no matched item found in a round, the matching fails immediately. Otherwise, matching will succeed if all items matched.

Obviously, this matching process depends on the children's order of the given list node. Therefore, this matching rule is just for completeness of design and is not recommended to be used in practice unless the children of list node in XML data have a fixed order.

Here is an example to show the matching process.

Round 1
Matcher current <list>　 - <hoge>123</hoge> → <hoge>{{regex:^[A-Z]\d{5}$}}</hoge> <hoge>{{ignore}}</hoge> </list> -		pick HTTP request - <list> ✖ <hoge>A12345</hoge> ✖ <hoge>here can be any string</hoge> ✔ <hoge>123</hoge> - </list>
Round 2
Matcher current <list>　 - <hoge>123</hoge> <hoge>{{regex:^[A-Z]\d{5}$}}</hoge> → <hoge>{{ignore}}</hoge> </list> -		pick HTTP request - <list> ✔ <hoge>A12345</hoge> <hoge>here can be any string</hoge> <hoge>123</hoge> - </list>
Round 3
Matcher current <list>　 - <hoge>123</hoge> <hoge>{{regex:^[A-Z]\d{5}$}}</hoge> <hoge>{{ignore}}</hoge> → </list> -		pick HTTP request - <list> skipped <hoge>A12345</hoge> ✔ <hoge>here can be any string</hoge> <hoge>123</hoge> - </list>

Finally, items on both sides are consumed, and the matching of this list is succeeded.

If we change the children's order of <list> as below, the matching will fail.

Round 1
Matcher current <list>　 - <hoge>{{ignore}}</hoge> → <hoge>123</hoge> <hoge>{{regex:^[A-Z]\d{5}$}}</hoge> </list> -		pick HTTP request - <list> ✔ <hoge>A12345</hoge> <hoge>here can be any string</hoge> <hoge>123</hoge> - </list>
Round 2
Matcher current <list>　 - <hoge>{{ignore}}</hoge> <hoge>123</hoge> → <hoge>{{regex:^[A-Z]\d{5}$}}</hoge> </list> -		pick HTTP request - <list> skipped <hoge>A12345</hoge> ✖ <hoge>here can be any string</hoge> ✔ <hoge>123</hoge> - </list>
Round 3
Matcher current <list>　 - <hoge>{{ignore}}</hoge> <hoge>123</hoge> <hoge>{{regex:^[A-Z]\d{5}$}}</hoge> → </list> -		pick HTTP request - <list> skipped <hoge>A12345</hoge> ✖ <hoge>here can be any string</hoge> skipped <hoge>123</hoge> - </list>

In the last round, there is no item found to match the given pattern (matcher expression) <hoge>{{regex:^[A-Z]\d{5}$}}</hoge>, which means the matching result of this list is failed.

Example

No.	Matcher Expression	HTTP request	Result
1	<xml> <hoge>123</hoge> </xml>	<xml> <hoge>123</hoge> </xml>	✔
2	<xml> <hoge>{{ignore}}</hoge> </xml>	<xml> <hoge>here can be any string</hoge> </xml>	✔
3	<xml> <hoge>{{regex:^[A-Z]\d{5}$}}</hoge> </xml>	<xml> <hoge>A12345</hoge> </xml>	✔
4		<xml> <hoge>a123</hoge> </xml>	✖
5	<list> <hoge>123</hoge> <hoge>{{regex:^[A-Z]\d{5}$}}</hoge> <hoge>{{ignore}}</hoge> </list>	<list> <hoge>123</hoge> <hoge>A12345</hoge> <hoge>here can be any string</hoge> </list>	✔
6		<list> <hoge>A12345</hoge> <hoge>here can be any string</hoge> <hoge>123</hoge> </list>	✔
7	<list> <hoge>{{ignore}}</hoge> <hoge>123</hoge> <hoge>{{regex:^[A-Z]\d{5}$}}</hoge> </list>	<list> <hoge>123</hoge> <hoge>A12345</hoge> <hoge>here can be any string</hoge> </list>	✖

Test (How to test it)

For xml_partial_match.go, more than one hundred unit test cases are added in xml_partial_match_test.go. Those test cases cover all input patterns for each function as possible.
In file ft_simulation_matching_test.go, we add test cases to make sure the matcher working properly in a simulation file.

[tommysitu]

This matcher sounds interesting. My only reservation is the naming, because what jsonPartial matcher does is to only compare a portion of a JSON document. So I would expect xmlPartial matcher would do something similar. For example, if a matcher value is:

<item>123<item>

Then it will match on any XML document with that node.

However this new matcher you are proposing still matches the entire XML, but with some part of it replaced with templates. It would be good to call it something else, to avoid the discrepancy with the JSON partial matcher.

[tommysitu]

how about xmlTemplated matcher?

[ri-tatsu]

Thanks for your explanation. xmlTemplated sounds good.

---

At first, we made it with features of

• template expressions of node values
• partial matching syntax of tree structure

Then, we realized that these two features may need very complex rules if we use them together. For example, we can not do the list matching described above, because partial matching means some nodes are already ignored.

Finally, to keep sample, we removed the second feature to avoid very complex matching rules.

[ri-tatsu]

OK. I changed all of the xmlPartial occurs in source code and test cases into xmlTemplated.

[tommysitu]

top notch testing, and no major issues. Happy to make those changes for you if you prefer, as I'm going to do a release in the coming day.

[tommysitu]

The Hoverfly response template function is in this format: {{ function_name arg1, arg2.. }}
So I would suggest replacing the : with space as delimiter such as {{ regex your_regex_string_value }} for consistency.

[ri-tatsu]

Thank you for merged it, and sorry for long time no commits.
I am a little busy recently. After we finish our tasks, I will add a commit for this.

[tommysitu]

👍 whenever you have some time.