Importing multiple simulation files initializes only the last sequence

[denisrosca]

Description of the bug

Importing multiple simulation files initializes only the sequences from the last imported file.
Note: I'm using docker for this

Steps to reproduce the issue

1. Define two simulation files:
   1.json

{
  "data": {
    "pairs": [
      {
        "request": {
          "method": [
            {
              "matcher": "exact",
              "value": "GET"
            }
          ],
          "path": [
            {
              "matcher": "exact",
              "value": "/1"
            }
          ],
          "requiresState": {
            "sequence:first": "1"
          }
        },
        "response": {
          "status": 200,
          "body": "Hello World 1!",
          "transitionsState": {
            "sequence:first": "2"
          }
        }
      }
    ]
  }
}

and 2.json:

{
  "data": {
    "pairs": [
      {
        "request": {
          "method": [
            {
              "matcher": "exact",
              "value": "GET"
            }
          ],
          "path": [
            {
              "matcher": "exact",
              "value": "/2"
            }
          ],
          "requiresState": {
            "sequence:second": "1"
          }
        },
        "response": {
          "status": 200,
          "body": "Hello World 2!",
          "transitionsState": {
            "sequence:second": "2"
          }
        }
      }
    ]
  }
}

2. Import them both: -webserver -import 1.json -import 2.json
3. Make a request to GET /1

Observed result

Hoverfly error messages seen (If none, say none)

api_mocks_1       | WARN[2019-05-13T11:31:30Z] Failed to find matching request from simulation  destination="localhost:8500" error="No match found" method=GET path=/1 query="map[]"
api_mocks_1       | ERRO[2019-05-13T11:31:30Z] There was an error when matching              error="Could not find a match for request, create or record a valid matcher first!\n\nThe following request was made, but was not matched by Hoverfly:\n\n{\n    \"Path\": \"/1\",\n    \"Method\": \"GET\",\n    \"Destination\": \"localhost:8500\",\n    \"Scheme\": \"http\",\n    \"Query\": {},\n    \"Body\": \"\",\n    \"Headers\": {\n        \"Accept\": [\n            \"*/*\"\n        ],\n        \"Accept-Encoding\": [\n            \"gzip, deflate\"\n        ],\n        \"Accept-Language\": [\n            \"en-US,en;q=0.5\"\n        ],\n        \"Connection\": [\n            \"keep-alive\"\n        ],\n        \"User-Agent\": [\n            \"Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0\"\n        ]\n    }\n}\n\nWhilst Hoverfly has the following state:\n\n{\n    \"sequence:second\": \"1\"\n}\n\nThe matcher which came closest was:\n\n{\n    \"path\": [\n        {\n            \"matcher\": \"exact\",\n            \"value\": \"/1\"\n        }\n    ],\n    \"method\": [\n        {\n            \"matcher\": \"exact\",\n            \"value\": \"GET\"\n        }\n    ],\n    \"requiresState\": {\n        \"sequence:first\": \"1\"\n    }\n}\n\nBut it did not match on the following fields:\n\n[state]\n\nWhich if hit would have given the following response:\n\n{\n    \"status\": 200,\n    \"body\": \"Hello World 1!\",\n    \"encodedBody\": false,\n    \"templated\": false,\n    \"transitionsState\": {\n        \"sequence:first\": \"2\"\n    }\n}" mode=simulate

Expected result

Request matches and returns correct result.

Additional relevant information

1. Hoverfly version:

↳ ./hoverctl version

+----------+--------------+
| hoverctl | master-3141  |
| hoverfly | v1.0.0-rc.2  |
+----------+--------------+

2. Anything that might help us to diagnose the problem

↳ ./hoverctl state get-all
State of Hoverfly:
"sequence:second"="1"

P.S. Note that I'm using the latest docker image available on docker hub and I was expecting the hoverfly version to be 1.0.0 instead of 1.0.0-rc2.

[JohnFDavenport]

I think you'll find Hoverfly only has the last simulation loaded. Hoverfly doesn't support the import of more than simulation (it would have to know how you want to merge them in case of identical requests). See #552

Currently if you want to import two simulations you need to merge them offline or capture everything in one go, or load two separate Hoverfly instances (assuming the simulations are distinct).

[tommysitu]

Have you tried pulling the latest docker image:

$ docker pull spectolabs/hoverfly:latest
latest: Pulling from spectolabs/hoverfly
Digest: sha256:c88efdc6c2e372d00e09383e60919b3b289e63c79f530344c4816b32802724d8
Status: Image is up to date for spectolabs/hoverfly:latest
docker.io/spectolabs/hoverfly:latest

The latest one should be v1.0.0: https://hub.docker.com/r/spectolabs/hoverfly/tags

[denisrosca]

I think you'll find Hoverfly only has the last simulation loaded. Hoverfly doesn't support the import of more than simulation (it would have to know how you want to merge them in case of identical requests). See #552

Currently if you want to import two simulations you need to merge them offline or capture everything in one go, or load two separate Hoverfly instances (assuming the simulations are distinct).

I believe that restriction applies only when importing via the API. You can clearly see from my error message that the simulation is loaded:

The matcher which came closest was:\n\n{\n    \"path\": [\n        {\n            \"matcher\": \"exact\",\n            \"value\": \"/1\"\n        }\n    ],\n    \"method\": [\n        {\n            \"matcher\": \"exact\",\n            \"value\": \"GET\"\n        }\n    ],\n    \"requiresState\": {\n        \"sequence:first\": \"1\"\n    }\n}\n\nBut it did not match on the following fields:\n\n[state]\n\n

it did not match because the sequence was not initialized

[denisrosca]

Have you tried pulling the latest docker image:

$ docker pull spectolabs/hoverfly:latest
latest: Pulling from spectolabs/hoverfly
Digest: sha256:c88efdc6c2e372d00e09383e60919b3b289e63c79f530344c4816b32802724d8
Status: Image is up to date for spectolabs/hoverfly:latest
docker.io/spectolabs/hoverfly:latest

The latest one should be v1.0.0: https://hub.docker.com/r/spectolabs/hoverfly/tags

I am indeed using the correct image:

v1.0.0: Pulling from spectolabs/hoverfly
Digest: sha256:c88efdc6c2e372d00e09383e60919b3b289e63c79f530344c4816b32802724d8
Status: Downloaded newer image for spectolabs/hoverfly:v1.0.0
Creating serenity_api_mocks_1 ... done
Attaching to serenity_api_mocks_1
api_mocks_1       | INFO[2019-05-14T11:20:05Z] Listen on specific interface                  host=0.0.0.0

I think the version bit is a non-issue, it was reporting the version of my local hoverfly instance instead of the one running in the docker container

[tommysitu]

@denisrosca, when you use either hoverfly -import or hoverctl import a simulation, it will always override the previous record and reset the states. That's why you only get one state initialized.

If you merge them offline, and import it once:

{
	"data": {
		"pairs": [{
				"request": {
					"method": [{
						"matcher": "exact",
						"value": "GET"
					}],
					"path": [{
						"matcher": "exact",
						"value": "/1"
					}],
					"requiresState": {
						"sequence:first": "1"
					}
				},
				"response": {
					"status": 200,
					"body": "Hello World 1!",
					"transitionsState": {
						"sequence:first": "2"
					}
				}
			},
			{
				"request": {
					"method": [{
						"matcher": "exact",
						"value": "GET"
					}],
					"path": [{
						"matcher": "exact",
						"value": "/2"
					}],
					"requiresState": {
						"sequence:second": "1"
					}
				},
				"response": {
					"status": 200,
					"body": "Hello World 2!",
					"transitionsState": {
						"sequence:second": "2"
					}
				}
			}
		]
	},
	"meta": {
		"schemaVersion": "v5",
		"hoverflyVersion": "v1.0.0",
		"timeExported": "2019-05-15T15:44:25+01:00"
	}
}

you should get both states initialized:

$ hoverctl state get-all
State of Hoverfly:
"sequence:first"="1"
"sequence:second"="1"

[JohnFDavenport]

I believe that restriction applies only when importing via the API

Unlikely. In general, at least where there's an interface, Hoverfly development has been API first.

[denisrosca]

@denisrosca, when you use either hoverfly -import or hoverctl import a simulation, it will always override the previous record and reset the states. That's why you only get one state initialized.

I know I am arguing against people that know more about hoverfly than me 😄 but it is obvious not all records are overriden.

You can clearly see in the error message that hoverfly knows about the records from the first imported file. Otherwise the error message wouldn't mention it.

I think the "override" part is only applied to states which is why the current behavior is confusing.

[tommysitu]

You're right @denisrosca. I wasn't looking the error message carefully. I looked into the simulation that is loaded into hoverfly when you do -webserver -import 1.json -import 2.json, and it has both simulations combined! 😮 This is different behaviour from doing hoverctl import.

We need to fix this inconsistency and initialize all sequences that are loaded in.

[JohnFDavenport]

Well, you could knock me over with a feather! I am pleasantly surprised. Thanks @denisrosca for ignoring me. It also appears Hoverfly will discard duplicate request/response pairs (assuming you use load the simulations in the same order) which should ensure that they are dealt with consistently.