Unable to edit alerts config if a fallback routing config is in use

[leth]

Identified by debugging a customer issue. The issue is somewhere around https://github.com/weaveworks/cortex/blob/master/pkg/alertmanager/multitenant.go#L440

[bboreham]

The context was that we saw messages like these when the user saved alerts:

time="2017-11-08T22:53:31Z" level=warning msg="MultitenantAlertmanager: invalid Cortex configuration for 5570: error parsing Alertmanager config: no route provided in config" source="multitenant.go:377"
time="2017-11-08T22:53:37Z" level=warning msg="MultitenantAlertmanager: invalid Cortex configuration for 5570: error parsing Alertmanager config: no route provided in config" source="multitenant.go:377"
time="2017-11-08T23:03:29Z" level=warning msg="MultitenantAlertmanager: invalid Cortex configuration for 5570: error parsing Alertmanager config: no route provided in config" source="multitenant.go:377"
time="2017-11-08T23:03:31Z" level=warning msg="MultitenantAlertmanager: invalid Cortex configuration for 5570: error parsing Alertmanager config: no route provided in config" source="multitenant.go:377"
time="2017-11-08T23:03:37Z" level=warning msg="MultitenantAlertmanager: invalid Cortex configuration for 5570: error parsing Alertmanager config: no route provided in config" source="multitenant.go:377"

But alerts are generated by ruler, which doesn't go through that code path.
I don't think MultitenantAlertmanager uses the RulesFiles member.
So this is an unfortunate log message but not actually a problem.