Should Curb Zone IDs be stable?

[BigIdeas]

In CDS, Curb Zones will be identified by their ID. Should these IDs remain stable per Curb Zone, or should a Curb Zone's ID change whenever the Curb Zone's properties change? If so, how significant should the change in properties be to trigger a new Curb Zone ID?

This question has been discussed in Should every curb polygon require a unique ID? and it was suggested that whenever a Curb Zone's geometry or policies change, that Curb Zone should be assigned a new ID. Arguments in favor of this approach include:

1. Providers and other interested parties could simplify their workflows by assuming that any time they see a Curb Zone with the same ID, they already know its policies and geometry, allowing them to reduce the frequency at which they need to re-inspect a Curb Zone's properties.
2. In some cases, the Curb Zone ID itself would not need to be stored anywhere because it could be generated on the fly by computing a hash of the Curb Zone's properties (although the Curb Zone's properties would still need to be stored somewhere).

However, non-stable Curb Zone IDs could present some challenges:

1. Events and Metrics map to the curb through Curb Zone ID. If a Curb Zone's ID changes when that Curb Zone's policies change, it would be impossible to compare metrics at that Curb Zone before and after the change (without some additional data structure tracking changes in Curb Zone IDs over time). In effect, a Curb Zone's history will be lost whenever its policies or geometry change.
2. A Curb Zone's policies and geometry must both be known and created at the same time. It would not be possible for one person to define and create a Curb Zone's geometry and then pass that to another person to create the policy.
3. Geography (i.e., Curb Zones) and policy are tightly coupled in a model where changes to a Curb Zone's policy result in a new ID, limiting the degree to which curb policy can be dynamic.

I'm sure there are additional pros, cons, and use cases to be discussed. I'm also curious about the relative lifecycles of Curb Zone policies and geometries. Do we expect a Curb Zone's policies to be roughly as stable as its physical geometry?

[alkhoury-elias]

Cities and agencies should determine whether a specific Curb ID needs to change or not. What would help though, is there is Date Modified or Date Updated field for each unique ID so it can be a reference point for data capture or analytics if a City or Agency wants to compare performance based on pre or post Modification / Updated date

[schnuerle]

Here are two possible solutions to add some stability while also having a history.

Common changes:

a) Propose adding a published_date to the zone object which would be a timestamp that the zone details were published. Matches MDS policy and naming.

b) Add an optional show_historic flag to return old zones/policies in the query response on each endpoint (so by default old/retired IDs are not returned).

1) Change IDs but have a way to look up old zones/policies.

Change the zone ID when there is an update to the details, or geography of a curb zone. Consumers would get a new ID if there was a change, but can look up the old ID in a new array.

c) Propose adding a prev_curb_zones UUID optional array of prior curb zones replaced by this one. Matches MDS policy concepts.

d) Propose adding published_date and prev_policies to /curb/policy as well.

2) Keep IDs but version them and have a way to look up old zones/policies.

Keep the same zone ID unless there is a significant change (ie, moving the zone, at discretion of city) to the geometry. Consumers get to keep the same ID, but know to there is a change and anyone can look up old info if needed.

c) Propose adding a last_updated date to the zone object which would be a timestamp that the zone was published. Would help notify consumers that there is a change. Naming matches parts of MDS.

d) Propose adding a zone_version integer field to the zone object which gets incremented when there is a change. We use this versioning in MDS Policy Requirements.

e) Propose adding published_date and last_updated and policy_version to /curb/policy as well.

[cartograham]

@schnuerle , I am in favor of approach 2. It aligns very closely to our versioned editing workflows for centerline edits and keeps the ID until the network changes.

I think one additional thing to consider (not sure if this has been discussed elsewhere): linkability to curb zones when the network does change, causing splitting and merging of zones. For example, a simple curb zone begins at the start of a block and ends at the end of that same street block. In this scenario, there would be two long curb zones with their own zone IDs (let’s call them ZoneID 1 & 2). A couple of years later, the city decides it wants to open an alley along the blockface that ZoneID 1 is on. What was once a single, unbroken curb zone now becomes two curb zones, presumably with new IDs as well: ZoneID 1a and 1b.

I am curious how the spec could allow users to analyze curb events across this split event.

Edit: also, how can we know that ZoneID 1a occupies a specific geographic portion of the historical ZoneID 1?

(apologies if this has been addressed elsewhere)

[schnuerle]

Here's an idea of where we can go with this to address the concerns brought up:

Changes to zone_id would only happen when there is a significant geometry/geography change.

To the curb_zone object:

• Add a published_date to the zone object which would be a timestamp that the zone details were published.
• Add a last_updated_date
• Add a prev_curb_zone_ids UUID optional array of prior curb zones replaced by this one. In order of adoption.
• Add prev_policies array:
  • policy_id
  • start_date
  • end_date

Curb zones would get a new policy_id when there is a new policy applied to the zone.

Policies are immutable. If you want to update a policy, you make a new policy with a new ID.

To the curb_policy object:

• Add published_date

To the endpoint query parameters:

• Add an optional show_historic flag to return old zones/policies in the query response on each endpoint (so by default old/retired IDs are not returned).

[jiffyclub]

Would prev_curb_zone_ids list any zone that previously overlapped this one at all? Or only those that were representing the exact same physical space?

I'm thinking about prev_policies and this particular situation: suppose one wants to figure out the set of curb rules governing a city on a specific date, July 17, 2021. I think CDS stipulates that there's only one zone applicable to a given geography at a time. So one finds all the zones that were active for that date and can be sure none overlap. For each of those, there could be policies that apply to July 17, 2021 in both the curb_policy_ids and prev_policies fields, so those essentially need to be combined to find all the policies applicable on July 17, 2021. To me that seems overly complicated, I'd rather that looking up zones that apply to a date also directly gives you the policies that apply on that date. But, prev_policies also seems tractable, if a bit unwieldy.

[schnuerle]

Yes a bit unwieldy, but it can work. I think this historic lookup complexity is needed if curb managers want to have a static curb zone ID even if the policies change for that zone over time, and this setup helps with tracking the built in CDS Metrics too to see effectiveness of policy changes over time.

[schnuerle]

All of this has been added to the spec now, with minor adjustments out of necessity. Please review here:

https://github.com/openmobilityfoundation/curb-data-specification/blob/feature-release-work-1/curbs/README.md#curb-zone

[jiffyclub]

I think IDs should be unique to a time, place, and set of rules. Maybe it's ok to keep the zone ID if a prior error is being corrected (as long as there is versioning) or if something trivial is being updated like a description. But zone IDs that reference a valid combination of time, space, and rules should never be reused. This is pretty much how it's already described.

When doing things like versioning I think it's useful to track metadata around when a prior version was replaced. Something like a date_deleted field that describes when a zone was replaced with a newer version. I guess you can kind of derive that from a published_date on a newer version, but I like explicitly packaging it with the "deleted" version.

[cartograham]

@jiffyclub , I agree wholeheartedly. When the network changes, versioning as you’ve described will keep things in order nicely. We can query for historical versions and reveal what once was there.

My main reason for bringing this up is because I can’t see how do historical queries to aggregate and summarize curb events in locations where there have been splits or merges. Using the example above (Zone 1 and its historical children, Zones 1a & 1b):

1. The split of Zone 1 occurs on December 7th (today)
2. At the end of this month, I’d like to generate a summary for my city showing weekly curb activity by curb zone.
3. Challenge: how can I reconcile the change that has occurred today? I’ll have parking events associated to curb zone ID 1 from December 1-6th. Then I’ll have parking events for both zone ID 1a & 1b for the remainder of the month.

Question: All 3 zones share the same geographic location but is there a way to easily allow users of CDS to group the current and historic curb zones without a lot of forensic reconstruction?

[schnuerle]

Does the proposal here resolve the needs you have listed?

#45 (reply in thread)

[alkhoury-elias]

Awsome! Thank you.

[cartograham]

@schnuerle - Yes, good idea. That works just fine. Thank you!

My main concern was for queries downstream, should a city ever implement the Metrics endpoint. For point-in-time, I think the proposed aggregation method works fine. If ever CDS were have a similar aggregation method for a timespan, (with a StartDate/EndDay), geometric splits and merges would need to be accounted for.