New media type or application/ld+json plus profile

[lanthaler]

JSON-LD's media type has a profile parameter which would allow to convey the same information as a separate media type. The processing model of application/ld+json isn't changed but the document conforms to a couple of additional restrictions which are defined by the profile.

[akuckartz]

👍

See #48 regarding Profile.

[jasnell]

I'm generally -1 on this. I prefer the more specific media type because it allows me to easily differentiate that this is an Activity Streams 2.0 document as opposed to some other JSON-LD based thing. I don't see how using JSON-LD's media type + a profile parameter provides better interop between AS2 implementations.

[akuckartz]

A brief summary of the reason why I support this proposal:

This is not about interoperability between AS2 implementations. This is about interoperability between different sorts of JSON-LD data. AS2 data is a subset of JSON-LD data and therefore is JSON-LD. One also does not have different audio mime types for different kinds of music or to differentiate between music or speech.

[elf-pavlik]

👍 this way one wouldn't need to request application/activity+json just for activities and then application/ld+json for all kind of other social data someone publishes online as JSON-LD

[akuckartz]

See also #53

[dret]

this feels a lot like the RDF world view that all that media types do is to communicate different serializations (while the actual model information is not visible on the media type at all). that's a very limiting way of looking of the diversity of things that exist on the web, and i'm with @jasnell here that we should be explicit about the model (this is AS), and not some generic metamodel (this is RDF in its JSON-LD serialization).

[akuckartz]

@dret Why?

I agree that the explicit information that certain data is a serialization of AS can be helpful. But as far as I understand media types (as defined by the IETF) their main purpose is to communicate the serialization format.

Can you provide examples of media types where the serialization format is not the primary information?

[dret]

most media types are explicit about what is being transported (not just how it is serialized), so that peers can negotiate and establish a shared understanding. application/atom+xml is one such example. in fact, the whole concept of media type structured syntax suffixes (https://tools.ietf.org/html/rfc6839) has been created to be able to communicate the relevant model information (it's atom), and some information about its serialization as well (it's based on XML).

[akuckartz]

@dret Yes, but in that case the information that Atom is based on XML is explicit. An equivalent for AS therefore would be application/activity+ld+json. But such media types do not seem to be legal. Or are they?

[dret]

they are, but they are not covered by RFC 6839, and so far when they were proposed or discussed, they were rejected. it also wouldn't be quite clear to me what you want to accomplish with this. why not make it application/activity+ld+json+unicode+bytes+bits, to take this to the extreme? ;-)
what you want to communicate is that you are using activity streams. the fact that activity streams is based on something is inherent to the activity streams format. the utility of the suffix in application/atom+xml is questionable and questioned quite a bit, and i have yet to see a real world scenario where application/atom would not have worked equally well.
the general idea behind the suffixes was that there would be some information about a media type's foundation, but since suffixes are neither used consistently nor enforced, so far there seems to be little actual use of them anywhere.

[jasnell]

Just a bit off topic side note.. but I've been seriously kicking around the idea of proposing a new json top level media type... e.g. "json/activity+ld" would actually be pretty nice as a media type here.

[dret]

i wouldn't be too optimistic about this being greeted with a lot of enthusiasm. recent proposals (i think "archive" was just proposed, but i haven't followed the discussion at IETF 91) never made it, and for the timeline we're on, i would not count on the top-level of the registry changing.

[akuckartz]

@jasnell 👍 for that idea. My first impression is that I like it. The application string in media types contains almost no information while ld+jsonis overloaded.

Do others agree that json/activity+ld would resolve this issue?

[akuckartz]

I forgot something obvious: A service can answer requests for several media types. That - btw - can also help to support both non-RDF and RDF-communities at the same time.

[elf-pavlik]

@jasnell @dret could you maybe try to get some feedback from IETF on that json/*+ld proposal? You know appropriate channels much better then me 😄

@lanthaler since AS requires http://www.w3.org/ns/json-ld#compacted #48 i guess such eventual AS specific profile would imply also it conforming to compacted profile? Or one could use multiple values for profile parameter?

@jasnell could you share with us some some clear examples where you see using profile could come as disadvantage? I would expect people to expose their data using application/ld+json in a first place and then very likely add additional support for more specific AS type. Using profile parameter makes it explicit that response has conforming application/ld+json and more specifically activity, if you use separate content type, you don't get an explicit information that you get application/ld+json in your response 😟

[dret]

i'll leave it up to @jasnell to propose a new top-level json/ media type branch on some IETF list. i would bet a lot on this not going anywhere, and personally, i am really not interested in spending time with this. my proposal would still be to pick a media type that makes it explicit that what's being represented is AS.
afaict, we are still operating under the assumption that any JSON-LD interpretation and processing is optional (that's how AS has been chartered), so to me, application/activity+json looks very much like what we're doing.

[elf-pavlik]

@dret just to clarify, do you see any use cases which require that application/activity+json response does NOT conform to JSON-LD spec? In other worlds, while we do NOT require JSON-LD processing, do we want it to fail if someone does try to process it as JSON-LD?

[dret]

@elf-pavlik, that's still the 90000lb gorilla in the WG debate. all i am trying to clarify right now is that we have a well-defined processing model so that everybody has the same "view" of AS. i am in favor of clean layering, but i think we're still mostly handwaving and ignoring the tricky questions of openness and extensibility when it comes to be well-defined in this department. http://dret.typepad.com/dretblog/2015/02/json-or-rdf-just-decide.html

[akuckartz]

This obviously is closely related to the information model question. @dret I have read your article with some interest. But I am still convinced that we can find a good solution which will make everybody happy.

A client can provide a server with a list of acceptable media types and a server can then decide what it serves. These media types could be requested:

• application/activity+json
• application/ld+json
• application/json
• json/activity+ld

The serialization can be identical for all these media types. Let us assume that this is what the server is doing.

Let us also assume that the client does not know anything about JSON-LD or RDF. In that case it does not seem to matter which of the four media types it receives. It is a AS2 client and expects AS2 and nothing else. If that assumption is not correct then I would like to know a relevant scenario.

The server also would not care because the serializations are identical. But there is one issue. An LDP server likely will not be aware of the two special AS2 media types. There should be a convincing reason to require these servers to know about these media types. Currently I am not aware of such a reason.

[dret]

we should try to separate these issues. the first question is what the "metamodel" of AS really is. is it some improvement/variation of AS1's JSON foundation, or is it JSON-LD and thus RDF? my claim is that it cannot reasonably be both. i have seen too many APIs trying to do the "i do not want to decide and make everybody happy" dance that @mnot describes so well, and they for sure all come out horrible, and most of them are simply broken as well. when we know what our processing model is, then we can discuss how to best signal that as a media type.

[akuckartz]

@dret Separating those issues makes sense - to some extent. In my view the main "problem" is the involvement of at least two communites with different interests. More precisely: the relative size or smallness of the intersection of those communities. The larger that intersection the better.

There was a time when some publishers of HTML-pages where very interested in correct grammar. And at the same time (most) other publishers very mainly or only interested that those pages looked nice when rendered by the most popular browsers. The situation now is that HTML5 specifies tag soup. (I suppose that the handling of media type application/xhtml+xml by IE - which did not support it for many years - played a significant role in this development. The advantages of XHTML therefore were not usable in a decisive time. And XHTML 2.0 did the rest...)

In any case: I hope that "Just Decide" is not seen as a Silver Bullet ;-)

[dret]

@akuckartz, the point i am making that it is a proven recipe for disaster to try to satisfy two metamodels within the same layer. "just deciding" means that you should pick one, and probably the one that happens to have more potential adopters using it. note that i am also making the point that you can still try to make both groups happy, but then it's much better to handle that in clean layers.

[jasnell]

Close due to lack of any action or further discussion

[akuckartz]

Added to the WG agenda on 2015-07-28:
Proposal to replace media type "application/activity+json" by "application/ld+json" plus profile => Reopen #52

[melvincarvalho]

@msporny great idea and comment, I've added this to the wiki:

https://www.w3.org/wiki/Socialwg/Media_type_for_AS2

Please feel free to add more points.

[jasnell]

@msporny ... unfortunately most media type parser implementations have issues with the double + suffix. Many drop the second suffix entirely and do not make it available to developers to see at all.

[BigBlueHat]

@msporny I think it'd be the other way around: applicaiton/activitystreams+ld+json as json is the foundational processing model, not activitystreams (which, afaik, doesn't have a different processing model other than JSON-LD or JSON).

In the Annotation WG F2F last week at TPAC, @azaroth42 and I were trending toward using application/ld+json;profile=http://www.w3.org/ns/anno.jsonld (where the profile value is also the value of @context within the document).

The data model for Annotations is RDF. The processing model SHOULD be JSON-LD, but MAY be JSON...if you know what the "magic strings" mean in the appropriate keys and values.

The expectation is that the response body will be a JSON-LD document matching the compacted form based on http://www.w3.org/ns/anno.jsonld as the context.

That seems to match what ActivityStreams 2.0 is and does also--JSON-LD that MAY be treated as JSON with it's terms (regardless) defined within a specific context file expressed in JSON-LD.

We'll be monitoring progress here (and plan to report back also) as we discover more.

Thanks!

[jasnell]

Compromise proposal: The spec can be updated to indicate that implementers SHOULD treat application/ld+json; profile="http://www.w3.org/ns/activitystreams" as being equivalent to application/activity+json. Not a MUST because there will be implementations that do not perform any JSON-LD processing at all and those should not be forced to indicate support for media type they do not actually support. For implementations that choose the full JSON-LD/RDF path, however, this equivalency gives them a path forward.

Note that @msporny's suggestion to use application/activitystreams+ld+json is currently not feasible due to limitations in many current media type parser implementations and the fact that the relevant specs (https://tools.ietf.org/html/rfc6839) do not actually define the meaning of multiple suffix attachments.

[azaroth42]

Given WG discussion regarding situations where valid AS2 documents can be invalid JSON-LD documents, +1 to activity+json with an equivalent profile. Propose that the features that would be lost when converting to JSON-LD be called out in some documentation, perhaps where ever the profile is defined.

[cwebber]

@azaroth42 I agree, I think that given the call today, @jasnell's suggestion there is the best way forward.

However, it would be good to know, what properties of AS2 might break under a pure json-ld expansion with implied vocabulary? GeoJSON is one. What are any others?

It would be a big win if we could figure out how to solve those, or at least know if they could be solved. Regarding the GeoJSON challenge of encoding issue (lists of lists) json-ld has some proposals on how to move forward.

[melvincarvalho]

Yes, it would be good to document somewhere (maybe the wiki?) the list of things in AS2 that break interop with JSON LD. It might just make sense for the smaller deployment to align with the bigger.

[jasnell]

None of the core vocabulary properties break with JSON-LD. AS2, however, does allow for extensions (such as GeoJSON) that are not JSON-LD compatible and currently warns against their use for exactly that reason (that's why the Place object has properties defined for instance). Even with those warnings, however, such non-JSON-LD compatible extensions are permitted under the AS2 "must ignore" policy for extensions that an implementation may not understand and implementations are required not to throw an error if such things are found.

[melvincarvalho]

Great that's pretty good news, and the JSON LD 1.1 may solve this use case.

I've added those comments to the wiki. What's missing is the section on the application/activity+json interop consideration (if there are any right now)

Looking roughly at the views, there's about

• 7 with arguments for application/ld+json
• 1 with arguments for application/activity+json
• 1 with arguments for both

Approximately 20 deployments using JSON LD documented.

Personally I dont feel strongly to +1 or -1 this.Id say consensus is leaning towards LD + profile param. Appreciate @jasnell gesture to compromise. It's something I could live with.

Please add extra info to the wiki if you have it.

[jasnell]

See PR #243

[akuckartz]

I appreciate the attempt of a compromise.

But I am not convinced. AS2 documents can not become invalid JSON-LD documents. That might sound paradox, but the only issue is that information might be lost by interpreting them as RDF triples.

JSON-LD does not require that all JSON data results in RDF triples. One can even construct pathological examples where a large JSON-LD document when interpreted as RDF is equivalent to zero triples.

That would be bad for LD users and SHOULD be avoided, but I do not see how a media type different from application/ld+json can prevent that.

[jasnell]

The media type does not prevent anything. It communicates the specific expectations, that is all. The compromise approach of treating the two as being equivalent ought to be enough to address the concern that has been raised.

[akuckartz]

@jasnell wrote:

None of the core vocabulary properties break with JSON-LD. AS2, however, does allow for extensions (such as GeoJSON) that are not JSON-LD compatible and currently warns against their use for exactly that reason (that's why the Place object has properties defined for instance). Even with those warnings, however, such non-JSON-LD compatible extensions are permitted under the AS2 "must ignore" policy for extensions that an implementation may not understand and implementations are required not to throw an error if such things are found.

This is a flawed argument. As already stated: GeoJSON is not JSON-LD incompatible. In any case a JSON-LD profile can "communicate specific expectations". Therefore this is yet another non-argument in favor of a specific new media type.

I would really like to understand why a specific media type would make sense and I am definitely prepared to change my mind. So far I have not seen any reason to do so.

[jasnell]

{
  "@context": {
    "@vocab": "_:",
  },
"type": "MultiLineString",
    "coordinates": [
        [ [100.0, 0.0], [101.0, 1.0] ],
        [ [102.0, 2.0], [103.0, 3.0] ]
      ]
}

When run through JSON-LD expansion, yields:

[
  {
    "_:coordinates": [
      {
        "@value": 100
      },
      {
        "@value": 0
      },
      {
        "@value": 101
      },
      {
        "@value": 1
      },
      {
        "@value": 102
      },
      {
        "@value": 2
      },
      {
        "@value": 103
      },
      {
        "@value": 3
      }
    ],
    "_:type": [
      {
        "@value": "MultiLineString"
      }
    ]
  }
]

Which, when compacted again results in :

{
  "@context": {
    "@vocab": "_:"
  },
  "coordinates": [
    100,
    0,
    101,
    1,
    102,
    2,
    103,
    3
  ],
  "type": "MultiLineString"
}

Which is obviously incorrect. The JSON-LD spec is quite clear that Arrays of Arrays are not valid in JSON-LD and example above shows that the JSON-LD algorithms are not capable of properly handling them.

I'm not trying to get you to change your mind. The compromise that I have proposed allows implementers to treat application/ld+json; profile="http://www.w3.org/ns/activitystreams" as being equivalent to the application/activity+json media type.

[akuckartz]

@jasnell That is more interesting than I expected. The JSON-LD spec contains this sentence in a "NOTE":

List of lists in the form of list objects are not allowed in this version of JSON-LD.
http://www.w3.org/TR/json-ld/#sets-and-lists

But that section is

• non-normative and
• referring to "list objects"

I will have a look at the example separately.

[akuckartz]

The issue in the example is not the list of lists but the combination with a JSON-LD context which tries to interpret that list of lists. Expanding that JSON-LD document without that context (or with a context for "type" only) works properly.

[jasnell]

"works properly" only in the sense that the "coordinates" property is ignored entirely and dropped from the expanded result. I wouldn't exactly call that "compatible". The JSON-LD algorithms quite clearly are not designed to handle arrays of arrays, regardless of how the @context is defined.

[akuckartz]

@jasnell I doubt that there exists any JSON-LD user or supporter who is not unhappy with that situation.

But the current issue is about the choice between creating a new media type and using application/ld+jsonplus profile. I still do not see how the "list of lists" problem is an argument in favor of a new media type or against the application/ld+jsonplus profile.

[melvincarvalho]

Good point about coordinates and lists of lists.

Do any of the user stories rely on this functionality?

[jasnell]

WG decided to accept the proposal to use application/activity+json as the media type and say that implementers SHOULD treat application/ld+json; profile="http://www.w3.org/ns/activitystreams#" as equivalent.

[melvincarvalho]

Thanks for everyone that contributed tho this discussion.

I suggest keeping the wiki page updated where possible as implementations progress

https://www.w3.org/wiki/Socialwg/Media_type_for_AS2

Casual observations (and this may be incorrect) of the gathered data, may lead one to believe that application/ld+json implementations significantly outnumber those of application/activity+json. I suspect we will know more in the fullness of time.

The WG may wish to consider marking application/activity+json at risk in case it fails to get the required traction, and then falling back to the existing REC may save time. Just a thought.