Splitting the concept of references into two? (Tree reference and Identifier Reference)

[sergioisidoro]

One of the features that I both love and struggle the most is references. And I think the main reason is because it was designed to reference a node from the tree, but has potential to become much more, and be misused exactly because of its potential.

My main problem comes from trying to use references with data passed from the server. While safeReferences partially solves this problem, safe references were designed to remove the data from the tree/array/map when they are not found. Which reinforces the idea that references are designed as client side references, not server references.

Let's imagine that the server response returns the following Todo response:

{
   "id": 2,
   "tittle": "hello world",
   "authorId": "3342"
}

Current problems in using authorId as a reference is:

• todo.authorId returns an object, instead of a number, as the property name suggests
• Author might not be in the tree, resulting in an error, or if safe reference used, authorId set to undefined
• We can duplicate the property, but it would require maintaining the two properties in sync.

A way that I've been using to bypass this issue is to make authorId a property, and create a view that resolves the reference to the User model. However that feels a bit like a hack and a waste of the references potential. This also solves the semantics problem (authorId returns the id, not the object), and allows for easy access to the reference

const Todo = types.model({
    id: types.identifier,
    title: types.string,
    authorId: types.string,
}).views(
    get author() {
      return resolveIdentifier(User, getRootStore(self), self.authorId)
    },
)

What I would love to be able to do, is have both client tree references, and general identifier references, that can later be used to reference a model in the tree.

const Todo = types.model({
    id: types.identifier,
    title: types.string,
    authorId: types.identifierReference(User)  // Reference to a User identifier, which would validate the type (string/number)
    author: types.safeReference("authorId")    // Reference based on the object property "authorId"
})

const TodoStore = types.model({
    todos: types.array(Todo),
    selectedTodo: types.reference(Todo)         // A reference to a node in the tree
})

In this case todo.authorId would return "3342", keeping the consistency of the name, and allow for todo.author safe reference to return undefiend if the node is not in the tree. This would work almost identically to many databases, where identifierReference would be analogous to a foreign key.

Are there any other ways of solving this issue?
Has anyone found a good way to use references with data passed from the server that might not yet (or even never - because of permissions) be in the state tree.

[coolsoftwaretyler]

Hey @sergioisidoro - I think I see where you're coming from with regard to the many ways references can and do get used, but I wonder if a call to the existing getSnapshot would at least resolve your issue with the difference between the objects and the primitive values.

Consider this code:

import { getSnapshot, types } from "mobx-state-tree";

const Todo = types.model({
  id: types.identifier,
  title: types.string
});

const TodoStore = types.model({
  todos: types.array(Todo),
  selectedTodo: types.reference(Todo) // A reference to a node in the tree
});

const t1 = Todo.create({
  id: "1",
  title: "First"
});

const t2 = Todo.create({
  id: "2",
  title: "Second"
});

const store = TodoStore.create({
  todos: [t1, t2],
  selectedTodo: "1"
});

console.log(store.selectedTodo); // {id: "1", title: "First"}
const snapshot = getSnapshot(store);
console.log(snapshot.selectedTodo); // "1"

You can see the output and play around with it in Codesandbox here: https://codesandbox.io/s/vigilant-sea-tpwquv?file=/src/index.ts

Just logging out store.selectedTodo will give you the node being referenced. But getSnapshot will serialize the store, and you'll get the actual reference value.

When I fetch data from the network and either put it into, or take it out of, MST, I usually do some kind of serialization with getSnapshot and applySnapshot, and I haven't run into any sharp edges. I will say, my use case doesn't have a lot of granular permissions, so perhaps there are some other scenarios where the serialization is insufficient.

What do you think? Does that get at the core of what you're thinking? Or is there something else you feel like you need from an additional identifier/reference type?

[sergioisidoro]

Hi @coolsoftwaretyler ! Thanks for taking the time to answer :)

Yes, indeed your suggestion is a quick way to get the primitive. But my issue extends to the async loading of data, and wanting to keep my api response and state as close as possible to each other, without having to write much glue code.

Coming back to my initial example, if my Todo API returns:

{
   "id": 2,
   "tittle": "hello world",
   "author": "3342"
}

and I have author as a reference in my model, chance is I might not have the user 3342 loaded in the tree, since there might be thousands of users i don't want to pre-load them all in my state (or even not have access to that particular user due to permissions).

I could inline serialise author, but then we need to create glue code, where I have to 2 step update the store, first adding the user to the userStore, and then modifying the snapshot, changing the inline user to its reference to insert it in the todosStore. And I cannot use safeReference because it will turn the reference to undefined, and hence lose data.

Maybe my main issue is losing the data with safeReference. But if I create a non destructive reference that returns undefined, the model won't get observed once the referenced object does appears in the tree.

This leaves me with custom references where I have to do a bit of glue code for loading missing references during resolution. I think this is the canonical way of solving my problem. But still very feels very boilerplatey, and doesn't solve the problem of not having permission to load that particular resource from the server

Sorry for the rambling, just putting my thoughts into words.

[coolsoftwaretyler]

No apologies necessary - I think the rambling provides good details!

I do think you've sort of found the conventional way to do this. I would offer two pieces of advice to either you or someone else who finds this thread:

1. I do wonder if you could do fewer intermediary steps by lifting your network request up higher in the tree, so that you get all the data you might need, figure out where it needs to be placed in the tree, and then apply it as a snapshot or patch to child stores. So in your example, your network request might come back with two sets of data: one for the TODOs, another for the relevant authors. Then you would add todos and authors in separate actions (or even just one particular patch/snapshot action) to child trees.
2. Alternatively (or maybe this works alongside that option), I find that glue code like this is easier to manage if you treat it like a Middleware. It doesn't stop you from having to sort of do a bunch of pre-and-post processing, but it does feel like a much more "conventional" place. We have a few places in my app that need to extend MST functionality, and it's nice to know that the edge cases almost all get handled as custom middlewares.

Other than that, I don't know if I have a better answer to your original prompt about the conceptual difference between identifier and tree nodes as references. Maybe someone else will!

[giaset]

Agreed this is a huge issue... I always end up doing something like this but it's so ugly

const Market = types
  .model('Market', {
    categoryId: types.string,
    id: types.identifier,
    participantId: types.string
  })
  .views(self => ({
    get category() {
      return this.rootStore.categories.get(self.categoryId);
    },
    get participant() {
      return this.rootStore.participants.get(self.participantId);
    },
    get rootStore() {
      return getRoot<{
        categories: ObservableMap<string, CategoryModelType>;
        participants: ObservableMap<string, ParticipantModelType>;
      }>(self);
    }
  }))