Add docs guide: Checking For the Existence of Relationships

[GeoffThorpeFT]

We are using v6 and have seen some significant performance issues with some of our relationship queries. We would like to avoid those issues when migrating to v7 but the documentation does not seem to cover our issue - or even suggest it may be an issue.

The v7 documentation provides the graphqQL syntax and examples for both Relationship Filtering and AggregationFiltering but it is unclear which of those to use when you just wish to filter by the presence, or absence, of relationships.

For example, we have approximately 250,000 infrastructure records stored in our Neo4J database that may, or may not, be linked to systems. We would like an efficient query to check for orphans 🤔

The v7 documentation provides Relationship filtering:

      {
          infrastructure_component (
            where: {
              attached_systems: {
                none: {
                  not: {
                    id: null
                  }
                }
              }
            }
          ) {
             id
         }
      }

But that seems to rely on us checking for none against the reverse of a positive connection - i.e. none of the connections are to null !

Or we can use an aggregation filter:

      {
        infrastructure_component (
          where: {
            attached_systems_connection: {
              aggregate: {
                count: {
                  nodes: {
                   eq: 0
                  }
                }
              }
            }
          }
        ) {
          id
        }
      }

But is that efficient, since it has to count all the connections before it can respond?

In v5 I think we could just check for the relationship field being null - ie. (where: { attached_systems: null } ).

In or v5/v6 tests we have seen:

• The { systemAggregate: { count: 0 } } syntax for n..1 takes multiple seconds to run whereas: { system: null } finishes in milliseconds.
• The {targetsAggregate: { count: 0 } } syntax for n...m takes multiple seconds to run whereas: {targets_NONE: {NOT:{code_EQ:null }}} finishes in milliseconds.

Could you please publish a recommendation/example using the new v7 syntax under a new heading entitled 'Checking For the Existence of Relationships'.
Many Thanks.

[angrykoala]

Hi @GeoffThorpeFT
Thanks for the feedback.
This document looks like a good idea, as this is a case that tends to be overlooked, as it is tricky to define what null should be in a GraphQL api.

As a rule of thumb, aggregations tend to be slower than normal queries, as you are seeing in your tests. But you are right we are not providing recommendations in our docs that could be helpful.

[GeoffThorpeFT]

Many thanks @angrykoala for your response 😄 Do you have a timescale to provide the preferred efficient v7 check for no relationships?

[angrykoala]

Many thanks @angrykoala for your response 😄 Do you have a timescale to provide the preferred efficient v7 check for no relationships?

I cannot guarantee a time for this, however I'll have a chat with the team this week about this, even if we cannot have the documentation ready I'll try to add some recommendations in this issue this week

[GeoffThorpeFT]

As part of an evaluation/experimentation with neo4j graphql v7 I can confirm that these two sample queries seem to be the fastest/most efficient way to check for the lack of, and presence of, relationships between parents and children:

  none: parents(where: {children: {none: {NOT: {}}}}) {          # or none: parents(where: {children: {none: {NOT: null }}}) {      
    code
  }
  some: parents(where: {children: {some: {NOT: {}}}}) {         # or some: parents(where: {children: {some: {NOT: null }}}) {
    code
  }

I know the NOT looks weird since it does not state what it is checking for but it does not really matter since its the none and some which are the key syntactical triggers for the generation and execution of the underlying cypher.

That resulting CYPHER looks like this for non-existence:

MATCH (this:Parent)
WHERE NOT (EXISTS {
  MATCH (this)-[:HAS_CHILD]->(this0:Child)
})
RETURN this { .id } AS this

and like this for existence:

MATCH (this:Parent)
WHERE EXISTS {
  MATCH (this)-[:HAS_CHILD->(this0:Child)
}
RETURN this { .id } AS this

If you supply details within the NOT object then they end up as clauses to the inner MATCH command which impacts the simplicity/effectiveness of the resulting CYPHER

[angrykoala]

Thanks for those examples.

Looking at that cypher, it looks like you already got the best performant query for this check that the current GraphQL library is going to generate, so you will be safe to use these queries with v7. I'll check if there are better versions of these queries (or if we can add them) so the actual check is a bit more intuitive, rather than an empty none: {NOT

[angrykoala]

Hi @GeoffThorpeFT
I've been double checking the API for v7, and these are both valid queries:

query {
  moviesConnection(where: { actors: { none: null } }) {
    edges {
      node {
        title
      }
    }
  }
  movies(where: { actors: { none: null } }) {
    title
  }
}

Both of these will return movies not connected to any actors (i.e. orphan nodes), using an efficient Cypher: WHERE NOT (EXISTS { MATCH ()-[]-() }). Which I think is the same as the Cypher in your version (effectively, the NOT is not necessary)

This would be the recommended approach for checking the non-existence of relationships, and I think will be the case you are trying to achieve. We will add this to the docs at some point, but cannot provide an exact timeline on that.

[GeoffThorpeFT]

Thanks @angrykoala.. I can confirm it works but it is a very interesting (confusing?) syntax choice.

I don't know how others feel but actors: { none: null } reads to me as "none of the actors are null" - meaning "some of the actors must be non-null" - which exactly the opposite of what we are trying to achieve - even though it does create CYPHER with the correct orphan checks.

The {none: {NOT: null }} syntax does emphasise the "none are not null" so "all must be null" 🤔

If you will allow me to make a syntax suggestion (enhancement?) then my preference would be:

movies(where: { actors: { empty } }) {
    title
  }

or

movies(where: { actors: { all: null } }) {
    title
  }

[i-like-robots]

Thanks @angrykoala - I wasn't aware of this syntax and it does produce the Cypher query I'd expect.

However, I agree with @GeoffThorpeFT that { none: null } reads as "non-null" which means the opposite to what the query syntax intends.

[angrykoala]

Hi @GeoffThorpeFT (and @i-like-robots )

This is a good point. I can see the confusion, as this API is more of a consequence of how the operations some all none work, more than anything.

The options you propose, on the surface, look reasonable, and it seems like either of these could be implemented (although some edge cases I can't think of right now may come up), I also agree those seem more intuitive.

Realistically, our bigger blocker on this atm is that both of these would be breaking changes, as both are valid queries currently (annoyingly, returning the exact opposite of the behaviour you'd expect), and changing that may break clients. After the release of 7, we are trying to minimize more breaking changes were possible.

I'll have a chat with the rest of the team, if we are able to come up with a non-breaking approach we may be able to add a better syntax (please, feel free to comment here if you have more ideas), but if not, we will probably sit this one out until it's time to release v8 with other breaking changes. The syntax may not be intuitive, but it does work efficiently, so the use case itself is not blocked by it

[angrykoala]

Hi @GeoffThorpeFT and @i-like-robots

After discussing with the team, this seems like an unintentional behavior of using null with none. As @GeoffThorpeFT pointed out, it doesn't read well, and it is not intuitive. Because of this, I've created an issue to build a better alternative, similar to the ones you proposed in the library. We will document the new alternative instead.

The behavior of { actors: { none: null } } will subsequently be deprecated once we have that alternative in place, so beware of using that syntax as it will be removed eventually

Thanks for pointing out this 🙏