RFC: A findBreakingChanges-type utility to iterate over queries actually being used

[joelgriffith]

When operating a large GraphQL server there's often portions of it that go unused. Utilities like findBreakingChanges offer incredible value to engineers, however there's times when a type needs to be updated (for instance to better reflect the capability of a REST api that might have gone missed), and though whilst breaking, won't actually break any clients (if a tree falls in the woods...).

In my own implementations I've begun recording/logging queries that come in so that those servers can eventually validate that changes will in fact not break existing clients. I believe all the pieces required to do this type of change detection are there, however there exists no obvious way to iterate over a collection of queries and validate that they will all in fact work against a schema.

I'm curious whether or not the community sees value in exposing a findBreakingChangesInQueries type of function that accepts either an array of queries (or a merged query), and a schema, and produces a list of breaking changes that will indeed break existing clients. I'd love to author this interface if others do indeed see value!

[wincent]

Thanks for bringing this up @joelgriffith. There is definitely value in building tools around this. The approach we take at Facebook is exactly the one you suggest (tracking actual field/type usage so that we can identify when otherwise "breaking" changes are safe to make, because the fields/types in question are never being exercised in practice).

In general I'd suggest that logging/validating entire queries might not be the way to go, because it is so expensive. More likely, you'll be interested in answering questions at a more granular level, such as, "How often is this field being resolved on this type"? So I think you most likely have a dependency on #284 (see also the related PR).

[leebyron]

I agree with @wincent - I would absolutely support the ability to pass additional metadata to findBreakingChanges, however list of queries would quickly become impractical. Instead, the metadata should describe if individual fields or types in the schema are actually being used or not (which you could derive from a collection of queries, or via logging tools like Apollo Optics)

[leebyron]

Closing this since this utility now exists!

[leebyron]

Ah sorry, misread. Nothing which handles the input of field/type usage as well.

[leebyron]

Perhaps something @sam-swarr could help with or at least help review if someone else were to take this on.

[leebyron]

I believe this is highly related to what @mjmahone is working on these days

[mjmahone]

As a note: I have code that does this internally but it's not in a state I'd feel good about open-sourcing it.

The basic premise is that we build up a map of "used types" as we visitWithTypeInfo all the documents.

This works really well, and we use the same idea for a variety of things, like building a schema to use at runtime. My version is much heavier-weight than it needs to be: you should be able to build this using the Schema AST nodes to produce a reduced Schema AST. My solution transitions between GraphQLSchema and IntrospectionSchema, which I don't recommend.

[jgcmarins]

We've implemented a bot to check that for us: https://github.com/entria/graphql-vigilant-bot