Expressions + Compute Functions

[gatesn]

Currently compute functions are separate from expressions. But they feel quite similar in lots of ways. The major difference is that compute functions are invoked with their input arguments, whereas expressions are given a scope and invoked with the whole expression tree. That is, expressions are responsible for evaluating their own inputs. This difference is perhaps important, e.g. for functions that wish to short-circuit.

Like most of Vortex though, we need to make both compute and expressions more extensible. Each compute engine that integrates with Vortex is likely to require slightly different semantics, and therefore benefit from an independent set of expressions (even if some base ones like logical operators are reused).

There is an existing ComputeFn trait that starts to experiment with this idea, although it's only implemented for Filter (which arguably shouldn't be a compute function since it's not scalar!!)

pub trait ComputeFn {
    /// The globally unique identifier for the compute function.
    fn id(&self) -> ArcRef<str>;

    /// Returns the function as the [`Any`] trait object.
    fn as_any(&self) -> &dyn Any;

    /// Invokes the compute function entry-point with the given input arguments and options.
    ///
    /// The entry-point logic can short-circuit compute using statistics, update result array
    /// statistics, search for relevant compute kernels, and canonicalize the inputs in order
    /// to successfully compute a result.
    fn invoke<'a>(&self, args: &'a InvocationArgs<'a>) -> VortexResult<Output>;

    /// Computes the return type of the function given the input arguments.
    ///
    /// All kernel implementations will be validated to return the [`DType`] as computed here.
    fn return_type<'a>(&self, args: &'a InvocationArgs<'a>) -> VortexResult<DType>;

    /// Returns whether the function operates elementwise, i.e. the output is the same shape as the
    /// input and no information is shared between elements.
    ///
    /// Examples include `add`, `subtract`, `and`, `cast`, `fill_null` etc.
    /// Examples that are not elementwise include `sum`, `count`, `min`, `fill_forward` etc.
    fn is_elementwise(&self) -> bool;
}

This trait works well, the Filter function can describe its behaviour, as well as have control of its entrypoint "invoke" function to perform pre/post validation. The implementation of invoke requires what we call kernels, however these are currently discovered by "asking" the encoding for a kernel. This prevents a third-party from registering a compute kernel for an encoding they don't own, even if it's for a compute function that they themselves defined

[gatesn]

Closing in favor of #3453