[WIP] Higher kinded types

[kpdonn]

This is the start of an attempt at creating a PR for adding support for higher kinded types (#1213). It's very incomplete but I managed to get the basics working enough that I figured it's worth getting some feedback on.

The only syntax I've added so far is the ability to create type parameters that have their own type parameters. I haven't gotten as far as adding any type of T<*> syntax yet so they have to be named right now. Essentially this is an implementation of the third option @Artazor proposed at #1213 (comment). I used that comment to get me started so thank you @Artazor, and also thanks to @gcanti for his blog post and fp-ts library which I've been using for reference.

I've only attempted using interfaces and classes as instantiations of generic type parameters so far. That is partially a result of not being finished, but also partially because internally those are implemented to be much easier to use with generics, so I'm not sure yet how well generic type aliases will eventually work with this.

Examples of what works right now:

interface Functor<Container<_T>, A> {
    map<B>(f: (a: A) => B): Container<B>;
}

 // FunctorX passes itself as a generic type parameter to Functor
interface FunctorX<A> extends Functor<FunctorX, A> {
    map<B>(f: (a: A) => B): FunctorX<B>;
    xVal: string;
}

interface FunctorY<A> extends Functor<FunctorY, A> {
    map<B>(f: (a: A) => B): FunctorY<B>;
    yVal: A;
}

declare const initialX: FunctorX<string>;
declare const initialY: FunctorY<string>;

function staticMap<F<_T> extends Functor<F, _T>, A, B>(fa: F<A>, f: (a: A) => B): F<B> {
    const result = fa.map(f);
    return result;
}

const resultX = staticMap(initialX, val => val.length);
const expectX: FunctorX<number> = resultX;

const resultY = staticMap(initialY, val => val.length);
const expectY: FunctorY<number> = resultY;

const resultX2 = staticMap(initialX, val => [val]);
const expectX2: FunctorX<string[]> = resultX2;

const resultY2 = staticMap(initialY, val => [val]);
const expectY2: FunctorY<string[]> = resultY2;

interface FMap<A, B> {
    (a: A): B
}

interface StaticFunctor<T<_T>> {
    <A, B>(a: T<A>, fmap: FMap<A, B>): T<B>;
}

interface LiftedResult<T<_T>> {
    <A, B>(f:  FMap<A, B>):  FMap<T<A>, T<B>>
}

declare function stringToNumber(a: string): number
declare function lift<T<_T>>(f: StaticFunctor<T>): LiftedResult<T>
declare const arrayFunctor: StaticFunctor<Array>

const liftedArray = lift(arrayFunctor);
const strArrayToNumArray = liftedArray(stringToNumber);

const arrayOfStrings = ["1", "2", "3"] // string[]
const arrayOfNumbers = strArrayToNumArray(arrayOfStrings); // number[]
const result: number[] = arrayOfNumbers; // assignment works

const expectError = strArrayToNumArray(arrayOfNumbers);
 // error as expected because number[] cant be passed to argument that needs string[]

Let me know what you think. I especially could use examples of patterns that people would expect to work with higher kinded types so I can create more tests.

[SimonMeskens]

I find that @masaeedu is an endless source of example code. Here's his GenericMaybe:

// Based on provided sample by Asad Saeeduddin

// just :: a -> T a
// none :: T a
// match :: { "just" :: a -> b, "none" :: b } -> T a -> b
const GenericMaybe = ({ just, none, match }) => {
    // Functor
    // map :: (a -> b) -> T a -> T b
    const map = f => match({ none, just: value => just(f(value)) });

    // Monad
    // of :: a -> T a
    const of = just;
    // chain :: (a -> T b) -> T a -> T b
    const chain = f => match({ none, just: f });

    return { map, of, chain };
};

// A concrete representation of a Maybe
{
    const MAYBE = Symbol("maybe");

    const just = value => ({ [MAYBE]: value });
    const none = {};

    const match = ({ just, none }) => maybe =>
        MAYBE in maybe ? just(maybe[MAYBE]) : none;

    const { map, chain, of } = GenericMaybe({ just, none, match });

    // Examples
    console.log([
        map((x: number) => x + 2)(of(42)),
        map((x: number) => x + 2)(none),
        chain((x: number) => (x > 40 ? none : of(x * 2)))(of(42)),
        chain((x: number) => (x > 40 ? none : of(x * 2)))(of(32)),
        map((x: number) => none)(of(42)),
        map((x: number) => of(x * 2))(of(42))
    ]);
}

I might have to checkout your PR myself to test the limits. Stellar work so far though, if all those examples type correctly already.

[jack-williams]

This works for me on your branch!

interface Fix<F<_T>> {
  unFix: F<Fix<F>>;
}

interface ListContainer<A> {
  h: number;
  rest: A;
};

type List = Fix<ListContainer>;

const nil: List = { unFix: (<ListContainer<never>>undefined) };
const y: List = {
  unFix: { h: 4, rest: nil }
};

function arrayToList(xs: number[]): List {
  return xs.reduce((rest, h) => {
    return { unFix: { h, rest } }
  }, nil);
}

function listToArray({ unFix: xs }: List): number[] {
  return (xs === undefined) ? [] : listToArray(xs.rest).concat(xs.h);
}

const result: number[] = listToArray(arrayToList([1, 2, 3]));

I'm not sure it's possible to make List generic because there is no way to partially apply generics, or create anonymous type lambdas.

[kpdonn]

Pushed some updated code today that I think is a good step forward in both implementation quality and functionality.

In the initial commits a few weeks ago, instantiating generic type parameters worked mostly by just blindly throwing type arguments into the new type without knowing that they actually belonged where they were being put. In the updated implementation the generic type parameter's arguments are neatly mapped into place in the new type in the same way as all other type parameters.

In terms of functionality, everything I changed was to make the following wall of code work correctly:

/*** Setup code ***/
declare function stringLength(strarg: string): number;

export interface Functor<A, Container<_T>> {
    map<B>(f: (a: A) => B): Container<B>;
}
declare class FunctorX<A> implements Functor<A, FunctorX> {
    map<B>(f: (a: A) => B): FunctorX<B> 
    uniqueMethodX(): A | undefined
}
declare class InvalidFunctor<A> {
    map<B>(f: (a: A) => B): FunctorX<B>
    uniqueInvalidFunctorMethod(): void
}

function lift<C<_T>>(someStaticFunctor: <A, B>(ca: C<A>, f: (a: A) => B) => C<B>):
    <A, B>(f: (a: A) => B) =>
        <C2<_T> extends C<_T>>(ca: C2<A>) => C2<B> {
    return f => ca => someStaticFunctor(ca, f);
}

function staticFunctor<C<_T> extends Functor<_T, C>, A, B>(ca: C<A>, f: (a: A) => B): C<B> {
    return ca.map(f);
}

declare const functorXString: FunctorX<string>
const stringArray = ["not explicitly declared to implement functor"]; // string[]
declare const invalidFunctor: InvalidFunctor<string>;

/*** Tests ***/

// lifts staticFunctor function, which has the constraint that it works with types that implement the Functor interface
const liftedStaticFunctor = lift(staticFunctor);

// liftedStringLength has type <C<*> extends Functor<*, C>>(ca: C<string>) => C<number>  
// (note the * is still pseudo-code for now)
const liftedStringLength = liftedStaticFunctor(stringLength);

const result1 = liftedStringLength(functorXString); // FunctorX<number>
result1.uniqueMethodX(); // we can call methods from FunctorX because result1 is still a FunctorX

const result2 = liftedStringLength(stringArray); // number[]
result2.filter(num => num > 10); // we can call methods from Array because result2 is still an Array

const expectError = liftedStringLength(invalidFunctor); // error

Making the expectError line actually have an error was the trickiest part. InvalidFunctor almost implements Functor because it has a map method that accepts a function <A, B>(a: A) => B and returns a container type that implements Functor, however that container is a FunctorX<B> not an InvalidFunctor<B>.

At runtime the expectError line would not fail, but allowing it would eventually cause a runtime error if you called expectError.uniqueInvalidFunctorMethod() because you thought you had an InvalidFunctor<number> when you really have a FunctorX<number>.

Still a bunch of things to do, probably the biggest one is figuring out what to do about cases where the number of type parameters don't match exactly, but I'm optimistic they are doable.

[kpdonn]

Also @jack-williams, I don't think your example actually works even though it doesn't have an error. It just looks like it works because of your type annotations. If you take them away you start getting any types. For example:

interface Fix<F<_T>> {
  unFix: F<Fix<F>>;
}

interface ListContainer<A> {
  h: number;
  rest: A;
};

type List = Fix<ListContainer>;

const nil: List = { unFix: (<ListContainer<never>>undefined) };
const y: List = {
  unFix: { h: 4, rest: nil }
};

function arrayToList(xs: number[]): List {
  return xs.reduce((rest, h) => {
    return { unFix: { h, rest } }
  }, nil);
}

function listToArray({ unFix: xs }: List) {
  return (xs === undefined) ? [] : listToArray(xs.rest).concat(xs.h);
}

// result has type any
const result = listToArray(arrayToList([1, 2, 3]));

However I actually don't think it should work. For example the part:

interface Fix<F<_T>> {
  unFix: F<Fix<F>>;
}

is not something I currently expect to be valid once I get around to actually adding validations. The reason is that you are defining a value that has a type without all its type parameters instantiated. Specifically the second F in F<Fix<F>> still needs its _T type parameter instantiated to something.

I'm guessing your intention was that the missing _T would come from instantiating F with something like Fix<Array<number>> where F becomes Array and _T becomes number (edit: although maybe that wasn't your intention since you did write type List = Fix<ListContainer> later). But the way it works right now and (I think) conceptually the way it should work is that F can only be instantiated with another uninstantiated generic type. So Fix<Array>, and the _T would come later.

The reason is so that the pattern from my example can work:

export interface Functor<A, Container<_T>> {
    map<B>(f: (a: A) => B): Container<B>;
}
declare class FunctorX<A> implements Functor<A, FunctorX> {
    map<B>(f: (a: A) => B): FunctorX<B> 
    uniqueMethodX(): A | undefined
}

note how it's implements Functor<A, FunctorX> instead of implements Functor<A, FunctorX<A>>. If the latter was allowed then _T would already be instantiated as A and Functor wouldn't be able to later instantiate it as B in map<B>(f: (a: A) => B): Container<B>. That's how I have been looking at things anyway, let me know what you think.

[sledorze]

Would love to have a feedback from the TS team on that PR (not the implementation details but the general stance on including that feature).

[jack-williams]

I'm not sure I understand everything 100%.

When you say:

The reason is that you are defining a value that has a type without all its type parameters instantiated. Specifically the second F in F<Fix<F>> still needs its _T type parameter instantiated to something.

Do you mean that it is not possible to pass a type of kind * => * as a generic parameter? In the class case, is the implements clause implements Functor<A, FunctorX> a special case? That appears to do a similar thing of passing a type (constructor) FunctorX without all it's parameters instantiated.

The type of the value for property unFix does have all it's type parameters instantiated (that is, it's a ground type of kind *). Though the type expression does have intermediate sub-expressions who's kind is not *, namely the inner F of kind * => *, where Fix has kind (* => *) => *.

I couldn't quite understand all of @Artazor 's proposal, but is there a difference between how things are treated in value positions (like the type of unFix), and in extends / implements clauses?

I'm guessing your intention was that the missing _T would come from instantiating F with something like Fix<Array<number>> where F becomes Array and _T becomes number (edit: although maybe that wasn't your intention since you did write type List = Fix<ListContainer> later). But the way it works right now and (I think) conceptually the way it should work is that F can only be instantiated with another uninstantiated generic type. So Fix<Array>, and the _T would come later.

The missing _T should be instantiated as the structure get's unfolded (at least that was my intention), so _T becomes Fix<ListContainer> again as you unfold one level. The type parameter of ListContainer doesn't represent the type of the values in the list, but it represents the type of the tail, which should also be a list. I couldn't make the list generic in its values because there is no way to partially apply a generic. But it should look something like this (if it were possible)

interface ListContainer<H,T> {
  h: H;
  rest: T;
};

type List<A> = Fix<ListContainer<A>>; // partially applied version
type List2<A> = Fix< <T>ListContainer<A,T> >; // annonymous type-abstraction version.

[kpdonn]

I'm not sure I understand everything 100%.

No worries because I'm positive I don't understand everything 100% when it comes to this stuff.

Do you mean that it is not possible to pass a type of kind * => * as a generic parameter? In the class case, is the implements clause implements Functor<A, FunctorX> a special case? That appears to do a similar thing of passing a type (constructor) FunctorX without all it's parameters instantiated.

This is a very good point. I probably need to start actually implementing the validations I've been visualizing sooner than later to make sure they don't end up conflicting with themselves, but I think the answer is going to end up being that in some way or another, values have a kind * meaning they have no type parameters that need instantiating.

That is assuming I can actually implement that anyway. Because while I don't feel like unFix: F<Fix<F>>; makes sense because of the uninstantiated _T, I do feel like something like declare const a: Functor<number, Array> theoretically makes sense (although isn't useful at all compared to Array<number>) because it never results in a value with uninstantiated type parameters. But without actually having it implemented I can't tell you at a glance why one would be allowed but not the other.

The missing _T should be instantiated as the structure get's unfolded (at least that was my intention), so _T becomes Fix<ListContainer> again as you unfold one level. The type parameter of ListContainer doesn't represent the type of the values in the list, but it represents the type of the tail, which should also be a list. I couldn't make the list generic in its values because there is no way to partially apply a generic.

I'm starting to understand I think. Are the type of values always going to be the same within the same list? I assume so since you are converting to and from an array and not a tuple. If so does the following make sense? (The following actually causes compile errors right now because I don't handle the mismatched number of type parameters at all yet)

interface Fix<T, LC<_T> extends ListContainer<_T, LC>> {
    unFix: LC<T> | undefined;
}

interface ListContainer<T, LC<_T> extends ListContainer<_T, LC>> {
  h: T;
  rest: Fix<T, LC>;
};

const nil = { unFix: undefined };
const y: Fix<number, ListContainer> = {
  unFix: { h: 4, rest: nil }
};

function arrayToList<T>(xs: T[]): Fix<T, ListContainer> {
  return xs.reduce((rest, h) => {
    return { unFix: { h, rest } }
  }, nil);
}

function listToArray<T>({ unFix: xs }: Fix<T, ListContainer>): T[] {
  return (xs === undefined) ? [] : listToArray(xs.rest).concat(xs.h);
}

const result = listToArray(arrayToList([1, 2, 3])); // result is number[]
const result2 = listToArray(arrayToList(["a", "b", "c"])); // result2 is string[]

Although I may still not be understanding because in that code the LC<_T> in Fix is always just instantiated as ListContainer and I don't see anywhere that something more specific could be inferred so there's really no advantage to what I wrote compared to plain:

interface Fix<T> {
    unFix: ListContainer<T> | undefined;
}

[typescript-bot]

Thanks for your contribution. This PR has not been updated in a while and cannot be automatically merged at the time being. For housekeeping purposes we are closing stale PRs. If you'd still like to continue working on this PR, please leave a message and one of the maintainers can reopen it.