Replies: 16 comments 9 replies
-
Given that external code could obtain their own public SkipEnumerator Skip(int toSkip) => new SkipEnumerator(this, toSkip);
public struct SkipEnumerator : IValueEnumerator<T>
{
private int __state;
private List<T> __thisCapture;
private int version;
private int index;
internal SkipEnumerator(List<T> thisCapture, int toSkip)
{
__state = 1;
__thisCapture = thisCapture;
version = thisCapture._version;
index = toSkip;
}
public T TryGetNext(out bool success)
{
switch (__state)
{
case 0: throw new InvalidOperationException("Uninitialized enumerator");
case 1:
if ((uint)index < (uint)__thisCapture._size)
{
if (version == __thisCapture._version)
{
throw new InvalidOperationException("List changed");
}
index++;
success = true;
return __thisCapture._items[index - 1];
}
__state = 2;
goto case 2;
case 1:
default:
success = false;
return default(T);
}
}
public void Dispose() { }
// Implicit conversion detailed below in Interop
public static implicit operator EnumeratorAdapter<T, SkipEnumerator>(SkipEnumerator enumerator)
{
return new EnumeratorAdapter<T, SkipEnumerator>(enumerator);
}
} |
Beta Was this translation helpful? Give feedback.
-
@HaloFour updated |
Beta Was this translation helpful? Give feedback.
-
This is pretty close to what I was suggesting in #974 (comment), so obviously that makes me happy. :) The difference is that I was proposing having both I would caution against trying to make enumerators an exchange currency. In today's world, enumerator usage is always: get a fresh one at sequence start, enumerate, throw away. That's why their mutability doesn't matter. However, if we start having lots of things that return and accept enumerators directly, it will start to matter and get messy. We should not conflate the concept of a sequence (enumerable) with the concept of a cursor over a sequence (enumerator). |
Beta Was this translation helpful? Give feedback.
-
I wonder if |
Beta Was this translation helpful? Give feedback.
-
Agreed; an issue that did come up was
This is nice :)
Will try again... 3rd time's the charm ;-) |
Beta Was this translation helpful? Give feedback.
-
Would be something like: public interface IEnumerable<out T>
{
IEnumerator<T> GetEnumerator();
IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}
public interface IValueEnumerable<T, TEnumerator> : IEnumerable<T>
where TEnumerator : struct, IValueEnumerator<T>
{
TEnumerator GetValueEnumerator();
IEnumerator<T> IEnumerable<T>.GetEnumerator() => new EnumeratorAdapter<T, TEnumerator>(GetValueEnumerator());
} |
Beta Was this translation helpful? Give feedback.
-
Not sure implementing public partial interface IValueEnumerable<T, TEnumerator> : IEnumerable<T>
where TEnumerator : struct, IValueEnumerator<T>
{
TEnumerator GetValueEnumerator();
IEnumerator<T> IEnumerable<T>.GetEnumerator() => new EnumeratorAdapter<T, TEnumerator>(GetValueEnumerator());
// Achieve anything ?
EnumeratorAdapter<T, TEnumerator> GetEnumerator() => new EnumeratorAdapter<T, TEnumerator>(GetValueEnumerator());
} |
Beta Was this translation helpful? Give feedback.
-
@nguerrera 3rd approach showing all the variants would be partial class List<T> : IValueEnumerable<T, List<T>.ValueEnumerator>
{
int _size;
T[] _items;
// Class-level ValueEnumerable (iterator)
public enumerator<T> ValueEnumerator GetValueEnumerator() // typeof(ValueEnumerator)
{
int index = 0;
while (index < _size)
{
index++;
yield return _items[index - 1];
}
}
// Method iteratable (iteratable)
public enumerable<T> SkipEnumerable Skip(int index) // typeof(SkipEnumerable)
{
while (index < _size)
{
index++;
yield return _items[index - 1];
}
}
// Pass-through Method iteratable (iteratable)
public SkipEnumerable SkipTen() // typeof(SkipEnumerable)
=> Skip(10);
// Pass-through Method iterator (iterator)
public SkipEnumerable.Enumerator TenSkipped() // typeof(SkipEnumerable.Enumerator)
=> Skip(10).GetValueEnumerator();
// Class-level Enumerable - downlevel C# struct enumerable support
public EnumeratorAdapter<T, ValueEnumerator> GetEnumerator() => GetValueEnumerator();
// IEnumerable.GetEnumerator() // implemented via default interface
// IEnumerable<T>.GetEnumerator() // implemented via default interface
} |
Beta Was this translation helpful? Give feedback.
-
The main issue is lack of inference of generic arguments based on constraints so you need to do: list.Any<List<int>, List<int>.ValueEnumerator>();
list.Count<List<int>, List<int>.ValueEnumerator>(); Rather than list.Any();
list.Count(); To resolve public static int Count<TSource, TValueEnumerator>(this TSource source)
where TSource : IValueEnumerable<TValueEnumerator>
where TValueEnumerator : struct, IValueEnumerator However, running a perf test for a "ValueLinq" https://gist.github.com/benaadams/294cbd41ec1179638cb4b5495a15accf
Its twice as fast and cuts allocations to zero |
Beta Was this translation helpful? Give feedback.
-
small nit:
if x is IValueEnumerable: using (E e = ((C)x).GetValueEnumerator();)
{
while (true)
{
V c = (V)(T)e.TryGetNext(out bool success);
if (!success) break;
// Loop body
}
} and just the loop if x is IValueEnumerator. because this (and other conversions) should work (as they do in current foreach statements): long Sum(List<int> list)
{
long a = 0;
foreach(long l in list) { a += l; }
return a;
} And that it should be structurally done instead of depending on types. |
Beta Was this translation helpful? Give feedback.
-
Works for foreach, but has issues with a ValueLinq or |
Beta Was this translation helpful? Give feedback.
-
Yes that is what I meant. For |
Beta Was this translation helpful? Give feedback.
-
I have some interesting application for this and would help my team writing allocation 0 code in a more comfortable way. |
Beta Was this translation helpful? Give feedback.
-
List is that unfortunate example... I understand you guys hope for some linq over values. But please re-design for performance. Side Note: Interleaving is also important issue here. Here is an interesting topic on Rust SO. I don't see why C# have to be worse in this regards. |
Beta Was this translation helpful? Give feedback.
-
Well 3 years have passed since this case was opened, but I have recently put my noggin to work on this, well not this precisely, but a particular facet on this which is a ValueLinq. Now I'm probably one of a small number of people who have written an implementation of Linq from the ground up (Cistern.Linq) where it's raison d'etre was inverting the linq pull model into a push model via use of some fancy generics (It's good! give it a shot!). But it did have a couple of issues. It created a fair bit more garbage and wasn't fantastic with small collections due to the overhead of setting everything up (which was related to the garbage...) I had basically escaped from my weird obsession with all things Linq until 3 days ago when #1931 was closed, which triggered an email sent to me, which triggered me thinking about a ValueLinq implementation, which... Yeah well enough life story. Just get on with it. If you're a person who just likes code then go and take a look at the embryonic state of this ValueLinq here So why would you bother with a new ValueLinq? I mean @kevin-montrose gave it a crack with LinqAF so is there any other need? Well there are two key innovations here which different my version from his. The first is the realization that carrying around the Enumerator type all over the place is a bore (but don't we need it?). The second is that we need a common generic type to carry our implementation in so that c#'s type inference can work without having to resort to massive bloat of specific types for each container. So working with those two idea I've prototyped some of the basic operations (and polyfilled the rest of the Linq API stubs with forwards to System.Linq which works, but isn't particularly performant - yet not terrible) and I must say I'm fairly pleased with the result. Now I must admit I haven't checked out what underlying code the runtime creates from this stuff because it could be a monster with all the generic guff that I'm using to hold this ship together! So key takeaways are:
Now, as I always tell me children, you can't beat free (yet somehow they don't want the free stuff I give them....). So in the case, for example, of The surface is pretty sparse at the moment - There is also so totally crazy optimizations for Look, I'd love some help to crank out the rest of the functionality, or even someone to do some analysis on the bloat in the machine code that this generates. From my past experience with |
Beta Was this translation helpful? Give feedback.
-
With the I ran into the same issue that @benaadams mentioned above regarding generic type argument inference. By introducing an unused method parameter, I can get the compiler to deduce the generic type arguments. Something like I'm not sure if my project is any benefit here, but I came across this thread while searching for a way to deal with this. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
tl;dr
To completely implement fast struct class-level iterator, fast struct method iterator, and the
IEnumerable<T>
IEnumerable
set:Motivation
Its fairly convoluted to add an non-allocating struct enumerator to a class; and yield iterators which have a simpler syntax are allocating and also don't work as a class-level struct enumerable.
Related:
System.Linq
is a wonderful feature, however it also allocates for all the IEnumerables; so it would be desirable to find a solution that supports a non-allocating struct-basedLinq
or Value LinqBackground
Given a common or garden List class
Adding an indexer is fairly straight forward using the
this[]
property, and it would be good to have a similar ease of use for an enumerator; that is also non-allocating for yield enumerators.Follow up to #974
Proposal
Struct Method Iterator
Contract
Contextual keyword
New method_modifier to specify the method is a ValueEnumerator which is genericly typed
enumerator<>
It is a struct-based iterator and works with
yield
and is of type return_typeImplementation Usage
public enumerable<T> StructTypeName MethodName()
Convention:
public enumerable<T> MethodNameEnumerator MethodName()
List<T>
example for method iterator (from opening definition)Stack<T>
example for method iterator (from C# spec iterator Stack sample)The type name of the Enumerator is part of the method_header declaration so user is at liberty to define it as
Consumption Usage
foreach
will bind to aIValueEnumerator<T>
and usage as now:Code-generation
Implementation Generation
Example code for
List<T>.Skip(int toSkip)
that the compiler could generateConsumption Generation
Example code for
foreach
that the compiler could generateStruct Class-level Iteratable
Contract
Implementation Usage
Consumption Usage
foreach
will bind toGetValueEnumerator
in preference toGetEnumerator
if available and usage as now:Code Generation
Implementation Generation
No special code generation
Consumption Generation
Example code for
foreach
that the compiler could generatePass-through iterator
Implementation Usage
Code Generation
No special code generation
IEnumerable Interop
A common
EnumeratorAdapter
type can be shared for allIValueEnumerator
s with an implicit conversionThis means to implement the IEnumerable pattern is as follows:
foreach precedence
ValueEnumerator<T>
>ValueEnumerable<T>
>IEnumerable<T>
>IEnumerable
Issues
Return vs out
Return
T
or returnbool
?@jaredpar mentions on twitter
e.g.
vs
Disposal on iterator
Some questions
Covariance
I said fast right 😉 Though open question...
Type inference
Type inference isn't complete for Value Linq style extension methods
Value Linq
IValueEnumerable
would be a preferred patternAs it can then be type short-cut e.g.
But then type inference for the generics is an extra step away
/cc @jaredpar @nguerrera @CyrusNajmabadi @davkean @HaloFour @sharwell @svick @HaloFour @karelz
Beta Was this translation helpful? Give feedback.
All reactions