C# Yield Return Demystified

Image By Author

One C# language feature that can initially seem like magic is yield return. This feature is very similar to generators in Python. Generators (in Python) and yield return (in C#) allow you to write very simple-looking functions to implement behavior that normally would require more complex stateful iterator objects. This post will explain the behavior of a function using yield return and then explain what code would be required to implement this behavior without yield return.

Implementing IEnumerable with yield return

Here is an example of a function that uses yield return:

As you can see this function returns an IEnumerable<string> which means it can be used in a foreach loop like this:

Running this will produce the following output:

To prove that the function is paused and resumed like this you can add Console.WriteLine statements to the function like this to help demonstrate the order of execution:

This will produce the following output:

As you can see from the order in which the messages are printed, the function really is paused between loop iterations and resumed when the next value is requested by the loop.

This pausing and resuming can initially seem like magic, but below I will show how you can manually implement the same thing (albeit with a lot more code).

Manually Implementing IEnumerable

For translating our yield return function into a manual implementation of IEnumerable<T> and IEnumerator<T>, the book analogy doesn't fit as well. We don't really have a "book" since the values are all generated on the fly. However, we still need to implement IEnumerable<T> if we want our object to be usable directly in foreach loops.

The following simple implementation of IEnumerable<T> can be used for our purposes:

This effectively just takes the two arguments that our YieldReturnDemo function takes, holds on to them in private fields, and passes them along to the constructor of StateMachineEnumerator (our IEnumerator<T> implementation).

You can mostly ignore the non-generic IEnumerable.GetEnumerator(), it is just a passthrough to the generic implementation. It is required due to IEnumerable<T> implementing IEnumerable which existed before generics were added to C#.

What State Needs To Be Preserved In Our IEnumerator<T>?

Let’s take another look at YieldReturnDemo and note the state that needs to be kept track of while the function is paused:

The following values need to be persisted across pauses ( yield return s) in the method:

• first and last because they are initialized before the first yield return, and then used between each of the loop yield returns.
• beginTime because it is initialized at the beginning and is used again in the final yield return
• i because the value needs to be maintained from one loop iteration to the next and we pause between each loop iteration due to the yield return in the loop body.

Note that endTime does not need maintained across any pauses of the method because it is introduced after the yield return in the loop and is not used again after the following yield return.

Manually Implementing IEnumerator

Here is our IEnumerator<T> implementation commented with explanations of what is going on. Don't worry if you don't follow it fully. The main point is that it is a state machine and that this code is difficult and error prone to write by hand.

Below is an annotated image of the state machine code and the yield return code put side-by-side. The annotations highlight the parts of the code that are are used to implement the same behavior. As you can see the mapping is not one-to-one. I had to split up the loop logic and call the code that is equivalent to the last yield return from a couple of different places to get the right behavior. It may be possible to refactor this to clean it up a bit, but this accurately demonstrates the types of challenges that you’ll run into when trying to build this kind of thing without the help of yield return.

Conclusion

If you are interested to read in more depth about the code the compiler generates and to understand how it handles things like exception handling and finally blocks, I would recommend reading Section 2.4 of the book “C# In Depth (Fourth Edition)” by Jon Skeet.

If you found this post interesting you may also like my post about C# generics.

Originally published at http://jonkuhn.com on January 10, 2021.