What’s It All About?

The C# compiler does an enormous amount of work so that we can write asynchronous code that looks almost identical to the synchronous version.

We pretty much just put the async and await keywords here and there, ending up with an asynchronous execution with all the benefits coming with that.

Gone are the days when we were manually passing callbacks as continuations leading to all sorts of complexities and maintainability issues.

This async/await convenience is almost magical as asynchronous programming is a lot different in its very nature. It consists of all sorts of challenges like attaching continuations, preserving state, flowing all kinds of contexts, propagating errors, etc.

I find the simplicity we end up with pretty fascinating. That’s why I decided to go for a deep dive and explore the topic in detail, accompanying the research with several articles.

For additional in-depth materials on async/await in C#, you can check this Pluralsight course.

The Articles

Let me give a brief summary of each article and describe how it contributes to the full picture.

Part #1 – The Awaitable Pattern

The Awaitable Pattern plays a vital role in the whole async/await workflow behind the scenes. The best way to understand how it works is to build your own awaitable type. This is what you will do in this post.

To reveal some of the content, we’ll go through every step of evaluating an await expression:

Key Concepts Covered In Part #1

• The core methods of the TaskAwaiter and their purpose
• The specific requirements to create your own awaitable type
• How do we attach continuations
• How is an await expression evaluated concretely
• Building your own awaitable type

Part #2 – Main Workflow and State Transitions

After getting familiar with the Awaitable Pattern, we take a high-level overview of the State Machine itself.

This article contains almost no code. However, it lays the crucial foundations of the whole async/await workflow.

Concretely, you will explore all the components of the State Machine and their interactions.

The source of truth for the discussion will be the following diagram:

Key Concepts Covered In Part #2

• How is the State Machine triggered
• How the await statements mark the different states of the State Machine
• The MoveNext method as the main driver of the State Machine
• How does the State Machine move through states, and how is the data preserved
• How and when are all the elements of the Awaitable Pattern being used

If you stop your exploration after reading this article – you will still have a pretty strong fundamental understanding of the async/await process.

Of course, I will advise you to take at least one step further and read the next post too.

With it, you will see some real and functional implementation of the async/await State Machine!

Part #3 – Conceptual Implementation

After getting enough high-level understanding of the async/await State Machine from Part #2, it’s time to get your hands dirty and see some actual implementation!

This implementation will be a little bit simplified. The reason is that the actual code is full of trickeries to achieve optimal performance and memory footprint.

Although these optimizations are quite substantial (and exciting to review), they don’t contribute to the general understanding of the State Machine. That’s why I decided to provide such a “conceptual implementation.”

Although I’ll simplify a thing or two, the implementation you’ll see in this post will be pretty close to the real one. What’s more – it will be fully functional!

Key Concepts Covered In Part #3

• How the initial async method is transformed so that it triggers the State Machine and returns the task
• The MoveNext implementation
• How is the state of the State Machine preserved
• How are the initial async method input parameters handled
• The exact mechanics of attaching continuations and moving between states
• What is the lifecycle of the resulting task

Getting through this article will leave you with an in-depth understanding of the async/await State Machine.

If you are not too interested in the advanced techniques in the actual implementation, you may stop here.

However, I’d advise you to at least walk through the next part. Quite possible you may discover something you’ve always been wondering about!

Part #4 – Concrete Implementation

This is the part when we get into the nitty-gritty details of the actual State Machine, produced by the compiler.

This is the kind of knowledge that you don’t need on a daily basis but truly get your skills to the next level.

The details you’ll see here will make you appreciate all the efforts and reasoning behind a real-world implementation used by millions of users.

I find such in-depth explorations as quite valuable. The takeaways are far more overreaching than just understanding some limited piece of functionality. It’s not only a lot of fun, but it takes you further on the path of becoming an expert in your programming ecosystem.

Key Concepts Covered In Part #4

• How and why the State Machine is kept on the execution stack if all the awaiters are already completed
• How is the State Machine boxed onto the heap exactly once in order to preserve its state across continuations
• What is the ExecutionContext, and how does it flow
• What are the unsafe methods in the async infrastructure, and how they help with performance

Part #5 – Synchronization Context

In this part, I’ll start exploring some of the most popular questions related to async/await.

Concretely, in the next couple of posts, I’ll clarify the misconceptions around the Synchronization Context and the usage of ConfigureAwait in nested and sequential async calls.

The Synchronization Context is an abstraction that lets you run a piece of code asynchronously without thinking about the specifics of the current environment. How does it work, and when is it invoked as part of the async workflow?

Key Concepts Covered In Part #5

• The concept of Synchronization Context and its’ implementations in different context-specific frameworks.
• How you can create and assign your own Synchronization Context.
• When and how the Synchronization Context is captured and propagated in the async infrastructure codebase.

Armed with the knowledge so far, I’ll move on with the series by answering some of the most common async/await questions related to nested async calls and the usage of ConfigureAwait.

Part #6 – Nested Async Calls and ConfigureAwait(false)

This is the part where I deal with the nitty-gritty details of nested async calls and the proper usage of ConfigureAwait(false).

From my experience, even if someone gains a pretty good understanding of the async/await workflow, there is still a lot of confusion around the behavior with nested async call chains.

This is especially valid when we raise the topic of when and how to set ConfigureAwait(false) in our libraries.

This article is meant to clean all of those uncertainties.

Concretely, following the diagram below, you’ll explore a very concrete example with an in-depth explanation of every execution step:

Key Concepts Covered In Part #6

• How is the Synchronization Context captured throughout a chain of nested async calls?
• Should you set ConfigureAwait(false) in your code at all?
• If you need ConfigureAwait(false), where should you set it in the call chain – on the first call, the last one, or everywhere?
• Is the default ConfigureAwait(true) a design flaw?

Part #7 – Stack Traces and Refactoring Pitfalls

Call stacks in async methods are fundamentally different compared to synchronous execution.

This leads to pitfalls where seemingly reasonable refactoring can make a method “disappear” from the Stack Trace.

Part #7 will explore these topics in detail.

Key Concepts Covered In Part #7

• How does the runtime prettify our async call stacks so they look as in a synchronous program?
• How and why are call stacks different in async methods compared to synchronous execution?
• How can we lose a method from the stack trace if we perform quite “reasonable” refactoring?
• How does the async/await machinery preserves the original call stack?

Summary

Going through the full series is an activity that would take you some time, and it requires concentration and focus.

However, I think the effort is entirely worth it.

This foundational knowledge will help you understand and answer most of the practical questions you may have had about async methods in C#.

Thanks for reading!

Resources

1. C# in Depth: Fourth Edition
2. Dissecting the async methods in C#