TPL Dataflow

Overview

The Task Parallel Library (TPL) was introduced in the .NET Framework 4, providing core building blocks and algorithms for parallel computation and asynchrony.  This work was centered around the System.Threading.Tasks.Task type, as well as on a few higher-level constructs.  These higher-level constructs address a specific subset of common parallel patterns, e.g. Parallel.For/ForEach for delightfully parallel problems expressible as parallelized loops.

While a significant step forward in enabling developers to parallelize their applications, this work did not provide higher-level constructs necessary to tackle all parallel problems or to easily implement all parallel patterns.  In particular, it did not focus on problems best expressed with agent-based models or those based on message-passing paradigms.  These kinds of problems are quite prevalent in technical computing domains such as finance, biological sciences, oil & gas, and manufacturing.

For TPL Dataflow (TDF), we build upon the foundational layer provided in TPL in .NET 4. TDF is a complementary set of primitives to those primitives delivered in TPL in .NET 4, addressing additional scenarios beyond those directly and easily supported with the original APIs.  TPL Dataflow utilizes tasks, concurrent collections, tuples, and other features introduced in .NET 4 to bring support for parallel dataflow-based programming into the .NET Framework.  It also directly integrates with new language support for tasks and asynchrony provided by both C# and Visual Basic, and with existing language support in .NET 4 for tasks provided by F#.

Background

In multiple vertical industries, including those in the technical computing domains, software systems are often best oriented around the flow of data.  These “dataflows” are often large, infinite, or unknown in size, and/or require complex processing, leading to high-throughput demands and potentially immense computational load.  To cope with these requirements, parallelism can be introduced to exploit system resources such as multiple cores.  However, the concurrent programming models of today’s .NET Framework (including .NET 4) are not designed with dataflow in mind, leading to verbose code and low-level locking that complicates the source code, reduces system throughput, and introduces concurrency issues that can otherwise be avoided with a model more befitting to reactive systems. Furthermore, much of the data that enters these systems is the result of I/O operations. Developers tend to process this I/O synchronously, as asynchronous programming has historically been an all-around difficult activity.  When a thread performs synchronous I/O, it essentially gives up control of the thread to the I/O device, often decreasing the responsiveness of the application and, again, hindering the overall throughput and scalability of the system.

Visual Studio 2010 targeted this problem space for native developers with the addition to the C Run-time (CRT) of the Asynchronous Agents Library (AAL).  This model has proven to be very attractive and useful, and also shows up in the Concurrency & Coordination Runtime (CCR) available separately as a library contained in the Microsoft Robotics Studio.  TDF can be thought of as a logical evolution of the CCR for non-robotics scenarios, incorporating CCR’s key concepts into TPL and augmenting the enhanced model with core programming model aspects of the Asynchronous Agents Library.  This combination yields a technology that targets a wide range of scenarios.  Most CCR-based code can be brought forward and ported to TDF with a straightforward translation, composing well with workloads implemented with other primitives from TPL.  Similarly, for developers desiring to port their native Asynchronous Agents-based code to managed, or to write a mixed-mode application that incorporates message passing on both sides of the divide, TPL Dataflow provides direct feature-to-feature mappings for most concepts.

TDF brings a few core primitives to TPL that allow developers to express computations based on dataflow graphs.  Data is always processed asynchronously.   With this model, tasks need not be scheduled explicitly – developers declaratively express data dependencies, and in return the runtime schedules work based on the asynchronous arrival of data and the expressed dependencies.  In this manner, TDF is a generator of tasks just as is the Parallel class and PLINQ.  In contrast, however, whereas Parallel and PLINQ are focused on more structured models of parallel computation, TDF is focused on unstructured, asynchronous models.

Astute readers may notice some similarities between TPL Dataflow and Reactive Extensions (Rx), currently available as a download from the MSDN Data developer center. Rx is predominantly focused on coordination and composition of event streams with a LINQ-based API, providing a rich set of combinators for manipulating IObservable<T>s of data.  In contrast, TPL Dataflow is focused on providing building blocks for message passing and parallelizing CPU- and I/O-intensive applications with high-throughput and low-latency, while also providing developers explicit control over how data is buffered and moves about the system.  As such, Rx and TPL Dataflow, while potentially viewed as similar at a 30,000 foot level, address distinct needs. Even so, TPL Dataflow and Rx provide a better together story.  (In fact, TPL Dataflow includes built-in conversions that allow its “dataflow blocks” to be exposed as both observables and observers, thereby enabling direct integration of the two libraries.)

Architecture

At its heart, TDF is based on two interfaces: ISourceBlock<T> and ITargetBlock<T>.  Sources offer data, and targets are offered data; nodes in a dataflow network may be one or the other, or both. A handful of concrete implementations of these interfaces are provided in the library, and developers may further build their own in order to target more advanced scenarios (though it is expected that the built-in implementations suffice for the majority of developer needs).  Put together, these interfaces and implementations enable patterns for parallel stream processing, including multiple forms of data buffering; greedily and non-greedily receiving, joining, and batching data from one or more sources; selecting a single datum from multiple sources; protecting concurrent operations without explicit locking by executing tasks within a declarative reader/writer model; automatically propagating data from one operation to another, all with as much concurrency as the system and developer jointly allow; and more.  Each of these patterns can be used standalone, or may be composed with others, enabling developers to express complex dataflow networks.  The primitives provided by TDF not only compose well at the programming model level with tasks, parallel loops, observables, and language features in support of asynchrony, they also compose at the system level to enable efficient execution that doesn’t oversubscribe the system.

While TDF is reminiscent of actor-oriented models, it is important to note that it makes no built-in guarantees of isolation.  It is an expressive and flexible messaging substrate and task generation system (using TPL as the scheduling infrastructure) on top of which higher-level models may be built.

Complete whitepaper

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: