You need TPL if you are developing a system that:

• Deals with large data
• Latency issues affect the application
• Parallel execution is required.
• Data larger than the hardware specifications (Supply 20GB data into an 8GB RAM)

When to NOT use TPL Dataflow

• Not suitable only for code organization and readability
• Micro optimization (anything executed in millisecond range)
• When the persistence State is important (TPL Dataflow mostly exists in memory)
• Run sequential problems in parallel
• If TPL overhead becomes an issue, then it is not ideal for the problem

It is a library that simplifies implementation for solutions that deal with large data, systems with latency issues or

just need high-performance parallel execution.

Characteristics:

• Execute branches in multiple cores
• Execute tasks asynchronously
• It is a push architecture, not a pull. The consumer is offered the Message.

TPL Dataflow is designed for developing applications with a non-linear execution circle. It is capable of running multiple branches of code at once, at multiple processor cores, and is also able to run different tasks asynchronously.

Asynchronous aspects:

• Accessing data/resources from different sources from the network
• Require context synchronization from tasks returning results.

Control Flow Programming vs Dataflow Programming

Imagine CFP to be a straight line with multiple workers, each deciding if should process every item. The Dataflow is a parallel line with multiple branches. The way we construct this line will determine which branch (worker) will deal with each item.

CFP:

Dataflow Programming

Task Level Parallelization

• Offers no resource duplication
• Auto Scaling
• Prone to run into problems

Pitfall of Parallel programming for small amounts of data:

• The more nested we get the more tasks we create
• The more tasks we create, the more threads we require from the Operating System
• The more threads we require, the more threads the Operating System has to manage

This is an expected behavior if the amount of data we process is significantly small making the thread creation and synchronization overhead larger than the gains of running.

Benefits of using Dataflow architecture design:

• Code readability
• Business mapping
• Global code orchestration
• Separation of business and execution logic

Task Parallel Library installation

It is a NuGet package published by System.Threading.Tasks.Dataflow