Vinicius Quinafelex Alves

Prefetching is a technique that starts loading data before it is needed, reducing total runtime at the risk of loading unnecessary data.

In coding, since there is more knowledge over what is needed to execute a method, it is easier to control when certain data should be loaded or not.

On C#, using async and tasks allow fetching data without interrupting the flow of code. This way, by prefetching data, the algorithm can dilute the downtime of I/O operations by working on other operations while waiting for the I/O result.

Below are some examples of prefetching with asynchony:

Prefetching one result

Calling an async method will immediately start its execution without interrupting the code flow. By holding the reference of an async task, it is possible to await the data only when it is actually needed.

public static async Task<string> GetHtmlAsync(string uri)
{
    using (var client = new HttpClient())
        return await client.GetStringAsync(uri);
}

// Start prefetching
var taskHtml = GetHtmlAsync("https://domain.com");
CodeWithoutHtml();

// Await and consume the result
var html = await taskHtml
CodeWithHtml(html);

Prefetching multiple results

Calling async methods in sequence without awaiting them is enough to fetch their data in parallel, without interrupting code flow, until an await is executed.

// Start pre-fetching
var taskHtml1 = GetHtmlAsync("https://domain1.com");
var taskHtml2 = GetHtmlAsync("https://domain2.com");
var taskHtml3 = GetHtmlAsync("https://domain3.com");

// Await results
var html1 = await taskHtml1;
var html2 = await taskHtml2;
var html3 = await taskHtml3;

Chaining prefetching

There are situations where an async method requires the result of another async method, creating a chain of fetches.

Tasks can be chained through the use of ContinueWith(), which invokes a method that starts executing as soon as a task is completed, and Unwrap(), which will expose the chained task being executed inside ContinueWith. Note that, since ContinueWith will only start when the task is completed, calling .Result property will not block the thread.

// Start pre-fetching
var taskUrl = RetrieveUrlAsync();

var taskStatusCode = taskUrl.ContinueWith(async (task) => 
{
    return await GetStatusCodeAsync(task.Result);
}).Unwrap();

var taskFavicon = taskUrl.ContinueWith(async (task) => 
{
    return await HasFaviconAsync(task.Result);
}).Unwrap();

// Await results
var statusCode = await taskStatusCode;
var hasFavicon = await taskFavicon;

Prefetching with IAsyncEnumerable

When using IAsyncEnumerable, it is also possible to prefetch the next result by manipulating the IEnumerator, as demonstrated by the following extension method. Benchmarks generated by BenchmarkDotNet.

public static async IAsyncEnumerable<T> WithPrefetch<T>(this IAsyncEnumerable<T> enumerable)
{
    await using(var enumerator = enumerable.GetAsyncEnumerator())
    {
        ValueTask<bool> hasNextTask = enumerator.MoveNextAsync();

        while(await hasNextTask)
        {
            T data = enumerator.Current;
            hasNextTask = enumerator.MoveNextAsync();
            yield return data;
        }
    }
}

// Prefetching 1 item
await foreach(var item in EnumerateAsync().WithPrefetch())
    Process(item);

There is a significant performance improvement prefetching data when fetch time and processing time are close. Attempting to use prefetch on data already in memory overloads the system with no gains.

There are no benefits prefeching more items when only a single item is processed at a time. Below is a code to prefetch more a single item:

public static IAsyncEnumerable<T> WithPrefetch<T>(this IAsyncEnumerable<T> enumerable, int prefetchDepth)
{
    while(prefetchDepth > 0)
    { 
        enumerable = enumerable.WithPrefetch();
        prefetchDepth--;
    }

    return enumerable;
}

// Prefetching 10 items
await foreach(var item in EnumerateAsync().WithPrefetch(10))
    Process(item);

Benchmarks demonstrate that prefetching more than one data causes a linear increase on processing time.

The method used as a reference sums 100 numbers from an enumerator when data is loaded synchronously, asynchronously or asynchronously with prefetch. 1 ms = 1,000,000 ns

Changing only processing time or fetching time without the other does not impact on the execution time of the function, regardless of the configured depth of prefetched data.

Increasing the prefetching depth also does not improve performance even when processing time and fetching time favor parallelism.