Last Updated on January 14, 2026 by Aram
In this tutorial, we will dive into the EF Core Tricks for Bulk reading Large Data Sets.
We will be highlighting the powerful read operations of the of the Entity Framework Extensions library by ZZZ projects.
In a previous article, I’ve blogged about EF Extensions capabilities from bulk writing perspective, so if you haven’t read that and interested to learn more, you can read the article here it is titled Exploring Bulk Operations in EF Core.
Bulk operations are not just about writes, reads break first with large ID lists and slow IN queries.
Most EF code works well, until data volume grows and integrations scale. Entity Framework Extensions addresses this exact problem.
It introduces dedicated bulk read APIs built for large datasets.
This article walks through the five methods for efficient bulk read operations, along with some highlights on the different ways of how custom joins are built using these bulk read operations.
You will also see a link to a DotNetFiddle which includes live code samples for the different methods of implementing bulk read operations using Entity Framework Extensions.
My special thanks goes to ZZZ Projects for building and supporting this wonderful library that is installed over 82 Million times. And thanks for sponsoring this article.
So let’s get introduced to the EF Core Tricks for Bulk Reading Large Data Sets using Entity Framework Extensions:
BulkRead
BulkRead loads data by joining an in memory list with a database table.
It behaves like a server side join, not a client side filter.
This method is designed for very large key lists and completely avoids IN clauses
var orders = Enumerable.Range(1, 10000).Select(i => new Order
{
Description = $"Order {i}",
CustomerName = $"Customer {i}",
Quantity = i * 2,
Amount = 10.5m * i,
CreatedDate = DateTime.UtcNow.AddDays(-i),
IsProcessed = i % 2 == 0,
Address = $"Address {i}, {i} + 1",
Notes = $"Notes {i}"
}).ToList();
var stopwatch = Stopwatch.StartNew();
await context.BulkInsertAsync(orders);
stopwatch.Stop();
var processedOrders = orders
.Where(o => o.IsProcessed)
.ToList();
var processedOrdersRecords = await context
.Orders
.BulkReadAsync(processedOrders);
WhereBulkContains
With the same underlying functionality as the BulkRead,
WhereBulkContains differs from BulkRead that it is a differed method, while BulkRead is an immediate method.
This method, same as other bulk read methods, produces an inner join statement rather than IN statement
ListcustomerNames = ["Customer 1", "Customer 3", "Customer 7", "Customer 9", "Customer 23"]; var ordersWithFivePrimeCustomerNames = context .Orders .WhereBulkContains(customerNames, o => o.CustomerName) .AsNoTracking() .ToList();
WhereBulkNotContains
WhereBulkNotContains returns rows that do not exist in the provided list.
Optimized alternative to NOT IN queries
This method outputs SQL query in the form of WHERE NOT EXISTS
Returns rows that do not exist in the provided list.
var lowQuantities = orders
.Where(o => o.Quantity <= 10000);
var ordersWithHighQuantities = await context
.Orders
.WhereBulkNotContains(lowQuantities)
.ToListAsync();
WhereBulkContainsFilterList
This is where you match against a complete list of records, not specific fields.
WhereBulkContainsFilterList is a very helpful to detect if certain items from a list exist in the database or not.
It is cleaner than manual joins
The filter list becomes a structured join condition.
This supports multi column matching.
var first100Batch = Enumerable
.Range(1, 100)
.Select(i => new Order
{
Description = $"Order {i}",
CustomerName = $"Customer {i}",
Quantity = i * 2,
Amount = 10.5m * i,
CreatedDate = DateTime.UtcNow.AddDays(-i),
IsProcessed = i % 2 == 0,
Address = $"Address {i}, {i} + 1",
Notes = $"Notes {i}"
}).ToList();
var first100BatchOfOrders = context
.Orders
.WhereBulkContainsFilterList(
first100Batch,
x => x.Quantity
);
WhereBulkNotContainsFilterList
Similar to the previous method, works on the complete records but it excludes the matching ones.
WhereBulkNotContainsFilterList is powerful for gap analysis and audit checks
Why it matters:
- Accurate exclusion logic
- Scales with complex keys
- No custom SQL needed
The below will return all orders except the last 500 records as specified in the in-memory list
var last1000Batch = Enumerable
.Range(9001, 10000)
.Select(i => new Order
{
Description = $"Order {i}",
CustomerName = $"Customer {i}",
Quantity = i * 2,
Amount = 10.5m * i,
CreatedDate = DateTime.UtcNow.AddDays(-i),
IsProcessed = i % 2 == 0,
Address = $"Address {i}, {i} + 1",
Notes = $"Notes {i}"
}).ToList();
var last1000BatchOfOrders = context
.Orders
.WhereBulkNotContainsFilterList(
last1000Batch,
x => x.Quantity);
Live Examples on DotNetFiddle
I am providing you a live code playground to try these methods yourself. Feel free to check this DotNetFiddle to see how powerful and flexible is the entity framework extensions library.
Custom Join
One of the powerful features of Entity Framework Extensions is that it enables you define a custom join, so that you can join on fields other than the primary key, where it would run natively on your DB provider.
Notice that to use a specific property or field name you need to make sure it exists in both your database entity and in the collection of items you are joining with to read from the database.
With Entity Framework Extensions custom joins are versatile and can be applied on the different bulk read methods, through the below options:
Default, using the default entity key
await context.Orders.BulkReadAsync(processedOrders);
Property name
await context.Orders.BulkReadAsync(processedOrders, "Description");
Lambda Expression
await context.Orders.BulkReadAsync(processedOrders, x => x.Quantity);
Composite, By List of Properties
await context.Orders.BulkReadAsync(processedOrders, new List{"CustomerName", "Quantity"});
Composite, By Anonymous Type
await context.Orders.BulkReadAsync(processedOrders, x => new {x.CustomerName, x.Quantity});
Limitations of Contains (LINQ/IN clause)
Supports only basic types like int or Guid.
You cannot use it directly with complex or anonymous types.
The list of values is limited by SQL parameter constraints.
Most databases (like SQL Server) cap how many parameters you can send, and larger lists can fail.
Does not support composite keys.
You can’t match on multiple fields with a single LINQ Contains.
Why these matter in practice
A simple LINQ Contains can break when you push it beyond trivial sets.
It fails on complex key scenarios, large lists, and anything beyond simple primitive filtering.
WhereBulkContains and the other bulk read methods remove these limitations by creating a temporary table and performing a JOIN instead of in-memory IN lists.
They handle unlimited items, support composite keys, and accept any list type (anonymous, entities, expando objects).
Final Thoughts
You read data in bulk because your system works in bulk, this is why efficient reads define how far your system can scale.
Using entity framework extensions will equip you with powerful and flexible bulk read capabilities, you choose the method, define the fields, and let this powerful library do its works at its best.
Used correctly, these patterns change your architecture, with fewer round trips, and fewer loops.
The end result is a predictable performance under load.
Entity Framework Extensions is the best library to accompany your EF Core implementation and extend its functions to include blazing fast bulk read and write operations.
Try it today: Entity Framework Extensions
References
Check out the amazing tools and libraries built by ZZZ Projects
Entity Framework Extensions Bulk Read and other methods documentation
Check out DotNetFiddle, the great tool for developers to test and benchmark any . by ZZZ Projects
Bonus
Enjoy this musical masterpiece of the baroque era by J.S.Bach Played by Yo-Yo Ma