ReFilter 1.2.0-alpha
See the version list below for details.
dotnet add package ReFilter --version 1.2.0-alpha
NuGet\Install-Package ReFilter -Version 1.2.0-alpha
<PackageReference Include="ReFilter" Version="1.2.0-alpha" />
paket add ReFilter --version 1.2.0-alpha
#r "nuget: ReFilter, 1.2.0-alpha"
// Install ReFilter as a Cake Addin
#addin nuget:?package=ReFilter&version=1.2.0-alpha&prerelease
// Install ReFilter as a Cake Tool
#tool nuget:?package=ReFilter&version=1.2.0-alpha&prerelease
ReFilter
A Package supporting Filtering and Pagination
This package is meant to support basic and advanced filtering, sorting and pagination scenarios for any queryable list or Entity Framework.
Basic features include search over string properties, filtering by built in mechanism or explicit custom filter definition, sorting by built in mechanism or explicit custom sort definition and finally pagination of results.
To use it, you should inject it inside Startup.cs like so:
services.AddReFilter(typeof(ReFilterConfigBuilder));
services.TryAddScoped<UserFilterBuilder>();
services.TryAddScoped<UserSortBuilder>();
This tells your DI to include the ReFilter bootstraper.
Only AddReFilter
is required. If you don't have any custom filters, you don't need to use anything else.
If you do, then you add them as any other service inside your DI.
For minimum implementation of required stuff check Filtering and Sorting Examples
And now you're ready to use ReFilter.
Basic Example
public async Task<PagedResult<User>> GetPaged(BasePagedRequest request)
{
var testQueryable = testList.AsQueryable(); // Any kind of User queryable
var pagedRequest = request.GetPagedRequest(returnResults: true); // Transformation
var result = await testReFilterActions.GetPaged(testQueryable, pagedRequest);
// Calling ReFilter GetPaged action
return result;
}
This would return a paginated list of users for the request you supply it.
BasePagedRequest
is the base request toward ReFilter and the main messanger object.
Main parts are Where, PropertyFilterConfigs and SearchQuery. The rest gets populated inside ReFilter.
/// <summary>
/// Where object for 1:1 mapping to entity to be filtered.
/// Only requirenment is that property names are same
/// </summary>
public JObject Where { get; set; }
/// <summary>
/// Defines rules for sorting and filtering
/// Can be left empty and in such way, the default values are used.
/// Default values are no sort and Equals comparer
/// </summary>
public List<PropertyFilterConfig> PropertyFilterConfigs { get; set; }
/// <summary>
/// String SearchQuery meant for searching ANY of the tagged property
/// </summary>
public string SearchQuery { get; set; }
Features
Pagination
Pagination is a basic feature so lets start with it.
It's all based on BasePagedRequest
.
Basic params are PageIndex
and PageSize
.
Index starts with 0.
It's enought to use the basic request from example and call any version of GetPaged
from IReFilterActions.
Pagination is applied as the last step, after setting up any filters and sorts to get the correct results.
new BasePagedRequest
{
PageIndex = 0,
PageSize = 10
}
Search
Search makes it possible to search over chosen string properties and fetch results. It's case insensitive and uses OperatorComparer.Contains
mechanism.
Setting up Search is quite easy and only requires setting an attribute over a property on database model:
[ReFilterProperty]
public string Address { get; set; }
This tells ReFilter that the property is available as a search parameter and it uses it in the query.
To trigger Search you need to pass the SearchQuery
value to ReFilter query and call any action from IReFilterActions.
Considerations and Limitations
Since V.1.1.0 it is possible to use Search values inside child entites and child collections.
ReFilter recognizes any object or array type marked with "UsedForSearchQuery" as another branch for going through Search value.
String search is expensive and going through a tree of entites searching for a string is very expensive.
Search itself is combined as an OR clause but is combined with every other feature as an AND clause.
Pending: Custom search provider in form of a Func<T, bool>.
Filtering
Filtering is achieved by using a Where
property, combined with PropertyFilterConfigs
, from BasePagedRequest
.
If PropertyFilterConfigs
is omitted then default filtering parameters are used: OperatorComparer.Contains
for string and OperatorComparer.Equals
for everything else.
Where is any arbitrary object you want.
Special case is RangeFilter
which filters out by provided range and falls back to basic property filtering.
All the filter options froem OperatorComparer
use built in ExpressionType
values such as: Contains, StartsWith, EndsWith, Equals, GreaterThan, LessThan, etc. .
The main thing to understand when using Filtering and Sorting is that you probably need a "container" object which mimics a part or the complete database model but has nullable properties in order to filter only by selected ones. That's because in most real life cases your database model has some required, non nullable, properties which then evaluate to default
values when the new object is created.
This makes it impossible to deduce which are the real filter values because null is the only "not set" value as such.
Example from test project:
class School
{
public int Id { get; set; }
public int IdRange { get; set; }
[ReFilterProperty]
public string Name { get; set; }
[ReFilterProperty]
public string Address { get; set; }
public Country Country { get; set; }
public List<string> Contacts { get; set; }
public List<Student> Students { get; set; }
public double Age { get; set; }
public DateTime FoundingDate { get; set; }
public DateOnly ValidOn { get; set; }
public bool IsActive { get; set; }
}
class SchoolFilterRequest : IReFilterRequest
{
public int? Id { get; set; }
public RangeFilter<int> IdRange { get; set; }
public string Name { get; set; }
public string Address { get; set; }
[ReFilterProperty(HasSpecialSort = true)]
public CountryFilterRequest Country { get; set; }
public List<string> Contacts { get; set; }
[ReFilterProperty(HasSpecialFilter = true)]
public List<Student> Students { get; set; }
public RangeFilter<double> Age { get; set; }
public RangeFilter<DateTime> FoundingDate { get; set; }
public RangeFilter<DateOnly> ValidOn { get; set; }
public bool? IsActive { get; set; }
}
Immediately Id is a problem. It will never be null. Any time you instantiate a School
object, Id will be 0 or any other value, meaning that it can't really be skipped when filtering.
That's why the FilterRequest version has every property as nullable. As such, any value it gets is used for filtering and there can be no doubt.
This is not a rule, but it's the most common case. If you need a property to always be defined you are free to set it up that way. If you don't need a nullable object for a filter, you don't need to set it up explicitly inside ReFilterConfigBuilder
, meaning your GetMatchingType
would fallback to default case.
Sorting
Sorting is a bit different from Filtering in a way that it only requires PropertyFilterConfigs
set up correctly.
To set them up correctly, the PFC needs to have a SortDirection
set and PropertyName
needs to match the case of the property name.
Sorting also supports for multiple sorts at the same time.
Filtering and Sorting Examples
Filtering and Sorting need to be setup in a central place inside your project.
For filtering that's your implementation of IReFilterConfigBuilder
and for sorting it's your implementation of IReSortConfigBuilder
.
Both serve the same purpose: matching nullable objects to database models and are nothing more than "controllers" for redirecting "requests".
Both have 2 methods: GetMatchingType
and GetMatching[Filter/Sort]Builder
.
They are intentionally separated since Sort and Filter don't need to have the same model.
Also, both implementations are <b>required</b> for ReFilter to work.
The minimum implementation (only filter is shown but filter and sort are mirrored) is as shown:
class ReFilterConfigBuilder : IReFilterConfigBuilder
{
public Type GetMatchingType<T>() where T : class, new()
{
switch (typeof(T))
{
default:
return typeof(T);
}
}
public IReFilterBuilder<T> GetMatchingFilterBuilder<T>() where T : class, new()
{
switch (typeof(T))
{
default:
return null;
}
}
}
Other than the basic sorting and filtering scenarios, there are also advanced ones using custom implementations provided by you.
The advanced custom scenarions are implemented via IRe[Filter/Sort]Builder
s. Also, these are the most complicated ones and provide all the options.
public interface IReFilterBuilder<T> where T : class, new()
{
/// <summary>
/// Gets all the custom filters and matches them to properties.
/// </summary>
/// <param name="filterRequest"></param>
/// <returns></returns>
IEnumerable<IReFilter<T>> GetFilters(IReFilterRequest filterRequest);
/// <summary>
/// Entry point of filter builder. Builds default query for that entity.
/// </summary>
/// <param name="filterRequest"></param>
/// <returns></returns>
IQueryable<T> BuildEntityQuery(IReFilterRequest filterRequest);
/// <summary>
/// First uses GetFilters and then applies them to the provided query.
/// </summary>
/// <param name="query"></param>
/// <param name="filterRequest"></param>
/// <returns></returns>
IQueryable<T> BuildFilteredQuery(IQueryable<T> query, IReFilterRequest filterRequest);
/// <summary>
/// Gets the list of Ids for the provided filter parameters in order to use it as an "IN ()" clause.
/// </summary>
/// <param name="filterRequest"></param>
/// <returns></returns>
List<int> GetForeignKeys(IReFilterRequest filterRequest);
}
A real life example of a UserFilterBuilder will be shown below.
We'll start with models and move upwards.
Entity Framework Code First is used as database provider for this solution. The models themselves don't need anything extra in this case (only Search attribute is used directly on EF models).
What it does require is the FilterRequest class that has a blueprint of what goes into the custom FilterBuilder.
This is the UserFilterRequest
. Notice that it inherits IReFilterRequest
and also defines special ReFilterProperty
attributes over some properties.
Those tell ReFilter how to handle them when they arrive via Where
property.
Once you're done with the configuration, you need to supply it with logic, in this case the UserFilterBuilder
which takes care of the general implementation for filtering the User.
For every property marked as HasSpecialFilter
you will need a dedicated filter which will be applied to the query. That's achieved via BuildFilteredQuery
and specific implementation of target filter showcased in RoleFilter
.
Finally, everything is connected inside ReFilterConfigBuilder
.
public class User : IdentityUser<Guid>
{
public DateTime DateCreated { get; set; }
public DateTime? DateModified { get; set; }
public DatabaseEntityStatus Status { get; set; }
public Person UserDetails { get; set; }
public List<UserRoles> UserRoles { get; set; }
public List<UserClaim> UserClaims { get; set; }
public List<UserRenewToken> UserRenewTokens { get; set; }
public List<CompanyUser> CompanyUsers { get; set; }
}
public class Person : DatabaseEntity<int>
{
[Required]
[StringLength(255)]
public string FirstName { get; set; }
[StringLength(255)]
public string MiddleName { get; set; }
[Required]
[StringLength(255)]
public string LastName { get; set; }
[StringLength(40)]
public string Mobile { get; set; }
[StringLength(40)]
public string Phone { get; set; }
[StringLength(40)]
public string Email { get; set; }
public Guid? UserId { get; set; }
public User User { get; set; }
}
public class UserFilterRequest : IReFilterRequest
{
public string Email { get; set; }
public string UserName { get; set; }
[ReFilterProperty(HasSpecialFilter = true)]
public int? CompanyId { get; set; }
[ReFilterProperty(HasSpecialFilter = true)]
public Guid? RoleId { get; set; }
[ReFilterProperty(HasSpecialFilter = true, HasSpecialSort = true)]
public string FirstName { get; set; }
[ReFilterProperty(HasSpecialFilter = true, HasSpecialSort = true)]
public string MiddleName { get; set; }
[ReFilterProperty(HasSpecialFilter = true, HasSpecialSort = true)]
public string LastName { get; set; }
[ReFilterProperty(HasSpecialFilter = true, HasSpecialSort = true)]
public string Mobile { get; set; }
[ReFilterProperty(HasSpecialFilter = true, HasSpecialSort = true)]
public string Phone { get; set; }
public bool? EmailConfirmed { get; set; }
[ReFilterProperty(HasSpecialFilter = true, HasSpecialSort = true)]
public bool? IsActive { get; set; }
[ReFilterProperty(HasSpecialFilter = true)]
public bool? IsSuperAdmin { get; set; }
}
internal class UserFilterBuilder : IReFilterBuilder<User>
{
private readonly IUnitOfWork unitOfWork;
private readonly ApplicationSettings appSettings;
// Since it's provided via DI, you can use any DI mechanism here
public UserFilterBuilder(IOptions<ApplicationSettings> appSettings, IUnitOfWork unitOfWork)
{
this.appSettings = appSettings.Value;
this.unitOfWork = unitOfWork;
}
public IQueryable<User> BuildEntityQuery(IReFilterRequest filterRequest)
{
var query = unitOfWork.GetDbSet<User>().AsQueryable();
query = BuildFilteredQuery(query, filterRequest);
return query;
}
public IQueryable<User> BuildFilteredQuery(IQueryable<User> query, IReFilterRequest filterRequest)
{
var filters = GetFilters(filterRequest).ToList();
filters.ForEach(filter =>
{
query = filter.FilterQuery(query);
});
return query;
}
public IEnumerable<IReFilter<User>> GetFilters(IReFilterRequest filterRequest)
{
List<IReFilter<User>> filters = new List<IReFilter<User>>();
if (filterRequest == null)
{
return filters;
}
var realFilter = (UserFilterRequest)filterRequest;
// Custom filter implementation for these properties
if (!string.IsNullOrWhiteSpace(realFilter.FirstName))
{
filters.Add(new FirstNameFilter(realFilter.FirstName));
}
if (!string.IsNullOrWhiteSpace(realFilter.LastName))
{
filters.Add(new LastNameFilter(realFilter.LastName));
}
if (!string.IsNullOrWhiteSpace(realFilter.MiddleName))
{
filters.Add(new MiddleNameFilter(realFilter.MiddleName));
}
if (!string.IsNullOrWhiteSpace(realFilter.Mobile))
{
filters.Add(new MobileFilter(realFilter.Mobile));
}
if (!string.IsNullOrWhiteSpace(realFilter.Phone))
{
filters.Add(new PhoneFilter(realFilter.Phone));
}
if (realFilter.RoleId.HasValue)
{
filters.Add(new RoleFilter(realFilter.RoleId.Value));
}
if (realFilter.IsActive.HasValue)
{
filters.Add(new IsActiveFilter(realFilter.IsActive.Value));
}
if (realFilter.CompanyId.HasValue)
{
filters.Add(new CompanyFilter(realFilter.CompanyId.Value));
}
if (realFilter.IsSuperAdmin.HasValue)
{
filters.Add(new SuperAdminFilter(realFilter.IsSuperAdmin.Value, appSettings.SuperAdminRole));
}
return filters;
}
public List<int> GetForeignKeys(IReFilterRequest filterRequest)
{
var query = unitOfWork.GetDbSet<Customer>().Where(u => u.Status == DatabaseEntityStatus.Active);
query = BuildFilteredQuery(query, filterRequest);
return query.Select(e => e.Id)
.Distinct()
.ToList();
}
}
internal class RoleFilter : IReFilter<User>
{
private readonly Guid roleId;
public RoleFilter(Guid roleId)
{
this.roleId = roleId;
}
public IQueryable<User> FilterQuery(IQueryable<User> query)
{
return query.Where(e => e.UserRoles.Any(ur => ur.RoleId == roleId));
}
}
class ReFilterConfigBuilder : IReFilterConfigBuilder
{
private readonly UserFilterBuilder userFilterBuilder;
public ReFilterConfigBuilder(UserFilterBuilder userFilterBuilder)
{
this.userFilterBuilder = userFilterBuilder;
}
public Type GetMatchingType<T>() where T : class, new()
{
return EntityTypeMatcher.GetEntityTypeConfig<T>().FilterRequestType ?? typeof(T);
}
public IReFilterBuilder<T> GetMatchingFilterBuilder<T>() where T : class, new()
{
return typeof(T) switch
{
Type user when user == typeof(User) => (IReFilterBuilder<T>)userFilterBuilder,
_ => null,
};
}
}
As a request you would use something like this:
{
'Where': {
'Email': 'user@email.com',
'UserName': 'username',
'FirstName': 'user',
'IsActive': true
}, // Supplies all the filter params to filter by
'PropertyFilterConfigs': [
{
'PropertyName': 'UserName',
'OperatorComparer': 1,
'SortDirection': 1
} // This way the request is configured to filter AND sort by UserName and apply Desc sort order and use 'StartsWith' built in filter mode
],
'SearchQuery': 'username' // This one wouldn't be applied since no property on User is marked as ReFilterProperty.UsedForSearchQuery = true
}
Finally, how it all looks inside the service:
// PagedResult is always the ReFilter result but you can return anything from your method
public async Task<ActionResponse<PagedResult<UserGridViewModel>>> GetPaged(BasePagedRequest request)
{
try
{
// Create any basic query you want to build upon
// NoTracking for speedy reading
var query = unitOfWork.GetDbSet<User>().AsQueryable().AsNoTracking();
// Transform PagedRequest to it's final form, supply it with mapping logic for end result (in that way you don't need to handle result transformation manually)
// Also, I encourage you to use ProjectTo by AutoMapper <3 becase it builds the select clause for just the stuff you need in your ViewModel
// If you compare it to Test project this really hides a lot of details under the hood and requires some advanced knowledge of AutoMapper <3 features
// However, it also provides the best QoL stuff
var pagedRequest = request.GetPagedRequest((IQueryable<User> x) => mapper.ProjectTo<UserGridViewModel>(x).ToList());
var pagedResult = await reFilterActions.GetPaged(query, pagedRequest);
return ActionResponse.Success(pagedResult);
}
catch (Exception ex)
{
var message = stringLocalizer.GetString(Resources.FetchError);
logger.LogError(message, ex, request);
return ActionResponse<PagedResult<UserGridViewModel>>.Error(Message: stringLocalizer.GetString(Resources.FetchError));
}
}
The only difference concearning sorting is the fact that it's configured using PropertyFilterConfig
. So, any config that is provided and has the SortDirection
set will be used to sort the query.
Again, it's necessary to configure UserSortBuilder
in order for everyting to apply correctly.
internal class UserSortBuilder : IReSortBuilder<User>
{
private readonly IUnitOfWork unitOfWork;
public UserSortBuilder(IUnitOfWork unitOfWork)
{
this.unitOfWork = unitOfWork;
}
public IQueryable<User> BuildEntityQuery()
{
return unitOfWork.GetDbSet<User>();
}
public IOrderedQueryable<User> BuildSortedQuery(IQueryable<User> query, PropertyFilterConfig propertyFilterConfig, bool isFirst = false)
{
var sorters = GetSorters(propertyFilterConfig);
if (sorters == null || sorters.Count == 0)
{
return (IOrderedQueryable<User>)query;
}
IOrderedQueryable<User> orderedQuery = (IOrderedQueryable<User>)query;
for (var i = 0; i < sorters.Count; i++)
{
orderedQuery = sorters[i].SortQuery(orderedQuery,
propertyFilterConfig.SortDirection.Value,
isFirst: (i == 0 && isFirst));
}
return orderedQuery;
}
public List<IReSort<User>> GetSorters(PropertyFilterConfig propertyFilterConfig)
{
List<IReSort<User>> sorters = new();
if (propertyFilterConfig != null)
{
if (propertyFilterConfig.PropertyName == nameof(UserFilterRequest.FirstName))
{
sorters.Add(new FirstNameSorter());
}
if (propertyFilterConfig.PropertyName == nameof(UserFilterRequest.MiddleName))
{
sorters.Add(new MiddleNameSorter());
}
if (propertyFilterConfig.PropertyName == nameof(UserFilterRequest.LastName))
{
sorters.Add(new LastNameSorter());
}
if (propertyFilterConfig.PropertyName == nameof(UserFilterRequest.Mobile))
{
sorters.Add(new MobileSorter());
}
if (propertyFilterConfig.PropertyName == nameof(UserFilterRequest.Phone))
{
sorters.Add(new PhoneSorter());
}
if (propertyFilterConfig.PropertyName == nameof(UserFilterRequest.IsActive))
{
sorters.Add(new IsActiveSorter());
}
}
return sorters;
}
}
internal class MiddleNameSorter : IReSort<User>
{
public IOrderedQueryable<User> SortQuery(IQueryable<User> query, SortDirection sortDirection = SortDirection.ASC, bool isFirst = true)
{
if (isFirst)
{
query = sortDirection == SortDirection.ASC ?
query.OrderBy(e => e.UserDetails.MiddleName) :
query.OrderByDescending(e => e.UserDetails.MiddleName);
}
else
{
query = sortDirection == SortDirection.ASC ?
((IOrderedQueryable<User>)query).ThenBy(e => e.UserDetails.MiddleName)
: ((IOrderedQueryable<User>)query).ThenByDescending(e => e.UserDetails.MiddleName);
}
return (IOrderedQueryable<User>)query;
}
}
Pending Features
Some pending features include recursive filtering over related entities along custom filtering and defining AND/OR clauses for specific filters.
Those would be breaking changes and it would take some time.
Additionally, check the Docs folder for examples of FE requests.
If needed, I can give you the AgGrid implementation as well.
Any kind of help or suggestions are welcome.
Thank you and I hope you enjoy using ReFilter
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net6.0 is compatible. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
-
net6.0
- LinqKit (>= 1.1.26)
- Microsoft.Extensions.DependencyInjection (>= 3.1.10)
- Newtonsoft.Json (>= 13.0.1)
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.
Version | Downloads | Last updated |
---|---|---|
2.0.1 | 147 | 1/14/2024 |
2.0.0 | 112 | 1/6/2024 |
1.2.0-alpha | 89 | 1/4/2024 |
1.1.1 | 500 | 5/9/2022 |
1.0.1 | 427 | 3/15/2022 |
1.0.0 | 402 | 3/6/2022 |
0.12.0-alpha | 179 | 3/3/2022 |
Initial version for filtering and pagination solution