Yotei.Tools.DynamicParser
0.5.6
dotnet add package Yotei.Tools.DynamicParser --version 0.5.6
NuGet\Install-Package Yotei.Tools.DynamicParser -Version 0.5.6
<PackageReference Include="Yotei.Tools.DynamicParser" Version="0.5.6" />
paket add Yotei.Tools.DynamicParser --version 0.5.6
#r "nuget: Yotei.Tools.DynamicParser, 0.5.6"
// Install Yotei.Tools.DynamicParser as a Cake Addin #addin nuget:?package=Yotei.Tools.DynamicParser&version=0.5.6 // Install Yotei.Tools.DynamicParser as a Cake Tool #tool nuget:?package=Yotei.Tools.DynamicParser&version=0.5.6
Yotei Dynamic Lambda Expression Parser
Yotei's Dynamic Parser is used to extract the logic of a DynamicLambda Expression (DLE) into a neutral and abstract format that doesn't depend on the capabilities of any underlying type, and that can be then used for a variety of purposes.
Introduction
Regular lambda expressions do not support using dynamic
arguments. Indeed,
if you try to do so the compiler will complain:
void Foo(Expression<Func<dynamic,...>> func);
In some scenarios this is a limitation. For instance, we may need to write some arbitrary logic without tying it to a concrete type. A canonical example is to write database code that refer to columns for which there are no corresponding properties in any type of the application code.
We often solved this situation by creating a multiplicity of DTO classes, or if this is not convenient (or sometimes not even possible), reverting to use plain strings with that code. But we all know that this approach has many disadvantages including, for instance, the possibility of writing dangerous code, or making it difficult to parse such code or to keep it in sync with what your favorite ORM is using.
Wouldn't it be nice to write something like the following, where the Id
name is not tied to any type, and then let that Where()
method parse the
expression into a neutral and abstract logic representation?
var result = database.Where(x => x.Id == "007");
DynamicParser to the rescue
And this is what the DynamicParser
class does. Its static Parse()
method
takes a single argument, a Func<dynamic, object>
dynamic lambda expression
that takes a single dynamic
argument, and emulates what otherwise the
compiler should have done by parsing that expression, and then capturing
the chain of dynamic operations binded against that dynamic
argument.
var parser = DynamicParser.Parse(x => ...);
var argument = parser.Argument;
var result = parser.Result;
The Parse
method returns an instance of DynamicParser
that contains two
main properties. The first one, Argument
, describes the dynamic
argument
used in the dynamic lambda expression. The second one, Result
, contains
the last node of the parsed chain of dynamic operations. It is an instance
of a class derived from the DynamicNode
type, and each class contains the
appropriate properties that permit your application to walk up that chain
as needed.
What is DynamicParser used for?
DynamicParser
is an infrastructure library. Its role is to provide the
ability of parsing arbitrarily complex dynamic lambda expressions into a
neutral and abstract logic representation (*).
This representation is based on classes that derive from the DynamicNode
type, and they are essentially POCO classes that can be easily manipulated
as needed.
(*): So, it does not provide the translation of that logic into any language
(such as your favorite SQL dialect). If you are interested in this use case,
please check the Yotei ORM
family of libraries.
Examples
Let's suppose we want to filter a set of results by checking if the values of an arbitrary field start with some letter or bigger. In our scenario, we have not an application-level type that contains that field, so we cannot write a Linq query or anything like that. We can easily write this logic in C# terms as follows:
var parser = DynamicParser.Parse(x => x.LastName >= "L");
var result = parser.Result;
But... wait a moment! We are comparing something whose type is unknown
against a string literal... which is something not tipically allowed by your
C# compiler. The magic here is that x
is a dynamic
object, and so, it
follows late bound rules. The compiler will delegate to the DLR (Dynamic
Language Runtime) the responsibility of dealing with that expression. When
parsed with DynamicParser
, its operations are then captured as
appropriate, without the need of any concrete type methods or members.
Often, we don't deal directly with DynamicParser
instances, nor with
their DynamicNode
results. As said before, this is an infrastructure
library that let you obtain a representation of that logic, that you can
later use as needed - for instance, translating it into an appropriate
SQL dialect.
DynamicParser
can parse much more complex dynamic lambda expressions.
Some examples are:
x => x.Id = x.Id + "_Whatever";
x => x.Indexed[x.Index + x.Beta];
x => x[7, "other"] = null!;
x => x(x.Argument);
x => x.MyMethod(x.Alpha, null, "other");
x => x(x.Alpha = x.Beta)(x.Beta = x.Alpha);
x => x.Alpha<int, string>(x.Beta);
x => x.Alpha(x.Alpha = x.Alpha(x.Beta = x.Alpha)[x.Alpha = x.Beta]);
x => x.Alpha == (x.Alpha = x.Beta);
x => x.Alpha && x.Beta;
x => x.Alpha += x.Beta;
x => x.Alpha = (string)x.Beta;
How it works
In a nutshells, by executing the dynamic lambda expression and intercepting
each of the dynamic operations, translating them into the appropriate
DynamicNode
instances.
DynamicNode
inherits from DynamicObject
, which provides the ability of
binding dynamic operations by overriding their associated methods. But this
approach only works well for a subset of operations (for the purposes of
DynamicParser
). So, because DynamicObject
implements the
IDynamicMetaObjectProvider
interface, DynamicNode
instances override its
GetMetaObject(...)
method to generate ad-hoc DynamicMetaObject
instances
that are the ones that, ultimately, bind the most complex dynamic
operations.
Now, for performance reasons, the DLR uses a sophisticated cache mechanism,
which is based on the type of the call site, and on the type of its
arguments. And this is unfortunate because DynamicParser
needs to produce
brand new results each time it parses a dynamic lambda expression, or any
of its intermediate results.
So, to prevent recycling previous results, each DynamicNode
is associated
with an internal version number. Then DynamicParser
forces the DLR to
invalidate the cached entries by using an ad-hoc BindingRestrictions
instance that validates the internal version and, if needed, updates it to
a new one.
Easy to say, hard to achieve because there are three special cases.
The first one relates to member set operations, as in 'x => x.Id = ...'
where, by default, the DLR 'forgets' the right side of the expression,
losing that part of the logic. To prevent this, set operations implement
a LastNode
hack whose value is checked and used if appropriate.
The second one relates to convert (or cast) operations, as in
'x => (string)x.Id'
. The thing here is that the DLR expects and validates
receiving an actual instance of that type, not a DynamicNode
one,
and so it would fail throwing an exception. To deal with this scenario,
DynamicParser
produces an intermediate result of the appropriate type,
boxed if necessary, a surrogate that is used as the key of a dictionary of
converted results. Then, when that surrogate result is later used, it is
substituted by the appropriate DynamicNodeConvert
instance.
And, finally, the third one relates to nested indexed and invoke operations,
as in 'x => x[x[x[...]]]'
, 'x => x(x(x(...)))'
, and their many
variations with and without members and methods. By default, the DLR would
return a simplified version of that chain of operations, which is not valid
for our purposes. DynamicParser
solves these scenarios thanks to both the
invalidation mechanism described above, and to the way that the
responsibility of binding dynamic operations is splited between the
DynamicObject
overrides, and the DynamicMetaObject
ones.
Considerations
Parsing dynamic lambda expressions is not fast. Firstly, no surprises,
the DLR itself is slower than regular C# code. And, secondly, the
DynamicParser
invalidation, last node, and surrogate mechanisms also
carry their own load.
The time required to parse a dynamic lambda expression increases as the complexity of that expression increases. This is often not a problem because, firstly, these expressions tend to be simple, and secondly, because this time is typically much less than the ones required by other elements (such as network latencies in accessing databases). In any case, you should be aware of it because of its potential performance impact.
Be careful if you want to cache DynamicParser
results. As explained
above, it works by actually executing the dynamic lambda expressions. So,
if any part of the expression invokes an external method, then the value
obtained is the one at the moment when the expression is parsed/executed.
In layman terms, if you try to cache parameterized expressions, as for
instance database ones, it won't work by default. The value of these
parameters will always be the one captured at the time when the expression
was parsed. Remember that DynamicNode
instance are immutable ones.
Having said that, it is not difficult to implement a visitor-alike solution
that rewrites the appropriate parts of a DynamicNode
results' chain. This
is actually part of the backlog for future versions.
Limitations
Coalesce Operations Do Not Work. Dynamic Lambda Expressions such as
'x => x.Alpha ?? x.Beta
' resolves straight into the left operand,
x.Alpha
in the example.
The interesting bit here is that the dynamic operation invoked when the dynamic lambda expression is executed is just a get member operation, but it is not then followed by a binary or comparison one.
The overriden DLR-related methods in the library are not even invoked.
Ternary Conditional Operations Do Not Work. Dynamic Lambda Expressions
such as 'x => x.Alpha ? x.Beta : x.Delta
' resolves into the last operand:
x.Delta
in the example.
Remember that DynamicParser
works by executing the Dynamic Lambda.
In this case, the method in charge of intercepting unary operations is
invoked to parse what essencially is a 'IsTrue(x.Alpha)
' operation.
DynamicParser
returns a false
value because the DLR is expecting
a boolean value as the result (we could have chosen true
as well).
But the net result is that the DLR chooses the second branch, hence
returning the last operand.
Note that the ternary conditional operation is essentially a compiler
trick, not a fundamental operator (in the sense of implementation).
DynamicParser
has currently no way of understanding this is ternary
conditional operation instead of an unary one, and so no way of invoking
it twice (one for the false branch, and then another for the true one).
Testing
To validate that the DLR rules are not reused, you will notice the tests contain the same member and method names over and over again. This is not a mistake but, rather, is done on purpose so that we validate we are preventing the DLR to reuse previous rules and intermediate results.
I have also noticed we shall also take in consideration what test engine we are using (in my case, Xunit). I've found that, when instructing Xunit to run tests in parallel, each is run in a kind-of private DLR context that don't reuse previous DLR rules, and passing all the test cases with no invalidations invoked. . As our intention is precissely the opposite (having rules to invalidate), it has forced me to write a console application that finds and executes these test methods in sequence.
History
The family of Yotei
libraries inherit from the previous work done for
the Kerosene
one, which still can be found in Code Project and
other sources.
[0.5.6]: Current version of the package. Includes support for dynamic conversion operations.
[0.2.0]: Almost a straight port from the Kerosene
version, but using
the newest .NET version.
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net7.0 is compatible. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
-
net7.0
- Yotei.Tools (>= 0.5.12)
- Yotei.Tools.Diagnostics (>= 0.5.12)
NuGet packages (2)
Showing the top 2 NuGet packages that depend on Yotei.Tools.DynamicParser:
Package | Downloads |
---|---|
Yotei.ORM
Yotei ORM |
|
Yotei.ORM.Relational
Yotei ORM Relational Engines |
GitHub repositories
This package is not used by any popular GitHub repositories.
Version | Downloads | Last updated |
---|