Thursday, January 26, 2012

Design a Business Expression with Interpreter Pattern


In enterprise applications, we sometimes need to provide users a flexible way to configure the application. In this article, I will demonstrate how to design a general purpose business expression with the Interpreter pattern for this requirement.

Business Expression

An expression is a finite combination of symbols that is well-formed according to rules that depend on the context. The business expression we want to provide to the user has the following grammar definition:
Business Expression ::= ‘{‘ Operator ‘@’ Operand { ‘;’ … } ‘}’
Operator ::= ‘ADD’ | ‘SUB’ | ‘MUL’ | ‘DIV’
Operand ::= Business Expression | Data
Data ::= ‘1’ | ‘2’ …
There are several business expression examples:
This business expression represents we want to get the result of 1 + 2.
This business expression represents we want to get the result of 3 – 2.
This business expression represents we want to get the result of 2 * 5.
This business expression represents we want to get the result of 4 / 2.
This is a more complicated business expression. It equals 2 * ((5 + 3) / (5 – 1)).

Applying the Interpreter Pattern

The definition of the Interpreter pattern is this: given a language, define a representation for its grammar along with an interpreter that uses the representation to interpret sentences in the language.
Interpreter Pattern
From the definition of the Interpreter pattern, we can see a business expression is a perfect candidate to apply the Interpreter pattern. We can define each part of the business expression as a business expression class that is used to represent the business expression grammar and put them together as a business expression hierarchy.
Business Expression
In the business expression hierarchy, we have a BusinessExpression class similar to the AbstractExpression in the pattern, have BusinessDataExpression similar to the TerminalExpression in the pattern, and haveBusinessCompositeExpression together with all its subclasses similar to the NonterminalExpression in the pattern. Therefore, client routing can use the business expression to evaluate a given business expression string.
Where is the Context class?
The Interpreter pattern diagram has a context class. It’s not in my business expression diagram. The reason that I didn’t include a context class is because the purpose of the context class in the Interpreter pattern is to contain the input string and the result. Therefore, it’s not a must have class. Without the context class, I can just pass the input string into a business expression with an input parameter and get the result back with a return value.

How to use a business expression

To use a business expression, we need to follow these steps:
  1. Translate the business expression string into a business expression object tree.
  2. Evaluate the business expression object tree to get the result.
Translating into a business expression object tree
The business expression is given as a string, such as {ADD@1;2}. The way to translate this string into a business expression object tree is by traversing through each character in the string and translating any recognized term into the corresponding business expression object. The Parse static method on the BusinessExpression class is designed for translating a business expression string into a business expression object tree. It uses a process stack internally to store the intermediate results during the process. An intermediate result is represented with theProcessAtom class.
Process Atom
There are three types of ProcessAtom: Raw Data, Incomplete Business Expression, and Completed Business Expression.
Raw Data: It’s a single character in a business expression string. When the processor can’t determine how to convert a single character into a business expression, it will just store the character as raw data and wait for more information.
Incomplete Business Expression: It’s a business expression that has not been processed completely. A business expression starts with the symbol ‘{‘ and ends with ‘}’. Before the ending symbol is reached, the processor will just store an incomplete business expression into the process stack and wait for the ending symbol.
Completed Business Expression: It’s a business expression that has been fully processed. A completed business expression is generated by the processor after processing both ‘{‘ and ‘}’.
Evaluating business expression
After we have the business expression object tree, we can call the evaluate method of the root business expression object to get the result of business expression string. Each business expression object knows how to evaluate itself or get the result from containing business expression. The root business expression object consolidates the result together and returns back to the client.

Related Design Patterns

The business expression is a perfect candidate to apply the Interpreter pattern, however, the Interpreter pattern can’t fulfill the requirement by itself, there are other patterns employed.
Composite Pattern
The business expression object tree is built with the Composite pattern, so we can have nested business expressions in a business expression. By doing this, a more complicated business expression can be defined.
Factory Pattern
The Parse method has a fixed algorithm on how the business expression string gets parsed into a business expression object tree, but we might want to add a new business expression sometimes to support a new operation. According to the single responsibility principle, we should not let the parser method directly create the business expression object. Instead, we want the business expression object creation be done in a factory class. Therefore, we can just add a new business class and change BusinessExpressionFactory to return the new business expression object into the parser.
public class BusinessExpressionFactory
    public BusinessExpressionFactory()

    public BusinessExpression CreateBusinessExpression(string type)
        BusinessExpression businessExpression = null;

        switch (type.ToLower())
            case "add":
                businessExpression = new BusinessAdditionExpression();
            case "sub":
                businessExpression = new BusinessSubstractionExpression();
            case "mul":
                businessExpression = new BusinessMultiplicationExpression();
            case "div":
                businessExpression = new BusinessDivisionExpression();

        return businessExpression;


A business expression is a typical requirement for enterprise applications. By utilizing the Interpreter pattern, we can provide an elegant solution to fulfill it.

Using the Code

The code is developed in Visual Studio 2010. You can add your own business expression class into it to see the clarity and extensibility in the example from using Design Patterns.

Saturday, January 14, 2012

Comparing Transparent Lazy Loading between NHibernate and Entity Framework


When I was trying Entity Framework after many years with NHibernate, I assume the Entity Framework has same transparent lazy loading behavior like NHibernate. But the assumption is not completely right. There are some minor behavior differences in Entity Framework. In this article, I will explain what lazy loading is, how NHibernate and Entity Framework implement it, and the same and differences.

What is Lazy Loading

Quote from Wikipedia:
Lazy loading is a design pattern commonly used in computer programming to defer initialization of an object until the point at which it is needed. It can contribute to efficiency in the program’s operation if properly and appropriately used. The opposite of lazy loading is Eager Loading.

When there is a domain model in application, entity in domain model is used to represent business entity in problem domain. Association between entities is equivalent to the relationship or interaction of business entities. The beauty of domain model is it can be used to mimic problem domain as much as we can. However, domain object in memory is volatility. Hence, we need tables in database to sustaining domain object. Domain object needs to be saved to database or loaded back into memory from database. The code for loading domain object from database is not trivial. Sometimes, this kind of code spends developer most time in application development. On the other hand, the code for loading domain object is very similar for any domain entity. Therefore, ORM framework is introduced to provide unified data access code. ORM framework relieves developer from doing nitty-gritty database access code and let them focus on more important business logic code. By using ORM framework, domain object is loaded or saved with developer friendly API.
Domain object in domain model is not independent usually. In most situations, multiple domain objects are linked with each other as a big domain object graph. When application got a particular domain object from database, we need to also load objects of related domain entities. Lazy loading is invented to load related domain objects on-demand. By doing this, developer will directly work with domain object and use it naturally without worry how the related domain objects are loaded. The easiest way to support lazy loading is to have customized code in properties and methods to initialize related domain object when it is going to be used. To add customized lazy loading code is a lot of work, time consuming, and error-prone. Fortunately, this kind of cross-cutting code can be injected by AOP framework, such as Windsor Castle AOP framework. This is called transparent lazy loading. The precondition for using transparent lazy loading is to make any property or method lazy loading friendly. This means marking property or method as virtual in C#. The benefit of lazy loading is deferring query executing to the moment application really need it and reduce application memory usage.

How transparent lazy loading works in NHibernate

NHiberenate uses Windsor Castle AOP framework to implement lazy loading. Let’s take a look how to get lazy loading works in NHibernate:
1. Make class properties and methods accessible and overload-able
The lazy loading can only be added for public or protect virtual property or method. Hence, if your class properties and methods are private, you can’t use lazy loading. Also lazy loading is not available on class fields
public class Employee
    public virtual int EmployeeID { get; set; }
    public virtual string Name { get; set; }
    public virtual Iesi.Collections.Generic.ISet<Task> Tasks { get; set; }

2. Turn on lazy loading in HBM mapping file
The lazy loading attribute is default to true, though, you can still explicitly specify it in HBM mapping file.

<?xml version="1.0" encoding="utf-8" ?>
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2" assembly="Demo" namespace="Demo">
  <class name="Employee">
    <id name="EmployeeID">
      <generator class="identity" />
    <property name="Name" />
    <set name="Tasks" lazy="true" inverse="true" cascade="all-delete-orphan">
      <key column="EmployeeID"/>
      <one-to-many class="Demo.Task"/>

3. Return domain object from session
The domain object must be returned back from session, such as using session.Get method with domain entity id or using session.CreateQuery with HQL query
var employee = session.Get<Employee>(1);
var employee = session.CreateQuery("from Employee as e where e.EmployeeID = 1").List<Employee>().FirstOrDefault();
If the domain object is attached into NHibernate session, you will not have lazy loading on it.
Internally, NHibernate uses Windsor Castle AOP framework to implement lazy loading. Windsor Castle AOP framework subclasses domain class and overrides all public and protect virtual properties and methods to provide logical for loading related domain object during the first time access. The subclass is named as <Class>Proxy.
Task Object in NHibernate
The session.Get method is the place letting NHibernate create an instance of subclass to replace the real domain object. The hereafter code using the created object is same to the proxy object and real domain object. Lazy loading process is transparent to developer, if you want to know whether the object is loaded, you can use NHibernateUtil.IsInitialized to verify it.

If you don’t want to have lazy loading enabled because N+1 performance issue or you knew you will use all related objects in your application, you can turn the lazy loading off by set lazy attribute to false.
<set name="Tasks" lazy="false" inverse="true" cascade="all-delete-orphan">
  <key column="EmployeeID"/>
  <one-to-many class="Demo.Task"/>
By set lazy attribute to false, NHibernate will do implicate eager loading for you.

How transparent lazy loading works in Entity Framework

The earlier version Entity Framework doesn’t have transparent lazy loading feature. This is introduced in version 4.0. Because there is no transparent lazy loading design upfront, this feature in Entity Framework is designed differently to backward compatible with Entity Framework older version. If you have POCO domain classes in your application, you are required to do following to have lazy loading enabled in Entity Framework:
1. Make class properties and methods override-able and accessible
This step is similar to NHibernate, Entity Framework uses subclass to inject additional code into domain class for lazy loading.
Note: this only applies to POCO style domain class, if your domain classes are generated with EntityModelCodeGenerator template, it should already contains explicit lazy loading code, so no subclass needed. Below is code snippet generated with EntityModelCodeGenerator template.
[EdmRelationshipNavigationPropertyAttribute("LazyLoadingModel", "FK_Task_Employee", "Task")]
public EntityCollection<Task> Tasks
        return ((IEntityWithRelationships)this).RelationshipManager.GetRelatedCollection<task>("LazyLoadingModel.FK_Task_Employee", "Task");
        if ((value != null))
            ((IEntityWithRelationships)this).RelationshipManager.InitializeRelatedCollection<task>("LazyLoadingModel.FK_Task_Employee", "Task", value);
2. Specify true for “Lazy Loading Enabled” in edmx properties
You can actually specify Lazy Loading Enabled in your context class directly. However, the default place is in edmx file. The Lazy Loading Enabled setting will eventually get into your context class if it’s automatically generated, like LazyLoadingEntities in my example.
public partial class LazyLoadingEntities : ObjectContext
    public const string ConnectionString = "name=LazyLoadingEntities";
    public const string ContainerName = "LazyLoadingEntities";

    #region Constructors

    public LazyLoadingEntities()
        : base(ConnectionString, ContainerName)
        this.ContextOptions.LazyLoadingEnabled = true;

    public LazyLoadingEntities(string connectionString)
        : base(connectionString, ContainerName)
        this.ContextOptions.LazyLoadingEnabled = true;

    public LazyLoadingEntities(EntityConnection connection)
        : base(connection, ContainerName)
        this.ContextOptions.LazyLoadingEnabled = true;


    #region ObjectSet Properties

    public ObjectSet<Employee> Employees
        get { return _employees  ?? (_employees = CreateObjectSet<employee>("Employees")); }
    private ObjectSet<employee> _employees;

    public ObjectSet<task> Tasks
        get { return _tasks  ?? (_tasks = CreateObjectSet<task>("Tasks")); }
    private ObjectSet<task> _tasks;

3. Return domain object from context
Domain object needs to be lazy loaded must be returned from context
A lazy loaded object looks like this in Visual Studio 2010 watch window
Task Object in Entity Framework
If you don’t want to have lazy loading in Entity Framework, you can turn it off by set LazyLoadingEnabled to false 
this.ContextOptions.LazyLoadingEnabled = false;
After the lazy loading is turned off, Entity Framework will not load related objects for you anymore. If you access a navigation property that is no loaded, you will get a NullReferenceException error. Or, when you access a navigation collection property that is not loaded, you will get an empty collection. This confuses me when I first use Entity Framework after years with NHibernate.
Task Object in Entity Framework with Lazy Loading False

Caveat on Using Lazy Loading

In both NHibernate and Entity Framework, lazy loading is enabled by default. This is the best practice for most situations. By enabling lazy loading, developer doesn’t need to worry how related objects are loaded. Also, untouched reference objects or collections do not occupy memory space. However, we need to be careful on N+1 problem incurred by lazy loading in some situations. The N+1 problem is when you have lazy loading enabled for a domain object that has a collection property contains N items, you will have one select query be executed to retrieve data for the domain object itself, and then you will have every single query be executed for each item in the collection. In total, you executed N+1 queries to get all data in the domain object and collection property instead of just one query. This causes too many queries to database. To avoid N+1 problem, we need to use profiler monitor execution of application to find the potential spot that has this problem, and then use explicit eager loading or batch loading to avoid or mitigate this problem.

Using the Code

The code is developed in Visual Studio 2010. You need to create database LazyLoading in SQL Server, and then run the attached CreateTables.sql script to create tables and data in it. Before running the code, remember to update connection string in configuration file to your local connection string.

Save Temporary Data with Entity Framework Self-Tracking Entity


In enterprise application, one situation faced frequently by developer is how to design temporary data saving feature. The temporary data saving feature will reduce the chance of data lost caused by session timeout, computer down, or network issue. Also, temporary data saving feature allow certain business process last for several business days without locking the existing data or slow down other business processes. There are many ways to design temporary data saving, such as saving the data into business table with status flag or saving the data into mirror table of your business table. In this article, I will show you how to use Entity Framework Self-Tracking Entity (acronym STE) to handle this situation that will not just keep the cleanness of your data model (database tables), also give enough extensibility for handling more temporary data saving request.

Temporary Data Saving

In a normal enterprise application, when there are a lot of data need to be entered into system at a time or the entered data must be verified and approved before seen by other users, user could face data lost problem or draft data issue. Thinking about this, if user typed data for several hours in a web based application, then hit submit button try to save the data, they will sometimes get “Session Timeout” instead of “Save Success”. How frustrated user will be? Also, thinking about this, when new customer data is entered into system by business analyst, before the customer data gets verified and approved by business department manager, other department user can already see and use this customer to do business. How serious consequence could be caused and how big the mess needs to be clean up later on? To resolve above mentioned issues, we can provide user a way to save data temporarily. The temporary saved data is only seeable by data input person or certain other people, this give data input person the ability to periodically save the work without wait to the end. Also prevent other department users from seeing and using the draft data.

Entity Framework Self-Tracking Entity

Before applying domain driven design, all the business data is processed with RecordSet in enterprise application, therefore, the most common solution for temporary data saving are either have status flag in business table to indicate the business data is not finalized or have a mirror table of business table to store temporary data. The drawback of both solutions are very cumbersome and requiring a lot of changes to existing application. Also, those solutions will result in big amount of additional data or tables that mess up the entire application. After applying domain driven design, we design application to have a centralized domain layer that uses domain entity to story business data. We can serialize the domain entity and save into a temporary location. How to support domain entity serialization and deserialization? We can design domain entity as Entity Framework Self-Tracking Entity. Entity Framework is an ORM framework from Microsoft. It is used to handle paradigm mismatch between domain model and data model. By using it, we will have an elegance domain model without worry about how to persistent it into database. There are many ways to design domain model. Such as POCO style domain model. The POCO style domain model is a simple .NET class that can be directly handled by ORM framework without any ORM required properties or methods. The cleanness and simplicity attracts many enterprise application developers to adopt it in their application. However, it needs more work on supporting change tracking and serialization, especially XML serialization. To have class supports change tracking and serialization, Entity Framework comes up with the Self-Tracking Entity idea. Strictly speaking, the Self-Tracking Entity is not pure POCO style domain entity because there are many properties and methods are added in domain entity to support self-tracking and serialization. But, because all the self-tracking related properties and methods and serialization/deserialization supporting are added at source code level. It is not tight with Entity Framework, so you should still be able to reuse the domain model with other ORM framework, such as NHibernate. The design purpose of Self-Tracking Entity is to allow domain entity to be serialized into XML in service application, and send over web service, and deserialize back in web service client application. In here, I am not really to use it this way. I will use the serializable ability of Self-Tracking Entity to help on temporary data saving.

The Idea

The idea is to use Self-Tracking Entity as domain entity, and have a table to store serialized XML of domain entity object. By doing this, we have following benefits:
1. Temporary data is completely separated with finalized data.
2. Easy to implement. Only one table need to be added and no change to any current business tables or codes.
3. Extensible. Add temporary data saving support for additional domain entity will not cause change to any existing tables or codes.

The Implementation

A simple ASP.NET MVC application is used here to demonstrate how to implement this idea. The demo application is designed as employee information management application. You can add, update, or delete employee.
Data Model
There are only two tables in demo application, one is business table Employee and another one is temporary data storage table TempData.
ER Diagram
The Data field in TempData table is used to store serialized business object XML. Obviously, the DataType is XML.
Visual Studio Solution Structure
There are four projects in demo application, Demo.Base, Demo.Data, Demo.Models, and Demo.Web.
Demo.Base: The common library of the demo application. It contains DataHelper to handle serialization and deserialization, and interface for dependency injection.
Demo.Data: The data access layer. It implements repository pattern to wrap up Entity Framework.
Demo.Models: The domain layer. It contains self-tracking entity style domain entities and data access interfaces.
Demo.Web: The web user interface layer. It’s built with ASP.NET MVC that has good separation of concerns and test friendly structure.
Entity Class and Context Class
The most important classes supporting temporary data saving reside in Demo.Data project and Demo.Models project. Let’s take a look how to generate them. First, we need to add EntityDataModel.edmx from data model (database) with Entity Framework Entity Data Model wizard.
Entity Data Model

By select “Add Code Generation Item…” in Entity Data Model context menu, you can add an ADO.NET Self-Tracking Entity Generator template into the project.
Add Code Generation Item

Give the name
Self-Tracking Entity Generator

After clicked Add button, you will see there are two files added in project, and The is the T4 template file of your domain entity classes and the is the T4 template file of Entity Framework context classes. Because Demo.Data project is data access layer, so need to move file into Demo.Models project. After moved it, a small change needed in file to point input file, EntityDataModel.edmx, to the original place.
Update Input File

At the same time, need to be updated to have the right Custom Tool Namespace, so the context classes can still reference to domain entity classes.

If you look at the generated Employee class, you can see there are some extra properties and methods in addition to those properties from Employee table. Those extra properties and methods added by Self-Tracking Entity template are for tracking change and serialization/deserialization purpose.
Employee Class

Also, the class has DataContract attribute on class and DataMember attribute on property to support serialization and deserialization.
Employee Class Code
Saving Temporary Data
The Demo.Web project uses Self-Tracking Entity classes and Context classes together to achieve the saving temporary data goal.
From the CreatTempData action method in Employee controller, we can see how we save the new created Employee object as temporary data
1. New an Employee object with user entered data.
2. Serialize the new Employee object to XML string.
3. Clear context.
Note: because we are using Unit-of-Work pattern in here to handle database operation, so we need to call Context.Clear() to close the current Unit-of-Work, therefore the following Context.Commit will be in a new Unit-of-Work. This will prevent objects other than TempData from be committed into database.
4. New a TempData object and store XML of the new Employee object into Data property. 5. Call Create method of repository to attach the new TempData object with context. 6. Commit the new created TempData object into database.

CreateTempData action method code snippet
        [AcceptParameter(Name="button", Value="Create Temp Data")]
        public ActionResult CreateTempData(EmployeeVM employeeVM)
            IContext context = Demo.Models.Registry.Context;
            ITempDataRepository tempDataRepository = Demo.Models.Registry.RepositoryFactory.GetTempDataRepository();

            if (ModelState.IsValid)
                // New an Employee oject with user entered data
                Employee employee = new Employee();
                employee.Name = employeeVM.Name;
                employee.Title = employeeVM.Title;
                employee.Address = employeeVM.Address;

                // Serialize the new Employee object to XML string
                string xml = DataHelper.SerializeEntity<Employee>(employee);

                // Clear context

                // New a TempData object and store XML of the new Employee object into Data property
                TempData tempData = new TempData();
                tempData.Data = xml;
                tempData.LastUpdated = DateTime.Now;

                // Call Create method of repository to attach the new TempData object with context

                // Commit the new created TempData object into database

                return RedirectToAction("Index", "Home");

From the SaveEmployee action method of Employee controller, we can see how to save temporary data into real business table.
1. Find the TempData object from database.
2. Deserialize the value of data property back to Employee object.
3. Attach Employee object into context.
4. Update the Employee object with the latest data entered by user.
5. Delete TempData object from context.
6. Commit the Employee object into database and delete TempData object from database.

SaveEmployee action method code snippet
        [AcceptParameter(Name = "button", Value = "Save Employee")]
        public ActionResult SaveEmployee(EmployeeVM employeeVM)
            IContext context = Demo.Models.Registry.Context;
            ITempDataRepository tempDataRepository = Demo.Models.Registry.RepositoryFactory.GetTempDataRepository();
            IEmployeeRepository employeeRepository = Demo.Models.Registry.RepositoryFactory.GetEmployeeRepository();

            if (ModelState.IsValid)
                // Find the TempData object from database
                TempData tempData = tempDataRepository.Find(td => td.TempDataID == employeeVM.LinkTempDataID);

                // Deserialize the value of data property back into Employee object
                Employee employee = DataHelper.DeserializeEntity<Employee>(tempData.Data);
                // Attach the Employee object into context
                // Update the Employee object with the latest data entered by user
                employee.Name = employeeVM.Name;
                employee.Title = employeeVM.Title;
                employee.Address = employeeVM.Address;

                // Delete TempData object from context

                // Commit the Employee object into database and delete TempData object from database

                return RedirectToAction("Index", "Home");

            return View(employeeVM);

Following is the sample Employee object XML stored in TempData table
<Employee xmlns="" xmlns:i="" xmlns:z="" z:Id="i1">
  <Address>Main Street, Edison, NJ</Address>
  <ChangeTracker z:Id="i2">
    <ExtendedProperties />
    <ObjectsAddedToCollectionProperties />
    <ObjectsRemovedFromCollectionProperties />
    <OriginalValues />


With Entity Framework Self-Tracking Entity, we can easily implement temporary data saving feature. This solution is simple, elegance, and extensible, also it has very minor impact on existing application. If you really don’t want to use Entity Framework Self-Tracking Entity, you can have POCO style domain entity and use DTO as the media to serialize and deserialize domain entity object to and from XML.

Using the Code

The code is developed in Visual Studio 2010. You need to firstly create database, SaveTempDataWithSTE, in SQL Server, and then run the attached SaveTempDataWithSTE.sql script to create Employee table and TempData in it. Before running the code, remember to update connection string in configuration file to your local connection string.