We are asked occasionally whether DevForce supports second level caching, that is, does DevForce have some means on the server of remembering previously queried entities between client requests.

This is rarely a real problem in a DevForce application because (a) DevForce applications are usually rich client applications and (b) the DevForce EntityManager cache typically delivers the performance benefits folks seek from a server side cache.

My response, although grounded in long experience, is not always persuasive. Second level caching should improve scalability in theory and theory often trumps reality.

In this post I discuss how to tell if you would benefit from server-side caching and how you might be able to use an Entity Framework second level cache to achieve it.

I haven't tried to install an Entity Framework second level cache. In this post I provide links to information about how to implement second level caching. The links come from reliable sources and it looks like this kind of caching should work. If you try it, please let me know how it goes. I haven’t tried it myself because I find that, for almost all of our customers, this is a solution looking for a problem. But if it makes sense for your application and you give it a try, I'm counting on you to get back to me with your results… and maybe contribute some guidance and code to help others.

Perceived Problem

You have a great many users who repeatedly query for the same entities. Those entities hardly ever change but for some reason they keep asking for them and for some reason you can’t cache them on the client.

You’ve measured and these queries account for a significant percentage of database hits. Moreover, they’re really bogging the database down. You’ve determined conclusively, after careful study of production traffic, that these repetitive database queries are choking your database. You’re pretty sure that server-side caching would provide significant relief.

Are you sure?

Honestly, I don’t think this happens often … which is why you should have the measurements that prove poor performance is traceable to this cause. Don’t guess that this is the problem. Don’t forecast that it is going to be a problem. You need proof.

The interest in second level caches arises most often among people who are evaluating DevForce and haven’t yet built an application with it. Such inquiries are typically speculative. Trust me, you can waste a lot of time investigating something that isn’t going to make any real difference in your application. It might make matters worse.

But suppose you’ve demonstrated that this is a real problem in your working application. You’ve established that the client app can’t cache these entities locally for some reason (perhaps it’s a web client) … which may be why you’re looking into caching on the server.

Maybe it really is time to consider EF “Second Level Caching”

You could try caching query results in a Query Interceptor. But that will require code you must write and if an EntityServer (aka, BOS) is involved you’ll have to make it thread safe. Consider EF “Second Level Caching” before rolling your own.

If there’s a 2nd, there must be a 1st

Time for some definitions. The “first level cache” is the local cache of entities retrieved by some persistence manager. The DevForce EntityManager is a first level cache on the client. EF’s ObjectContext is a first level cache on the server.

When writing with EF Code First, you create a DbContext which is wrapper around an ObjectContext. You can think of your DbContext as a first level cache if you wish.

On the EntityServer (aka, the BOS) DevForce creates a new ObjectContext for each client request.

This EntityServer is in-process in a 2-tier deployment.

These first level caches do a great job of holding frequently requested entities. But they (and their entities) disappear when the EntityManager or ObjectContext disappear. The EntityManager on the client can live a long time, the life of the user session perhaps. The ObjectContext, on the other hand, evaporates after each client request.

If you had a “second level cache” it would sit outside of the EF ObjectContext. It would outlive the ObjectContext and would be shared by multiple instances of ObjectContexts. When your query reaches a new, empty ObjectContext, EF makes a request to the database. A second level cache could intercept that database request and satisfy it with previously retrieved results.

That sounds like the perfect resolution to your problem. If you had a second level cache, it could hold query results for the entities that clients are clamoring for … and the database pressure would be reduced.

Unfortunately EF doesn’t have an out-of-the-box second level cache.

From time to time you’ll hear someone argue that NHibernate is better than EF because NHibernate does have a second level cache. Often the person making this argument (a) is unaware of the limitations of a second level cache, (b) doesn’t realize that DevForce client-side caching usually eliminates the need for a second level cache and (c) has no evidence that the application would benefit from a second level cache. There’s the whiff of FUD in the air.

Let’s deodorize. As it happens, there is an open source second level cache for EF!

In brief, it’s a plug-in that intercepts EF requests to the EF “Store Provider” (the component that turns EF store queries into SQL queries on a database, be it SQL Server, Oracle, or something else).

The second level cache checks if it’s holding results for that query; if so, it returns them from its cache, short circuiting the call to the database; if not, the query passes through to the Store Provider. Then the second level cache intercepts and caches the returned results for next time. I’m simplifying of course; you’ll want to dig into the resources mentioned below for full details.

Notice that the component includes a tracing interceptor as well.

You don’t have to design your application for second level caching up front. You can add the second level cache component later … when you know you need it. It’s presence (or absence) is largely transparent to DevForce and EF. You’re just “wrapping” the Store Provider in this caching component; it looks like a normal Store Provider to EF.

Learn about Second Level Caching in EF

If this approach sounds like it would help, you can learn more about it from these sources:

Second-Level Caching in the Entity Framework and Windows Azure (Julie Lerman, Sept 2011)

EF Caching with Jarek Kowalski's Provider (EF Team, Sept 2010)

Using tracing and caching provider wrappers with Code First

In briefYour application can stall for several minutes while Entity Framework gathers the information it needs to perform queries and saves, a lengthy process it performs twice: before the first query and before the first save. Those minutes pile up, wasting developer time and angering your customers. You can drastically reduce these delays by pre-compiling the Entity Framework’s “views” of your model … as I explain in this post and demonstrate in its 14 minute accompanying video.



Costly EF startup If you’ve used Entity Framework for a line-of-business application model, you’ve suffered a lengthy delay before the first query completes and a similar delay before the first save completes. Subsequent queries and saves finish are much quicker, completing in an amount of time commensurate with the request.

The delay is a non-linear function of the number of entities in the model. It often feels exponential. You probably won’t notice it in a toy model (every demo you’ll ever see) because the delay is lost in the wash of everything else that you’re thinking and learning about. But when the model grows to normal size – 100 or more entities – the delay mushrooms to a minute, two minutes or more. And you suffer this delay every time run the application … which you do all day, every day during development. Multiply that by the number of developers on the project and you’re wasting a lot of time … and money.

The cost is far worse than the time lost. Make a developer wait two or three minutes per iteration and she’s bound to forget why she ran the app in the first place. Two minutes is a long time. The mind wanders. The mind turns to email, Twitter, and Facebook. Productivity is shot.

Now I don’t think you should be going near a database during normal development iterations. I recommend that you toggle the app to run against an in-memory representation of your data layer such as the DevForce “Fake Backing Store”. But maybe you’ll disregard my suggestion. And everyone has to hit the database occasionally just to confirm that the app works end-to-end.

So the development cost is terrible no matter what you do … unless your developers’ time is free; perhaps you price them at zero dollars and you’re response to every productivity decline is to hire more developers. Your second instinct is to outsource.

What about your customers and internal end users? If the app runs 2-tier, they suffer the delay every time they launch the app. Does their time matter to you? I’ll bet someone will make sure it matters to you.

You won’t field customer complaints if your application runs n-tier (e.g., in a Silverlight application) because the Entity Framework runs on the server. The startup penalty is paid only by the first user to query and save. If you run n-tier and you don’t care about developer productivity, turn the page and move along.

Pre-compiled Views to the rescueI’ve been wondering what to do about this for a long time. I’d heard that “Entity Framework Pre-compiled Views” might help. I also had heard that it was troublesome and might not work. It seemed like one more thing to get around to someday.

Then one of our professional services customers called and complained. His project had started fine but hit the wall at around 200 entity types. The first query and first save each took about 50 seconds on most machines. Team productivity had sunk, morale was sinking, and he was catching serious political flak internally. Our own staff confirmed that the problem was real. Since we (IdeaBlade) had recommended EF Code First, we had to do something.

My colleague, Steven Schmitt, did the leg work that proved EF pre-compiled views (a) work for Code First models, (b) were easy to create, and (c) improved performance dramatically: the 50 second first query dropped to seven seconds; the 50 second first save dropped to less than one second.

He deserves the credit … I’m taking the glory by blogging about it.

The accompanying 14 minute video shows EF’s slow launch times for a 200+ entity model, demonstrates how to create pre-compiled Views, and explains a bit about how they work.

I produced the video to spare you a parade of screen shots. I think it also conveys the seriousness of the problem and the practical benefit of pre-compiled Views more effectively than I can in spare prose.

The EF view generation tool does not work with EF 4.3 yet. Microsoft sources report that an update is in the works.

For those of you who want just the facts, here they are:

1. Ensure that SQL Server Express is installed. You can get around it with a DefaultConnectionFactory but it’s such a pain. Save your energy for better things and just install the thing.
   
   
2. In Visual Studio 2010, open the Extension Manager (Tools | Extension Manager).
   
   
3. Search for “Entity Framework Power Tools”. The version as I write is “Entity Framework Power Tools CTP1 0.5.0.0”.
   
   
4. [optional] Review the online information about it. These tools do more than pre-compile EF views.
   
   
5. Locate your custom DbContext class in Solution Explorer [note: we’re describing how to pre-compile views for an EF Code First model. You follow a similar approach for an EDMX-based model although I haven’t tried it personally.]
   
   
6. Make sure that your DbContext class has a public parameterless constructor … or the tool will fail in a mysterious way.
   
   
7. Select your DbContext class, right-click, and select “Entity Framework”
   
   
8. Select the “Optimize Entity Data Model” sub-item
   
   
9. Wait … the tool takes a while to compile the "views”.
   
   
10. When it’s done, your DbContext has a companion DbContext.Views class file.
    
    
11. Build and run.

You should notice an immediate improvement in start time. There is still a delay before the first query completes. But it should be a fraction of the former delay … around 1/7th of the time. The delay for the first save should be gone; it takes no longer than the second save.

DevForce Developer NotesYour DevForce application benefits from EF Pre-compiled views when you follow these steps. A DevForce Code First model doesn’t have to have a custom DbContext class … but you will have to create one to use this tool.

DevForce developers typically don’t define a parameterless constructor because DevForce wants a constructor that takes a connection string. Add the parameterless constructor anyway. Don’t worry, we will pickup the appropriate constructor at runtime.

When the model changesEntity Framework detects if your entity model classes have changed since you compiled the EF views class. When you attempt your first query, you’ll get a clear runtime exception telling you to re-compile the views class.

Only database-related changes to persisted data and navigation properties matter. You can add UI hint attributes (e.g., [Display…]) and non-persisted custom properties (e.g., FullName) without triggering an exception. Any change that would affect the mapping between your entity classes and the database will trigger the exception.

How does EF know that the model has changed? I’m not certain but I have a pretty good guess. Ignore the views class filename and look at the name of the views class itself. It will be something like “ViewsForBaseEntitySets72E6108A34B7DB042DBA3C465F35B967B4E3C76051DFBAB958B69CB0D23EA8B7”.

The hex suffix at the end looks like a hash. I’m guessing it is a hash of your entity model classes and that Entity Framework spends the initial seconds before the first query reflecting over and hashing your entity model classes before comparing that hash to this views class suffix. Inside the class itself are a couple more hash values. Maybe it's using those too or instead. Someday I'll find out. It's evident that its doing some kind of comparison between the entity model classes and this views class to ascertain if there is a disconnect.

Anyway, at runtime, if EF detects a difference, it throws an exception which should terminate your app. You'll encounter the exception quickly and unmistakeably when your app first requests data. I presume that will be before you push to production :). Just re-run the tool and you should be back in business.

At IdeaBlade we’re looking into a way to detect the views/model incompatibility at build time and regenerate the pre-compiled views automatically.

Meanwhile, it’s good to know that EF fails fast when the pre-compiled views and your model are out of sync … and the remedy is as simple as re-running the tool.

Hope this helps real-world EF developers everywhere.

Update - March 23My buddy Steve Schmitt reminds me of a few more points.

• Rowan Miller and the EF team deserve credit for developing the EF Power Tools; we just downloaded it.
• There’s a bit more info about the tool online.
• If you don’t want to regenerate the views for whatever reason, you can just delete the views file and you’re back to “normal”.

Update - April 6The EF team published this month an important white paper on performance in EF 4 and 5 that bears on pre-compiled views and other tactics that could make a significant difference for your project.