Figure 3 Caching

The amount of data you can reach at any level is different, but the right doses are determined on a per-application basis.
Also different from layer to layer is the time needed to retrieve data. Session, in most cases, is an in-process and in-memory object. Nothing could be faster. Keeping Session lean is critical because it is duplicated for each connected user. For quick access to data that can be shared between users, nothing is better than Cache or Application. Cache is faster and provides for automatic decay and prioritization. Relatively large amounts of frequently used static data can be effectively stored in any of these containers.
Disk files serve as an emergency copy of data. Use them when you don't need or can't afford to keep all the data in memory, but when going to the database is too costly. Finally, DBMS views are just like virtual tables that represent the data from one or more tables in an alternative way. Views are normally used for read-only data, but under certain conditions they can be updateable.
Views can also be used as a security mechanism to restrict the data that a certain user can access. For example, some data can be available to users for query and/or update purposes, while the rest of the table remains invisible. And table views can constitute an intermediate storage for preprocessed or post-processed data. Therefore, accessing a view has the same effect for the application, but doesn't cause preprocessing delays or place any locks on the physical table.

XML Server-side Data Islands
Caching is particularly useful when you have a large amount of data to load. However, when the amount of data is really huge, any technique—either on the client or the server—can hardly be optimal. When you have one million records to fetch, you're out of luck. In such situations, you can reduce the impact of the data bulk by using a layered architecture for caching by bringing the concept of client-side data islands to the server. An XML data island is a block of XML that is embedded in HTML and can be retrieved through the page's DOM. They're good at storing read-only information on the client, saving round-trips.
Used on the server, an XML data island becomes a persistent bulk of information that you can store in memory, or (for scalability) on disk. But, how do you read it back? Typically, in .NET you would use DataSet XML facilities to read and write. For lots of data (say, one million records), caching this way is not effective if you don't need all records in memory. Keeping all the records in a single file makes it heavier for the system. What about splitting records into different XML files that are organized like those in Figure 4? This expands the level of XML disk files shown in Figure 3.


Figure 4 Dividing Records for Performance

You can build up an extensible tree of XML files, each representing a page of database records. Each time you need a block of non-cached records, you fetch them from the database and add them to a new or existing XML data island. You would use a special naming convention to distinguish files on a per-session basis, for example, by appending a progressive index to the session ID. An index file can help you locate the right data island where a piece of data is cached. For really huge bulks of data, this minimizes the processing on all tiers. However, with one million records to manage there is no perfect tool or approach.

Automatic Cache Bubble-up
Once you have a layered caching system, how you move data from one tier to the next is up to you. However, ASP.NET provides a facility that can involve both a disk file and a Cache object. The Cache object works like an application-wide repository for data and objects. Cache looks quite different from the plain old Application object. For one thing, it is thread-safe and does not require locks on the repository prior to reading or w