The Sitecore Pie: strategic slicing for better implementations

The Sitecore Pie

pie
At some point I want to catalog the full set of features a Sitecore Content Delivery and Content Management server can manage in a project.  My goal would be to identify all the elements that can be split apart into independent services.  This post is not a comprehensive list of those features, but serves to introduce the concept.

Think of Sitecore as a big blueberry pie that can be sliced into constituent parts.  Some Sitecore sites can really benefit from slicing the pie into small pieces and letting dedicated servers or services manage that specific aspect of the pie.  Too often, companies don’t strategize around how much different work their Sitecore solution is doing.

An example will help communicate my point: consider IIS and how it serves as the execution context for Sitecore.   Many implementations will run logic for the following through the same IIS server that is handling the Sitecore request for rendering a web page.  These are all slices of the Sitecore pie for a Sitecore Content Delivery server:

  1. URL redirection through Sitecore
  2. Securing HTTP traffic with SSL
  3. Image resizing for requests using low-bandwidth or alternative devices
  4. Serving static assets like CSS, JS, graphics, etc
  5. Search indexing and query processing (if one is using Lucene)

If you wanted to cast a broader net, you could include HTTP Session state for when InProc mode is chosen, Geo-IP look-ups for certain CD servers, and others to this list of pie slices.  Remember, I didn’t claim this was an exhaustive list.  The point is: IIS is enlisted in all this other work besides processing content into HTML output for Sitecore website visitors.

Given our specific pie slices above, one could employ the following alternatives to relieve IIS of the processing:

  1. URL Redirection at the load balancer level can be more performant than having Sitecore process redirects
  2. Apply SSL between the load balancer and the public internet, but not between the IIS nodes behind your load balancer — caled “SSL Offloading” or “SSL Termination”
  3. There are services like Akamai that fold in dynamic image processing as part of their suite of products
  4. Serving static assets from a CDN is common practice for Sitecore
  5. Coveo for Sitecore is an alternative search provider that can take a lot of customer-facing search aspects and shift it to dedicated search servers or even Coveo’s Cloud.  One can go even further with Solr for Sitecore or still other search tiers if you’re really adventurous

My point is, just like how we hear a lot this election season about “let Candidate X be Candidate X” — we can let Sitecore be Sitecore and allow it to focus on rendering content created and edited by content authors and presenting it as HTML responses.  That’s what Sitecore is extremely valuable for.

Enter the Cache

I’ve engaged with a lot of Sitecore implementations who were examining their Sitecore pie and determining what slices belong where . . . and frequently we’d make the observation that the caching layer of Sitecore was tightly coupled with the rest of the Sitecore system and caching wasn’t a good candidate for slicing off.  There wasn’t a slick provider model for Sitecore caches, and while certain caches could be partially moved to other servers, it wasn’t clean, complete, or convenient.

That all changed officially with the initial release of Sitecore 8.2 last month.  Now there is a Sitecore.Caching.DefaultCacheManager class, a Sitecore.Caching.ICache interface, and other key extension points as part of the standard Sitecore API.  One can genuinely add the Sitecore cache to the list of pie slices one can consider for off-loading.

In my next post, I will explore using this new API to use Redis as the cache provider for Sitecore instead of the standard in-memory IIS cache.

Sitecore and TTL Index Heresy for MongoDB

Background

Part of our role at Rackspace is to be pro-active in tuning and optimizing Sitecore implementations for our customers.  Sometimes, the “optimizations” come more in terms of cost-savings instead of performance benefits, and sometimes there is a correlation.

Here’s an optimization that is more oriented to cost-savings.  We have customers using MongoDB for both private and shared session state in Sitecore, as well as xDB data collections; they’re using a hosted MongoDB service through ObjectRocket which provides provisioning elasticity, good performance, and access to really strong MongoDB pros.  We noticed old HTTP session data accumulating in the session collections for one customer, and couldn’t find an obvious explanation.  These session records shouldn’t be leaking through the MongoDB clean-up logic that’s part of the Sitecore API (Sitecore.SessionProvider.MongoDB.MongoSessionStateProvider), but we wanted to correct it and understand why.

The catch was, the customer’s development had several higher priority issues they were working on, and sifting through a root cause analysis of something like this could take some time.  In the interim, we addressed how to clean out the old session data from MongoDB — if the customer was using SQL Server for sessions state, a SQL Server agent deleting expired sessions could be easily employed . . . but that isn’t an option for MongoDB.

Research

At Rackspace, we began evaluating options for a MongoDB equivalent to the “clean up old session records” process and so I started by reviewing everything I could find about data retention for MongoDB and Sitecore.  It turns out there isn’t much.  There isn’t mention of data clean-up in the Sitecore documentation on shared and private session for MongoDB.  There isn’t an explicit <agent> configured anywhere, either.  With Reflector, I started looking through the Sitecore.SessionProvider.MongoDB.MongoSessionStateProvider class and I didn’t see any logic related to data clean-up.

I did finally find some success, however, in reviewing the Sitecore.SessionProvider.MongoDB.MongoSessionStateProvider class that extends the abstract Sitecore.SessionProvider.SitecoreSessionStateStoreProvider class.

The abstract class uses a timer (controlled by the pollingInterval set for the provider); it runs OnProcessExpiredItems . . . and then GetExpiredItemExclusive . . . and eventually RemoveItem for that record.  That finally calls through to the MongoDB API with the delete query:

RemoveItemThis was all helpful information to the broader team working to determine why the records were remaining in MongoDB, but we needed a quick non-invasive solution.

TTL Indexes

MongoDB supports “Time to Live (TTL) Indexes” which purge data based on time rules.  Data older than 1 week, for example, could automatically be removed with this index type.  That’s surely acceptable for sessions state records that leak through the Sitecore API.  We coordinated with the ObjectRocket team and are setting the customer up with TTL Indexes instead of the default; this should dramatically reduce the data storage footprint.

Heresy

While pursuing this effort on behalf of session state management, I realized this could be an intriguing solution to data retention challenges with Sitecore’s xDB.  Using MongoDB TTL indexes for the xDB collections would prevent that data from growing out of control.  Set a TTL value of 180 days, for example, and make use of just the most recent 6 months of user activity as part of content personalization, profiling, etc.  Of course, one sacrifices the value of the old data if one expires it after a set time.  Remember, I’m acknowledging this is heresy! 🙂

I really wonder, though, how many Sitecore implementations intend to store all the xDB data indefinitely and have a strategy for making use of all that old, ever-accumulating, data?  I think the promise of xDB analytics-based presentation rules is far greater than the reality.  I see many organizations lacking a cohesive plan for what data to gather into xDB, and how to effectively make use of it.

I think TTL Indexes for MongoDB would be a good safety net for those companies still sorting out their path with xDB and analytics, without having to bank massive volumes of data in MongoDB during the maturation process.

One final note: since conserving disk space is a priority, performing a weekly data compaction for MongoDB is a good idea.  This fully reclaims the space for all the expired documents.

Understanding Sitecore Session Timeouts

This post describes the configuration of general session timeouts in Sitecore.  Implementation teams are often interested in how to approach session management, cautious of situations where authors are logged-off after working with a piece of content for an extended amount of time (such as authoring in a RichTextField for 30+ minutes ). As I explain below, focusing on just http session timeouts will only partly address the problem. Internally, Sitecore has both regular http session timeouts and authentication timeouts that factor into this.  Some teams are tempted to trigger ajax calls to the server to extend the session lifetime . . . but as you’ll see, this is already handled by Sitecore and further customization of this process could weaken the security of the system.

For additional background: Sitecore made significant improvements to the session management logic with Sitecore 6.6 Update-7; proper support for sliding expiration was introduced, and full respect for standard ASP.Net authentication configuration was added to the platform. This may add to the confusion on this subject, as work-arounds and various custom patches may have been necessary for an older Sitecore implementation to achieve what is now out-of-the-box session support.

The Universal Editing Concern: Lost Work Scenario

Consider the following steps taken by a Sitecore author in the Content Management environment:

  1. Content Author creates new item or opens an existing one
  2. Author starts editing text in a RichText field
  3. The editing takes a long time, so the author remains working on that field for longer than the session timeout value (or they might take a 20 minute coffee brake in the middle, etc)
  4. Eventually the Content Author clicks Save to move on with their work

In a typical non-Sitecore ASP.Net application, after session expiration, all the work will be lost as the user will need to log back in to the system.  This is exactly the lost work scenario we’ll aim to avoid.

On Content Delivery Servers

For the Sitecore CD servers, one controls session timeout in the regular ASP.Net way, via the sessionState configuration node in web.config:

<sessionState timeout="180" ...

The timeout value is in minutes, so 180 equals 3 hours. Sitecore recommends a short session timeout for CD servers to conserve server resources. A default of 20 minutes is usually adequate, but each implementation is different. This has no relevance to the lost work scenario as CD servers aren’t where authors work with editing content, but I include this note in the interest of showing the complete configuration for Sitecore and session timeout management.

Content Authoring Servers

HTTP Session Timeouts
The sessionState configuration in web.config governs session duration in Sitecore CM servers just as it does for CD servers. HTTP session timeouts are defined as follows:

<sessionState timeout="180" ...

Authentication Timeouts
Along with HTTP Session timeouts, when dealing with the Sitecore Client on CM servers, it’s important to know how Authentication Timeouts come into play as these are less obvious to most implementations. To understand Authentication Timeouts, know that there are two types of Sitecore authentication sessions:
1) Not-persistent auth session
2) Persistent auth session
A “Persistent” auth session is created when the user logs in and has the “Remember Me” box checked on the login form; the “Not-Persistent” auth session is when this is not checked. See the login screen below for reference.
loginremember
The two types of auth sessions use different locations to configure timeouts. Both locations are, by default, in the web.config file of the Sitecore webroot.

Not-Persistent Authentication Session

The lifetime of the “not-persistent” auth session is determined by settings in the authentication node as follows:

<authentication mode="None">
  <forms name=".ASPXAUTH" cookieless="UseCookies" timeout="180" slidingExpiration="true" />
</authentication>

The value of the “timeout” attribute controls the lifetime of the auth session in minutes. If it’s not specified in the configuration, the default timeout value is 30 minutes. For clarity, it’s a good practice to explicitly set a value for this property.
The value of the “slidingExpiration” attribute controls whether sliding expiration is enabled.

From the MSDN documentation on slidingExpiration:
“Sliding expiration resets the expiration time for a valid authentication cookie if a request is made and more than half of the timeout interval has elapsed. If the cookie expires, the user must re-authenticate. Setting the SlidingExpiration property to false can improve the security of an application by limiting the time for which an authentication cookie is valid, based on the configured timeout value.”

Note that the slidingExpiration value defaults to true, but for clarity it’s a good practice to explicitly include it in the forms definition xml node.

Persistent Authentication Session

Lifetime of “persistent” Sitecore auth sessions is controlled by the “Authentication.ClientPersistentLoginDuration” setting; the value is in days.

<setting name="Authentication.ClientPersistentLoginDuration" value="180"/>

Note: prior to Sitecore 6.2, this setting was known as Authentication.TicketTimeout

KeepAlive.aspx

Sitecore prevents the lost work scenario, caused by regular http session expiration, via a page called “KeepAlive.aspx.” When the Content Editor is open, a short while before a session is going to expire there is Sitecore javascript code that makes a request to [webroot]/sitecore/service/keepalive.aspx to extend the session. The browser connects to the server with a simple http request to prevent session expiration.

The KeepAlive.aspx page doesn’t, however, extend the authentication expiration; it only extends the http session. This is because it can be a security vulnerability to perpetuate the lifetime of an authentication ticket automatically for an idle browsesr. The KeepAlive.aspx page can be customized to extend the authentication expiration, but it’s important to understand the repercussions. It’s possible for an editor to never have their authentication ticket expire if modifications like the following are made:

Out-of-the-box, KeepAlive.aspx uses this logic in the Page_Load event to extend the lifetime of the http session:

protected override void OnLoad(EventArgs e)
{
  if (Tracker.IsActive)
  {
    Tracker.CurrentPage.Cancel();
  }
}

To extend the authentication expiration, one could use this logic instead in KeepAlive.aspx:

protected override void OnLoad(EventArgs e)
{
  if (Tracker.IsActive)
  {
    Tracker.CurrentPage.Cancel();
  }

  //Start of customization. Not sure you really wan't to do this . . .
  HttpCookie cookie = HttpContext.Current.Request.Cookies[FormsAuthentication.FormsCookieName];
  if (cookie != null)
  { 
    Sitecore.Web.Authentication.TicketManager.Relogin    (Sitecore.Web.Authentication.TicketManager.GetCurrentTicketId());
  }
  //end customization
}

Again, this is not generally recommended as authors could potentially never have their authentication tickets expire when their computer is idle, but happens to still have a browser open to the Sitecore Client.

Active Directory and Authentication Session

If an implementation is using Active Directory integration (the standard Ldaplogin.aspx with the Sitecore AD Module), it’s programmatically the same as not checking the “remember me” checkbox and so it uses not-persistent auth sessions. If you’re using standard Sitecore AD integration, the authentication and forms xml nodes are most relevant to configuring authentication timeouts.

Conclusion

Session timeout and authentication session timeout are two different concepts, but often get combined together when talking about ASP.Net web application “sessions” with Sitecore.

Sitecore recommends using a session timeout (defined in the sessionState configuration) that’s as short as can be while still meeting the needs of the implementation. The default of 20 minutes is a reasonable starting point and Sitecore’s javascript calls to KeepAlive.aspx will keep that session from expiring if an author is working in the Sitecore Client but not making http requests to the web server directly due to user actions.

For the Sitecore Content Management servers, authentication session expiration is more likely to cause an author to be redirected to the login page and cause the lost work scenario. Sitecore introduces no KeepAlive functionality for authentication sessions as it could be a security vulnerability.

      There are two locations to set an authentication timeout value:

    1. The authentication node in web.config has a Forms timeout attribute (for not-persistent auth sessions, also used with standard Sitecore Active Directory module integration)
    2. The Authentication.ClientSessionTimeout setting in web.config (for persistent auth sessions)

Confusion can be caused by testing with a regular Sitecore account (not Active Directory) and one checks the “remember me” checkbox on the login page . . . then the user is testing with a persistent auth session and not exercising the configuration relevant to Sitecore’s AD module.

I should also add that there’s a session timeout bug in recent Sitecore versions; authors can be mistaken for robot traffic and so their session is terminated prematurely. It’s discussed on the Sitecore KB item https://kb.sitecore.net/articles/135940<?a> but, I’ve encountered two projects where the KB resolution wasn’t sufficient in solving the problem. In those situations, one needs to remove the suggested KB article configuration patch and modify the system.webServer section of the Sitecore web.config (if you’re using IIS Classic mode you’d use the httpModules section instead). The blue colored configuration change resolves the issue:

...<add type="Sitecore.Web.HttpModule,Sitecore.Kernel" name="SitecoreHttpModuleExtensions" />
      <add type="Sitecore.Support.FixHttpSessionTimeout.CausedByRobotDetection,Sitecore.Support.414299" name="RevertingAnalyticsBotSessionDuration" />

Armed with this knowledge, Sitecore implementations can make the best use of the session timeout settings.

Out-of-process Session Options for Sitecore 7.2

Most Sitecore implementations I work with are using multiple active Content Delivery (CD) servers; more rarely, a project will use multiple Content Management (CM) servers.  These multi-server setups are considered “scaled” in Sitecore terminology, and the foundation for setting up a scaled implementation is the Sitecore Scaling Guide.

ScaledThat Scaling Guide is a good reference, but can gloss over certain details or doesn’t speak directly to a customer’s scenario.  Managing HTTP Session state is one such topic. I’ve compiled something like the following analysis for a couple Sitecore customers in the last few weeks, so I’m going to post a general version here since it reflects the state of handling Sitecore session state as of right now, February 2015, for a customer running Sitecore 7.2 today but considering upgrades in the next year or two. The customer isn’t running on Windows Azure, but they are weighing it as an option going forward in the years to come — and this is relevant as you’ll see below.

Overview
By default, Sitecore is configured to use an in-process (“InProc”) session provider for convenience. This is the fastest of all session state providers (don’t just take my word for it), and it’s understandable since any session information remains on the specific IIS server that generated it. I won’t go into the well-trodden path of why one may not want this InProc session provider, since I take it as a given that if you’ve read this far you’re curious about the alternatives. So, without further ado, here is a look at your alternatives to the default InProc session management in Sitecore:

Out-of-process Session State Options for CD Servers in Sitecore 7.2
For Sitecore Content Delivery (CD) servers, out-of-process session state is fully supported. Changing from the default InProc setting to use an out-of-process provider is the same as with any other ASP.Net web application; the “mode” attribute of the sessionState element in the web.config file governs this functionality. See https://msdn.microsoft.com/en-us/library/ms178586%28v=vs.140%29.aspx?f=255&MSPPError=-2147217396 for documentation on StateServer or SQLServer (the two main choices for out-of-process session). Besides just configuring the sessionState mode setting, you should know that there may be other provisioning and setup (the Microsoft documentation I linked to covers it thoroughly).

In addition to StateServer and SQLServer, one could use the “Custom” mode to use something like Redis. The Microsoft.Web.Redis.RedisSessionStateProvider is popular because of fast performance metrics; in the words of the movie Zoolander, that Redis is so hot right now!  Redis is a relatively new option for session state management, however, and Sitecore has not been fully regression tested with it. Still, I know of some customers anecdotally reporting a positive experience with the RedisSessionStateProvider in spite of it being considered “experimental” by Sitecore support.

I mention Redis because it’s a popular topic right now, but for a customer cautious of using officially “experimental” approaches in Sitecore, I think you have to avoid Redis unless you’re on Azure. Furthermore, as I’ll go into, there are session considerations in Sitecore 7.5 and 8 that for the moment make Redis less attractive.

Out-of-process Session State Options for CM Servers for Sitecore 7.2
For Content Management (CM) servers, support for out-of-process session in Sitecore 7.2 is considered “experimental” as it hasn’t been fully regression tested. See https://kb.sitecore.net/articles/901069 for the official statement from Sitecore on this, but my recommendation is to use the default InProc session provider for their CM Server and configure a sticky session for Content Management operations. This is usually not too big a pill to swallow for companies, and know that in the months to come this blog post will no longer be relevant as Sitecore is continuing to work on liberating CM session state.

Sitecore 7.5 and 8 Changes
Starting with Sitecore 7.5 and continued for the future of the platform, the notion of a Private Session State provider and a Shared Session State provider have been introduced. A session provider must support the Session_End event to be compatible with these versions of Sitecore, and Sitecore ships with two session providers that satisfy this requirement:

  • Sitecore.SessionProvider.MongoDB.MongoSessionStateProvider
  • Sitecore.SessionProvider.Sql.SqlSessionStateProvider

One can implement their own provider that will support the Session_End event and use it (for example, Microsoft provides the RedisSessionStateProvider, but it does not support the Session_End event . . . so one would need to extend it to support the Session_End behaviour).

In Summary

Most organizations prefer to run on a platform fully supported by Sitecore without customization unless absolutely necessary, and so this limits the session state options. SQL Server is faster than MongoDB in the performance tests for session state relevant to Sitecore, so SQL Server is the out-of-process provider I’m recommending for a customer who is currently on 7.2 and has their sites set on 7.5, 8, and beyond in the medium term (next year or two).

Returning to the hypothetical organization determining their session state approach for 7.2, it’s a pretty cut-and-dried decision in the abstract since there is only one out-of-process provider that satisfies both Sitecore 7.2 and Sitecore 7.5+ requirements today: SQL Server the session provider. I say “in the abstract” since there are businesses with investments in Memcached out there who would probably not blink an eye at using a custom out-of-proc provider with Sitecore and adding a session_end to their Memcached approach, for them this could be an advantageous use of their resources and in-house expertise . . . but my recommendation here is for an average company running the conventional Sitecore stack of ASP.Net, IIS, SQL Server.

Final Configuration Notes

Let me add to the end of this some less documented advice about out-of-process configuration for Sitecore. Assuming you have CD servers that aren’t using InProc, it makes sense to make the following changes:

<setting name="PageStateStore" value="Sitecore.Web.UI.DatabasePageStateStore, Sitecore.Kernel" />

Also, change the ViewStateStore to DatabasePageStateStore for CD servers:

<setting name="ViewStateStore" value="Sitecore.Data.DataProviders.DatabaseViewStateStore, Sitecore.Kernel" />

Also, disable the caching of ViewState for CD servers:

<setting name="Caching.CacheViewState" value="false" />