How to use Redis in place of Sitecore in-memory caches

In this previous post I discussed the case for separating functions typically combined into a single out-of-the-box Sitecore instance into distinct parts, supporting better scalability, performance, availability, etc.  I concluded that piece with a reference to using Redis cache for Sitecore caches, instead of the default IIS in-memory cache.  Prior to Sitecore 8.2, this wasn’t entirely possible (although one could alter some Sitecore caching aspects through extensive customization of certain Sitecore assemblies).  Implementations looking to free up IIS resources, to get more out of their limited Sitecore IIS licenses, might consider swapping the in-memory caches for Redis.

Major caveat: This specific blog post doesn’t analyze the performance implications of doing this; my focus here is to cover how to accomplish the swap of the in-memory cache instead of examining all the nuances of why or why not to do it.  That being said, I do expect the use of Redis for Sitecore caches to be a surgical decision made based on the particulars of each project.  It’s not a one-size-fits-all solution.  There are also questions of which caches to put into Redis, and where Redis should be running (considerations of connectivity, Redis perf, and the whole landscape of Redis can also be brought into this conversation).  This will make for future material I cover, probably in conjunction with the big time Redis pros at ObjectRocket (who, conveniently, are part of my family at Rackspace).  I hope and expect others in the greater Sitecore community will expand on this topic, too!

Customizing Cache Configuration with Sitecore 8.2 +

Sitecore 8.2 includes a new .config file at App_Config\Include\CacheContainers.config — as the name suggests, this is where one can identify cache containers that will replace the default ones Sitecore uses.  For my example, I’m going to specify a custom container for the website[html] cache; any cache can be included here (I refer you to the old reliable /sitecore/admin/cache.aspx page — the name column enumerates all the Sitecore caches):

cache

With my customization, my CacheContainers.config file looks like this:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
 <sitecore>
  <cacheContainerConfiguration>
   <cacheContainer name="website[html]" type="Rackspace.Sc.Caching.ReferenceRedisCache, Rackspace.Sc.Caching">
    <param desc="name">$(0)</param>
    <param desc="maxSize">$(1)</param>
    <cacheSizeCalculationStrategy type="Sitecore.Caching.DefaultCacheSizeCalculationStrategy, Sitecore.Kernel" />
   </cacheContainer>
  </cacheContainerConfiguration>
 </sitecore>
</configuration>

I’m not going to regurgitate the comments from the default CacheContainers.config file, but it’s certainly worth reviewing.

Once the CacheContainers.config is pointing to the custom class and assembly (in my case type=”Rackspace.Sc.Caching.ReferenceRedisCache, Rackspace.Sc.Caching”) it’s necessary to create the custom assembly to provide the Redis cache behaviour.  The foundation for this is to implement the new Sitecore.Caching.ICache interface (which requires Sitecore.Caching.ICacheInfo and Sitecore.Caching.Generics.ICache<string> interfaces. It can look like a lot of work, but it actually isn’t so bad.

One way I found to make it easier was to reference the out-of-the-box Sitecore Sitecore.Caching.MemoryCacheAdapter in the Sitecore.Kernel assembly.  One can consider the MemoryCacheAdapter for an example of the implementation of these interfaces:

cache2

Reflector, or whatever your preferred decompilation tool of choice, for the win!

In my case, I create a ReferenceRedisCache for this. . .

public class ReferenceRedisCache : Sitecore.Caching.ICache

. . . and the nuts and bolts come down to swapping Redis in place of the System.Runtime.Caching.MemoryCache object from the MemoryCacheAdapter implementation.  There are other considerations, of course, but this is the big picture.

I’m not going to post all the code here as mine is still a bit of a work in progress, and I need to evaluate it in terms of performance etc.  You’re much safer using the decompiled Sitecore source as a guide, at this point.  With this post, I did want to share how one can approach this powerful Sitecore customization; you can selectively change from Sitecore’s in-memory caching to just about any option you’d like this way.

There is another avenue that I’ve not explored with this.  Sitecore provides a Sitecore.Caching.BaseCacheConfiguration class that can be overridden to introduce customizations.  Sitecore.Caching.DefaultCacheConfiguration is the standard approach you could use as an example; this is an alternative way of substituting your own caching logic, but I haven’t dug into it yet.

I’ll clean up my code, do some profiling and evaluations, and come back with the full details of my custom Rackspace.Sc.Caching.ReferenceRedisCache implementation soon.

The Sitecore Pie: strategic slicing for better implementations

The Sitecore Pie

pie
At some point I want to catalog the full set of features a Sitecore Content Delivery and Content Management server can manage in a project.  My goal would be to identify all the elements that can be split apart into independent services.  This post is not a comprehensive list of those features, but serves to introduce the concept.

Think of Sitecore as a big blueberry pie that can be sliced into constituent parts.  Some Sitecore sites can really benefit from slicing the pie into small pieces and letting dedicated servers or services manage that specific aspect of the pie.  Too often, companies don’t strategize around how much different work their Sitecore solution is doing.

An example will help communicate my point: consider IIS and how it serves as the execution context for Sitecore.   Many implementations will run logic for the following through the same IIS server that is handling the Sitecore request for rendering a web page.  These are all slices of the Sitecore pie for a Sitecore Content Delivery server:

  1. URL redirection through Sitecore
  2. Securing HTTP traffic with SSL
  3. Image resizing for requests using low-bandwidth or alternative devices
  4. Serving static assets like CSS, JS, graphics, etc
  5. Search indexing and query processing (if one is using Lucene)

If you wanted to cast a broader net, you could include HTTP Session state for when InProc mode is chosen, Geo-IP look-ups for certain CD servers, and others to this list of pie slices.  Remember, I didn’t claim this was an exhaustive list.  The point is: IIS is enlisted in all this other work besides processing content into HTML output for Sitecore website visitors.

Given our specific pie slices above, one could employ the following alternatives to relieve IIS of the processing:

  1. URL Redirection at the load balancer level can be more performant than having Sitecore process redirects
  2. Apply SSL between the load balancer and the public internet, but not between the IIS nodes behind your load balancer — caled “SSL Offloading” or “SSL Termination”
  3. There are services like Akamai that fold in dynamic image processing as part of their suite of products
  4. Serving static assets from a CDN is common practice for Sitecore
  5. Coveo for Sitecore is an alternative search provider that can take a lot of customer-facing search aspects and shift it to dedicated search servers or even Coveo’s Cloud.  One can go even further with Solr for Sitecore or still other search tiers if you’re really adventurous

My point is, just like how we hear a lot this election season about “let Candidate X be Candidate X” — we can let Sitecore be Sitecore and allow it to focus on rendering content created and edited by content authors and presenting it as HTML responses.  That’s what Sitecore is extremely valuable for.

Enter the Cache

I’ve engaged with a lot of Sitecore implementations who were examining their Sitecore pie and determining what slices belong where . . . and frequently we’d make the observation that the caching layer of Sitecore was tightly coupled with the rest of the Sitecore system and caching wasn’t a good candidate for slicing off.  There wasn’t a slick provider model for Sitecore caches, and while certain caches could be partially moved to other servers, it wasn’t clean, complete, or convenient.

That all changed officially with the initial release of Sitecore 8.2 last month.  Now there is a Sitecore.Caching.DefaultCacheManager class, a Sitecore.Caching.ICache interface, and other key extension points as part of the standard Sitecore API.  One can genuinely add the Sitecore cache to the list of pie slices one can consider for off-loading.

In my next post, I will explore using this new API to use Redis as the cache provider for Sitecore instead of the standard in-memory IIS cache.

Sitecore RemoteRebuild Strategy Best Practices

I spent entirely too long troubleshooting a customer’s approach to the RemoteRebuild indexing strategy for Sitecore.  The official documentation is fairly straight-forward, but there are some significant gaps left up to the reader to infer or figure out.

I think the heading “Best Practice” on that documentation page is great, and I hope Sitecore continues to expand those notes to be as thorough as possible.  That being said, I would include the following example patch configuration showing how to apply the change without manually editing Sitecore base install files:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <contentSearch>
      <configuration type="Sitecore.ContentSearch.ContentSearchConfiguration, Sitecore.ContentSearch">
        <indexes hint="list:AddIndex">
           <index id="sitecore_web_index" type="Sitecore.ContentSearch.LuceneProvider.LuceneIndex, Sitecore.ContentSearch.LuceneProvider">
            <strategies hint="list:AddStrategy">
              <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/remoteRebuild" />
            </strategies>
          </index>
        </indexes>
      </configuration>
    </contentSearch>
  </sitecore>
</configuration>

This patch should be applied on the Sitecore CD servers where you want to perform the Lucene re-indexing operations.  There are no configuration changes necessary to the Sitecore CM servers.

Speaking of CM server, one might think the posted Sitecore prerequisites cover it:

  • The name of the index on the remote server must be identical to the name of the index that you forced to rebuild.
  • You must enable the EventQueue.
  • The database you assign for system event queue storage (core by default) must be shared between the Sitecore instance where the rebuild takes place and the other instances.

But the biggest addition to this I found was that the Indexing Manager feature in the Control Panel does not call the proper API to trigger the RemoteRebuild activity, so this screen is not where one initiates the remote rebuild:

Won’t work:

remoterebuild

The only way to properly activate the RemoteRebuild is via the Developer ribbon in the Sitecore Content Editor

This works:

remoterebuild2

See this quick video on how to enable this Developer area in the Content Editor, in case this is new to you.

Apparently this dependence on the Developer ribbon is a bug in Sitecore and scheduled to be corrected in a future release.  I know I spent several hours troubleshooting configuration changes and attempted many permutations of the configuration for the RemoteRebuild strategy before Sitecore support confirmed this fact (as did the ever-helpful Ivan Sharamok).

The only other detail I’ll share on the Sitecore RemoteRebuild strategy is that one should verify if the Lucene index adding RemoteRebuild to is using Sitecore.ContentSearch.LuceneProvider.SwitchOnRebuildLuceneIndex or just Sitecore.ContentSearch.LuceneProvider.LuceneIndex.  The patch .config file one uses should reference the appropriate type attribute.

A final note I’ll add about rationale for the RemoteRebuild . . . this customer has several Sitecore Content Delivery servers spread in different data centers, and is currently reliant on the Lucene search provider for Sitecore.  One could make a strong case the customer should be using Solr instead, and indeed this is on their implementation road map, but in the meantime we suggested the RemoteRebuild as a way for them to centrally manage the state of indexes on all their disparate CD servers.  The alternative would be depending on the onPublishEndAsync strategy (which works well, but has limited applications in certain cases), or doing some administrative connection to each CD server (via browser or PowerShell etc) and doing something along the lines of the following when they need to rebuild specific indexes:

public void Page_Load(object sender, EventArgs e)
{
        Server.ScriptTimeout = 3600; //one hour in case this takes a while
        ContentSearchManager.GetIndex("your_index_name").Rebuild();
}

This quickly becomes unwieldy and hard to maintain, however, so getting RemoteRebuild wired up in this case is going to be a valuable addition to the implementation.

Sitecore Live Session Agents

It can be fun when an hour-long meeting is cancelled at the last minute; you suddenly have a chance to tackle your personal “back log” right?

This note on xDB processing is part of my “back log” so without further ado . . . let me begin by noting how Sitecore is getting better at designing modular elements of functionality. They’ve been at it for years.

The Menu of Sitecore Server Roles

The notion of a “Sitecore server” being a single defined resource is a thing of the past.   Things used to be either Content Management (CM) or Content Delivery (CD); now, CM or CD servers can be supplemented with dedicated publishing servers, the xDB 3 amigos (processing, aggregation, and reporting servers), search work-horses (via Solr, Coveo, or ?).  One can combine elements together so you could have processing + aggregation, or your dedicated publishing server could run Solr for search.  I’m not going to discuss the databases that support all of this, MongoDB or SQL Server, but the count of databases and collections is also ever-increasing.

Then, there are the less well-publicized “mini server roles” like the one that does GeoIP resolution work in the background.  This post is mostly about another specific “mini” Sitecore server known as the Live Session agent.

There is a clear pattern from Sitecore that “servers” are combinations of several slices of different functionality.  I think this shows a maturation in the product, and realization that different implementations will be making use of Sitecore components in different ways.  It makes for flexibility and lots of tuning opportunities.  It keeps a Sitecore Architect employed, I suppose!  I do worry about the stability of all these permutations, however, and along with this pattern is the bloat of configuration files and the potential for misconfiguration.  There are some noble efforts at improving this, though, and I think the complexity of configuration will improve in future Sitecore releases.

The Live Session “Role”

While not truly a “role” in the standard Sitecore sense, the Live Session Agent extends a Content Delivery (CD) node to also process xDB automation states.  This is to assist in monitoring and processing the queue of contacts with automation states with timeouts. One could run a dedicated server just for this Live Session Agent activity, I suppose, or employ a set of CD servers to share the load.  From the documentation:

You can enable the background agent only on a subset of instances, such as on a dedicated instance or a set of instances that only process the timeout and not HTTP requests.

In reviewing the granular tuning settings exposed by Sitecore, Live Session Agent optimization could be a bit of a dark art.  Should be some fun ahead, though, and a chance to enlist additional server resources to alleviate a potential bottleneck around engagement automation state processing.

The Live Session Agent depends on two config files (Sitecore.EngagementAutomation.LiveSessionAgent.config and Sitecore.EngagementAutomation.LiveSessionAgent.Processing.config) and a single .dll (Sitecore.EngagementAutomation.LiveSessionAgent.dll) that can be downloaded under the Resources section here.  The documentation is sufficient to get you started and, while I find it light on practical guidance, there is an obvious effort to expose all the knobs and switches one might require for this sort of background process management (threads to allocate etc).

I suspect this Live Session Agent will become a mainstay in the performance and scaling efforts for more complicated Sitecore xDB implementations — but most any xDB project could benefit from additional resources devoted to engagement automation state processing.  I think the trick is not overwhelming the CD server that you ask to pitch-in and help.

Now, with the other 30 minutes I found by that meeting being cancelled, I’m going to review what all is going on with Sitecore.EngagementAutomation.LiveSessionAgent.dll with Reflector . . .

Basic Azure Benchmarking for Sitecore

We used this approach to run through some database performance benchmarking for Azure for a real production Sitecore implementation (Sitecore version 8.0).  The idea is to use real resource utilization statistics to estimate DTU requirements and Azure SQL database tiers for if/when this workload shifts to Azure.  This was for a real site, not LaunchSitecore or JetStream or Habitat 🙂

We ran the tests for an hour, then six hours, then at different times of the day . . . yet the results were consistent each time.  The Standard S2 Tier was the recommendation based on our observed workload for every database (Core, Master, and Web databases).  This implementation used MongoDB for session state, however, so this doesn’t include SQL Server session state into the calculation.

There were usually around 9% of the time when the DTUs would spike and exceed the Standard S2 capacity, so I’m a bit concerned and curious about what that would translate into if we were running 100% in Azure (probably slow perf for those times — which roughly appeared to align with Sitecore publishing — which means the Standard S2 Tier might not really cut it).  We could see Content Authors complaining of slow perf, stalled content updates, etc . . . so this is just a starting point.

Sitecore Config for Publishing to Remote Data Centers

A couple months back, I shared our Rackspace publishing strategy for Sitecore in multiple data centers.  It’s fast, reliable, and we’ve had good experiences taking that approach.

This diagram shows what I’m talking about:2016-02-23-Sitecore-Enterprise-Architecture-For-Global-Publishing

That post at developer.Rackspace.com is only part of the picture, however, and I wanted to return to this topic since I recently had a chance to revisit our Sitecore configuration for a customer running Sitecore 7.2.  Referring to the diagram above, we needed to patch the client Sitecore configuration to accommodate the addition of the proxyWebForHongKong and proxyWebForDallas databases; we were finishing up the provisioning of a clean environment, and had replication running, but we needed to connect the two remote data centers so they fully participated with publishing.

Wading through the Sitecore documentation to find these key architectural pieces is tough, and you would need to synthesize at least the Search Scaling Guide, the Scaling Guide, and the Search and Indexing Guide to get to all this material.  For my own benefit, probably more than for any potential reader, I’m consolidating this here.

It’s important when one does a publish in Sitecore that it triggers index updates on the remote servers, as well as move the content to the appropriate Web database.  This is the crux of what I’m up to here.

At a high level, we had to complete these steps:

  1. Add the database connection strings for the two proxy databases serving our two remote data centers
  2. Add the two proxy database definitions under the <sitecore> node so they can be referenced as Sitecore Publishing Targets etc
  3. Add an index update strategy specific for each proxy database
    • This only needs to be on the environments where the index is used; for example, sitecore_web_index configurations would only be relevant on the CD servers
  4. Connect these new indexing strategies to indexes that need to be kept consistent between data centers
    • This customer is making some use of Lucene, so there is a sitecore_web_index that’s relevant here
    • This customer is using an alternative search provider, too, but I’m focusing on the out-of-the-box configuration as it relates to Lucene here

To implement all of the above, we would typically steer customers to a Sitecore patch configuration strategy such as this one by Bryan “Flash” Furlong.  For simplicity, however, we at Rackspace created a subdirectory App_Config/Include/Rackspace and placed a single config patch file in it.  We don’t own the solution structure, and work to isolate our changes as much as possible from a customer implementation.  So having a single file we can reference is handy for our use case — but not what I’d consider an absolute best practice.

Here’s a nice summary on using subdirectories for Sitecore configurations, in case that’s new to you; here’s Sitecore’s official configuration patch documentation if you’d like to review that: https://sdn.sitecore.net/upload/sitecore6/60/include_file_patching_facilities_sc6orlater-usletter.pdf.

Unfortunately, the <connectionStrings> definitions are not under the Sitecore node and so we can’t patch those the same way, but everything else is in the Sitecore configuration patch below.  I’m not going to detail the following XML, but summarize by saying steps 2 through 4 above are handled here.  Other relevant settings, such as EnableEventQueues etc, are set elsewhere as the Sitecore scaling guide has been broadly applied to this setup already . . . the following is just the changes we needed to make in this particular case, but I’m hoping the example is helpful:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <!-- 
  We cannot patch in a connection string, so be sure to explicitly add connectionstrings
  pointing to the appropriate proxy databases
  <add name="ProxyWebForHongKong" connectionString="Trusted_Connection=True;Data Source=SQLCluster;Database=ProxyWebForHongKong" />
  <add name="ProxyWebForDallas" connectionString="Trusted_Connection=True;Data Source=SQLCluster;Database=ProxyWebForDallas" />

-->
  <sitecore>

    <!-- Add the database configurations that match the connectionstring entries
    (The id of the <database> node must be equal to the name of your connection string node)-->
    <databases>
      <database id="ProxyWebForHongKong" singleInstance="true" type="Sitecore.Data.Database, Sitecore.Kernel">
        <param desc="name">$(id)</param>
        <icon>Network/16x16/earth.png</icon>
        <securityEnabled>true</securityEnabled>
        <dataProviders hint="list:AddDataProvider">
          <dataProvider ref="dataProviders/main" param1="$(id)">
            <disableGroup>publishing</disableGroup>
            <prefetch hint="raw:AddPrefetch">
              <sc.include file="/App_Config/Prefetch/Common.config" />
              <sc.include file="/App_Config/Prefetch/Webdb.config" />
            </prefetch>
          </dataProvider>
        </dataProviders>
        <proxiesEnabled>false</proxiesEnabled>
        <proxyDataProvider ref="proxyDataProviders/main" param1="$(id)" />
        <archives hint="raw:AddArchive">
          <archive name="archive" />
          <archive name="recyclebin" />
        </archives>
        <Engines.HistoryEngine.Storage>
          <obj type="Sitecore.Data.SqlServer.SqlServerHistoryStorage, Sitecore.Kernel">
            <param connectionStringName="$(id)" />
            <EntryLifeTime>30.00:00:00</EntryLifeTime>
          </obj>
        </Engines.HistoryEngine.Storage>
        <Engines.HistoryEngine.SaveDotNetCallStack>false</Engines.HistoryEngine.SaveDotNetCallStack>
        <cacheSizes hint="setting">
          <data>20MB</data>
          <items>10MB</items>
          <paths>500KB</paths>
          <itempaths>10MB</itempaths>
          <standardValues>500KB</standardValues>
        </cacheSizes>
      </database>
      <database id="ProxyWebForDallas" singleInstance="true" type="Sitecore.Data.Database, Sitecore.Kernel">
        <param desc="name">$(id)</param>
        <icon>Network/16x16/earth.png</icon>
        <securityEnabled>true</securityEnabled>
        <dataProviders hint="list:AddDataProvider">
          <dataProvider ref="dataProviders/main" param1="$(id)">
            <disableGroup>publishing</disableGroup>
            <prefetch hint="raw:AddPrefetch">
              <sc.include file="/App_Config/Prefetch/Common.config" />
              <sc.include file="/App_Config/Prefetch/Webdb.config" />
            </prefetch>
          </dataProvider>
        </dataProviders>
        <proxiesEnabled>false</proxiesEnabled>
        <proxyDataProvider ref="proxyDataProviders/main" param1="$(id)" />
        <archives hint="raw:AddArchive">
          <archive name="archive" />
          <archive name="recyclebin" />
        </archives>
        <Engines.HistoryEngine.Storage>
          <obj type="Sitecore.Data.SqlServer.SqlServerHistoryStorage, Sitecore.Kernel">
            <param connectionStringName="$(id)" />
            <EntryLifeTime>30.00:00:00</EntryLifeTime>
          </obj>
        </Engines.HistoryEngine.Storage>
        <Engines.HistoryEngine.SaveDotNetCallStack>false</Engines.HistoryEngine.SaveDotNetCallStack>
        <cacheSizes hint="setting">
          <data>20MB</data>
          <items>10MB</items>
          <paths>500KB</paths>
          <itempaths>10MB</itempaths>
          <standardValues>500KB</standardValues>
        </cacheSizes>
      </database>
    </databases>


    <!-- Add the specific update strategies for each data center-->
    <contentSearch>
      <indexUpdateStrategies>
        <onPublishEndAsyncProxyHongKong type="Sitecore.ContentSearch.Maintenance.Strategies.OnPublishEndAsynchronousStrategy, Sitecore.ContentSearch">
          <param desc="database">ProxyWebForHongKong</param>
          <CheckForThreshold>true</CheckForThreshold>
        </onPublishEndAsyncProxyHongKong>
        <onPublishEndAsyncProxyDallas type="Sitecore.ContentSearch.Maintenance.Strategies.OnPublishEndAsynchronousStrategy, Sitecore.ContentSearch">
          <param desc="database">ProxyWebForDallas</param>
          <CheckForThreshold>true</CheckForThreshold>
        </onPublishEndAsyncProxyDallas>
      </indexUpdateStrategies>




      <!-- Add the additional indexing strategies to the sitecore_web_index, so our additional regions get the messages to update Lucene-->
      <configuration type="Sitecore.ContentSearch.ContentSearchConfiguration, Sitecore.ContentSearch">
        <indexes hint="list:AddIndex">
          <index id="sitecore_web_index" type="Sitecore.ContentSearch.LuceneProvider.LuceneIndex, Sitecore.ContentSearch.LuceneProvider">
            <strategies hint="list:AddStrategy">
              <strategy ref="contentSearch/indexUpdateStrategies/onPublishEndAsyncProxyHongKong" />
              <strategy ref="contentSearch/indexUpdateStrategies/onPublishEndAsyncProxyDallas" />
            </strategies>
          </index>
        </indexes>
      </configuration>
      
      
    </contentSearch>
  </sitecore>
</configuration>

Faster Sitecore Publishing for Global Implementations

My post over at the main Rackspace developer blog about Sitecore publishing strategies was something I’d like to link to from here.

The crux of it is this diagram illustrating how we can make use of SQL Replication to move Sitecore content between data centers, but still take advantage of Sitecore publishing to promote content to the live site.  The blog at Rackspace discusses it in some detail, so I’ll just point out that the idea is to let SQL Server Replication do the hard (and potentially slow) work of synchronizing content around the planet, while Sitecore publishing can be the plain old HTTP publish operation we know and love from Sitecore:

2016-02-23-Sitecore-Enterprise-Architecture-For-Global-Publishing

Programmatically Setting the Sitecore Admin Password (and how to secure it)

For an enterprise building out a lot of Sitecore environments, where they’re setting up elastic-scaling or dynamic provisioning of Sitecore CM or CD servers, the default settings of the Sitecore install can leave something to be desired.

One such “something to be desired” is the handling of the password for the sitecore\admin account.  It’s puzzling why a parameter isn’t included in the Sitecore installer to set the sitecore\admin account password.  It feels trivial, but the process of logging in and manually changing the password through the Sitecore Client after a new installation can be a big pain in an automated setting.  Consider doing that for 10 CM environments . . . or beyond . . .

While there are ways to do it with a library like Sitecore Powershell Extensions (Set-UserPassword), we were looking for a method that didn’t require that Sitecore Powershell dependency.  I’m a big fan of minimalism when it comes to including modules and 3rd party dependencies, maybe to a fault, but having a method of setting a secure and unique Sitecore admin password that used the plain out-of-the-box Sitecore API was most desirable.

So we crafted this utility .ASPX page to drop into our sitecore/admin directory on the web server — and we’ll make a call to that page near the end of our environment provisioning setup.  If we want to be really fancy, we could delete this page after we’ve run it once, to tidy up after ourselves.  It’s designed, however, to only alter the original “b” password and won’t perform changes beyond making that initial change.

Without further ado, here’s the full .aspx code.  I’ll dive into a bit more detail afterwards.

1:  <%@ Language="C#" %>  
2:  <HTML>  
3:         
4:        public void Page_Load(object sender, EventArgs e)  
5:        {  
6:          object newPwd = Request.QueryString.Get("p");  
7:          if( object.ReferenceEquals( newPwd, null ) )  
8:          {  
9:            //no parameter provided  
10:          }  
11:          else  
12:          {  
13:            System.Web.Security.MembershipUser user = GetUser();      
14:            Sitecore.Diagnostics.Assert.IsNotNull((object) user, typeof (Sitecore.Security.Accounts.User));  
15:            user.ChangePassword("b", newPwd.ToString());  
16:            lblSummary.Text="New password set to " + newPwd.ToString() + " for UserName " + user.UserName;  
17:          }  
18:        }  
19:        System.Web.Security.MembershipUser GetUser()  
20:        {  
21:           return System.Web.Security.Membership.GetUser(@"sitecore\admin");  
22:        }  
23:      
24:    <body>  
25:     <form id="MyForm" runat="server">  
26:       <asp:Label runat="server" ID="lblSummary">  
27:         Pass the password you'd like to define for the sitecore\admin user to this page in the querystring as the "p" parameter.  
28:         <br /><br />  
29:         For example http://[domain-name]/sitecore/admin/SetPassword.aspx?p=NewPassword1  
30:         <br /><br />  
31:       </asp:Label>  
32:     </form>  
33:    </body>  
34:  </HTML>  

Some explanation:

  1. First off, why a lame old .ASPX page when there’s cool MVC approaches or at least code-behind techniques proven effective for over a decade of .Net development?  This page is a self-contained unit that doesn’t need compiled .dlls or anything else, so it’s an easily deployed option. It’s not fancy, but it doesn’t cause a recompilation if one adds it to an existing site and is the most simple tool for the job.
  2. I full qualify all the references instead of including assemblies at the top (maybe with the exception of the Request reference on line #6 — I guess I could have used HttpContext.Current.Request instead, to be fully consistent 🙂 ).  This is a habit I developed a long time ago, where other developers are frequently using the code as a reference or building on it as a proof-of-concept.  By using System.Web.Security.MembershipUser on line #13 I make it totally explicit where the MembershipUser lives in the API (same with Sitecore.Security.Accounts.User etc).  This page will be easier for other developers to work with as a result.
  3. On line #15, the call to user.ChangePassword, is something I pulled straight from the SitecoreMembershipProvider class defined in the Sitecore.Security namespace.  To find this method, I used a .Net reflection tool to inspect the Sitecore assemblies and poked around the MembershipProvider code in the Sitecore.Kernel.  I can’t say enough good things about using a decompiler to work with Sitecore — it’s an absolute must for serious Sitecore work.  Here’s a screen-shot of the decompiled output, for example:

Capture.JPG

This .aspx code intentionally works only on the sitecore\admin account, and it’s hard-coded to only change the default “b” password.  There are methods to extend this and make it truly dynamic, and make it more secure, but for our limited purpose of facilitating a one-time change to the Sitecore admin password for a freshly minted Sitecore CM server, this approach is clean and minimal.  This page could be called as a post install step, loading http://%5Bdomain-name%5D/sitecore/admin/SetPassword.aspx?p=NewPassword1 for example, and display the new password in plain text this one time for the new administrator.  I think, if one were to use this page in “the wild” it makes sense to escape that querystring value and not write it blindly to the page (avoiding potential cross-site scripting vulnerabilities), but there’s a lot more to say about security with this entire .ASPX page . . .

Securing this sort of thing

As Kam Figy pointed out, sending the password in plain text as a querystring value will — by default — have the querystring value added to the server IIS logs in plain text.  This is certainly not ideal, but the idea here is to establish a random strong password for a fresh Sitecore install.  Our assumption is that the site administrators would change the password from our randomly generated one, and maybe we can help them by forcing them to change it on first login (adding a <loggingin> processor to force a change, maybe?).   We could also look at passing the password some other way, in a custom host-header or via database.  Maybe we generate the new password in our ASPX page and communicate it back to our provisioning system some other way?  There are many avenues here, but since our real goal is to prevent the standard “b” password from allowing anyone to access a site that is freshly provisioned . . . I’m not too worried about it.  If it’s days or weeks until somebody actually logs in to the CM server, leaving the standard “b” password alone is a much more significant security liability.

Just one final note about securing utility pages like this, since I think it follows from the above.  We intentionally didn’t use the approach below because we want to allow an anonymous HTTP request to change the admin password.  That’s most likely a terrible idea for most any other use-case, so here’s what you could do to lock access to this page down:

Add the following to the top of the page:

1:  <%@ Language="C#" Inherits="Sitecore.sitecore.admin.AdminPage" %>  
2:  <HTML>  
3:    <script runat="server" language="C#">  
4:        protected override void OnInit(EventArgs args)  
5:        {  
6:          CheckSecurity(true);  
7:          base.OnInit(args);  
8:        }  

Line #1 includes the instruction to inherit the page from Sitecore.sitecore.admin.AdminPage class (that’s another good one to check out in a decompilation tool — it’s in the Sitecore.Client assembly!).  Here’s what the CheckSecurity method shows in that decompiled AdminPage:

Capture

On line #6 of my code for “Securing this sort of thing” I call to CheckSecurity(true) and we can see that the boolean true controls if we allow those in the Sitecore Developer role to also have access to this resource, or not.  The logic of this check is to redirect the visitor to the admin login screen if they’re not authenticated, effectively locking this page down from unauthorized access.  This is something I would include in nearly every administrative script I introduce for Sitecore; whether one allows developers or not, via the isDeveloperAllowed boolean parameter, is something to determine based on the purpose of the page.

Sitecore and Solr Cloud Implementations

I’ve been working on a project with Chris Sulham of Velir, supporting some of his Solr efforts for a mutual customer. He’s got a great series of posts on Solr Cloud and Sitecore; I recommend checking those out for a thorough exploration of Solr Cloud. I worked with another project using the Solr search provider and Patrick Perrone of Arke wrote up some good notes on the topic with Sitecore 8.  So much Solr!

Those two separate projects have a lot of Solr buzzing around in my head right now, and since official documentation from Sitecore is light on the topic, I wanted to record a few of my own thoughts relating to Solr and Sitecore:

  1. Chris writes how “Sitecore’s Solr search provider layer does not support this [querying Solr Cloud], so a load balancer is necessary” . . . while technically it’s true that the Sitecore search provider doesn’t support it, it’s more an issue with Solr.net than Sitecore.As I explained in a write-up for another project (yes, a 3rd Solr project recently!):
    To integrate a ZooKeeper Ensemble (Solr Cloud) with Sitecore, you point the ServiceBaseAddress setting in Sitecore ContentSearch configuration to the loadbalancer (19.20.21.22 in the below example):
        <setting name="ContentSearch.Solr.ServiceBaseAddress">
            <patch:attribute name="value">http://19.20.21.22:8983/solr</patch:attribute>
          </setting>
    
    The Sitecore Solr provider communicates with Solr through Solr.Net, which doesn’t support the failover feature in the case where the provided ServiceBaseAddress is not available.  To achieve failover protection, it’s recommended to use a load balancer and use the load balanced address in the ContentSearch.Solr.ServiceBaseAddress setting. This limitation is noted as a feature request for Solr.Net known as “Client side switching”: https://github.com/mausch/SolrNet/issues/96
    
  2. Since I’m speaking to issues of friction between Sitecore and Solr, I should mention that if the Solr server in the ContentSearch.Solr.ServiceBaseAddress setting is unavailable when the Sitecore app pool starts, this causes Sitecore to short-circuit the application start-up process on both the Content Authoring or Content Delivery environments.  All requests will be rejected making Sitecore unavailable.To clarify, this is only when Sitecore recycles the application pool and the Solr server is unavailable at the address set in ContentSearch.Solr.ServiceBaseAddress. The expected behaviour (empty search results since Solr isn’t running, etc) is not the current experience. If Solr becomes unavailable and Sitecore does not recycle the application pool, the system continues to function as expected (understandably with search and Solr related exceptions).Sitecore support can craft a workaround (one could reference ID 409005 with Sitecore); it’s worth a further discussion whether the support assembly is worth incorporating into an implementation.At a minimum, any system-wide restarts or disaster recovery documentation should mention that the Solr server be operational before the Sitecore application (or a Sitecore app pool recycle take place after confirmation of the availability of the Solr server).
  3. Customizations must be made to the SwitchOnRebuildSolrSearchIndex class to work with a Solr Cloud implementation.  Let me again copy my notes from another write-up:
    Sitecore includes a SwitchOnRebuildSolrSearchIndex feature that guards against inconsistent search results when a full reindex of Solr takes place; this feature doesn’t work with Solr Cloud, however, so care should be taken to avoid SwitchOnRebuildSolrSearchIndex when using Solr Cloud or pursue a customization to address this limitation.
    
    Specifically, the SWAP command used by the SwitchOnRebuildSolrSearchIndex is not supported by Solr Cloud.  One observes and error like the following . . .
    
       org.apache.solr.common.SolrException: 
        No such core: sitecore_core at 
        org.apache.solr.core.SolrCores.swap
        (SolrCores.java:215) 
    
    Additional info:
    Without Solr Cloud, the terms 'collection' and 'core' were matched 1-to-1. When configuring search Sitecore approached it as follows:
    index (for ex 'sitecore_web_index') <=> collection (for ex 'itembuckets') <=> core ('itembuckets')
    There is a relation 1-to-1-to-1. In this case, when using the SwitchOnRebuildSolrIndex class, one has the ability to switch cores when the full rebuild is being executed, it uses the SWAP action. It works because the relation is 1-to-1-to-1.  But in Solr Cloud, the relation can be 1-to-1-to-many, which means that a single collection can be represented by several cores. In this case, the SWAP action doesn't work properly. 
    
    As a workaround for this SwitchOnRebuildSolrSearchIndex limitation, one could create a custom SolrSearchIndex that can handle the switch with SolrCloud. One could use a system based on Aliases that are pointed to the inactive core for rebuilding. It works but know that Sitecore doesn’t make it easy due to limited extensibility options with the current SolrSearchIndex.  For example, the code that initializes the cores filters on known Sitecore types, this leads to having to create a dummy index configuration to have the cores included.  
    
  4. Another minor Solr gotcha is around the Sitecore Media Framework (SMF); SMF assumes Lucene will be the search provider, which means there is a config change necessary to enable the Solr provider to work smoothly with SMF. One needs to replace the Sitecore.ContentSearch.MediaFramework.Brightcove.config that comes with the framework with this one Sitecore.Support.ContentSearch.MediaFramework.Brightcove.SOLR.config — note that WordPress won’t let me upload .config files, so that links to a .doc file that you should rename to .config.Be sure the Sitecore.Support.ContentSearch.MediaFramework.Brightcove.SOLR.config file is located after the main Solr config file (Sitecore.ContentSearch.Solr.Indexes.config by default) in the App_config/Include folder. Also, in the attached .config file, change the core name to the one that matches your project:
    <index id="mediaframework_brightcove_index" type="Sitecore.ContentSearch.SolrProvider.SolrSearchIndex, Sitecore.ContentSearch.SolrProvider" patch:before="index[1]">
      <param desc="name">$(id)</param>
      <param desc="core">ChangeInYourImplementation</param>

Reviewing these 4 points . . .

  1. Client side switching limitation with Solr.net
  2. Solr must be running when Sitecore starts
  3. SwitchOnRebuildSolrSearchIndex customizaton
  4. Solr-specific Media Framework config file

. . . shouldn’t discourage projects from using the Solr provider or Solr Cloud with Sitecore. Solr overcomes a lot of the limitations of Lucene, freeing CM and CD servers from organizing indexes and processing query results. Solr helps you get more from your Sitecore IIS servers and is a smart choice for many multi-server Sitecore environments.

Working around the rough edges of Solr with Sitecore is usually well worth the effort! Solr Cloud, furthermore, is the enterprise approach to Solr and the direction we encourage implementations to take when it comes to Solr (vs the older master/slave approach with Solr). Chris has a great overview on the topic of Solr Cloud and Sitecore, so I think it’s an appropriate note to end on.

GeoIP Resolution For Sitecore Explained

A Sitecore implementation can use MaxMind lookups for GeoIP resolution; this is valuable information for marketers as it provides for geography-based segmentation in reports and decision making in the Sitecore Analytics package.  The Analytics.PerformLookup setting in the /App_Config/Include/Sitecore.Analytics.config file governs the lookup activity, and is true with an OOTB installation config file (so be careful!).

It’s recommended to only set this setting to true for one server in an implementation; set this setting to false for the other servers.  For scaled (multi-server) Sitecore installations, this is a common issue since the default config file enables Analytics.PerformLookup.

You can enable lookups on either a CM or a CD instance.  The Sitecore instance should be able to access the MaxMind web service (via the public internet) in order to complete the work.  Having more than one server configured as true for this setting can cause SQL deadlocks and other problems, as well as duplicating your traffic to MaxMind (and using any lookups you’re paying for).

Under the covers, this is what this PerformLookup setting is doing:

  • Sitecore periodically starts a task on the server to perform GeoIP lookups if the setting Analytics.PerformLookup is true in Sitecore.Analytics.config
    1. This task is considered the “Lookup Worker” task
    2. The frequency of the Lookup Worker running is governed by the Analytics.PerformLookup.Interval setting in Sitecore.Analytics.config
    3. The Lookup Worker takes records in the Sitecore “Visit” table and processes any with an EmptyLocationID value in the LocationID column
      • the EmptyLocationId is a special GUID that is generated on each environment, check the first record in the Locations table with a blank business name, country, etc if you want to see what GUID your system is using
    4. The goal of the Lookup Worker is to process Visit records with an EmptyLocationID and populate them with data, so they are no longer empty.  It does this by communicating with MaxMind.

This is only part of the picture, though, as it explains how the background agent determines GeoIP  information for Visits.

What about when a page loads that makes decisions based on the visitor’s GeoIP information; for example, news sites might target different Sitecore content to European visitors compared with Asian visitors.

The process of resolving a visitor’s GeoIP information is as follows:

  1. Attempt to retrieve the GeoIP information from the memory cache
  2. If there’s no data from step 1, retrieve GeoIP information from the Analytics database
  3. If there’s no data from step 2, request GeoIP information from the MaxMind geolocation service

Each of the above steps gets progressively slower; reading from memory >  reading from the database > reading from a remote web service.

When it comes to step 3, if a request is made to MaxMind during a visitor’s HTTP request, Sitecore doesn’t make that a synchronous call and wait for a response.  I’ve seen many websites perform slowly due to blocking web service calls, so this is a smart design decision by Sitecore.  If, however, you’re in a situation where you want to wait for the GeoIP data, refer to this KB article from Sitecore for two methods of addressing it: https://kb.sitecore.net/articles/320734.

If you don’t want to pursue either work-around in the Sitecore KB article, but are using geolocation targeting in your implementation, you should plan for a graceful fallback.  Provide for a generic set of data for non-geolocation targeted visitors.

Additional references on GeoIP resolution with Sitecore include: