Sitecore and Solr Cloud Implementations

I’ve been working on a project with Chris Sulham of Velir, supporting some of his Solr efforts for a mutual customer. He’s got a great series of posts on Solr Cloud and Sitecore; I recommend checking those out for a thorough exploration of Solr Cloud. I worked with another project using the Solr search provider and Patrick Perrone of Arke wrote up some good notes on the topic with Sitecore 8.  So much Solr!

Those two separate projects have a lot of Solr buzzing around in my head right now, and since official documentation from Sitecore is light on the topic, I wanted to record a few of my own thoughts relating to Solr and Sitecore:

  1. Chris writes how “Sitecore’s Solr search provider layer does not support this [querying Solr Cloud], so a load balancer is necessary” . . . while technically it’s true that the Sitecore search provider doesn’t support it, it’s more an issue with Solr.net than Sitecore.As I explained in a write-up for another project (yes, a 3rd Solr project recently!):
    To integrate a ZooKeeper Ensemble (Solr Cloud) with Sitecore, you point the ServiceBaseAddress setting in Sitecore ContentSearch configuration to the loadbalancer (19.20.21.22 in the below example):
        <setting name="ContentSearch.Solr.ServiceBaseAddress">
            <patch:attribute name="value">http://19.20.21.22:8983/solr</patch:attribute>
          </setting>
    
    The Sitecore Solr provider communicates with Solr through Solr.Net, which doesn’t support the failover feature in the case where the provided ServiceBaseAddress is not available.  To achieve failover protection, it’s recommended to use a load balancer and use the load balanced address in the ContentSearch.Solr.ServiceBaseAddress setting. This limitation is noted as a feature request for Solr.Net known as “Client side switching”: https://github.com/mausch/SolrNet/issues/96
    
  2. Since I’m speaking to issues of friction between Sitecore and Solr, I should mention that if the Solr server in the ContentSearch.Solr.ServiceBaseAddress setting is unavailable when the Sitecore app pool starts, this causes Sitecore to short-circuit the application start-up process on both the Content Authoring or Content Delivery environments.  All requests will be rejected making Sitecore unavailable.To clarify, this is only when Sitecore recycles the application pool and the Solr server is unavailable at the address set in ContentSearch.Solr.ServiceBaseAddress. The expected behaviour (empty search results since Solr isn’t running, etc) is not the current experience. If Solr becomes unavailable and Sitecore does not recycle the application pool, the system continues to function as expected (understandably with search and Solr related exceptions).Sitecore support can craft a workaround (one could reference ID 409005 with Sitecore); it’s worth a further discussion whether the support assembly is worth incorporating into an implementation.At a minimum, any system-wide restarts or disaster recovery documentation should mention that the Solr server be operational before the Sitecore application (or a Sitecore app pool recycle take place after confirmation of the availability of the Solr server).
  3. Customizations must be made to the SwitchOnRebuildSolrSearchIndex class to work with a Solr Cloud implementation.  Let me again copy my notes from another write-up:
    Sitecore includes a SwitchOnRebuildSolrSearchIndex feature that guards against inconsistent search results when a full reindex of Solr takes place; this feature doesn’t work with Solr Cloud, however, so care should be taken to avoid SwitchOnRebuildSolrSearchIndex when using Solr Cloud or pursue a customization to address this limitation.
    
    Specifically, the SWAP command used by the SwitchOnRebuildSolrSearchIndex is not supported by Solr Cloud.  One observes and error like the following . . .
    
       org.apache.solr.common.SolrException: 
        No such core: sitecore_core at 
        org.apache.solr.core.SolrCores.swap
        (SolrCores.java:215) 
    
    Additional info:
    Without Solr Cloud, the terms 'collection' and 'core' were matched 1-to-1. When configuring search Sitecore approached it as follows:
    index (for ex 'sitecore_web_index') <=> collection (for ex 'itembuckets') <=> core ('itembuckets')
    There is a relation 1-to-1-to-1. In this case, when using the SwitchOnRebuildSolrIndex class, one has the ability to switch cores when the full rebuild is being executed, it uses the SWAP action. It works because the relation is 1-to-1-to-1.  But in Solr Cloud, the relation can be 1-to-1-to-many, which means that a single collection can be represented by several cores. In this case, the SWAP action doesn't work properly. 
    
    As a workaround for this SwitchOnRebuildSolrSearchIndex limitation, one could create a custom SolrSearchIndex that can handle the switch with SolrCloud. One could use a system based on Aliases that are pointed to the inactive core for rebuilding. It works but know that Sitecore doesn’t make it easy due to limited extensibility options with the current SolrSearchIndex.  For example, the code that initializes the cores filters on known Sitecore types, this leads to having to create a dummy index configuration to have the cores included.  
    
  4. Another minor Solr gotcha is around the Sitecore Media Framework (SMF); SMF assumes Lucene will be the search provider, which means there is a config change necessary to enable the Solr provider to work smoothly with SMF. One needs to replace the Sitecore.ContentSearch.MediaFramework.Brightcove.config that comes with the framework with this one Sitecore.Support.ContentSearch.MediaFramework.Brightcove.SOLR.config — note that WordPress won’t let me upload .config files, so that links to a .doc file that you should rename to .config.Be sure the Sitecore.Support.ContentSearch.MediaFramework.Brightcove.SOLR.config file is located after the main Solr config file (Sitecore.ContentSearch.Solr.Indexes.config by default) in the App_config/Include folder. Also, in the attached .config file, change the core name to the one that matches your project:
    <index id="mediaframework_brightcove_index" type="Sitecore.ContentSearch.SolrProvider.SolrSearchIndex, Sitecore.ContentSearch.SolrProvider" patch:before="index[1]">
      <param desc="name">$(id)</param>
      <param desc="core">ChangeInYourImplementation</param>

Reviewing these 4 points . . .

  1. Client side switching limitation with Solr.net
  2. Solr must be running when Sitecore starts
  3. SwitchOnRebuildSolrSearchIndex customizaton
  4. Solr-specific Media Framework config file

. . . shouldn’t discourage projects from using the Solr provider or Solr Cloud with Sitecore. Solr overcomes a lot of the limitations of Lucene, freeing CM and CD servers from organizing indexes and processing query results. Solr helps you get more from your Sitecore IIS servers and is a smart choice for many multi-server Sitecore environments.

Working around the rough edges of Solr with Sitecore is usually well worth the effort! Solr Cloud, furthermore, is the enterprise approach to Solr and the direction we encourage implementations to take when it comes to Solr (vs the older master/slave approach with Solr). Chris has a great overview on the topic of Solr Cloud and Sitecore, so I think it’s an appropriate note to end on.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s