Sitecore and Solr Cloud Implementations

I’ve been working on a project with Chris Sulham of Velir, supporting some of his Solr efforts for a mutual customer. He’s got a great series of posts on Solr Cloud and Sitecore; I recommend checking those out for a thorough exploration of Solr Cloud. I worked with another project using the Solr search provider and Patrick Perrone of Arke wrote up some good notes on the topic with Sitecore 8.  So much Solr!

Those two separate projects have a lot of Solr buzzing around in my head right now, and since official documentation from Sitecore is light on the topic, I wanted to record a few of my own thoughts relating to Solr and Sitecore:

  1. Chris writes how “Sitecore’s Solr search provider layer does not support this [querying Solr Cloud], so a load balancer is necessary” . . . while technically it’s true that the Sitecore search provider doesn’t support it, it’s more an issue with Solr.net than Sitecore.As I explained in a write-up for another project (yes, a 3rd Solr project recently!):
    To integrate a ZooKeeper Ensemble (Solr Cloud) with Sitecore, you point the ServiceBaseAddress setting in Sitecore ContentSearch configuration to the loadbalancer (19.20.21.22 in the below example):
        <setting name="ContentSearch.Solr.ServiceBaseAddress">
            <patch:attribute name="value">http://19.20.21.22:8983/solr</patch:attribute>
          </setting>
    
    The Sitecore Solr provider communicates with Solr through Solr.Net, which doesn’t support the failover feature in the case where the provided ServiceBaseAddress is not available.  To achieve failover protection, it’s recommended to use a load balancer and use the load balanced address in the ContentSearch.Solr.ServiceBaseAddress setting. This limitation is noted as a feature request for Solr.Net known as “Client side switching”: https://github.com/mausch/SolrNet/issues/96
    
  2. Since I’m speaking to issues of friction between Sitecore and Solr, I should mention that if the Solr server in the ContentSearch.Solr.ServiceBaseAddress setting is unavailable when the Sitecore app pool starts, this causes Sitecore to short-circuit the application start-up process on both the Content Authoring or Content Delivery environments.  All requests will be rejected making Sitecore unavailable.To clarify, this is only when Sitecore recycles the application pool and the Solr server is unavailable at the address set in ContentSearch.Solr.ServiceBaseAddress. The expected behaviour (empty search results since Solr isn’t running, etc) is not the current experience. If Solr becomes unavailable and Sitecore does not recycle the application pool, the system continues to function as expected (understandably with search and Solr related exceptions).Sitecore support can craft a workaround (one could reference ID 409005 with Sitecore); it’s worth a further discussion whether the support assembly is worth incorporating into an implementation.At a minimum, any system-wide restarts or disaster recovery documentation should mention that the Solr server be operational before the Sitecore application (or a Sitecore app pool recycle take place after confirmation of the availability of the Solr server).
  3. Customizations must be made to the SwitchOnRebuildSolrSearchIndex class to work with a Solr Cloud implementation.  Let me again copy my notes from another write-up:
    Sitecore includes a SwitchOnRebuildSolrSearchIndex feature that guards against inconsistent search results when a full reindex of Solr takes place; this feature doesn’t work with Solr Cloud, however, so care should be taken to avoid SwitchOnRebuildSolrSearchIndex when using Solr Cloud or pursue a customization to address this limitation.
    
    Specifically, the SWAP command used by the SwitchOnRebuildSolrSearchIndex is not supported by Solr Cloud.  One observes and error like the following . . .
    
       org.apache.solr.common.SolrException: 
        No such core: sitecore_core at 
        org.apache.solr.core.SolrCores.swap
        (SolrCores.java:215) 
    
    Additional info:
    Without Solr Cloud, the terms 'collection' and 'core' were matched 1-to-1. When configuring search Sitecore approached it as follows:
    index (for ex 'sitecore_web_index') <=> collection (for ex 'itembuckets') <=> core ('itembuckets')
    There is a relation 1-to-1-to-1. In this case, when using the SwitchOnRebuildSolrIndex class, one has the ability to switch cores when the full rebuild is being executed, it uses the SWAP action. It works because the relation is 1-to-1-to-1.  But in Solr Cloud, the relation can be 1-to-1-to-many, which means that a single collection can be represented by several cores. In this case, the SWAP action doesn't work properly. 
    
    As a workaround for this SwitchOnRebuildSolrSearchIndex limitation, one could create a custom SolrSearchIndex that can handle the switch with SolrCloud. One could use a system based on Aliases that are pointed to the inactive core for rebuilding. It works but know that Sitecore doesn’t make it easy due to limited extensibility options with the current SolrSearchIndex.  For example, the code that initializes the cores filters on known Sitecore types, this leads to having to create a dummy index configuration to have the cores included.  
    
  4. Another minor Solr gotcha is around the Sitecore Media Framework (SMF); SMF assumes Lucene will be the search provider, which means there is a config change necessary to enable the Solr provider to work smoothly with SMF. One needs to replace the Sitecore.ContentSearch.MediaFramework.Brightcove.config that comes with the framework with this one Sitecore.Support.ContentSearch.MediaFramework.Brightcove.SOLR.config — note that WordPress won’t let me upload .config files, so that links to a .doc file that you should rename to .config.Be sure the Sitecore.Support.ContentSearch.MediaFramework.Brightcove.SOLR.config file is located after the main Solr config file (Sitecore.ContentSearch.Solr.Indexes.config by default) in the App_config/Include folder. Also, in the attached .config file, change the core name to the one that matches your project:
    <index id="mediaframework_brightcove_index" type="Sitecore.ContentSearch.SolrProvider.SolrSearchIndex, Sitecore.ContentSearch.SolrProvider" patch:before="index[1]">
      <param desc="name">$(id)</param>
      <param desc="core">ChangeInYourImplementation</param>

Reviewing these 4 points . . .

  1. Client side switching limitation with Solr.net
  2. Solr must be running when Sitecore starts
  3. SwitchOnRebuildSolrSearchIndex customizaton
  4. Solr-specific Media Framework config file

. . . shouldn’t discourage projects from using the Solr provider or Solr Cloud with Sitecore. Solr overcomes a lot of the limitations of Lucene, freeing CM and CD servers from organizing indexes and processing query results. Solr helps you get more from your Sitecore IIS servers and is a smart choice for many multi-server Sitecore environments.

Working around the rough edges of Solr with Sitecore is usually well worth the effort! Solr Cloud, furthermore, is the enterprise approach to Solr and the direction we encourage implementations to take when it comes to Solr (vs the older master/slave approach with Solr). Chris has a great overview on the topic of Solr Cloud and Sitecore, so I think it’s an appropriate note to end on.

Sitecore MVC ViewRenderings From the Ground Up

I was working with a customer who had no real .Net expertise, but needed to get up and running quickly with Sitecore. The concepts of MVC were familiar to the customer, so it made sense to introduce them to Sitecore ViewRenderings and all that can be done without compiling any custom code in Visual Studio.  Sitecore ViewRenderings connect with Sitecore content to define a Model item by convention, requiring no custom configuration on the part of the customer.

What follows is a basic example illustrating what can be done with Sitecore ViewRenderings without any custom code compilation.  I take things from the top with a clean Sitecore 8 install, since the customer was new to the Sitecore platform . . .

1. Define Sitecore Templates

Taking a fresh installation of Sitecore 8.0 rev 150223, one defines the Sitecore templates that will represent our content. As the picture below shows, we’ll add three templates under the Sitecore/Templates/User Defined area of the content tree: News, Word, and WordPage.

templates

  1. The News template will be for an example of querying a list of Sitecore content, and for simplicity we’ll just define a single field named “Headline” for that template.
  2. The Word template will contain fields for Title, Subtitle, and Image – in our example we’ll pick from a collection of Word items to associate with our sample page.
  3. Finally, the WordPage template will be be the container that represents the page on the website. This shows a “UI Template” approach that is common; implemenations of Sitecore will usually have templates defining the structure of the data (such as our News and Word templates) as well as UI only (such as our WordPage template).

Once this example is finished, it should all make more sense.

The News template should have a single Headline field defined as a Single-Line-Text type:

template1

The Word template should have three fields as follows:

template2

The WordPage template should have two fields as follows:

template3

To customize the icon used for these templates, I used the Icon button on the Configure ribbon in Sitecore. This is optional as the icon doesn’t impact the functionality, but it makes life easier for content authors.

2 Add Media To Media Library

In the Word template we defined an Image field and specified the source as /sitecore/media library/word images. We need to create that folder under the Media Library and upload some sample images into it. The following screen capture shows that I placed 4 images in that directory. For what we’re doing, you can disregard the red “1 warning” messages as this relates to image alt text on the media items.

media

3 Create Sample Sitecore Content

In the Sitecore content tree, under sitecore/content I create a new folder named “Global” to hold data that is common to the entire implementation. I create News and Words folders under this Global folder.

verdant

I created a few sample items under the News folder of the /sitecore/templates/User Defined/News type, and sample items under the Words folder of the /sitecore/templates/User Defined/Word type. Populate each field of these sample items with data so our example has content to display. Be sure to name one of the News items “Story1” as that’s hard-coded into our ViewRendering (discussed later on).

Under the /sitecore/home location in the content tree, I created an item based on our /sitecore/templates/User Defined/WordPage type and populated the Summary and Body with sample text.

samplepage

After saving this SamplePage item, if you do a Sitecore Publish, this page will be accessed by the URL http://%5Bserver name]/SamplePage. At this stage in the process, however, there are no Sitecore Presentation Layout Details that associate HTML markup with this content. All we have right now is the skeleton. One would see the Sitecore error page if one tried to view http://%5Bserver name]/SamplePage at this juncture.

4 Create Files With HTML

On the file system, under the Website directory where Sitecore installed to will be a Views directory. We’re going to add the following:

  • DictionarySample directory with a file named “WordPage.cshtml”
  • Layouts directory with a file named “Dictionary Layout.cshtml”
  • A special Web.config file that enables MVC for this Views directory, I will send you a .zip file with the Web.config to use.

5 DictionarySample/WordPage.cshtml

Use Notepad, or any text editor, to place the following in WordPage.cshtml file:

 <!--  we're using Sitecore’s default RenderingModel with the ViewRendering. It provides two key preoperties:  
       -the data source is available as a property Model.Item  
       -the context item is available as a property Model.PageItem  
 -->  
   <div style="width:900px;background-color:#ededed;">  
     Hard-coded text in the WordPage.cshtml file . . .  
     <h1>@Html.Sitecore().Field("Summary", Model.PageItem)</h1>  
     <div style="float:right;width:300px;background-color:#dfdfdf;padding:20px;">  
       Datasource word: <b>@Html.Sitecore().Field("Title", Model.Item)</b>  
       <br><br>  
       <div style="float:right;padding:10px;border:1px solid #000;">  
         @Html.Sitecore().Field("Image", Model.Item)  
       </div>  
       @Html.Sitecore().Field("SubTitle", Model.Item)  
     </div>  
     @Html.Sitecore().Field("Body", Model.PageItem)  
     <hr>  
     <h1>News</h1>  
     <!-- examples of getting content not linked to this context item or datasource in any way -->  
     Single item: @Sitecore.Configuration.Factory.GetDatabase("web").GetItem("/sitecore/content/global/news/story1")["Headline"]  
     <ul>  
       @foreach (var newsItem in @Sitecore.Configuration.Factory.GetDatabase("web").SelectItems("/sitecore/content/global/news/*"))  
       {  
       <li>  
         <a href="#">@Html.Sitecore().Field("Headline", newsItem)</a>  
       </li>  
       }  
     </ul>  
   </div>  

I used Visual Studio .net to author this file, as the color highlighting is helpful, but this could be done in any text editor.
I’ll hold off on explaining each part of this file for the time being, but this is the only logic one needs to use the default MVC ViewRendering in Sitecore. No C# class files and no compilation is necessary.

6 Layouts/Dictionary Layout.cshtml
Use Notepad, or any text editor, to place the following in Dictionary Layout.cshtml:

 <!DOCTYPE html>  
 <html>  
 <head>  
   <title>Dictionary Title</title>  
   <style type="text/css">  
     body{  
       font-family:Tahoma;  
       font-size:.9em;  
     }  
   </style>  
  </head>  
  <body>  
    <div style="width:100%;background-color:#f1edd1;text-align:center;padding:20px;">  
      This could be your static header text  
    </div>     
  <form method="post" runat="server" id="mainform">  
    <div class="container">  
    @Html.Sitecore().Placeholder("main-content")  
    </div>  
  </form>  
    <div style="width:100%;background-color:#f1edd1;text-align:center;padding:20px;">  
      This could be your static footer text  
    </div>     
  </body>  
 </html>  

I’ll hold off on explaining each part of this file for the time being, but this defines the basic HTML for the site and through the “main-content” placeholder our WordPage.cshtml can be inserted.

7 Define Presentation Items in Sitecore
Under sitecore/Layout/Layouts create a new item of the /sitecore/templates/System/Layout/Layout type and specify the path to be /Views/Layouts/Dictionary Layout.cshtml.
layout

Under sitecore/Layout/Renderings create a folder and a new item named WordPageViewRendering of the /sitecore/templates/System/Layout/Renderings/View rendering type under your custom folder:
folder

8 Associate Presentation Layout Details in Sitecore
With the physical .cshtml files in place and the Sitecore presentation items defined (the layout and the ViewRendering), we will set the Presentation Layout Details on the /sitecore/content/Home/SamplePage item.
In the Sitecore Client, on the Presentation ribbon is the button for “Details:”
details

Clicking this button will display the Layout Details page. Click on “Edit” on the Default device to assign details. This screen will load:
blank

From the dropdownlist, select our custom Layout and click OK:
dict

Switch to the Controls tab of the Device Editor and click Add:
device

Choose the WordPageViewRendering and specify “main-content” as the Placeholder:
viewrendering

Click Select. From the Device Editor screen, click Edit:
deviceedit

On the Control Properties screen, choose to define a Data Source via the “Browse” link. Select a Word in the sitecore/Content/Global/Words folder. This is how one defines a Datasource in Sitecore, and the ViewRendering will make that selected Word item available via the Model.Item property in the .cshtml file.

Click OK after choosing a Word. Click OK a few more times to accept all the changes we made to the Presentation Details.

9 View The Page
We need to do a Sitecore Publish to make this SamplePage visible to the public. We will do a Smart Publish from the Publish ribbon > Publish site button. Once the publish is finished, depending on the text you entered for content in the sample items, and the Word you set as Datasource for the WordPageViewRendering, you will see something like this when requesting SamplePage from a browser:
page