I’ve been engaged on a few more Sitecore Commerce builds (Commerce 8.2.1 still as these have carried over from 2017) and found an interesting wrinkle the other day. At first, it looked like a MongoDB issue as contacts weren’t being properly added to Sitecore ListManager “Lists” for use in EXM, but after scratching beneath the surface it was a lot more interesting. I used a utility sent my way by Sitecore support — it’s a .zip that has a Sitecore 8.2 update-5 specific tool for seeing Sitecore Lists and their status in terms of what’s in MongoDB and what’s in content search indexes (Solr in my case).
The tool made it pretty clear that the data was being stored properly in MongoDB but NOT in the search index (the screenshot below shows “Contacts in index: 3” which is after we corrected the problem — initially the Contacts in index would only ever show 0 and that’s what helped to isolate the problem to Sitecore Content Search):
Another piece of evidence, in the Sitecore UI when we’d try to add a new contact to ListManager we’d see this message:
Please note that contacts in the list are currently being indexed, so not all contacts are available to view at this time. 0 out of 3 contacts are currently indexed.
Once we enabled verbose logging for search and examined the Search.log output, we see messages like this in the logs:
INFO Solr Query - ?q=(type_t:(contact) AND contact.tags_sm:(ContactLists\:\{B76B0E74-E94D-4EBB-F219-6A347C75520D\}))&start=0&rows=20&fl=contact.contactid_s,contact.identifier_t,contact.firstname_t,contact.surname_t,contact.preferredemail_t,contactscount_tl,_uniqueid,_datasource&fq=_indexname:(sitecore_analytics_index)
I bolded the contact.tags_sm criteria in the query as that turned out to be key. This is the query that Sitecore issues to Solr when trying to obtain contacts for ListManager.
Through considerable trial and error, Solr schema inspection, and just determination (and I think Dana [https://twitter.com/thesoftwarejedi] was the one who finally yelled “bingo” and discovered this), when we run this query directly against Solr, we would find our missing ListManagement contact:s
http://solr-server:8983/solr/sitecore_analytics_index/select?q=(type_t:(contact) AND contact.tags_tm:(ContactLists\:\{B76B0E74-E94D-4EBB-F219-6A347C75520D\}))&start=0&rows=20&fl=contact.contactid_s,contact.identifier_t,contact.firstname_t,contact.surname_t,contact.preferredemail_t,contactscount_tl,_uniqueid,_datasource&fq=_indexname:(sitecore_analytics_index
The contact.tags_tm is bold above, and that was the crux of our challenge.
Sitecore was indexing contacts using tags_tm while Sitecore queries were looking for tags_sm.
In Sitecore config file CommerceServer\CommerceServer.ContentSearch.Solr.DefaultIndexConfiguration.config is a fragment of XML like the following:
<typeMatches hint="raw:AddTypeMatch">
<typeMatch typeName="idCollection" type="System.Collections.Generic.List`1[[Sitecore.Data.ID, Sitecore.Kernel]]" fieldNameFormat="{0}_sm" multiValued="true" settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, Sitecore.ContentSearch.SolrProvider" />
<typeMatch typeName="textCollection" type="System.Collections.Generic.List`1[System.String]" fieldNameFormat="{0}_tm" multiValued="true" settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, Sitecore.ContentSearch.SolrProvider" />
The typeMatch for typeName=”textCollection” was the issue, along with how it duplicates the mapping for the System.Collections.Generic.List`1[System.String] type — and the many places that were using the textCollection returnType that depended on this typeMatch. I removed the typeMatch from the config file and updated any dependency on textCollection to use stringCollection instead and . . . magic . . . the contacts properly indexed into Solr and the contact.tags_sm criteria would match the new data.
According to Sitecore Support, this is a defect in the way Commerce search indexing is setup and it’s overlap with Sitecore ListManager (EXM in our case). Commerce should probably use a custom configuration section instead of modifying the default index configuration, but we’ll have to wait and see how this is implemented in a future patch or release.
For the time being, I’ve created the following Sitecore patch configuration file to remove the textCollection elements. This is preferable to editing the standard Sitecore configuration files that come with the product and will make for easier Sitecore upgrades or adjustments when (or if?) a true correction for this defect is released by Sitecore:
<?xml version="1.0" encoding="utf-8" ?> <configuration xmlns:patch="http://www.sitecore.net/xmlconfig/"> <sitecore> <contentSearch> <indexConfigurations> <defaultSolrIndexConfiguration type="Sitecore.ContentSearch.SolrProvider.SolrIndexConfiguration, Sitecore.ContentSearch.SolrProvider"> <fieldMap type="Sitecore.ContentSearch.SolrProvider.SolrFieldMap, Sitecore.ContentSearch.SolrProvider"> <typeMatches hint="raw:AddTypeMatch"> <typeMatch typeName="textCollection"> <patch:delete /> </typeMatch> </typeMatches> <fieldNames> <field fieldName="instocklocations"> <patch:attribute name="returnType">stringCollection</patch:attribute> </field> <field fieldName="outofstocklocations"> <patch:attribute name="returnType">stringCollection</patch:attribute> </field> <field fieldName="orderablelocations"> <patch:attribute name="returnType">stringCollection</patch:attribute> </field> <field fieldName="commerceancestornames"> <patch:attribute name="returnType">stringCollection</patch:attribute> </field> </fieldNames> <fieldTypes hint="raw:AddFieldByFieldTypeName"> <fieldType fieldTypeName="catalog selection control"> <patch:attribute name="returnType">stringCollection</patch:attribute> </fieldType> <fieldType fieldTypeName="child categories list control"> <patch:attribute name="returnType">stringCollection</patch:attribute> </fieldType> <fieldType fieldTypeName="child products list control"> <patch:attribute name="returnType">stringCollection</patch:attribute> </fieldType> <fieldType fieldTypeName="parent categories list control"> <patch:attribute name="returnType">stringCollection</patch:attribute> </fieldType> <fieldType fieldTypeName="relationship list control"> <patch:attribute name="returnType">stringCollection</patch:attribute> </fieldType> <fieldType fieldTypeName="variant list control"> <patch:attribute name="returnType">stringCollection</patch:attribute> </fieldType> </fieldTypes> </fieldMap> <documentOptions> <fields hint="raw:AddComputedIndexField"> <field fieldName="instocklocations"> <patch:attribute name="returnType">stringCollection</patch:attribute> </field> <field fieldName="outofstocklocations"> <patch:attribute name="returnType">stringCollection</patch:attribute> </field> <field fieldName="orderablelocations"> <patch:attribute name="returnType">stringCollection</patch:attribute> </field> <field fieldName="commerceancestornames"> <patch:attribute name="returnType">stringCollection</patch:attribute> </field> </fields> </documentOptions> </defaultSolrIndexConfiguration> </indexConfigurations> </contentSearch> </sitecore> </configuration>
Sitecore Architecture 