EXM 3.1.1 installation

Getting this version (for Sitecore 8.0 update 4) of the Email Experience Manager to work takes a few more steps than are in the provided documentation. These steps can be found throughout the internet but I thought I would collect them here.

Subscription Form type is unavailable

EXM comes with a sublayout called the Subscription Form. This is an out-of-the-box control that will enter the user into a newsletter subscription engagement plan with a two step sign up process. Unfortunately the type for the view (Subscription Form.ascx) is not available in the EXM assemblies. The code-behind file is provided, but it has errors in it and will not compile.

To fix this we just need to fix the cs file (Subscription Form.ascx.cs) or provide this type elsewhere. The solution for this can be found at https://community.sitecore.net/developers/f/10/t/1923#pi214filter=all&pi214scroll=false

Change this:

protected override void OnInit(EventArgs e)
this.recipientRepository = EcmFactory.GetDefaultFactory().Bl.RecipientRepository;

to this:

protected override void OnInit(EventArgs e)
this.recipientRepository = RecipientRepository.GetDefaultInstance();

And fully qualify the type of the Tracker.Current.Contact.ContactId:

protected virtual ID ContactId
if (!Email.Visible || string.IsNullOrEmpty(Email.Text))
return new ID(Sitecore.Analytics.Tracker.Current.Contact.ContactId);

You may also need to add a reference to Sitecore.Analytics.Model

Rendering webforms controls in an MVC site

This problem is not specific to EXM, but if you’re site uses MVC it is required to get the Subscription Form into a page. This solution can be found at http://www.chrisvandesteeg.nl/2014/02/11/usercontrol-renderings-in-a-sitecore-mvc-website-wffm-for-mvc/

Add the UserControlRenderer and GetUserControlRenderer classes to your solution, and patch the mvc.getRenderer pipeline with mvc.usercontrolrenderer.config and you’re good to go.

The source can be found at https://github.com/efocus-nl/sitecore-mvc-usercontrolrenderer

Contact locking in MongoDB

Our developer machines will typically have multiple sites running on the same instance of MongoDB. Production environments frequently  have multiple delivery servers. Each Sitecore instance/cluster will need to provide a key to identify the owner of each document. These keys are set in App_Config/Include/Sitecore.Analytics.Tracking.config. Using the host name is sufficient, although any string may be used for the cluster name.

The solution for this problem can be found at https://kb.sitecore.net/articles/965127

<setting name="Analytics.ClusterName" value="siteHostName" />

If you started using the site with an empty string for this setting, the existing Mongo databases will need to be dropped and recreated (automatically) in order for the Cluster Name to take effect.

Backbone javascript error

This only happened after I installed the Launch Sitecore module, but it is worth noting. Click on the Create button on the EXM Start Page, choose a one-time message, select a template and enter a message name. Clicking on the Create button in this dialog resilts in an error in backbone:

The solution for this problem can be found at https://community.sitecore.net/developers/f/10/p/2623/7709#7709. The casing of the linkManager Urls and the javscript that uses those values need to be in sync with one another.

        <add name="sitecore">
            <x:attribute name="lowercaseUrls">true</x:attribute>

If lowercaseUrls is true, this script needs to be fixed:

Setting lowercaseUrls to false may fix this problem, but it may also cause new errors to arise. I left the value as true and fixed the code that was failing as a result.

There is a switch that expects specific casing:

switch (createType) {
    case "ExistingTemplate":
        this.createExistingTemplate(contextApp, eventInfo);
    case "ExistingPage":
        this.createExistingPage(contextApp, eventInfo);
    case "ImportHtml":

It needs to be changed so the casing is ignored:

switch (createType.toLowerCase()) {
    case "existingtemplate":
        this.createExistingTemplate(contextApp, eventInfo);
    case "existingpage":
        this.createExistingPage(contextApp, eventInfo);
    case "importhtml":

With all of these fixes in place the module will behave as expected. Happy coding!


News Mover for the Web Database

I recently needed to create a process that would run in the CD (content delivery) environment. This process will create items that need to be organized with folders and the CD environment only has access to the Web database.

We decided to use the News Mover module to accomplish this. it does a great job of organizing items by using the value of a date field on the item to create configurable folders that correspond to the date. Unfortunately it does not work on the Web database. Well, it didn’t. Until now.

The Problem

The configuration for News Mover adds an item:saved event handler. This handler has a <database> node that is set to master by default. Unfortunately this setting is not actually used by the code at any time.

<event name="item:saved">
	<handler type="Sitecore.Sharedsource.Tasks.NewsMover, Sitecore.Sharedsource.NewsMover" method="OnItemSaved">
		<templates hint="raw:AddTemplateConfiguration">
			<template id="user defined/newsarticle" sort="Descending">
				<YearTemplate formatString="yyyy">Common/Folder</YearTemplate>
				<MonthTemplate formatString="MMMM">Common/Folder</MonthTemplate>
				<DayTemplate formatString="dd">Common/Folder</DayTemplate>

Here we see that the database name is never set to anything by the code:

The Solution

The News Mover template settings are consumed via the Configuration Factory. For flexibility I decided to add a database property to each template definition so they can be configured in a more granular manner. Now each TemplateConfiguration has its own Database property that it will use for its operations.

Updated config:

<event name="item:saved">
	<handler type="Sitecore.Sharedsource.Tasks.NewsMover, Sitecore.Sharedsource.NewsMover" method="OnItemSaved">
		<templates hint="raw:AddTemplateConfiguration">
			<template id="user defined/newsarticle" sort="Descending">
				<YearTemplate formatString="yyyy">Common/Folder</YearTemplate>
				<MonthTemplate formatString="MMMM">Common/Folder</MonthTemplate>
				<DayTemplate formatString="dd">Common/Folder</DayTemplate>

The TemplateConfigurationBuilder will read this new property and use the value to get a Sitecore Database that will be passed into the TemplateConfiguration constructor. Now each template definition can be set to use any database that the environment has access to.

Source Code

I’ve committed these changes in a fork of the News Mover source: https://github.com/NotIsNull/NewsMover. A pull request has been submitted to the original author so it can be incorporated into the official source: https://github.com/JimmieOverby/NewsMover.

Sitecore MVC – Multiple Forms

With Sitecore, forms can be presented to the page as either a View Rendering or a Controller Rendering. This article will focus on forms in Controller Renderings.

Sitecore MVC renders the page with different renderings, potentially including multiple controller renderings. In ‘pure’ ASP.Net MVC a form is always posted to a specific action, which is marked with the [HttpPost] attribute. This is not possible in Sitecore because the page is rendered through the rendering pipeline and not through a single action. While there are a few ways to do this with Sitecore, they get away from the ASP.Net MVC way of doing things.

Two forms on one page is a problem

When two (or more) forms are on a MVC page and one of them is submitted, all available controller actions are evaluated and those decorated with the [HttpPost] ActionMethodSelector will be selected over all other actions. This causes all form submission actions to be handled when we only want to handle the one that was submitted by the user.

Fear not! There is a way to validate each post action to determine if it is the correct action for the current form post.

Create an extension method that will render a hidden input with the uniqueId of the current rendering:

public static class SitecoreHelperExtensions
    public static MvcHtmlString RenderingToken(this SitecoreHelper helper)
        if (helper.CurrentRendering == null) return null;

        var tagBuilder = new TagBuilder("input");
        tagBuilder.Attributes["type"] = "hidden";
        tagBuilder.Attributes["name"] = "uid";
        tagBuilder.Attributes["value"] = helper.CurrentRendering.UniqueId.ToString();

        return new MvcHtmlString(tagBuilder.ToString(TagRenderMode.SelfClosing));

Call this method inside an MVC form to add a hidden input with the rendering’s uniqueId to the form:


The result of this method looks like this:

<input name="uid" type="hidden" value="c151068e-9bb9-4899-a470-27560f551338"/>

Create an attribute that will check this uniqueId value to make sure that the form posts to the correct action:

public class ValidRenderingTokenAttribute : ActionMethodSelectorAttribute
    public override bool IsValidForRequest(ControllerContext controllerContext, System.Reflection.MethodInfo methodInfo)
        var rendering = RenderingContext.CurrentOrNull;
        if (rendering == null) return false;

        Guid postedId;
        return Guid.TryParse(controllerContext.HttpContext.Request.Form["uid"], out postedId) && postedId.Equals(rendering.Rendering.UniqueId);

Decorate POST actions with the ValidRenderingToken attribute:

public ActionResult Index(FormModel model)
    model.Message = "Valid rendering token found here";
    return View(model);

Ensure that the default actions for your controllers do not have the [HttpGet] attribute, as this will confuse the Sitecore rendering pipeline and an error will be thrown indicating that it cannot find an action that permits a post. It either needs an unmarked action or an action with [HttpPost]

// note: no attribute here
public ActionResult Index()
    var model = GetModel();
    return View(model);

For information on Sitecore Forms see Martina Welanders article on Posting Forms in Sitecore MVC: Part 1 – View Renderings and part 2 – Controller Renderings.

Get response from custom Solr requestHandler with .NET

This is a follow-up to my previous post: Configuring Solr to provide search suggestions

So everything is set up in Solr, but now we need to get the response in our .NET code.

Assuming the following:

using Newtonsoft.Json.Linq;
using SolrNet.Impl;
var solrRoot = "http://localhost:3664/solr";
var suggestIndexName = "suggest_index";
var requestHandlerName = "/suggest";
var suggestUrl = string.Format("/{0}{1}", suggestIndexName, requestHandlerName);

All we need to do is set up the querystring parameters, instantiate the SolrConnection and do a GET.

var query = new SuggestQuery
   SuggestTerm = "partial string to get suggestions for",
   PageSize = 10
var parameters = new Dictionary<string, string>
   {"q", query.SuggestTerm}, // the string we're getting suggestions for
   {"wt", "json"}      // the response format, can also be "xml"
var solrConnection = new SolrConnection(solrRoot);

var response = solrConnection.Get(suggestUrl, parameters);

In this case the response is a json string. I used Newtonsoft.Json to parse the response into a JObject. Then I can traverse the fields/nodes and finally cast the suggestions into a concrete type.

Since I have two suggesters set up (a Fuzzy suggester and an Infix suggester), I’ll get two sets of suggestions. My SolrSuggest Suggestion type is set up with overrides for Equals and GetHashCode to facilitate the comparing of these items. This way I can do a .Union on the two lists to remove duplicate terms, then it can be ordered and truncated (page size or max count).

var response = new SolrSuggest(); // custom type defined below

var responseHeader = JObject.Parse(solrSuggestReponse); // Newtonsoft.JObject
var suggestions = responseHeader["suggest"];

var infixSuggester = suggestions["infixSuggester"];
var infixTermSuggestions = infixSuggester[query.SuggestTerm]; // this field will have the name of the term that was searched.

var infix = jsDeserializer.Deserialize<SolrSuggest>(infixTermSuggestions.ToString()); // Deserialize to custom Type

var fuzzySuggester = suggestions["fuzzySuggester"];
var fuzzyTermSuggestions = fuzzySuggester[query.SuggestTerm]; // this field will have the name of the term that was searched.

var fuzzy = jsDeserializer.Deserialize<SolrSuggest>(fuzzyTermSuggestions.ToString());

// .Union will add items from the second list to the first list while comparing them to prevent the addition of duplicate items
response.suggestions = infix.suggestions.Union(fuzzy.suggestions).OrderByDescending(s => s.weight).Take(query.PageSize).ToList();

response.numFound = response.suggestions.Count;

return response;

And this is the structure I used to deserialize the TermSuggestions to C# objects:

public class Suggestion
   public string term { get; set; }
   public int weight { get; set; }
   public string NormalizedTerm
      // The infix suggester will wrap the suggest term within the suggestions in <b> tags
      get { return term.Replace("<b>", string.Empty).Replace("</b>", string.Empty); }

   public override bool Equals(object otherSuggestion)
      var other = (Suggestion)otherSuggestion;
      return other != null && other.NormalizedTerm == NormalizedTerm;

   public override int GetHashCode()
      return NormalizedTerm.GetHashCode();

public class SolrSuggest
   public int numFound { get; set; }
   public List<Suggestion> suggestions { get; set; }
   public SolrSuggest()
      suggestions = new List<Suggestion>();

Thanks for reading, I hope this helps someone out there.

Configuring Solr to provide search suggestions

I needed to provide search term suggestions based on characters that the user has typed into the search box. Doing this is pretty easy with Solr, an open source enterprise search platform, powered by Java, Apache and Lucene.

If you’re using a version prior to 4.8, this can be accomplished using the SpellCheckComponent. See this document for details.

As of 4.8 a new component is available, the solr.SuggestComponent. This post will go through the steps to configure an index to provide search suggestions using this component. In my case I created a separate index to handle this, it could be combined into an existing index such as sitecore_web_index (or any other custom indexes you may be using), depending on what your needs are.

Define the schema for the index:

In order to create smaller documents I trimmed the fields down to the bare minimums. This is done in schema.xml.

    <field name="_content" type="text_general" indexed="true" stored="false" />
    <field name="_database" type="string" indexed="true" stored="true" />
    <field name="_uniqueid" type="string" indexed="true" stored="true" required="true" />
    <field name="_name" type="text_general" indexed="true" stored="true" />
    <field name="_indexname" type="string" indexed="true" stored="true" />
    <field name="_version" type="string" indexed="true" stored="true" />
    <field name="_version_" type="long" indexed="true" stored="true" />

Then I added two fields that will be used by the suggester. One to store the suggestion text and another to store the weight of that suggestion. The suggestion field should be a text type and the weight field should be a float type. Both need to be stored in the index. In this case these fields get their values form corresponding fields in our sitecore instance. These fields can be added to documents based on your specific indexing strategy.

<field name="term" type="text_general" indexed="true" stored="true" />
<field name="weight" type="float" indexed="true" stored="true" />

Define a custom field type for the suggest component:

Next we need to add a new type that the suggester will use to analyze and build the suggestion fields. This particular type will remove all non alphanumeric characters and be case-insensitive as well as tokenizing the contents of the field. This is not strictly necessary, existing types may be used. Again, this is done in schema.xml.

<fieldType name="suggestType" class="solr.TextField" positionIncrementGap="100">
        <charFilter class="solr.PatternReplaceCharFilterFactory" pattern="[^a-zA-Z0-9]" replacement=" " />
        <tokenizer class="solr.WhitespaceTokenizerFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>

Define the suggest component for the index:

Now that we have the schema set up, we need to define a searchComponent that will do the suggesting. This is done in solrconfig.xml.

Add the following to the <config> node:

<searchComponent name="suggest" class="solr.SuggestComponent">
    <lst name="suggester">
        <str name="name">fuzzySuggester</str>
        <str name="lookupImpl">FuzzyLookupFactory</str>
        <str name="storeDir">fuzzy_suggestions</str>
        <str name="dictionaryImpl">DocumentDictionaryFactory</str>
        <str name="field">term</str>
        <str name="weightField">weight</str>
        <str name="suggestAnalyzerFieldType">suggestType</str>
        <str name="buildOnStartup">false</str>
        <str name="buildOnCommit">false</str>
    <lst name="suggester">
        <str name="name">infixSuggester</str>
        <str name="lookupImpl">AnalyzingInfixLookupFactory</str>
        <str name="indexPath">infix_suggestions</str>
        <str name="dictionaryImpl">DocumentDictionaryFactory</str>
        <str name="field">term</str>
        <str name="weightField">weight</str>
        <str name="suggestAnalyzerFieldType">suggestType</str>
        <str name="buildOnStartup">false</str>
        <str name="buildOnCommit">false</str>


In this case we’re setting up a suggest component that has two suggester data sources available to it.

  • The first uses the FuzzyLookupFactory: a FST-based sugester (Finite State Transducer) which will match terms starting with the provided characters while accounting for potential misspellings. This lookup implementation will not find terms where the provided characters are in the middle.
  • The second uses the AnalyzingInfixLookupFactory: which will look inside the terms for matches. Also the results will have <b> highlights around the provided terms inside the suggestions.

Using a combination of methods, we can get more complete results. Additional suggester implementations are available:

  • WFSTLookup: offers more fine-grained control over results ranking than FST
  • TSTLookup: “a simple, compact trie-based lookup”. Whatever that means.
  • JaspellLookup: see the Jaspell source.

See the Suggester Documentation for more details on the different types of Lookup Implementations. They each have properties unique to their implementation.

storeDir and indexPath

These parameters define the directory where the suggester structure will be stored after it’s built. This parameter should be set so the data is available on disc without rebuilding.


The field to get the suggestions from. This could be a computed or a copy field.


As of Solr 5.1 this field is optional. In previous versions this field is required. If no proper weight value is available, a workaround is to define a float field in your schema and use that. Even if this field is never added to a document the code will compensate.

threshold (not used in this example)

A percentage of the documents a term must appear in. This can be useful for reducing the number of garbage returns due to misspellings if you haven’t scrubbed the input.


This parameter is set to the fieldType that will process the information in the defined ‘field’. I suggest starting simple and adding complexity as the need arises.

  • This fieldType is completely independent from the analysis chain applied to the field you specify for your suggester. It’s perfectly reasonable to have the two fieldTypes be much different.
  • The “string” fieldType should probably not be used. If a “string” type is appropriate for the use case, the TermsComponent will probably serve as well and it is much simpler.

buildOnStartup and buildOnCommit

Building the suggester data involves re-reading, decompressing and and adding the field from every document to the suggester. These two settings should both generally be set to “false”. On Startup happens every time Solr is started. On Commit happens every time a document is committed. In the case of a smaller list of potential suggestions, the latter is acceptable.

Define a requestHandler for the Suggest Component

<requestHandler name="/suggest" class="solr.SearchHandler" startup="lazy" >
    <lst name="defaults">
        <str name="suggest">true</str>
        <str name="suggest.dictionary">infixSuggester</str>
        <str name="suggest.dictionary">fuzzySuggester</str>
        <str name="suggest.onlyMorePopular">true</str>
        <str name="suggest.count">10</str>
        <str name="suggest.collate">true</str>
    <arr name="components">

The “name” of the requestHandler defines the url that will be used to request suggestions. In this case it will be http://”localhost”:8983/solr/index_name/suggest. Your port number may be different.

The requestHandler definition contains two parts:


These are settings that you would like to apply to each request. They may be provided in the querystring if different values are necessary.

Multiple “suggest.dictionary” values may be used. Each one will have it’s own section of results. The values are the names of the suggesters that were defined in the Suggest Component.


The name of the Suggest Component is set here. This connects the handler to the component.

See the documentation for more details on configuring search components and request handlers.

Actually getting suggestions

Once all of this is set up, using it is very simple. Assuming a solr index url like this:

  • Build the suggester:
    Issue http://localhost:8983/solr/index_name/suggest?suggest.build=true.

    • Until you do this step, no suggestions are returned.
    • The two build settings (buildOnStartup and buildOnCommit) can be used to avoid this, but consider the size of your index and the time and cpu that will be required to build the suggest index automatically.
  • Ask for suggestions:
    Issue http://localhost:8983/solr/index_name/suggest?suggest.q=whatever

    • Additional parameters can be included, such as the count, the desired format (json or xml) or a specific suggest.dictionary.
    • Use “wt” and “indent” parameters to format your results into json or xml and apply indenting. e.g.: &wt=json&indent=true
    • The response will contain a “suggest” field. This field will contain fields for each of the suggest.dictionaries that was used. Each of these dictionary fields will have a “numFound” field as well as a “suggestions” field containing an array of the found suggestions and their weights.

Response Format:

  suggest: {
    suggester_name: {
       suggest_query: { numFound:  .., suggestions: [ {term: .., weight: .., payload: ..}, .. ]} 

I hope you find this information useful. See the Suggester documentation for more details.

Thanks for reading!

Detailed Package Installer

Not too long ago I was deploying changes through the various environments that our client has. We did not have a tool like TDS or Razl to help with this, so we used Sitecore packages. Due to multiple languages and extensive changes to templates and content these packages were very large with thousands of items. Installing packages of this size can take a very long time, and the only way to know what is happening is to read the log files. I’ve seen solutions that use a separate PowerShell window to ‘tail’ the log file, but I wanted something that was integrated into the Installation Wizard dialog. At first I looked into modifying the installation Job to provide information about itself, but as I dug deeper it became apparent that this would require extensive customization to the Sitecore code. So, I began writing code to monitor the log file. In order to accomplish this, I need a few things:

  • INFO logging enabled
  • The current log path
  • Total items in the package

Sitecore uses log4net as its default logger. This will indicate whether INFO logging is enabled:

((Hierarchy) LogManager.GetLoggerRepository()).Root.IsEnabledFor(Level.INFO);

If INFO logging is not enabled, it is possible to programmatically change it:

var heirarchy = ((Hierarchy) LogManager.GetLoggerRepository());
InitialLoggingLevel = heirarchy.Root.Level.ToString();
heirarchy.Root.Level = Level.INFO;

And to set it back:

if (string.IsNullOrEmpty(InitialLoggingLevel))

var heirarchy = ((Hierarchy)LogManager.GetLoggerRepository());
var levelValue = heirarchy.LevelMap[InitialLoggingLevel];
heirarchy.Root.Level = levelValue;

Sitecore does a lot of logging, and the path can change. Fortunately it is possible to get the current log path:

var rootAppender = ((Hierarchy)LogManager.GetLoggerRepository())

var path = rootAppender != null ? rootAppender.File : string.Empty;

Sitecore packages follow a specific pattern, so it is possible to open them to read their contents and determine the number of items in the package:

int itemCount;
var filename = Installer.GetFilename(PackageFile.Value);

using (var archive = ZipFile.OpenRead(filename))
    var packageEntry = archive.Entries.FirstOrDefault(e => e.FullName == InnerPackageFileName);
    if (packageEntry == null)

    using (var packageStream = packageEntry.Open())
        using (var package = new ZipArchive(packageStream, ZipArchiveMode.Read))
            itemCount = package.Entries.Count(e => e.FullName.StartsWith(PackageEntryItemPrefix));

Now that we’re set up, we can monitor the log and use the wealth of information it provides to present the status of the job to the client. Logging follows a pattern, so I created some constants that are used to filter the log entries and determine how they should be interpreted.

A client-side timer allows us to post requests back to the server so we can monitor the log at a specific interval. The current log path and cursor position are saved in the view to allow us to read the log delta each time the timer fires and update the status information.

Each log delta is split into a list of lines, which can then be interrogated to determine what they tell us. We basically care about three things:

  • What is the installer currently doing
  • How many items out of the total have been installed
  • Have any warning or errors occurred during the process.

A log line will be one of a handful of things:

  • Starting / Ending a section of work
  • Installing Item
  • Warning or Error

The Installer will do its work in three groups: Install Items, Install Security, Update Index. The view has three boxes that are used to indicate when one of the jobs is working: when a ‘Starting’ line is read, its corresponding box is turned to green. When an ‘Ending’ line is read, its corresponding box is turned gray.

Item installation progress is indicated by a visual progress bar and a text message. Each ‘Installing Item’ line will increment a counter, which is then used along with the total count (from reading the package) to indicate the progress.

A text box contains the log entries that have happened since the install started. Only INFO logs pertaining to the install will be shown, but all WARN and ERROR logs are shown because they might have an impact on the installation process. The lines will also be color coded: INFO = green; WARN = yellow; ERROR = red.

The full module can be found here: https://marketplace.sitecore.net/en/Modules/Detailed_Package_Installer.aspx

I hope the information in this article helps you, and I appreciate your taking the time to read it.