Adding Search Metadata to Publishing Site Pages in SharePoint 2010

Scenario

You have a SharePoint publishing site with a number of pages that display dynamic content based on a query string. You followed a process similar to Crawling Publishing Sites in SharePoint 2010 to configure SharePoint search to index the dynamic page content. Now you’d like to enrich the items in the search index with additional metadata that can be used for property restriction queries or for adding custom refiners.

Solution

Add dynamically generated META tag to the page. SharePoint will automatically create a crawled property of type Text under in the Web category using the name attribute of the META tag as the crawled property name. You can then map the crawled property to a new managed property that will get its value populated with the content attribute value of the META tag.

Example

I’ll use the web part and pages created in my previous blog post and will simply extend the web part to generate a META tag.

[ToolboxItemAttribute(false)]
public class ProductInformation : WebPart
{
    protected override void CreateChildControls()
    {
        // get the model number from query string
        string modelNumber = Page.Request.QueryString["ModelNumber"];
        if (!string.IsNullOrEmpty(modelNumber))
        {
            // assign a product category based on the model number
            string productCategory = string.Empty;
            switch (modelNumber)
            {
                case "M300":
                case "M400":
                case "M500":
                case "X200":
                case "X250":
                    productCategory = "Digital Camera";
                    break;
                case "X300":
                case "X358":
                case "X400":
                case "X458":
                case "X500":
                    productCategory = "Digital SLR";
                    break;
            }

            // set the page title
            ContentPlaceHolder contentPlaceHolder = (ContentPlaceHolder)Page.Master.FindControl("PlaceHolderPageTitle");
            contentPlaceHolder.Controls.Clear();
            contentPlaceHolder.Controls.Add(new LiteralControl() { Text = string.Format("{0} {1}", modelNumber, productCategory) });

            // add the model number and product category to the page as an H2 heading
            Controls.Add(new LiteralControl() { Text = string.Format("<h2>{0} {1}</h2>", modelNumber, productCategory) });

            // generate a META tag
            Page.Header.Controls.Add(new HtmlMeta() { Name = "modelnumber", Content = modelNumber });
        }
    }
}

If we refresh one of the product information pages after deploying the code change above, we should be able to see the META tag in the page source.

<meta name="modelnumber" content="M300" />

Now run a full crawl and then verify that the crawled property was created by going to Central Administration > Search Service Application > Metadata Properties > Crawled Properties (for SharePoint Search) or to Central Administration > Query SSA > FAST Search Administration > Crawled property categories > Web (for FAST Search).

Next, create a new managed property of type Text and add a mapping to the crawled property above. If using FAST Search, also check the Query property and Refiner property checkboxes.

Run another full crawl and the managed property is now ready to be used for property restriction queries or as a refiner.

Let’s test it by running the following query first:
PropertyRestrictionQuery

You can now also use the new managed property as a refiner.
CustomPropertyRefiner

Using Content Enrichment Web Service Callout in SharePoint 2013 Preview

SharePoint 2013 Preview release intoduced a new functionality called content enrichment web service callout. It provides the ability to inspect and manipulate managed property values for each item before it’s added to the search index. Prior to SharePoint 2013, the only way to accomplish something similar was in FAST Search for SharePoint by extending the item processing pipeline. Clients using SharePoint server search were out of luck as the functionality was not available to them.

The process of building and configuring a web service callout is relatively straight forward. These are the high-level steps to follow:

  1. Build a web service by implementing the IContentProcessingEnrichmentService interface. Add logic to manipulate managed property values.
  2. Run PowerShell commands to configure the callout web service endpoint address, input and output managed properties, trigger condition and a few other things.
  3. Execute a full crawl.

In this blog post I’ll show an example of developing a web service that populates a new managed property value which is then used as a refiner on the search results page. Let’s say we have a number of project sites in SharePoint where each site contains information about a specific bike model.

Each bike model belongs to a product category such as Mountain Bikes, Road Bikes and Touring Bikes. We’d like to be able to refine search results by product category but unfortunately that metadata is not available in SharePoint at this point. What we are going to do next is create a new managed property called ProductCategory and build a web service to populate the managed property values based on our custom business logic. The ProductCategory managed property can then be used as a refiner on the search results page.

To create the managed property, navigate to Central Administration > Search Service Application > Search Schema > New Managed Property.

  • Property name: ProductCategory
  • Type: Text
  • Searchable: checked
  • Queryable: checked
  • Retrievable: checked
  • Refinable: Yes – active
  • Token Normalization: checked

In Visual Studio 2012, create the web service: New Project > Visual C# > WCF > WCF Service Application.

Delete the Service1 created by default or rename it to EnrichmentService. Delete the IService1 or IEnrichmentService interface.

Add an assembly reference to C:\Program Files\Microsoft Office Servers\15.0\Search\Applications\External\microsoft.office.server.search.contentprocessingenrichment.dll.

Open EnrichmentService.svc.cs, add the following using statements:

using Microsoft.Office.Server.Search.ContentProcessingEnrichment;
using Microsoft.Office.Server.Search.ContentProcessingEnrichment.PropertyTypes;

Replace the class implementation:

public class EnrichmentService : IContentProcessingEnrichmentService
{
    private Dictionary<string, string> productModels = new Dictionary<string, string>()
    {
        {"mountain-100", "Mountain Bikes"},
        {"mountain-500", "Mountain Bikes"},
        {"road-150", "Road Bikes"},
        {"road-450", "Road Bikes"},
        {"touring-1000", "Touring Bikes"},
        {"touring-2000", "Touring Bikes"}
    };

    public ProcessedItem ProcessItem(Item item)
    {
        ProcessedItem processedItem = new ProcessedItem();
        processedItem.ItemProperties = new List<AbstractProperty>();

        AbstractProperty pathProperty = item.ItemProperties.Where(p => p.Name == "Path").FirstOrDefault();
        if (pathProperty != null)
        {
            Property<string> pathProp = pathProperty as Property<string>;
            if (pathProp != null)
            {
                foreach (var productModel in productModels)
                {
                    if (pathProp.Value.Contains(productModel.Key))
                    {
                        Property<string> modelProp = new Property<string>()
                        {
                            Name = "ProductCategory",
                            Value = productModel.Value
                        };
                        processedItem.ItemProperties.Add(modelProp);
                    }
                }
            }
        }

        return processedItem;
    }
}

Now the web service is ready and the next step is to configure SharePoint to call the web service during the crawl. That is done using PowerShell. To minimize the performance impact of the web service callout, we only want it to be called under a certain condition – this condition is defined in the Trigger property. More information about the syntax can be found in the Trigger expression syntax article on MSDN. The expected input and output managed properties are configured via the InputProperties and OutputProperties. When debugging the web service, the DebugMode property value can be set to $true in which case SharePoint will ignore the InputProperties value and will send all available managed properties for each item to the service. Any managed property values returned by the web service in debug mode are ignored by SharePoint.

$ssa = Get-SPEnterpriseSearchServiceApplication
$config = New-SPEnterpriseSearchContentEnrichmentConfiguration
$config.DebugMode = $false
$config.Endpoint = "http://localhost:64401/EnrichmentService.svc"
$config.FailureMode = "WARNING"
$config.InputProperties = "Path"
$config.OutputProperties = "ProductCategory"
$config.SendRawData = $false
$config.Trigger = 'StartsWith(Path,"http://intranet.contoso.com/adventureworks/models/")'
Set-SPEnterpriseSearchContentEnrichmentConfiguration –SearchApplication $ssa –ContentEnrichmentConfiguration $config

Finally, launch the web EnrichmentService created earlier and start a new full crawl. Once the crawl is complete, the ProductCategory managed property should be populated and searchable:

The final step is to add a Product Category search refiner. Edit the search results page, edit the Refinement web part, click the Choose Refiners… button within the Properties for Search Refinement section, select the ProductCategory managed property in the Available refiners list and press the Add > button. Move the ProductCategory to the top of the Selected refiners list, then scroll down and set the Display name to Product Category and save your changes.

Run a search for “bike” and you should now be able to refine the search results by the product categories:

References:

Adding a Custom Property Search Refiner in SharePoint 2010

In the previous blog post Extending FAST Search Processing Pipeline we created a new custom managed property called Project and extended the FAST Search processing pipeline to populate the property with values. In this article we’ll give users the ability to refine results based on the new property values by adding a new refiner to the search results page in the search center.

First we’ll navigate to the results.aspx page by submitting a search query.

Next we need to Edit the page and bring up the Refinement Panel properties.

The properties we are interested in are the Filter Category Definition and Use Default Configuration. Go ahead and copy the Filter Category Definition property content into your favorite text editor. Insert the following element into the original xml content. Note that the managed property name set in the MappedProperty attribute must be in all lower case letters.

<Category Title="Project" Description="Project number" Type="Microsoft.Office.Server.Search.WebControls.ManagedPropertyFilterGenerator" MetadataThreshold="1" NumberOfFiltersToDisplay="4" MaxNumberOfFilters="20" ShowMoreLink="True" MappedProperty="project" MoreLinkText="show more" LessLinkText="show fewer" ShowCounts="Count" />

Then make sure to uncheck the Use Default Configuration checkbox, hit Apply to submit the web part changes and Save the page. You should see similar results as in the screenshot below:

Now the users have the ability to easily refine search results by Project without having to go back and add any additional metadata to existing SharePoint content – the Project property values are generated dynamically based on the location of the document within SharePoint site hierarchy by the custom processing pipeline module implemented in the earlier blog post.