Category Archives: General

Sitecore content search and LINQ (part 2)

This is part 2 of my post about Sitecore content search and LINQ. For the first part see https://sionict.wordpress.com/2015/09/19/sitecore-content-search-and-linq-part-1/.

In the first part we set up our custom index and created a piece of code to use it. Now let’s say that we don’t only have products but also services to list on our site. We created a Sitecore template for them and want our Service items to be in the index as well. So we added our Service template ID to the index as we did before with the product template ID:

<serviceTemplateId>{guid}</serviceTemplateId>

Don’t forget to regenerate your index and check with Luke the contents are there after publishing your first Service items! Sometimes you may have to restart IIS and regenerate again to get the index working. If it keeps failing clear the whole custom index folder in the Sitecore data folder and try to regenerate again.

We create a class for our service entries in our code:

using Sitecore.ContentSearch.SearchTypes;
namespace ScPredicateBuilderApp.DataObjects
{
   public class ServiceSearchResult:SearchResultItem
   {
      public string ServiceName { get; set; }
      public string Description { get; set; }
      public double Rate { get; set; }
   }
}

and some code to retrieve them from the index similar as we did for products but with the above class for type parameter in GetQueryable(). Of course C# generics can be very convenient to prevent a lot of duplicate code so you may want to change the SearchProducts() method of the previous part into something more generic:

/// <summary>
/// Get the index contents using search
/// </summary>
public List<T> SearchIndexContent<T>() where T : SearchResultItem
{
   ISearchIndex myIndex = ContentSearchManager.GetIndex("sitecore_myindex");
   using (var context = myIndex.CreateSearchContext())
   {
      var query = context.GetQueryable<T>();
      var result = query.ToList();
      return result;
   }
}

Now when calling this method pass either ProductSearchResult or ServiceSearchResult as type parameter to get a list of the corresponding type.

However… as said before Lucene entries are generic (“documents”) and Sitecore has no way of knowing what entry belongs to what type (class) you specified based on the index alone. It will try to map ALL entries to the type you passed, leaving empty properties for fields it cannot map. So the first thing we have to do when using contentsearch is add a filter on template to the Lucene query.

Adding a template filter

Where and how we pass the template ID to filter on is a matter of conventions in your working environment or personal preference. You can put a base class between SearchResultItem and your content search classes where you retrieve the template ID (and to define common properties), or pass it as a parameter when calling the above method. In this case I do the latter. So we add a parameter to the method:

public List<T> SearchIndexContent<T>(string templateId) where T : SearchResultItem

In this case I defined some configuration somewhere for my project and have the Sitecore IDs as string, passing them as such for parameter. Now we add a line in our using block and extend our query line with a filter on the template. In other words replace the “var query = ....” line with:

var tId=new ID(templateId);
var query = context.GetQueryable<T>().Filter(t=>t.TemplateId==tId);

Filter() is part of the LINQ interface Sitecore has implemented for contentsearch. There is also a Where() implementation. The difference is that Where returns a result taking Lucene’s scoring system into account whereas Filter just returns the result set. Since we don’t do anything with the scoring here we use Filter.

As with any LINQ implementation on a datasource you have to assign external parameters to a local variable before entering them into a LINQ expression to avoid so-called “modified closure” issues. These issues can cause nasty bugs that won’t give you a warning or error but can return incorrect results. Also, if the parameter in the Filter() expression is the result of a function or calculation you have to assign it first to a local variable or it will cause runtime exceptions. For example combining the above into something like:

var query = context.GetQueryable<T>().Filter(t=>t.TemplateId== new ID(templateId));

will NOT work. So always use a local variable as parameter in LINQ expressions.

You can chain LINQ methods just like any other LINQ implementation. Sitecore will take the expression and convert it to a Lucene query, returning the result as an IQueryable. Let’s say we want our products sorted on creation date, so something like:

var query = context.GetQueryable<T>().Filter(t=>t.TemplateId==tId).OrderByDescending(p=>p.CreatedDate);

will work. CreatedDate comes from a computed field that was already defined by Sitecore in our index configuration file as “__smallcreateddate”. It is one of the predefined properties on SearchResultItem.

Since our generic method takes SearchResultItem as type, we don’t have our product- or service specific properties here. However you can refer to fields directly using the Lucene field names and indexer. Let’s say we have a boolean field “Available” added to our Product template, we can do something like:

var query = context.GetQueryable<T>().Filter(t=>t.TemplateId==tId).Filter(p=>p["available"]=="1");

Note that using the Fields collection for the parameter (p=>p.Fields[“available”]) instead of the indexer uses a function get_Item() internally, causing an exception. It is one of many quirks you have to be aware of when using LINQ for contentsearch. Also you’re referring values as they are in Lucene this way, which means they’re not type-converted and thus might not give the results you expect. The above is therefore a string comparison.

Of course we now also have a problem for our ServiceSearchResult entries since they don’t have a field “available”. It results in a null value for the field and thus the above expression is always false.

Building LINQ expressions dynamically

So what if we want to have expressions depending on our item type? We could go back to making type-specific search functions, having to duplicate the template filtering expression (and probably more) in each one. Moreover, in real-world scenario’s we may not know in advance what expressions may be needed (a user may or may not select a specific search option). In short, we want to build our search expression from separate parts.

In comes Sitecore’s PredicateBuilder. This class resides in the Sitecore.ContentSearch.Linq.Utilities namespace and can be used to create and manipulate (partial) LINQ expressions for a given type. You start by either calling the True() or False() method, where you use True() for operations than need to be combined using logical “And “ and False() for logical “Or”. Then you use the And() or Or() methods for the comparison expression. The current version also has a Create<T>() method but at his point I cannot confirm it works the same as the True/And and False/Or method combinations under all circumstances.

These methods return an object of type System.Linq.Expressions.Expression. To make this all clear it is best to show an example. Say we put the expression for available products in a separate method, it will look like this:

/// <summary>
/// Return an expression for filtering on available products only
/// </summary>
/// <returns></returns>
public Expression<Func<ProductSearchResult, bool>> GetAvailableProductsExpression()
{
   var predicate = PredicateBuilder.True<ProductSearchResult>();    //True for "And"
   predicate = predicate.And(p => p.Available == true);
   return predicate;
}

Now we add a parameter of type Expression<Func<T, bool>> to our SearchIndexContent method, and pass the result of the above in when calling the search for products. For services we don’t pass in a specific expression and use a dummy expression in our SearchIndexContent method. It now looks like this:

/// <summary>
/// Get the index contents using search
/// </summary>
public List<T> SearchIndexContent<T>(string templateId, Expression<Func<T, bool>> expression = null) where T : SearchResultItem
{
   ISearchIndex myIndex = ContentSearchManager.GetIndex(Constants.IndexName);
   using (var context = myIndex.CreateSearchContext())
   {
      var tId = new ID(templateId);
      // Dummy if null
      var exp = expression ?? PredicateBuilder.False<T>().Or(p => true);
      var query = context.GetQueryable<T>().Filter(t => t.TemplateId == tId).Filter(exp);
      var result = query.ToList();
      return result;
   }
}

So to get our products we call it with:

var products = SearchIndexContent<ProductSearchResult>(Constants.ProductTemplateId, GetAvailableProductsExpression());

And to get our services:

var services = SearchIndexContent<ServiceSearchResult>(Constants.ServiceTemplateId);

(Note for this example code I defined the IDs of the templates as strings in a static Constants class.)

Predicate expressions can be combined and nested. Instead of putting a boolean expression in the And() or Or() methods directly you can put the result of another PredicateBuilder in. This way you can logically combine “And” and “Or” expressions, create large complex expressions from smaller parts and “inject” expressions based on user filter selections for example.

Using PredicateBuilder in general

As mentioned before the resulting expression of a PredicateBuilder operation is of a .NET type and not a Sitecore type. The PredicateBuilder is not tied to Sitecore’s content search itself. As far as I know you can use it with any type that can be cast using AsQueryable(). This can be quite handy, for example to do post-search filtering using the facets from the query.getResults() method mentioned in part 1 of this article. Or to perform post-search operations on the result set that Sitecore’s LINQ parser cannot translate to a search query.

There’s a lot more to Sitecore’s content search, LINQ and the PredicateBuilder than can be covered by this article. Unfortunately Sitecore’s documentation about it is far from complete, so you’ll have to look around on the web and experiment to figure out all that’s there.

Getting the demo code

I have uploaded a sample application to GitHub at https://github.com/mcrvdriel/ScPredicateBuilderDemo. It contains a simple Visual Studio 2013 solution, and a folder containing a package you can import into Sitecore to get the templates and some items. Since Sitecore libraries are proprietary software they are not included and you have to add them to the solution yourself. You need to have a working Sitecore 8 on your environment to be able to use this code, and set the demo project to publish to it.

Advertisements

ASP.NET session state and authentication

A few weeks after rebuilding a security implementation of an existing ASP.NET webforms system, I got a call from my client saying one of their customers lost their in-session data and was confronted with defaults from the system. A quick look at the logs showed the customer in this case had left the system idle for a while, then returned after a session timeout had occured. As expected the user was redirected to the logon screen, but managed somehow to get back into the system bypassing the logon (although of course not as a different user).

Explanation

After some research looking into the configuration I found the installation of a third-party component we used in the system had added a line to the web.config:

<sessionState timeout=”20” …….

As for the authentication, the system used ASP.NET forms authentication with the default timeout, which is 30 minutes. At this point it is important to realize that ASP.NET authentication is not connected to a particular session. ASP.NET configured for forms authentication creates an authentication ticket with a timeout that is usually stored in an authentication cookie (with default name “.ASPXAUTH”). Setting the timeout on the forms authentication does NOT set the session timeout, something that is often misunderstood or overlooked in ASP.NET applications.

Apparently the user in this case had a session timeout but after being redirected to the logon page used the browser’s back button BEFORE the authentication timeout occured. The difference between session state timeout and authentication timeout had left a 10 minute window where a user without a session was still authenticated. Since the user still had a valid authentication ticket, the system just created a new session but of course the previously stored session information was lost, presenting the user with default settings.

Synchronize session and authentication

To avoid the above situation from happening, first of all set the authentication timeout and the session timeout to at least the same values. By default authentication uses a sliding expiration unless configured not to, meaning the counter is reset on user activity (but not necessarily after each request). For session state this is always the case.

Depending on your requirements you can choose a strategy to avoid getting sessions out of sync with authentication. One way is to just reinitialize the session if it was expired and the user is still logged in. This is easy if no or little information is kept in relation to the session. Another way is to make sure session ending does end the authentication and vice versa.

Part 1: end authentication when session is expired

To implement ending the authentication after session expiration, first make sure the session sticks by entering something into it, otherwise the session will get renewed on every request. To do this, directly after authenticating the user store the session ID in a session variable. So in a logon form (ASP.NET webforms) or ASP.NET MVC controller it will look something like:

...
//Authentication, validation etc.
....

FormsAuthentication.SetAuthCookie(UserName, false);
Session["__MyAppSession"] = Session.SessionID;
..

Since we are going to bind our authentication to the session, it is pointless to set the createPersistentCookie parameter to true of course.

Now we can check on any request if we still have the session active, and if not log out the user. The exact place to do this can be tricky and causes a lot of questions on forums and such, but arguably the best location is in the Application_AcquireRequestState event in global.asax.

Since there’s no guarantee we have a valid user or session in this event, we need to do a lot of null-checking. The code will look like this:

void Application_AcquireRequestState(object sender, EventArgs e)
 {
 var session = System.Web.HttpContext.Current.Session;
 if (session == null || string.IsNullOrWhiteSpace(session.SessionID)) return;
 var userIsAuthenticated = User != null &&
 User.Identity != null &&
 User.Identity.IsAuthenticated;
 if (userIsAuthenticated && !session.SessionID.Equals(Session["__MyAppSession"]))
 {
 Logoff();
 }
 // part 2 gets here
 }
private void Logoff()
{ 
    FormsAuthentication.SignOut(); 
    var authCookie = new HttpCookie(FormsAuthentication.FormsCookieName, string.Empty) { Expires = DateTime.Now.AddYears(-1) }; 
    Response.Cookies.Add(authCookie); 
    FormsAuthentication.RedirectToLoginPage(); 
}

Now, if a request is sent while the session expired with the user still authenticated, the stored ID (or actually the session variable “__MyAppSession”) will no longer be present and the user will be logged of. The part with the cookie I will explain below.

Part 2: end session when authentication ends (timeout)

This can be added quite easily. After the comment in the above event code, add the following:

if (!userIsAuthenticated && session.SessionID.Equals(Session["__MyAppSession"]))
{ 
    ClearSession();
}

And the ClearSession method:

private void ClearSession()
{
    Session.Abandon();
    var sessionCookie = new HttpCookie("ASP.NET_SessionId", string.Empty) { Expires = DateTime.Now.AddYears(-1) };
    Response.Cookies.Add(sessionCookie);
}

Now if a session exists with session information while the user is no longer authenticated, the session will be abandoned.

Part 3: full user logout

When a user actively logs off we have to clear both the authentication and the session. Since this is usually a single “fire and forget” operation that can be called from various places it’s usually implemented best as a static operation in a logical place.

Per recommendation it is best not to only call FormsAuthentication.SignOut() and session.Abandon(), but to actively overwrite the cookies with ones having expired dates. So a full logoff will look like this:

public static void Logoff()
{
    FormsAuthentication.SignOut();
    Session.Abandon();
    var authCookie = new HttpCookie(FormsAuthentication.FormsCookieName, string.Empty) { Expires = DateTime.Now.AddYears(-1) };
    Response.Cookies.Add(authCookie);
    var sessionCookie = new HttpCookie("ASP.NET_SessionId", string.Empty) { Expires = DateTime.Now.AddYears(-1) };
    Response.Cookies.Add(sessionCookie);
    FormsAuthentication.RedirectToLoginPage();
}

In the above Session and Response are of course from the context (controller or form).

A note on ASP.NET MVC: per recommendation MVC applications should be stateless as much as possible, and not store information in the session. In this case it doesn’t really matter if the session gets renewed automatically while a user is still authenticated since there shouldn’t be any persistent information in there anyway.

My first post

So, I finally got around to starting a blog. As so many things nowadays on the net, it is surprisingly easy to set up. So why didn’t I start one before?

Main reason is that in my opinion there’s already an overload of blogs, articles, forums and what not on the net that add little or nothing to the world. “Information overload” is one of the biggest issues we have to deal with today. So I never had the urge to add just more to that. However, lately I found myself increasingly in need of “publishing space” for information that others told me could be interesting to share.

My intention is to write posts just for sharing thoughts but also posts that will be of a quite technical nature. In fact the main reason to start this blog now comes from a fellow software engineer who gave me the advice to publish something about a technical issue I solved recently on a project. So here it is: my own blog.