B2B eCommerce, Sitecore and Digital Marketing

As a regular online shopper, when we think about Online Shopping or eCommerce, we mostly picture about B2C eCommerce like Amazon. But,  a huge part of the Online eCommerce is B2B eCommerce. According to Forrester Research, in 2014 the B2B sale in US was $692 billion and that was a very conservative number. This year the prediction is that, it will reach $780 billion and the sale will reach more than a trillion dollar by 2020.

pic1

So, what is the role that Content Management System like Sitecore can play to influence the sales of a B2B eCommerce company. Both B2B and B2C eCommerce need some sort of CMS for managing the content, media or digital assets etc. for the eCommerce Site. Those are mostly similar in functionalities. The real difference how the Digital Marketing or in case of Sitecore, how the Experience Platform will be used to drive the B2B sales. But, before diving into that discussion, let’s look at what B2B businesses think about the importance and effectiveness of Content Management System. Below is a survey result I picked up from a Salesforce blog.

pic2

As per that survey, Marketing Analytics (Experience Analytics in Sitecore), Content Management (Sitecore), Marketing Automation (Engagement Planning in Sitecore) and Predictive Intelligence (Personalization in Sitecore) are very critical and effective. The good thing is Sitecore is one of the Market Leaders on those areas and Sitecore is capable of putting B2B business in advantageous position if used effectively.

For last few years, I am working mainly in the area of integrating eCommerce with Sitecore and spent lots of time understanding B2B clients’ business and requirements. The B2B commerce is different and most important difference is that, a Business buys products from B2B site for reselling or running it’s business. Where as in B2C an individual buys products for his/her use. In both cases, a user buys products, but in B2B, user buys it for his/her company. That’s why in B2B the buying is less impulsive and that’s make a huge difference on how the Perosonalization features should be used. For example, based on what a B2B user has bought or looked at in the past, a Quick List can be recommended instead of Product. Quick List is a list of Products that can be added in the Cart quickly from the list. It’s effective because it speeds up the buying process for the customer.

Lots of time the B2B user is a Customer Service Representative (CSR) and buys products for the Customers he/she is assigned to serve. A CSR not only buys products for the customer, they also educate them, fill up their questions and create a long term relationship with them. The B2B site can be and should personalized for the CSRs. For example, it will be helpful for CSRs if they can quickly find the documentation of the products. Based on what customer he/she is helping at that time and customer’s past buying data, documents, articles etc. can be made available to CSRs in the personalized website. The CSRs are knowledgeable person and they usually use Quick Order tools to buy products. The Quick Order Tool should be made more accessible to CSRs than regular user. These are just few examples of how personalization can be achieved in B2B site.

More and more B2B businesses are entering in the digital market. The B2B businesses are going from printed catalog to EMail Marketing. Most of ours clients see the power Personalized EMail Marketing and return of investing on Intelligent Targeted Marketing campaign using software like Sitecore. The user base is changing too. More and more people from young generation are entering in the B2B space. On their personal time they are online shopper and they expect similar experience from B2B business too. B2B business must change to satisfy this generation.

The software like Sitecore will make huge impact on the B2B market in the coming years because of its capability to make the business reach to more customers, using less resources and less time. Furthermore it will make shopping experience intriguing  and satisfying for the user and thus it will attract more customers. I am looking forward to integrating more and more B2B business to Sitecore.

Posted in Commercce | Tagged , | 1 Comment

Some Sitecore Habitat Setup Issues

If you haven’t already checked out Sitecore’s Habitat Framework, you should take a look at the solution. Whether you like the Framework architecture or not, it is worth checking the solution. I see it as Sitecore Accelerator  Solution Framework and it is going to be useful for my future projects. You can start from here in Github. Fantastic Architecture discussion and Getting Started videos from Thomas Eldblom (@TEldblom).

This blog is not about introducing Habitat. I recently setup the application in my machine and faced few issues. I will discuss them because they are not mentioned in the Habitat Wiki.

Visual Studio 2015 vs Visual Studio 2013

Habitat Solution requires you to have VS 2015. If you have VS 2015 or you can install it in your machine, just do that before opening up the solution. For me, I have several reasons why I cannot install VS 2015. I did a little hack to make it work for VS 2013.

  • Open the solution file (Habitat.sln) in notepad++ and following lines
    Microsoft Visual Studio Solution File, Format Version 12.00
    # Visual Studio 14
    VisualStudioVersion = 14.0.23107.0
    MinimumVisualStudioVersion = 10.0.40219.1

    WithMicrosoft Visual Studio Solution File, Format Version 12.00
    # Visual Studio 2013
    VisualStudioVersion = 12.0.31101.0
    MinimumVisualStudioVersion = 10.0.40219.1
  • In notepad++ replace 14.0 with 12.0 in all .csproj files in the SRC folder.
  • Now you can open the solution in VS 2013. Open the file gulpfile.js file in VS 2013 IDE and replace 14.0 with 12.0 in the file. There is only one reference. This is required for the valid MSbuild path.

NuGet Package Issue

I had trouble restoring the xUnit.net packages. It seems the xUnit.net packages were created with the more recent version of NuGet. I had to get latest version of NuGet Extension for Visual Studio. After that there was no problem restoring the packages.

Gulp Issues

Habitat uses Gulp to build and publish the files. If you don’t have Gulp already installed in your machine, you have to first install Gulp . In Visual Studio open the NuGet Package Manager Console. Run following two commands.

npm install –global gulp

npm install –save-dev gulp

If you open the Task Runner window, you still may not see the tasks. Click on the refresh button. If that doesn’t work close and open the solution in Visual Studio.

Unicorn

I had some trouble serializing items in Unicorn Control Panel. I must say, I am very new to Unicorn and it was probably my lack of understanding.

  • The Framework Project items has to be serialized first, then the Domain Projects items and lastly Website items. Start with Habitat.Framework.Serialization.
  • Do not click on the Reserialize until you Sync first. For me it wiped out the .yml files.

That’s all the issues I faced so far. I hope it helps some of you out there. Happy coding.

Posted in Framework, Sitecore | Tagged , | Leave a comment

Sitecore MVC and Unity

Using a Dependency Injection (DI) framework in ASP.NET MVC project is most common thing. I have used DI in almost all my MVC project and the one I use most is Unity. I know, Ninject is cool and most used DI statistically :). Unity has come a long way and there are many improvements done recently. When it comes to use Unity in a MVC project, what I most like is its ability to resolve the Types based on name matching. No need to register the Types in code or configuration, just use convention.

This blog post is not about ASP.NET MVC and Unity though. This is about using Unity as DI for Sitecore MVC project. We know Sitecore interferes with MVC framework and handle things differently. If you quickly want to know how Sitecore MVC works, watch Martina Welander’s video presentation Sitecore MVC – Getting Started. But, when it comes to using DI with Sitecore MVC any DI framework can work because Sitecore ultimately delegates the resolution of controllers to ASP.NET MVC. Recently, Mickey Rahman wrote a blog post about, how Sitecore 8.1 is now using Current DependencyResolver to resolve controllers: Changes to Dependency Injection in Sitecore 8.1. So, as long as I set the DependencyResolver to UnityDependencyResolver Sitecore MVC controllers should be resolved properly. The fact is, I even don’t have to do that. All should happen as I install the Unity NuGet package.

Here are the steps.

  • Open up the NuGet Package Manager in you Sitecore MVC solution and search for Unity. Install ‘Unity bootstrapper for ASP.NET MVC’.
    NuGet Package
  • Once this is installed you should see two files added to the App_Start folder, UnityMvcActivator.cs and UnityConfig.cs. If you open the UnityMvcActivator.cs file, you can see how the DependencyResolver is set to UnityDependencyResolver.UnityWebActivator
  • The next step is to open up the UnityConfig.cs file and register the types in the RegisterTypes method. You can register the types one at a time like below
    container.RegisterType<IProductService, ProductService>();
    But, this becomes difficult when your project has too many Types and chain of dependencies. As I said before, I like most the way Unity resolves the Types by convention. To do that you need to use container.RegisterTypes method like below.
    container.RegisterTypes(AllClasses.FromAssemblies(typeof(Services.BaseService).Assembly,
    typeof(ApiClient.BaseClient).Assembly), WithMappings.FromMatchingInterface,
    WithName.Default);

    One thing you have to be careful about. Do not use AllClasses.FromLoadedAssemblies(). This will try to resolve all types in the loaded assemblies and Sitecore will throw error. In most cases we know where the classes are that we want DI to resolve. Use AllClasses.FromAssemblies option and include the assemblies that contain your Types.

That’s all for today. Happy coding.

Posted in Uncategorized | Tagged , , , , | 2 Comments

Sitecore Powershell Extention : How Find-Item solved the performance issue

In my last blog post I mentioned that after loading the images in Sitecore with the help of Sitecore Powershell Extension (SPE), I needed to associate the products to the images. I didn’t show that code because it’s a specific requirement. At that time, I thought number of products is close to number of images, that is, 10000. When we started loading actual data from the eCommerce system, client started complaining about the performance of my Powershell Script. I realized actual number of products is way more than 10K, close to 70K.

Following is the portion of the code

		#Associate Product to Image.
		$ImageName=$Image.TrimEnd(".$FileExtension")
		$products=Get-Item master: -Query "/sitecore/content/Product Repository/Products//*[@@templatename='Custom Insite Product' and @Image Name='$ImageName']"
		if($products)
		{
			foreach($product in $products)
			{
				$product.Image=Get-Item $MediaImagePath.TrimEnd(".$FileExtension")
				$productName=$product.DisplayName
				Write-Host "Assigned image $ImageName to '$productName' ..."
			}
		}
		else
		{
			Write-Host "No product found for this image $Image ..."
		}

After some debugging, I found that the highlighted code is taking most of the time. That code is getting all the products that are supposed to use the image the script just loaded. Is it a SPE thing or Sitecore query problem? To find that, I took the query and ran it in XPath builder. Result shown below.

Sitecore Query Result

The query was taking 104 seconds. That’s for one image. For 10K images this script will never finish.

To improve the performance, I started looking at SPE commands to see if I can query the products from the search index, because, the products are already indexed by another process. I found Find-Item and replace the code with the following code.

		#Associate Product to Image.
		$ImageName=$Image.TrimEnd(".$FileExtension")
		$products=Find-Item -Index commerce_products_master_index -Criteria @{Filter = "Equals"; Field = "_templatename"; Value = "Custom Insite Product"}, @{Filter = "Equals"; Field = "Image Name"; Value = "$ImageName"} | Select-Object -Property Path | Get-Item

		if($products)
		{
			foreach($product in $products)
			{
				$product.Image=Get-Item $MediaImagePath.TrimEnd(".$FileExtension")
				$productName=$product.DisplayName
				Write-Host "Assigned image $ImageName to '$productName' ..."
			}
		}
		else
		{
			Write-Host "No product found for this image $Image ..."
		}

The result was amazing. Now the query returns products immediately. For 10K images and 70K products, it’s taking little more than half an hour. That’s acceptable performance for this kind of batch process.

 

Posted in Powershell | Tagged , , | Leave a comment

Bulk Loading Images in Sitecore Media Library Using Sitecore Powershell Extension (SPE)

With the blessings from Sitecore Powershell gurus Adam NajmanowiczMichael West and Mike Reynolds, this is my first blog post on Sitecore Powershell Extension (SPE). I mean it. When I had a problem to solve and I wanted to use SPE for that, I just dropped a question in Twitter and the help was more than what I expected.  They helped me all the way to solve my problem. These guys are awesome. I love Sitecore community.

So, what problem I was trying to solve using SPE? We have an application where, we have more than 10 thousands product items. We have to load and associate the images for those product items in Media Library. Why use SPE? Because, I cannot imagine a content editor loading images using Media Library uploader. Media Library does upload images from zip file. In my case the requirement doesn’t stop on just loading the images. We need to associate images to the product items, which I did using my SPE script. Beside this, since this a SPE is script, we can run it via Sitecore Schedule Task. All the user needs to do is drop the images in the designated folder.

For this blog, I will only describe how loading images in bulk works. Rest of the program of associating images to product items is very specific to our requirement and better not discussed for the sake of keeping length of blog smaller. While working on this problem, I faced couple of issues. Those were solved with the help of SPE team. I described the issues at the end of the blog, in case someone looking for answers.

My research started with this article written by Adam. This describes, how a remote SPE script can be run to load image from the file system to media library. I needed to extend this functionality. The first thing I did was created my own Powershell function that loads multiple images from the file system. Here is the script.

#You need to include cmdlets from following location before using this function
#Invoke-Script 'master:/sitecore/system/Modules/PowerShell/Script Library/Platform/Functions/Remoting'
function Upload-MultipleImages {
    [CmdletBinding()]
    param(
        #Remote site host name
        [Parameter(Position=0, Mandatory=$true)]
        [ValidateNotNullOrEmpty()]
        [string]$SiteUrl,
        
        #Remote site username
        [Parameter(Position=1, Mandatory=$true)]
        [ValidateNotNullOrEmpty()]
        [string]$Username,

        #Password for above username
        [Parameter(Position=2, Mandatory=$true)]
        [ValidateNotNullOrEmpty()]
        [string]$Password,

        #Folder location in the File System where images are located
        [Parameter(Position=3, Mandatory=$true)]
        [ValidateNotNullOrEmpty()]
        [string]$SourcePath,

        #Image file extension jpg, png etc.
        [Parameter(Position=4, Mandatory=$true)]
        [ValidateNotNullOrEmpty()]
        [string]$FileExtension,

        #Media Library location
        [Parameter(Position=5, Mandatory=$true)]
        [ValidateNotNullOrEmpty()]
        [String]$MediaPath

    )
    #Connect to Remote Sitecore site
    Set-SitecoreConfiguration $SiteUrl $Username $Password
    #Get all the Image file names
    $images=Get-ChildItem "$SourcePath*.$FileExtension" -name
    foreach($image in $images)
    {
        $MediaImagePath="$MediaPath$image"
        #Check if the image already exists in the Media Library location.
        #If exists, skip it but log in the Sitecore log file and in the console
        #If doesn't exists, upload the image and log the message
        if(Test-Path $MediaImagePath.TrimEnd(".$FileExtension"))
        {
            Write-Log "Image $Image already exists... skipping"
            Write-Host "Image $Image already exists... skipping"
        }
        else
        {
            Write-Log "Uploading Image $Image ..."
            Write-Host "Uploading Image $Image ..."
            Upload-SitecoreFile -remotePath $MediaImagePath -File "$SourcePath$image"
            Write-Log "Uploading Image $Image ... done."
            Write-Host "Uploading Image $Image ... done."
        }
    }
}

Once this function was created, I created below script to call the above function.

#Include SPE remoting scripts
Invoke-Script 'master:/sitecore/system/Modules/PowerShell/Script Library/Platform/Functions/Remoting'
#Include my function
Invoke-Script 'master:/sitecore/system/Modules/PowerShell/Script Library/Nishtech/Functions/Upload-MultipleImages'
$StartTime=Get-Date
#Get context item
$item=Get-Item .
#Get full path of the context item (media folder)
$MediaPath=$item.FullPath
Upload-MultipleImages 'http://myapp.local.com' 'admin' 'b' 'C:\Work\Project Docs\myapp\Images\' 'jpg' "$MediaPath/"
$EndTime=Get-Date
#Show total time taken
new-timespan -Start $StartTime -End $EndTime

I added the above script in the location showed in the following image.

 ScriptLocation

The reason to store the script there was to use Context Menu, like below. I can right click on any media folder and run the script from the context menu.

ContextMenu

I also added a rule for the script so that the context menu only shows up for the Media Library folders.

Rule

The context menu will not show up if the Module was not activated. Make sure the following option checked for the Module where script has been stored.

ModuleActivation

Now, if I run the script from the context menu, it runs and shows me the following progress box.

ProgressBox

If I click on the ‘View Script Results’ link, I can see the result in the SPE console window snapshot.

ConsoleI wanted to run the same script from the Schedule Task. I had to create a separate script for that because, in this case there is no context media folder. I need to pass the destination folder in from the script itself. Same script, only change is the following line.

Upload-MultipleImages ‘http://myapp.local.com&#8217; ‘admin’ ‘b’ ‘C:\Work\Project Docs\Myapp\Images\’ ‘jpg’ ‘master:/sitecore/media library/Images/Products’

Here is the scheduled task settings

ScheduledTask

That’s all I had to do for loading images.

Issues:

As I mentioned in the beginning that I faced couple of issues working on this. I will discuss them now. I learned few things from these issues.

Routing Issue:

One of my application included in the Sitecore desktop has the path that contains /Console/. After installing SPE, I found that this application stopped working and I was getting following error.

RoutingError

I opened up Cognifide.PowerShell.Core.Processors.RewriteUrl in DotPeek and found that following code in the preprocessorRequest pipeline is looking for any route that includes ‘/Console/’ and re-writing it with “/sitecore modules/PowerShell/”

        Uri url = arguments.Context.Request.Url;
        string localPath = url.LocalPath;
        if (localPath.StartsWith("/Console/", StringComparison.OrdinalIgnoreCase))
          WebUtil.RewriteUrl(new UrlString()
          {
            Path = localPath.ToLowerInvariant().Replace("/console/", "/sitecore modules/PowerShell/"),
            Query = url.Query
          }.ToString());

The solution was to write my own processor and replace the SPE processor with that. I just changed the code to look for more specific route “/Console/Services/”. Simple fix.

        Uri url = arguments.Context.Request.Url;
        string localPath = url.LocalPath;
        if (localPath.StartsWith("/Console/Services/", StringComparison.OrdinalIgnoreCase))
          WebUtil.RewriteUrl(new UrlString()
          {
            Path = localPath.ToLowerInvariant().Replace("/console/", "/sitecore modules/PowerShell/"),
            Query = url.Query
          }.ToString());

 No Rules Available in the Rules Editor:

When I tried to set the Rule for my script I found that there is no out of the box Sitecore rules in the Rules Editor. Adam pointed me that there is patch exists in the Sitecore Marketplace download area for this issue. After I installed that patch, the rules started showing up.

RulesPatch

Final Words:

This is my first encounter with Sitecore Powershell and I am extremely excited by realizing it’s usefulness in Sitecore projects. I will be using it in future and keep blogging about it.

References:

Posted in Powershell, Sitecore | Tagged , | 2 Comments

TDS Timeout and TDSService.asmx 500 error

The title of this blog post was exactly what I googled when I was troubleshooting the TDS deployment timeout issue for one of my projects. Sad, there are not much help out there on how to troubleshoot this issue. There are some suggestions from Hedgehog in the FAQ section. I tried to apply that but that didn’t help much. There was not much information in any log files, except below in the IIS log (500 error).

2015-04-08 17:59:59 10.0.1.6 POST /_DEV/TdsService.asmx – 80  – – 500 0 0 233538

The problem was sporadic. Sometime it’s 500 error, sometime TDS Timeout and sometime it worked. But, most of the time build was failing. Although there was 500 error in the IIS Log, site was working fine. Pretty frustrating and nothing to look into to get more information.

I wanted to know what exactly going on in the IIS Process. Maybe that will give me some clue. So, I took a memory dump of the Application Pool and opened up the memory dump in Windbg. The first thing I needed to check, what threads were taking most CPU time. Usually that sheds some idea about what’s running in the process. The command to do that in windbg is !runaway. Showing top 8 threads below

0:000> !runaway
User Mode Time
Thread      Time
7:4b4         0 days 0:01:43.265
6:82c         0 days 0:01:12.625
23:adc       0 days 0:00:16.187
21:11d0     0 days 0:00:14.125
22:1088    0 days 0:00:13.828
24:12b4    0 days 0:00:13.578
10:110c     0 days 0:00:07.500
11:8d0       0 days 0:00:06.046
….

Looks like thread# 7 was consuming most CPU time. Let’s look at the stack for that thread. Again, there were lots of information here but, this shows that the thread was executing the code from Application_Start method from global.asax and it was trying to open connection to the SQL server. It got my attention immediately because failure in Application_Start does throw 500 error if not handled gracefully. So was there any error? What database connection was it opening?

0:007> !clrstack
OS Thread Id: 0x4b4 (7)
        Child SP               IP Call Site
00000005b8507dd8 00007fff25140dca [InlinedCallFrame: 00000005b8507dd8] <Module>.SNIReadSyncOverAsync(SNI_ConnWrapper*, SNI_Packet**, Int32)
00000005b8507dd8 00007fff0c4bcb71 [InlinedCallFrame: 00000005b8507dd8] <Module>.SNIReadSyncOverAsync(SNI_ConnWrapper*, SNI_Packet**, Int32)
00000005b8507db0 00007fff0c4bcb71 DomainNeutralILStubClass.IL_STUB_PInvoke(SNI_ConnWrapper*, SNI_Packet**, Int32)
00000005b8507e80 00007fff0c4a350f SNINativeMethodWrapper.SNIReadSyncOverAsync(System.Runtime.InteropServices.SafeHandle, IntPtr ByRef, Int32)
00000005b8507ef0 00007fff0c4a322e System.Data.SqlClient.TdsParserStateObject.ReadSniSyncOverAsync()
00000005b8507fa0 00007fff0c4a3140 System.Data.SqlClient.TdsParserStateObject.TryReadNetworkPacket()
00000005b8507fe0 00007fff0c4a3918 System.Data.SqlClient.TdsParserStateObject.TryPrepareBuffer()
00000005b8508010 00007fff0c4a67a2 System.Data.SqlClient.TdsParserStateObject.TryReadByte(Byte ByRef)
00000005b8508050 00007fff0c4a57d6 System.Data.SqlClient.TdsParser.TryRun(System.Data.SqlClient.RunBehavior, System.Data.SqlClient.SqlCommand, System.Data.SqlClient.SqlDataReader, System.Data.SqlClient.BulkCopySimpleResultSet, System.Data.SqlClient.TdsParserStateObject, Boolean ByRef)
00000005b85081f0 00007fff0c4a53f4 System.Data.SqlClient.TdsParser.Run(System.Data.SqlClient.RunBehavior, System.Data.SqlClient.SqlCommand, System.Data.SqlClient.SqlDataReader, System.Data.SqlClient.BulkCopySimpleResultSet, System.Data.SqlClient.TdsParserStateObject)
00000005b8508250 00007fff0c4a5285 System.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean)
00000005b85082e0 00007fff0c49ff10 System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(System.Data.SqlClient.ServerInfo, System.String, System.Security.SecureString, Boolean, System.Data.ProviderBase.TimeoutTimer, Boolean)
00000005b8508370 00007fff0c49f1eb System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(System.Data.SqlClient.ServerInfo, System.String, System.Security.SecureString, Boolean, System.Data.SqlClient.SqlConnectionString, System.Data.SqlClient.SqlCredential, System.Data.ProviderBase.TimeoutTimer)
00000005b8508510 00007fff0c49ed6d System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(System.Data.ProviderBase.TimeoutTimer, System.Data.SqlClient.SqlConnectionString, System.Data.SqlClient.SqlCredential, System.String, System.Security.SecureString, Boolean)
00000005b85085c0 00007fff0c49e9ce System.Data.SqlClient.SqlInternalConnectionTds..ctor(System.Data.ProviderBase.DbConnectionPoolIdentity, System.Data.SqlClient.SqlConnectionString, System.Data.SqlClient.SqlCredential, System.Object, System.String, System.Security.SecureString, Boolean, System.Data.SqlClient.SqlConnectionString, System.Data.SqlClient.SessionData)
00000005b8508660 00007fff0c49e5fa System.Data.SqlClient.SqlConnectionFactory.CreateConnection(System.Data.Common.DbConnectionOptions, System.Data.Common.DbConnectionPoolKey, System.Object, System.Data.ProviderBase.DbConnectionPool, System.Data.Common.DbConnection, System.Data.Common.DbConnectionOptions)
00000005b8508770 00007fff0c49e0b8 System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(System.Data.ProviderBase.DbConnectionPool, System.Data.Common.DbConnection, System.Data.Common.DbConnectionOptions, System.Data.Common.DbConnectionPoolKey, System.Data.Common.DbConnectionOptions)
00000005b85087e0 00007fff0c49de75 System.Data.ProviderBase.DbConnectionPool.CreateObject(System.Data.Common.DbConnection, System.Data.Common.DbConnectionOptions, System.Data.ProviderBase.DbConnectionInternal)
00000005b85088a0 00007fff0c49dbb7 System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(System.Data.Common.DbConnection, System.Data.Common.DbConnectionOptions, System.Data.ProviderBase.DbConnectionInternal)
00000005b85088f0 00007fff0c49d2ec System.Data.ProviderBase.DbConnectionPool.TryGetConnection(System.Data.Common.DbConnection, UInt32, Boolean, Boolean, System.Data.Common.DbConnectionOptions, System.Data.ProviderBase.DbConnectionInternal ByRef)
00000005b85089d0 00007fff0c49cfe4 System.Data.ProviderBase.DbConnectionPool.TryGetConnection(System.Data.Common.DbConnection, System.Threading.Tasks.TaskCompletionSource`1<System.Data.ProviderBase.DbConnectionInternal>, System.Data.Common.DbConnectionOptions, System.Data.ProviderBase.DbConnectionInternal ByRef)
00000005b8508a60 00007fff0c49cac7 System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(System.Data.Common.DbConnection, System.Threading.Tasks.TaskCompletionSource`1<System.Data.ProviderBase.DbConnectionInternal>, System.Data.Common.DbConnectionOptions, System.Data.ProviderBase.DbConnectionInternal, System.Data.ProviderBase.DbConnectionInternal ByRef)
00000005b8508b20 00007fff0c49c48b System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(System.Data.Common.DbConnection, System.Data.ProviderBase.DbConnectionFactory, System.Threading.Tasks.TaskCompletionSource`1<System.Data.ProviderBase.DbConnectionInternal>, System.Data.Common.DbConnectionOptions)
00000005b8508b80 00007fff0c499ff9 System.Data.SqlClient.SqlConnection.TryOpenInner(System.Threading.Tasks.TaskCompletionSource`1<System.Data.ProviderBase.DbConnectionInternal>)
00000005b8508c20 00007fff0c499eb6 System.Data.SqlClient.SqlConnection.TryOpen(System.Threading.Tasks.TaskCompletionSource`1<System.Data.ProviderBase.DbConnectionInternal>)
00000005b8508c90 00007fff0c4999bf System.Data.SqlClient.SqlConnection.Open()
00000005b8508d00 00007fff0ee34733 System.Web.DataAccess.SqlConnectionHolder.Open(System.Web.HttpContext, Boolean)
00000005b8508d80 00007fff0ee33859 System.Web.DataAccess.SqlConnectionHelper.GetConnection(System.String, Boolean)
00000005b8508e10 00007fff0ef4a3b5 System.Web.Security.SqlRoleProvider.GetAllRoles()
00000005b8508ed0 00007ffec193af3e Sitecore.Data.DataProviders.NullRetryer.Execute[[System.__Canon, mscorlib]](System.Func`1<System.__Canon>, System.Action)
....
....
00000005b850b630 00007ffebdb3ab65 InsiteForSitecore.Web.MvcApplication.Application_Start()
....

Let’s see if there was any exception in this stack. The command for that is !pe. There was only one error. It’s a NullReferenceException and it didn’t seem to create the problem because the code continued after that method call.

0:007> !pe
Exception object: 00000005b9700ea0
Exception type:   System.NullReferenceException
Message:          Object reference not set to an instance of an object.
InnerException:   <none>
StackTrace (generated):
    SP               IP               Function
    00000005B850B568 00007FFEBDB3B19D Microsoft_Practices_ServiceLocation!Microsoft.Practices.ServiceLocation.ServiceLocator.get_Current()+0xd
    00000005B850B570 00007FFEBDB3B056 InsiteForSitecore_Web!InsiteForSitecore.Web.MvcApplication.ConfigureServiceLocation()+0x46

StackTraceString: <none>
HResult: 80004003

But, it seemed the code was trying to open a SQL Connection and that was taking long time to execute. To find the SQL Connection , I decided to dump the stack object using the command !dso and found the System.Data.SqlClient.SqlConnectionString object in the stack. Dumping this object and then dumping the _usersConnectionString object shows me the connectionstring that it was trying to open the core database. Also, the time out value is set to 15 seconds (default value).

0:007> !do 00000006bb9cfab0
Name:        System.Data.SqlClient.SqlConnectionString
MethodTable: 00007fff0c514950
EEClass:     00007fff0c3594f8
Size:        208(0xd0) bytes
File:        C:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
00007fff1bc80bd8  40002cd        8        System.String  0 instance 00000007b99846e0 _usersConnectionString
00007fff1bc84b98  40002ce       10 ...ections.Hashtable  0 instance 00000006bb9cfb80 _parsetable
00007fff0c516670  40002cf       18 ...mon.NameValuePair  0 instance 00000006bb9cfd20 KeyChain
00007fff1bc7f370  40002d0       28       System.Boolean  1 instance                1 HasPasswordKeyword
00007fff1bc7f370  40002d1       29       System.Boolean  1 instance                0 UseOdbcRules
00007fff1bc82458  40002d2       20 ...ity.PermissionSet  0 instance 00000006bb9d0dc8 _permissionset
00007fff1abd8228  40002c9      2d8 ...Expressions.Regex  0   shared           static ConnectionStringValidKeyRegex
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:NotInit  0000000a10219980:NotInit  <<
00007fff1abd8228  40002ca      2e0 ...Expressions.Regex  0   shared           static ConnectionStringValidValueRegex
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:NotInit  0000000a10219980:NotInit  <<
00007fff1abd8228  40002cb      2e8 ...Expressions.Regex  0   shared           static ConnectionStringQuoteValueRegex
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:NotInit  0000000a10219980:NotInit  <<
00007fff1abd8228  40002cc      2f0 ...Expressions.Regex  0   shared           static ConnectionStringQuoteOdbcValueRegex
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:NotInit  0000000a10219980:NotInit  <<
00007fff1bc7f370  40010e7       2a       System.Boolean  1 instance                0 _integratedSecurity
00007fff1bc7f370  40010e8       2b       System.Boolean  1 instance                1 _connectionReset
00007fff1bc7f370  40010e9       bc       System.Boolean  1 instance                0 _contextConnection
00007fff1bc7f370  40010ea       bd       System.Boolean  1 instance                0 _encrypt
00007fff1bc7f370  40010eb       be       System.Boolean  1 instance                0 _trustServerCertificate
00007fff1bc7f370  40010ec       bf       System.Boolean  1 instance                1 _enlist
00007fff1bc7f370  40010ed       c0       System.Boolean  1 instance                0 _mars
00007fff1bc7f370  40010ee       c1       System.Boolean  1 instance                0 _persistSecurityInfo
00007fff1bc7f370  40010ef       c2       System.Boolean  1 instance                1 _pooling
00007fff1bc7f370  40010f0       c3       System.Boolean  1 instance                0 _replication
00007fff1bc7f370  40010f1       c4       System.Boolean  1 instance                0 _userInstance
00007fff1bc7f370  40010f2       c5       System.Boolean  1 instance                0 _multiSubnetFailover
00007fff1bc837c8  40010f3       2c         System.Int32  1 instance               15 _connectTimeout
00007fff1bc837c8  40010f4       98         System.Int32  1 instance                0 _loadBalanceTimeout
00007fff1bc837c8  40010f5       9c         System.Int32  1 instance              100 _maxPoolSize
00007fff1bc837c8  40010f6       a0         System.Int32  1 instance                0 _minPoolSize
00007fff1bc837c8  40010f7       a4         System.Int32  1 instance             8000 _packetSize
00007fff1bc837c8  40010f8       a8         System.Int32  1 instance                1 _connectRetryCount
00007fff1bc837c8  40010f9       ac         System.Int32  1 instance               10 _connectRetryInterval
00007fff0c520418  40010fa       b0         System.Int32  1 instance                0 _applicationIntent
00007fff1bc80bd8  40010fb       30        System.String  0 instance 00000008b87ed558 _applicationName
00007fff1bc80bd8  40010fc       38        System.String  0 instance 00000005b87d1420 _attachDBFileName
00007fff1bc80bd8  40010fd       40        System.String  0 instance 00000005b87d1420 _currentLanguage
00007fff1bc80bd8  40010fe       48        System.String  0 instance 00000006bb9cfcf8 _dataSource
00007fff1bc80bd8  40010ff       50        System.String  0 instance 0000000000000000 _localDBInstance
00007fff1bc80bd8  4001100       58        System.String  0 instance 00000005b87d1420 _failoverPartner
00007fff1bc80bd8  4001101       60        System.String  0 instance 00000006bb9cfe28 _initialCatalog
00007fff1bc80bd8  4001102       68        System.String  0 instance 00000006bb9d01a0 _password
00007fff1bc80bd8  4001103       70        System.String  0 instance 00000006bb9d00e0 _userID
00007fff1bc80bd8  4001104       78        System.String  0 instance 00000005b87d1420 _networkLibrary
00007fff1bc80bd8  4001105       80        System.String  0 instance 0000000000000000 _workstationId
00007fff0c5326a8  4001106       b4         System.Int32  1 instance             2008 _typeSystemVersion
00007fff1bc85590  4001107       88       System.Version  0 instance 00000007b99881f8 _typeSystemAssemblyVersion
00007fff0c52a450  400110a       b8         System.Int32  1 instance                0 _transactionBinding
00007fff1bc80bd8  400110b       90        System.String  0 instance 0000000000000000 _expandedAttachDBFilename
00007fff1bc84b98  40010e5      ef0 ...ections.Hashtable  0   shared           static _sqlClientSynonyms
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:00000007b9988238 0000000a10219980:00000005b889f280 <<
00007fff1bc84b98  40010e6      ef8 ...ections.Hashtable  0   shared           static _netlibMapping
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:0000000000000000 0000000a10219980:0000000000000000 <<
00007fff1bc85590  4001108      f00       System.Version  0   shared           static constTypeSystemAsmVersion10
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:00000007b99881f8 0000000a10219980:00000005b889f240 <<
00007fff1bc85590  4001109      f08       System.Version  0   shared           static constTypeSystemAsmVersion11
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:00000007b9988218 0000000a10219980:00000005b889f260 <<

0:007> !do 00000007b99846e0
Name:        System.String
MethodTable: 00007fff1bc80bd8
EEClass:     00007fff1b611aa8
Size:        266(0x10a) bytes
File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
String:      Data Source=(local);Initial Catalog=Sitecore_core;Integrated Security=False;User ID=******;Password=******
Fields:
              MT    Field   Offset                 Type VT     Attr            Value Name
00007fff1bc837c8  40000ab        8         System.Int32  1 instance              120 m_stringLength
00007fff1bc81f38  40000ac        c          System.Char  1 instance               44 m_firstChar
00007fff1bc80bd8  40000ad       18        System.String  0   shared           static Empty
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:NotInit  0000000a10219980:NotInit  <<

At this point I know that the longest running thread was trying to open a connection to the core database, the connection timeout was 15 seconds and the thread was running longer than that. There was no timeout error in the stack but, I thought increasing timeout might fix the issue. I increased the timeout and that fixed the issue.

Final Words

I wrote this blog not to tell what I found but to talk about an approach to troubleshoot this kind of problem. I hope this helps others with similar issues.

References

All about Windbg
SOS.dll (SOS Debugging Extension)

Posted in Debugging, Sitecore | Tagged , , , | Leave a comment

Dynamic Multilist with Search

Recently, in my project, I had a requirement where, I had to show Products in a Carousel. The Content Editor would choose some Products for the Carousel from a “MultiList with Search” field in the Carousel item. The Products are stored in a Bucket. The “MultiList with Search” is very useful in this case because the Products can be search in the control and also, in the Carousel template for the Source of the “MultiList with Search” field, we can give a StartSearchLocation and Filters like below, so that Sitecore will search from that location and apply the filters.

StartSearchLocation={92DC0456-1A2A-4AB5-A650-7053ACD119F5}&Filter=_name:*

The problem with my requirement was that, the Item Id or the path for the StartSearchLocation could be different for different website. The Products bucket for websites are located under the website node. So, I need a “Multilist with Search” , where the StartSearchLocation can change or in other words, I need to query to find the StartSearchLocation of the Products bucket for the website. Not too difficult to do this. Here is how I did it.

BucketList is the class I needed to extend. It is a FieldType and it is derived from SearchList. The Source property of the FieldTypes is what I had to change. This property is located in MultiListEx which is the parent class of SearchList. The StartSearchLocation now can contain query in addition to ItemId. If it contains the word ‘query:’ that means the StartSearchLocation ItemId has to be found from query. The following code is doing exactly that. If the word ‘query:’ is not present code assumes that StartSearchLocation contains the ItemId and just return the base.Source.

using Sitecore.Buckets.FieldTypes;
namespace Sitecore.SharedControls.Buckets.FieldTypes
{
    public class DynamicBucketList : BucketList
    {
        public new string Source
        {
            get { return base.Source; }
            set
            {
                if (value.Contains("query:"))
                {
                    var sourceString = value;
                    var queryString = sourceString.Substring(sourceString.IndexOf("query:"),
                        sourceString.Contains('&')
                            ? sourceString.IndexOf('&') - sourceString.IndexOf("query:")
                            : sourceString.Length - sourceString.IndexOf("query:"));
                    //Find out the item from the query. The query should return only one item.
                    var contextItem = global::Sitecore.Context.ContentDatabase.Items[this.ItemID];
                    var queryItem = contextItem.Axes.SelectSingleItem(queryString.Substring("query:".Length));
                    base.Source = value.Replace(queryString, queryItem.ID.ToString());
                }
                else
                {
                    base.Source = value;
                }
            }
        }
    }
}

After this I needed to create the new “Dynamic Multilist with Search” FieldType in core database like below.

Dynamic Multilist with Search

and added the following in a config file

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <controlSources>
      <source patch:after="*[@namespace='Sitecore.Buckets.FieldTypes']" mode="on" namespace="Sitecore.SharedControls.Buckets.FieldTypes" assembly="Sitecore.SharedControls" prefix="contentExtension" />
    </controlSources>
  </sitecore>
</configuration>

Selected the Field Type like below in the Carousel Template. The StartSearchLocation was a query.
StartSearchLocation=query:./ancestor-or-self::*[@@templatename=’Website’]/Product Repository/Products

Carousel

That’s it. Now, I can use a query to dynamically select the StartSearchLocation for the Multilist with Search.

Posted in Commerce Connect, Sitecore | Tagged , , , , | 1 Comment

Sitecore Custom LinkProvider hangs Sitecore Indexing

This blog is to discuss an indexing hang issue I found recently while I was working on a Commerce Connect Synchronization implementation. In my last blog post I mentioned, how a custom synchronization provider can be used to re-index items after importing the items from external commerce system. This was working fine but, recently I found that, as I run the synchronization, the synchronization status progress box spins for ever. To see what the progress box is doing, I started Fiddler and found that it’s checking the status of the job every 1000 milliseconds and the job is on hung state.

Not sure what’s going on, I took a memory dump of the worker process. I opened the hang dump in Windbg and started looking into clr stack of all threads running in the process. There are two stacks that got my attention. The one below showed that the custom link provider was called by index crawler and the thread was going to the wait state, waiting to acquire the lock it released.

 

        Child SP               IP Call Site
00000082f9f6d448 00007ffb0fe41c2a [GCFrame: 00000082f9f6d448]
00000082f9f6d518 00007ffb0fe41c2a [HelperMethodFrame_1OBJ: 00000082f9f6d518] System.Threading.Monitor.ObjWait(Boolean, Int32, System.Object)
00000082f9f6d630 00007ffb024aab2c System.Threading.ManualResetEventSlim.Wait(Int32, System.Threading.CancellationToken)
00000082f9f6d6f0 00007ffb024a366b System.Threading.Tasks.Task.SpinThenBlockingWait(Int32, System.Threading.CancellationToken)
00000082f9f6d780 00007ffb02ce080a System.Threading.Tasks.Task.InternalWait(Int32, System.Threading.CancellationToken)
00000082f9f6d880 00007ffb02ced556 System.Threading.Tasks.Task`1[[System.__Canon, mscorlib]].GetResultCore(Boolean)
00000082f9f6d8c0 00007ffaa73f5b99 Src.Smartworks.ApiClient.Public.Catalog.CatalogsClient.GetCatalogTreeForCatalog(Int32)
00000082f9f6d930 00007ffaa73f59f1 Src.Smartworks.Web.Services.EcommerceLinksService.GetExternalCatalog(Int32)
00000082f9f6d9c0 00007ffaa73f591f Src.Smartworks.Web.Services.EcommerceLinksService.GetExternalCatalog(Sitecore.Data.Items.Item)
00000082f9f6da30 00007ffaa73f57e5 Src.Smartworks.Web.Services.EcommerceLinksService.GetCatalogUrl(Sitecore.Data.Items.Item)
00000082f9f6da80 00007ffaa73d2be3 Src.Smartworks.Web.Services.EcommerceLinksProvider.GetCatalogUrl(Sitecore.Data.Items.Item)
00000082f9f6db10 00007ffaa6f7659a Src.Smartworks.Web.Services.EcommerceLinksProvider.GetItemUrl(Sitecore.Data.Items.Item, Sitecore.Links.UrlOptions)
00000082f9f6dbc0 00007ffaa733b4b3 Sitecore.ContentSearch.ComputedFields.UrlLink.ComputeFieldValue(Sitecore.ContentSearch.IIndexable)
00000082f9f6dc00 00007ffaa733890f Sitecore.ContentSearch.LuceneProvider.LuceneDocumentBuilder.AddComputedIndexFields()
00000082f9f6dce0 00007ffaa7332804 Sitecore.ContentSearch.LuceneProvider.LuceneIndexOperations.GetIndexData(Sitecore.ContentSearch.IIndexable, Sitecore.ContentSearch.IProviderUpdateContext)
00000082f9f6dd50 00007ffaa73323c4 Sitecore.ContentSearch.LuceneProvider.LuceneIndexOperations.BuildDataToIndex(Sitecore.ContentSearch.IProviderUpdateContext, Sitecore.ContentSearch.IIndexable)
00000082f9f6dda0 00007ffaa7332090 Sitecore.ContentSearch.LuceneProvider.LuceneIndexOperations.Update(Sitecore.ContentSearch.IIndexable, Sitecore.ContentSearch.IProviderUpdateContext, Sitecore.ContentSearch.ProviderIndexConfiguration)
00000082f9f6ddf0 00007ffaa7331a9b Sitecore.ContentSearch.SitecoreItemCrawler.DoUpdate(Sitecore.ContentSearch.IProviderUpdateContext, Sitecore.ContentSearch.SitecoreIndexableItem)
00000082f9f6de90 00007ffaa7330ef6 Sitecore.ContentSearch.HierarchicalDataCrawler`1[[System.__Canon, mscorlib]].CrawlItem(System.Tuple`3<System.__Canon,Sitecore.ContentSearch.IProviderUpdateContext,CrawlState`1<System.__Canon,System.__Canon>>)
00000082f9f6df80 00007ffb024a30ee System.Threading.Tasks.Task.Execute()
00000082f9f6dff0 00007ffb02408355 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
00000082f9f6e150 00007ffb024080c9 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
00000082f9f6e180 00007ffb024a33c5 System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef)
00000082f9f6e260 00007ffb024a2a65 System.Threading.Tasks.Task.ExecuteEntry(Boolean)
00000082f9f6e2a0 00007ffaa73d23b1 Sitecore.ContentSearch.LimitedConcurrencyLevelTaskScheduler.<NotifyThreadPoolOfPendingWork>b__3(System.Object)
00000082f9f6e310 00007ffb0248b193 System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
00000082f9f6e360 00007ffb0248a22a System.Threading.ThreadPoolWorkQueue.Dispatch()
00000082f9f6e8b8 00007ffb03db0453 [DebuggerU2MCatchHandlerFrame: 00000082f9f6e8b8]
00000082f9f6ea58 00007ffb03db0453 [ContextTransitionFrame: 00000082f9f6ea58]
00000082f9f6ec78 00007ffb03db0453 [DebuggerU2MCatchHandlerFrame: 00000082f9f6ec78]

 

The second thread that got my attention was the one below. This thread was started to refresh the index by the synchronization provider. This thread is going into sleep and holding the lock released by the above thread.

        Child SP               IP Call Site
00000082f723d358 00007ffb0fe419ba [HelperMethodFrame: 00000082f723d358] System.Threading.Thread.SleepInternal(Int32)
00000082f723d450 00007ffb0248f399 System.Threading.Thread.Sleep(Int32)
00000082f723d480 00007ffaa7330837 *** ERROR: Module load completed but symbols could not be loaded for Sitecore.ContentSearch.dll
Sitecore.ContentSearch.HierarchicalDataCrawler`1[[System.__Canon, mscorlib]].UpdateHierarchicalRecursive(Sitecore.ContentSearch.IProviderUpdateContext, System.__Canon, System.Threading.CancellationToken)
00000082f723d540 00007ffaa733011d Sitecore.ContentSearch.HierarchicalDataCrawler`1[[System.__Canon, mscorlib]].RefreshFromRoot(Sitecore.ContentSearch.IProviderUpdateContext, System.__Canon, Sitecore.ContentSearch.IndexingOptions, System.Threading.CancellationToken)
00000082f723d5c0 00007ffaa7304feb *** ERROR: Module load completed but symbols could not be loaded for Sitecore.ContentSearch.LuceneProvider.dll
Sitecore.ContentSearch.LuceneProvider.LuceneIndex.PerformRefresh(Sitecore.ContentSearch.IIndexable, Sitecore.ContentSearch.IndexingOptions, System.Threading.CancellationToken)
00000082f723d660 00007ffaa7304d05 Sitecore.ContentSearch.LuceneProvider.LuceneIndex.Refresh(Sitecore.ContentSearch.IIndexable)
00000082f723d690 00007ffaa732d6b8 *** WARNING: Unable to verify checksum for Src.Smartworks.Sitecore.Connector.dll
*** ERROR: Module load completed but symbols could not be loaded for Src.Smartworks.Sitecore.Connector.dll
Src.Smartworks.Sitecore.Connector.Providers.SmartworksProductSynchronizationProvider.RefreshIndexes()
00000082f723d810 00007ffaa73074a3 Src.Smartworks.Sitecore.Connector.Providers.SmartworksProductSynchronizationProvider.SynchronizeArtifacts(Sitecore.Commerce.Services.Products.SynchronizationRequest)
00000082f723dc38 00007ffb03db0453 [DebuggerU2MCatchHandlerFrame: 00000082f723dc38] 
00000082f723df38 00007ffb03db0453 [HelperMethodFrame_PROTECTOBJ: 00000082f723df38] System.RuntimeMethodHandle.InvokeMethod(System.Object, System.Object[], System.Signature, Boolean)
00000082f723e0b0 00007ffb023dc4c5 System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(System.Object, System.Object[], System.Object[])
00000082f723e120 00007ffb02490603 System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)
00000082f723e1a0 00007ffb0248f413 System.Reflection.MethodBase.Invoke(System.Object, System.Object[])
00000082f723e1e0 00007ffaa72f918b DynamicClass.(System.Object, System.Object[])
00000082f723e220 00007ffaa47f546d Sitecore.Pipelines.CorePipeline.Run(Sitecore.Pipelines.PipelineArgs)
00000082f723e2b0 00007ffaa7303515 Sitecore.Jobs.Job.ThreadEntry(System.Object)
00000082f723e340 00007ffaa47dd385 Sitecore.Threading.ManagedThreadPool.ProcessQueuedItems()
00000082f723e3b0 00007ffb02408355 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
00000082f723e510 00007ffb024080c9 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
00000082f723e540 00007ffb024080a7 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
00000082f723e590 00007ffb0248da71 System.Threading.ThreadHelper.ThreadStart()
00000082f723e898 00007ffb03db0453 [GCFrame: 00000082f723e898] 
00000082f723ebe8 00007ffb03db0453 [DebuggerU2MCatchHandlerFrame: 00000082f723ebe8] 
00000082f723ed88 00007ffb03db0453 [ContextTransitionFrame: 00000082f723ed88] 
00000082f723efa8 00007ffb03db0453 [DebuggerU2MCatchHandlerFrame: 00000082f723efa8] 

 

After after looking into the local variables of the HierarchicalDataCrawler.UpdateHierarchicalRecursive method in the debugger, it seems, the following highlighted code is going into infinite loop.

      Assert.ArgumentNotNull((object) context, "context");
      Assert.ArgumentNotNull((object) indexable, "indexable");
      HierarchicalDataCrawler<T>.CrawlState<T> crawlState = new HierarchicalDataCrawler<T>.CrawlState<T>(this.index, cancellationToken)
      {
        ItemAction = new Action<IProviderUpdateContext, T>(((Crawler<T>) this).DoUpdate),
        ItemActionErrorMessage = "Crawler : UpdateRecursive UpdateItem failed - {0}"
      };
      try
      {
        this.currentCrawlOperations[crawlState.Id] = crawlState;
        lock (crawlState)
          ++crawlState.PendingCrawlCount;
        this.CrawlItem(new Tuple<T, IProviderUpdateContext, HierarchicalDataCrawler<T>.CrawlState<T>>(indexable, context, crawlState));
        while ((crawlState.PendingCrawlCount > 0L || crawlState.CurrentTasks.Count > 0) && (!crawlState.IsCancelled && !crawlState.CancellationToken.IsCancellationRequested))
          Thread.Sleep(50);
      }
      finally
      {
        this.currentCrawlOperations.Remove(crawlState.Id);
      }

 

If I disable the custom link provider, indexing works fine again. I haven’t found any reference to this issue in internet. I would like to know if anyone saw this issue and if there is a solution for this. The version of Sitecore is 7.5 rev.141003.

Posted in Commerce Connect, Debugging, Sitecore | Tagged , | Leave a comment

Custom Synchronization Provider for Sitecore Commerce Connect

I have been using Sitecore Commerce Connect for some time and customized it for project’s need. This is my first blog to share my knowledge about Commerce Connect and I am hoping many more to come in future.

Before I start talking about Commerce Connect, let make an announcement. I am thrilled to announce that, Sitecore awarded me as Technology MVP 2015.  I am one of 167 MVPs this year worldwide. I am so honored  to be part of such a talent group of people.

Back to Sitecore Commerce Connect. If you haven’t heard about Sitecore Commerce Connect or hadn’t have a chance to work on it, Commerce Connect is a Framework for connecting external eCommerce system with Sitecore. This integration of eCommerce System with Sitecore is very important because, a vast amount of online shopping websites out there uses Sitecore for Content Management and bringing eCommerce data in Sitecore empowers the marketers to increase the site’s visibility, specially, by using Sitecore Experience Platform (aka, DMS).

The Sitecore Commerce Connect is a separate module, available in SDN here. You will need Commerce Connect license to use this module. If you use partner license for the development, it is already included. Once you install Commerce Connect module and create a Product Repository folder from the Commerce Connect branch template, you should see following buttons on the Ribbon if you select the Product Repository folder item in the Content Editor.

Commerce Connect buttons

Clicking on the ‘Synchronize all products’ button will start synchronization of products and other related items. This doesn’t happen magically. We need to write many pipeline processors to get data from the external eCommerce and many configuration changes are needed. But, the basic flow is, clicking on ‘Synchronize all products’ calls a sitecore command, the command call the Commerce Connect’s Synchronization Provider and Synchronization Provider calls the pipelines where our code is to connect to the external eCommerce system and get the data.

This blog is to talk about, how we can replace the default Synchronization Provider with our custom Synchronization Provider. There can be many reason to do that. In my case I wanted to re-index the Product Repository immediately after importing the data. I said, re-index not rebuild index. To know more about how to programmatically re-index Sitecore items from a selected node, see this blog article that I wrote before.

The first thing we need to do is to change the ‘Synchronize all products’ command to call my command class. Below is the change that we need.

    <commands>
      <command name="synchronization:allproducts" type="My.Command.SynchronizeAllProducts, My.Command"
               patch:instead="command[@type='Sitecore.Commerce.Commands.Products.SynchronizeAllProducts, Sitecore.Commerce']"/>
    </commands>

My.Command.SynchronizeAllProducts needs to be derived from Sitecore.Commerce.Commands.Products.SynchronizeAllProducts.

The Execute method in My.Command.SynchronizeAllProducts class gets called. We will assign the custom Synchronization Provider here.

        public override void Execute(CommandContext context)
        {
            this.ProductSynchronizationProvider = new MyCustomProductSynchronizationProvider(context.Items[0]);
            base.Execute(context);
        }

Now, about the Custom Synchronization Provider. Here we are pausing the indexes, stopping events etc. and then calling the Product Synchronization pipeline. Once the products are synchronized, we are re-indexing the indexes selectively.

    public class MyCustomProductSynchronizationProvider : ProductSynchronizationProvider
    {
        private readonly Item _productRepositoryItem;

        public MyCustomProductSynchronizationProvider (Item productRepositoryItem)
        {
            _productRepositoryItem = productRepositoryItem;
        }

        public override ServiceProviderResult SynchronizeProducts(SynchronizationRequest request)
        {
            try
            {
                if (ContentSearchManager.SearchConfiguration != null)
                {
                    if (ContentSearchManager.SearchConfiguration.Indexes != null)
                    {
                        if (Enumerable.Any<KeyValuePair<string, ISearchIndex>>((IEnumerable<KeyValuePair<string, ISearchIndex>>)ContentSearchManager.SearchConfiguration.Indexes))
                        {
                            foreach (ISearchIndex searchIndex in ContentSearchManager.Indexes)
                            searchIndex.PauseIndexing();
                        }
                    }
                }
            }
            catch
            {
            }
            Settings.Indexing.Enabled = false;
            ServiceProviderResult serviceProviderResult;
            using (new SecurityDisabler())
            {
                using (new EventDisabler())
                {
                    using (new ProxyDisabler())
                    {
                        using (new DatabaseCacheDisabler())
                        {
                            using (new BulkUpdateContext())
                            {
                                request.Properties.Add("ContextItem", _productRepositoryItem);
                                serviceProviderResult = this.RunPipeline<SynchronizationRequest, ServiceProviderResult>("commerce.synchronizeProducts.synchronizeProducts", request);
                            }
                        }
                    }
                }
            }
            CacheManager.ClearAllCaches();
            Settings.Indexing.Enabled = true;
            try
            {
                if (ContentSearchManager.SearchConfiguration != null)
                {
                    if (ContentSearchManager.SearchConfiguration.Indexes != null)
                    {
                        if (Enumerable.Any<KeyValuePair<string, ISearchIndex>>((IEnumerable<KeyValuePair<string, ISearchIndex>>)ContentSearchManager.SearchConfiguration.Indexes))
                        {
                            foreach (ISearchIndex searchIndex in ContentSearchManager.Indexes)
                            {
                                searchIndex.ResumeIndexing();
                                if (searchIndex.Name.ToLower().Equals("sitecore_master_index"))
                                {
                                    searchIndex.Refresh(new SitecoreIndexableItem(this._productRepositoryItem));
                                }
                                else if (searchIndex.Name.ToLower().Equals("commerce_products_master_index"))
                                {
                                    searchIndex.Refresh(new SitecoreIndexableItem(this._productRepositoryItem));
                                }
                            }
                        }
                    }
                }
            }
            catch
            {
            }
            return serviceProviderResult;
        }

    }

That’s all, Commerce Connect calls the Custom Synchronization Provider where I can do the custom processing, before I call the Product Synchronization Pipeline.

Posted in Commerce Connect | Tagged , , , | 1 Comment

2014 in review

It’s been a very active year of blogging for me. Started on May this year, I added 16 blog articles and contributed a module (youtube media connector) in Sitecore Marketplace. Here is the 2014 annual report for my blog generated by WordPress.com. Wishing all my readers and followers a very happy and prosperous 2015. Looking forward to another year of awesome knowledge sharing with fantastic Sitecore community.

Click here to see the complete report.

Posted in General, Sitecore | Leave a comment