2015 in review

Wish you a very happy and prosperous 2016. The WordPress.com stats helper monkeys prepared a 2015 annual report for this blog.

Here’s an excerpt:

The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 13,000 times in 2015. If it were a concert at Sydney Opera House, it would take about 5 sold-out performances for that many people to see it.

Click here to see the complete report.

Posted in Uncategorized | Leave a comment

Continuous Integration with Gulp and Sitecore Powershell Extensions Part 1

Sitecore development and code deployment is quite different than normal ASP.Net Web Application development because of the fact that we need to deal with the versioning of Sitecore items. In this regard Team Development for Sitecore (TDS) is a huge help. There are two main things that TDS does, it publishes the files to the application folder and deploys Sitecore items in the application. Actually, TDS does lot more than that, but those two things are most important to me.

Last few days I was contemplating if can I achieve these two deployment functionalities using Gulp and Sitecore Powershell Extensions (SPE), because, Gulp is great for publishing project files and SPE is great for dealing with Sitecore items. So, the journey started and I am happy that it works. This is going to be a blog series. This one is for deployment of Sitecore application in developer’s local environment.

Sitecore Setup

This is easy. I used SIM to install Sitecore 8.1 and 3.3 version of SPE from Sitecore Marketplace. To work on this, I need a real Sitecore application. I named it ‘Cobbler’. For local environment the hostname is cobbler.local. After setup is done, I had to create the templates and items for my application. Below is a screenshot from the content editor. All the items are available in Github along with the source code.


Only change I made in the default installation, I changed the serialization folder in the Sitecore.config. I did this because I want to version control the serialized Sitecore items. Following line changed

<setting name=”SerializationFolder” value=”C:\Github\Cobbler\data\serialization” />

Project Setup

Following is a screenshot of the Cobbler solution in Visual Studio.



By no means it is a fully working solution. I mean, after setting up everything and do the deployment, a nice website will not be launched. The idea is to show how deployment works. Being said that, I tried to add different kinds of projects in the solution to represent how a real application solution looks like. We have two website projects. Cobbler.Web is the main web application and Elf.WebApi is a restful API project. Cobbler talks Elf via restful API. Cobbler.Design project contains all UI design related files (css, font etc.). I separated it to show that files from project other than the Cobbler.Web can be published in Cobbler website. In real life this is often the case. The Elf website is a separate IIS site and deployment process recognize that and publishes to that website.

Gulp Setup

To use Gulp in Visual Studio solution you first need to install Node and npm. Visit nodejs.org and click on the big green button to download and then install. Npm comes with the Node and separate installation is not needed. After installation you can run following two command in the command prompt window to see if they were installed properly. This will show you the version.

node -v

npm -v

Once Node is installed, open Package Manager Console in the Cobbler Visual Studio solution and run following two commands. Since the solution already has a package.json, all gulp packages used in the gulp scripts will be downloaded.

npm install -g gulp

npm install –save-dev gulp

You will also need to install Task Runner Explorer to run the gulp tasks. You can down load from here. After installing Task Runner Explorer, close of Visual Studio and launch again. I found that the gulp tasks doesn’t show up in Task Runner unless I relaunch again. At this point if you right click on the gulpfile.js and click on Task Runner Explorer, you should see all the gulp tasks on the left.


Sitecore Powershell Extension Setup

Other than installing SPE Package in Sitecore, you need to download Remoting for SPE 3.3 Package from Marketplace. Copy the files in C:\Users\<username>\Documents\WindowsPowershell\Modules\SPE folder. The script Invoke-RemoteScript.ps1 in this folder will be use to run scripts remotely in the Cobbler Sitecore application.


That should be all for setup. Let’s look at the scripts.

Powershell Scripts

This script reads the items path from the SitecoreItemPath.txt file and serialize the files in the Serialization folder C:\Github\Cobbler\data\serialization. If a line in the SitecoreItemPath.txt marked as -DeployOnce, that item will be skipped if it already exists.

#Change the Powershell Execution Policy so that script can be run from Visual Studio
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
$session = New-ScriptSession -Username &quot;admin&quot; -Password &quot;b&quot; -ConnectionUri &quot;http://cobbler.local&quot;
$ItemPaths=Get-Content &quot;C:\Github\Cobbler\SitecoreItemPath.txt&quot;
foreach($ItemPath in $ItemPaths)
$ItemPath=$ItemPath -replace &quot;/&quot;, &quot;\&quot;
if($ItemPath.Length -gt 0)
if(-not (($ItemPath.ToLower().Contains(&quot;-deployonce&quot;)) -and (Test-Path &quot;$($RootFolder)$($ItemPath.ToLower().TrimEnd(&quot;-deployonce&quot;).Trim())$($ItemExtn)&quot;)))
Write-Host &quot;Serializing Item: $ItemPath&quot;
$script = {
Get-Item -path $params.Path | export-item -ItemPathsAbsolute -Root $params.RootFolder
$args = @{
&quot;Path&quot; = &quot;master:$($ItemPath)&quot;
&quot;RootFolder&quot; = &quot;$RootFolder&quot;
Invoke-RemoteScript -ScriptBlock $script -Session $session -ArgumentList $args

This script is opposite to SerializeItems.ps1 script. It reads the items path from the SitecoreItemPath.txt and create or update the items in Sitecore. Similar to the other script, this script also skips the item if the item is marked as -DeployOnce.

#Change the Powershell Execution Policy so that script can be run from Visual Studio
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
$session = New-ScriptSession -Username &quot;admin&quot; -Password &quot;b&quot; -ConnectionUri &quot;http://cobbler.local&quot;
$ItemPaths=Get-Content &quot;C:\Github\Cobbler\SitecoreItemPath.txt&quot;
foreach($ItemPath in $ItemPaths)
$IsDeployOnceItem = $FALSE
#$ItemPath=$ItemPath -replace &quot;/&quot;, &quot;\&quot;
if($ItemPath.Length -gt 0)
$IsDeployOnceItem = $TRUE
Write-Host $ItemPath
$script = {
if(Test-Path $params.Path)
if(-not ($Params.DeployOnceFlag))
Get-Item -path $params.Path | Import-item -ForceUpdate
Write-Log &quot;Skipped&quot;
Get-Item -path $ParentPath | Import-item -Recurse
$args = @{
&quot;Path&quot; = &quot;master:$($ItemPath)&quot;
&quot;DeployOnceFlag&quot; = $IsDeployOnceItem
Invoke-RemoteScript -ScriptBlock $script -Session $session -ArgumentList $args

Gulp Scripts

I am not going to paste all the gulp script here but just describe the purpose of the scripts.

Publish-Cobbler-Local –> Publishes code to the Cobbler website

Publish-Elf-Local –> Publishes code to the Elf webapi site

Publish-Cobbler-Design-Local –> Publishes design files to Cobbler website

Serialize-Cobbler-Items-Local –> Runs SerializeItems.ps1

Deserialize-Cobbler-Items-Local –> Runs DeSerializeItems.ps1

DeployAll-Local –> Runs multiple scripts to deploy files and Sitecore Items to the website. Full Deployment.

Deploy-Projects-Local –> Publishes only the deployment files. No Sitecore item gets deployed.

Following scripts runs when solution is launched in Visual Studio and watch changes in the css, javascripi and MVC Views. As soon as file(s) change, they get deployed to the website.


Final Word

This is working fine for me in most cases. What this does not provide that I get from TDS is the Sync functionality and interactive way to serialize items in Visual Studio. In the next part, I will integrate this deployment process with Team City and deploy application to remote server.







Posted in Gulp, Powershell, Sitecore | Tagged , , , | 2 Comments

Social Media, Sitecore and B2B Commerce

In my last blog post I discussed about role of Sitecore in B2B Commerce and how big of an opportunity their to use Sitecore to increase the business revenue. In this blog I will discuss why B2B Commerce needs Sitecore for content generation and distributing the content via different channels to increase the business prospect.

I would like to start with a particular experience I had working with one of our B2B Commerce clients. This was beginning of a project and we were talking about requirements. In the Product Detail mock up there were some social media buttons for Facebook, Twitter etc. Our client thought none of their users will be posting the product link in the their Facebook or Twitter pages. I thought same way too. For example, if I am a Customer Representative and I buy products for customers, why would I post product link in my social media pages? I don’t think I would. But, that incident got me thinking about the role of social media for the B2B Commerce.

Before we discuss, how social media can be used in B2B Commerce, let’s discuss about the difference between a B2B Commerce and B2C Commercce user. In B2C Commerce the user is the customer and buys products for personal use. For B2C commerce user, posting the product link on user’s own social media page makes sense because user owns the product and probably would like to share the purchase with user’s followers. On the other hand in B2B, user often is not the customer. For B2B user the lead time for a purchase is often longer than B2C user. B2B user usually tend to do more research by reading blogs, articles that posted by either the manufacturer or by some independent bloggers. They need to do more research because most cases the B2B products are complex and some time it is configured product. What is important for the B2B Commerce site is, to make these research materials easily available for the users through social media channels and forums. This is where Sitecore can help with content generation and pushing contents using Experience Platform.

Before we talk about how or what Sitecore can do, let’s look at this interesting statistics below about a survey conducted on 2013 to understand how B2B decision makers are using social media.


You can find more about this chart in this article. Although, the chart is two years old the situation has not changed a lot. It is clear that B2B business can do a lot to distribute content about their product via social media.

Blogging using Sitecore WeBlog module

The B2B businesses needs to provide information about their products through blogs, articles either in the eCommerce site itself or via links to independent bloggers. Sitecore Marketplace has a great blogging module called WeBlog and it is free. You can find this module here. With the help of WeBlog, the blogging or Product articles can be added with minimum effort. WeBlog comes with many features like tagging, RSS feed generation, commenting, spam filtering, CAPTCHA etc. What is sweet though, it comes with Social sharing ShareThis or AddThis, and other Facebook and Twitter widgets. By having Sitecore and WeBlog module, the B2B business can easily generate blog contents and distribute those through different media channels.

Sitecore Media Framework

I am a serious hobbyist photographer. When I look for a new photo gear and I need to research about it, I sometime prefer to watch videos about the gear, instead of reading long articles. It is a great idea to add videos in the B2B site to easily communicate the product information to the user. In this regard Sitecore Media Framework can be a great help. Sitecore Media Framework integrates with Brightcove and Ooyala professional video streaming service. The B2B site built on Sitecore can take advantage of these media connectors two manage the videos. Some time back, I wrote a Media Connector for integrating Sitecore with Youtube. This module is available in Sitecore Marketplace. I wrote some blog posts too. You can find more here Sitecore Youtube Connector.

Sitecore Experience Platform

Finally, Sitecore Experience Platform can be a great use for selecting appropriate blogs/articles/videos for the B2B user. Based on the user and customer profile and what kind of products the user is interested in, related blogs/articles/videos/Independent blog links can be presented to the user. Each content will provide user the option to share content in the social media. The distribution of content via social media will attract more customer to the B2B site.


B2B business has yet to use the full potential of social media. With the power Sitecore, an integrated B2B eCommerce platform with Sitecore can achieve this goal very quick and it can add great value to the business.

Posted in Commercce, Sitecore, Uncategorized | Tagged , , , , , | Leave a comment

B2B eCommerce, Sitecore and Digital Marketing

As a regular online shopper, when we think about Online Shopping or eCommerce, we mostly picture about B2C eCommerce like Amazon. But,  a huge part of the Online eCommerce is B2B eCommerce. According to Forrester Research, in 2014 the B2B sale in US was $692 billion and that was a very conservative number. This year the prediction is that, it will reach $780 billion and the sale will reach more than a trillion dollar by 2020.


So, what is the role that Content Management System like Sitecore can play to influence the sales of a B2B eCommerce company. Both B2B and B2C eCommerce need some sort of CMS for managing the content, media or digital assets etc. for the eCommerce Site. Those are mostly similar in functionalities. The real difference how the Digital Marketing or in case of Sitecore, how the Experience Platform will be used to drive the B2B sales. But, before diving into that discussion, let’s look at what B2B businesses think about the importance and effectiveness of Content Management System. Below is a survey result I picked up from a Salesforce blog.


As per that survey, Marketing Analytics (Experience Analytics in Sitecore), Content Management (Sitecore), Marketing Automation (Engagement Planning in Sitecore) and Predictive Intelligence (Personalization in Sitecore) are very critical and effective. The good thing is Sitecore is one of the Market Leaders on those areas and Sitecore is capable of putting B2B business in advantageous position if used effectively.

For last few years, I am working mainly in the area of integrating eCommerce with Sitecore and spent lots of time understanding B2B clients’ business and requirements. The B2B commerce is different and most important difference is that, a Business buys products from B2B site for reselling or running it’s business. Where as in B2C an individual buys products for his/her use. In both cases, a user buys products, but in B2B, user buys it for his/her company. That’s why in B2B the buying is less impulsive and that’s make a huge difference on how the Perosonalization features should be used. For example, based on what a B2B user has bought or looked at in the past, a Quick List can be recommended instead of Product. Quick List is a list of Products that can be added in the Cart quickly from the list. It’s effective because it speeds up the buying process for the customer.

Lots of time the B2B user is a Customer Service Representative (CSR) and buys products for the Customers he/she is assigned to serve. A CSR not only buys products for the customer, they also educate them, fill up their questions and create a long term relationship with them. The B2B site can be and should personalized for the CSRs. For example, it will be helpful for CSRs if they can quickly find the documentation of the products. Based on what customer he/she is helping at that time and customer’s past buying data, documents, articles etc. can be made available to CSRs in the personalized website. The CSRs are knowledgeable person and they usually use Quick Order tools to buy products. The Quick Order Tool should be made more accessible to CSRs than regular user. These are just few examples of how personalization can be achieved in B2B site.

More and more B2B businesses are entering in the digital market. The B2B businesses are going from printed catalog to EMail Marketing. Most of ours clients see the power Personalized EMail Marketing and return of investing on Intelligent Targeted Marketing campaign using software like Sitecore. The user base is changing too. More and more people from young generation are entering in the B2B space. On their personal time they are online shopper and they expect similar experience from B2B business too. B2B business must change to satisfy this generation.

The software like Sitecore will make huge impact on the B2B market in the coming years because of its capability to make the business reach to more customers, using less resources and less time. Furthermore it will make shopping experience intriguing  and satisfying for the user and thus it will attract more customers. I am looking forward to integrating more and more B2B business to Sitecore.

Posted in Commercce | Tagged , | 1 Comment

Some Sitecore Habitat Setup Issues

If you haven’t already checked out Sitecore’s Habitat Framework, you should take a look at the solution. Whether you like the Framework architecture or not, it is worth checking the solution. I see it as Sitecore Accelerator  Solution Framework and it is going to be useful for my future projects. You can start from here in Github. Fantastic Architecture discussion and Getting Started videos from Thomas Eldblom (@TEldblom).

This blog is not about introducing Habitat. I recently setup the application in my machine and faced few issues. I will discuss them because they are not mentioned in the Habitat Wiki.

Visual Studio 2015 vs Visual Studio 2013

Habitat Solution requires you to have VS 2015. If you have VS 2015 or you can install it in your machine, just do that before opening up the solution. For me, I have several reasons why I cannot install VS 2015. I did a little hack to make it work for VS 2013.

  • Open the solution file (Habitat.sln) in notepad++ and following lines
    Microsoft Visual Studio Solution File, Format Version 12.00
    # Visual Studio 14
    VisualStudioVersion = 14.0.23107.0
    MinimumVisualStudioVersion = 10.0.40219.1

    WithMicrosoft Visual Studio Solution File, Format Version 12.00
    # Visual Studio 2013
    VisualStudioVersion = 12.0.31101.0
    MinimumVisualStudioVersion = 10.0.40219.1
  • In notepad++ replace 14.0 with 12.0 in all .csproj files in the SRC folder.
  • Now you can open the solution in VS 2013. Open the file gulpfile.js file in VS 2013 IDE and replace 14.0 with 12.0 in the file. There is only one reference. This is required for the valid MSbuild path.

NuGet Package Issue

I had trouble restoring the xUnit.net packages. It seems the xUnit.net packages were created with the more recent version of NuGet. I had to get latest version of NuGet Extension for Visual Studio. After that there was no problem restoring the packages.

Gulp Issues

Habitat uses Gulp to build and publish the files. If you don’t have Gulp already installed in your machine, you have to first install Gulp . In Visual Studio open the NuGet Package Manager Console. Run following two commands.

npm install –global gulp

npm install –save-dev gulp

If you open the Task Runner window, you still may not see the tasks. Click on the refresh button. If that doesn’t work close and open the solution in Visual Studio.


I had some trouble serializing items in Unicorn Control Panel. I must say, I am very new to Unicorn and it was probably my lack of understanding.

  • The Framework Project items has to be serialized first, then the Domain Projects items and lastly Website items. Start with Habitat.Framework.Serialization.
  • Do not click on the Reserialize until you Sync first. For me it wiped out the .yml files.

That’s all the issues I faced so far. I hope it helps some of you out there. Happy coding.

Posted in Framework, Sitecore | Tagged , | Leave a comment

Sitecore MVC and Unity

Using a Dependency Injection (DI) framework in ASP.NET MVC project is most common thing. I have used DI in almost all my MVC project and the one I use most is Unity. I know, Ninject is cool and most used DI statistically :). Unity has come a long way and there are many improvements done recently. When it comes to use Unity in a MVC project, what I most like is its ability to resolve the Types based on name matching. No need to register the Types in code or configuration, just use convention.

This blog post is not about ASP.NET MVC and Unity though. This is about using Unity as DI for Sitecore MVC project. We know Sitecore interferes with MVC framework and handle things differently. If you quickly want to know how Sitecore MVC works, watch Martina Welander’s video presentation Sitecore MVC – Getting Started. But, when it comes to using DI with Sitecore MVC any DI framework can work because Sitecore ultimately delegates the resolution of controllers to ASP.NET MVC. Recently, Mickey Rahman wrote a blog post about, how Sitecore 8.1 is now using Current DependencyResolver to resolve controllers: Changes to Dependency Injection in Sitecore 8.1. So, as long as I set the DependencyResolver to UnityDependencyResolver Sitecore MVC controllers should be resolved properly. The fact is, I even don’t have to do that. All should happen as I install the Unity NuGet package.

Here are the steps.

  • Open up the NuGet Package Manager in you Sitecore MVC solution and search for Unity. Install ‘Unity bootstrapper for ASP.NET MVC’.
    NuGet Package
  • Once this is installed you should see two files added to the App_Start folder, UnityMvcActivator.cs and UnityConfig.cs. If you open the UnityMvcActivator.cs file, you can see how the DependencyResolver is set to UnityDependencyResolver.UnityWebActivator
  • The next step is to open up the UnityConfig.cs file and register the types in the RegisterTypes method. You can register the types one at a time like below
    container.RegisterType<IProductService, ProductService>();
    But, this becomes difficult when your project has too many Types and chain of dependencies. As I said before, I like most the way Unity resolves the Types by convention. To do that you need to use container.RegisterTypes method like below.
    typeof(ApiClient.BaseClient).Assembly), WithMappings.FromMatchingInterface,

    One thing you have to be careful about. Do not use AllClasses.FromLoadedAssemblies(). This will try to resolve all types in the loaded assemblies and Sitecore will throw error. In most cases we know where the classes are that we want DI to resolve. Use AllClasses.FromAssemblies option and include the assemblies that contain your Types.

That’s all for today. Happy coding.

Posted in Uncategorized | Tagged , , , , | 2 Comments

Sitecore Powershell Extention : How Find-Item solved the performance issue

In my last blog post I mentioned that after loading the images in Sitecore with the help of Sitecore Powershell Extension (SPE), I needed to associate the products to the images. I didn’t show that code because it’s a specific requirement. At that time, I thought number of products is close to number of images, that is, 10000. When we started loading actual data from the eCommerce system, client started complaining about the performance of my Powershell Script. I realized actual number of products is way more than 10K, close to 70K.

Following is the portion of the code

		#Associate Product to Image.
		$products=Get-Item master: -Query "/sitecore/content/Product Repository/Products//*[@@templatename='Custom Insite Product' and @Image Name='$ImageName']"
			foreach($product in $products)
				$product.Image=Get-Item $MediaImagePath.TrimEnd(".$FileExtension")
				Write-Host "Assigned image $ImageName to '$productName' ..."
			Write-Host "No product found for this image $Image ..."

After some debugging, I found that the highlighted code is taking most of the time. That code is getting all the products that are supposed to use the image the script just loaded. Is it a SPE thing or Sitecore query problem? To find that, I took the query and ran it in XPath builder. Result shown below.

Sitecore Query Result

The query was taking 104 seconds. That’s for one image. For 10K images this script will never finish.

To improve the performance, I started looking at SPE commands to see if I can query the products from the search index, because, the products are already indexed by another process. I found Find-Item and replace the code with the following code.

		#Associate Product to Image.
		$products=Find-Item -Index commerce_products_master_index -Criteria @{Filter = "Equals"; Field = "_templatename"; Value = "Custom Insite Product"}, @{Filter = "Equals"; Field = "Image Name"; Value = "$ImageName"} | Select-Object -Property Path | Get-Item

			foreach($product in $products)
				$product.Image=Get-Item $MediaImagePath.TrimEnd(".$FileExtension")
				Write-Host "Assigned image $ImageName to '$productName' ..."
			Write-Host "No product found for this image $Image ..."

The result was amazing. Now the query returns products immediately. For 10K images and 70K products, it’s taking little more than half an hour. That’s acceptable performance for this kind of batch process.


Posted in Powershell | Tagged , , | Leave a comment

Bulk Loading Images in Sitecore Media Library Using Sitecore Powershell Extension (SPE)

With the blessings from Sitecore Powershell gurus Adam NajmanowiczMichael West and Mike Reynolds, this is my first blog post on Sitecore Powershell Extension (SPE). I mean it. When I had a problem to solve and I wanted to use SPE for that, I just dropped a question in Twitter and the help was more than what I expected.  They helped me all the way to solve my problem. These guys are awesome. I love Sitecore community.

So, what problem I was trying to solve using SPE? We have an application where, we have more than 10 thousands product items. We have to load and associate the images for those product items in Media Library. Why use SPE? Because, I cannot imagine a content editor loading images using Media Library uploader. Media Library does upload images from zip file. In my case the requirement doesn’t stop on just loading the images. We need to associate images to the product items, which I did using my SPE script. Beside this, since this a SPE is script, we can run it via Sitecore Schedule Task. All the user needs to do is drop the images in the designated folder.

For this blog, I will only describe how loading images in bulk works. Rest of the program of associating images to product items is very specific to our requirement and better not discussed for the sake of keeping length of blog smaller. While working on this problem, I faced couple of issues. Those were solved with the help of SPE team. I described the issues at the end of the blog, in case someone looking for answers.

My research started with this article written by Adam. This describes, how a remote SPE script can be run to load image from the file system to media library. I needed to extend this functionality. The first thing I did was created my own Powershell function that loads multiple images from the file system. Here is the script.

#You need to include cmdlets from following location before using this function
#Invoke-Script 'master:/sitecore/system/Modules/PowerShell/Script Library/Platform/Functions/Remoting'
function Upload-MultipleImages {
        #Remote site host name
        [Parameter(Position=0, Mandatory=$true)]
        #Remote site username
        [Parameter(Position=1, Mandatory=$true)]

        #Password for above username
        [Parameter(Position=2, Mandatory=$true)]

        #Folder location in the File System where images are located
        [Parameter(Position=3, Mandatory=$true)]

        #Image file extension jpg, png etc.
        [Parameter(Position=4, Mandatory=$true)]

        #Media Library location
        [Parameter(Position=5, Mandatory=$true)]

    #Connect to Remote Sitecore site
    Set-SitecoreConfiguration $SiteUrl $Username $Password
    #Get all the Image file names
    $images=Get-ChildItem "$SourcePath*.$FileExtension" -name
    foreach($image in $images)
        #Check if the image already exists in the Media Library location.
        #If exists, skip it but log in the Sitecore log file and in the console
        #If doesn't exists, upload the image and log the message
        if(Test-Path $MediaImagePath.TrimEnd(".$FileExtension"))
            Write-Log "Image $Image already exists... skipping"
            Write-Host "Image $Image already exists... skipping"
            Write-Log "Uploading Image $Image ..."
            Write-Host "Uploading Image $Image ..."
            Upload-SitecoreFile -remotePath $MediaImagePath -File "$SourcePath$image"
            Write-Log "Uploading Image $Image ... done."
            Write-Host "Uploading Image $Image ... done."

Once this function was created, I created below script to call the above function.

#Include SPE remoting scripts
Invoke-Script 'master:/sitecore/system/Modules/PowerShell/Script Library/Platform/Functions/Remoting'
#Include my function
Invoke-Script 'master:/sitecore/system/Modules/PowerShell/Script Library/Nishtech/Functions/Upload-MultipleImages'
#Get context item
$item=Get-Item .
#Get full path of the context item (media folder)
Upload-MultipleImages 'http://myapp.local.com' 'admin' 'b' 'C:\Work\Project Docs\myapp\Images\' 'jpg' "$MediaPath/"
#Show total time taken
new-timespan -Start $StartTime -End $EndTime

I added the above script in the location showed in the following image.


The reason to store the script there was to use Context Menu, like below. I can right click on any media folder and run the script from the context menu.


I also added a rule for the script so that the context menu only shows up for the Media Library folders.


The context menu will not show up if the Module was not activated. Make sure the following option checked for the Module where script has been stored.


Now, if I run the script from the context menu, it runs and shows me the following progress box.


If I click on the ‘View Script Results’ link, I can see the result in the SPE console window snapshot.

ConsoleI wanted to run the same script from the Schedule Task. I had to create a separate script for that because, in this case there is no context media folder. I need to pass the destination folder in from the script itself. Same script, only change is the following line.

Upload-MultipleImages ‘http://myapp.local.com&#8217; ‘admin’ ‘b’ ‘C:\Work\Project Docs\Myapp\Images\’ ‘jpg’ ‘master:/sitecore/media library/Images/Products’

Here is the scheduled task settings


That’s all I had to do for loading images.


As I mentioned in the beginning that I faced couple of issues working on this. I will discuss them now. I learned few things from these issues.

Routing Issue:

One of my application included in the Sitecore desktop has the path that contains /Console/. After installing SPE, I found that this application stopped working and I was getting following error.


I opened up Cognifide.PowerShell.Core.Processors.RewriteUrl in DotPeek and found that following code in the preprocessorRequest pipeline is looking for any route that includes ‘/Console/’ and re-writing it with “/sitecore modules/PowerShell/”

        Uri url = arguments.Context.Request.Url;
        string localPath = url.LocalPath;
        if (localPath.StartsWith("/Console/", StringComparison.OrdinalIgnoreCase))
          WebUtil.RewriteUrl(new UrlString()
            Path = localPath.ToLowerInvariant().Replace("/console/", "/sitecore modules/PowerShell/"),
            Query = url.Query

The solution was to write my own processor and replace the SPE processor with that. I just changed the code to look for more specific route “/Console/Services/”. Simple fix.

        Uri url = arguments.Context.Request.Url;
        string localPath = url.LocalPath;
        if (localPath.StartsWith("/Console/Services/", StringComparison.OrdinalIgnoreCase))
          WebUtil.RewriteUrl(new UrlString()
            Path = localPath.ToLowerInvariant().Replace("/console/", "/sitecore modules/PowerShell/"),
            Query = url.Query

 No Rules Available in the Rules Editor:

When I tried to set the Rule for my script I found that there is no out of the box Sitecore rules in the Rules Editor. Adam pointed me that there is patch exists in the Sitecore Marketplace download area for this issue. After I installed that patch, the rules started showing up.


Final Words:

This is my first encounter with Sitecore Powershell and I am extremely excited by realizing it’s usefulness in Sitecore projects. I will be using it in future and keep blogging about it.


Posted in Powershell, Sitecore | Tagged , | 2 Comments

TDS Timeout and TDSService.asmx 500 error

The title of this blog post was exactly what I googled when I was troubleshooting the TDS deployment timeout issue for one of my projects. Sad, there are not much help out there on how to troubleshoot this issue. There are some suggestions from Hedgehog in the FAQ section. I tried to apply that but that didn’t help much. There was not much information in any log files, except below in the IIS log (500 error).

2015-04-08 17:59:59 POST /_DEV/TdsService.asmx – 80  – – 500 0 0 233538

The problem was sporadic. Sometime it’s 500 error, sometime TDS Timeout and sometime it worked. But, most of the time build was failing. Although there was 500 error in the IIS Log, site was working fine. Pretty frustrating and nothing to look into to get more information.

I wanted to know what exactly going on in the IIS Process. Maybe that will give me some clue. So, I took a memory dump of the Application Pool and opened up the memory dump in Windbg. The first thing I needed to check, what threads were taking most CPU time. Usually that sheds some idea about what’s running in the process. The command to do that in windbg is !runaway. Showing top 8 threads below

0:000> !runaway
User Mode Time
Thread      Time
7:4b4         0 days 0:01:43.265
6:82c         0 days 0:01:12.625
23:adc       0 days 0:00:16.187
21:11d0     0 days 0:00:14.125
22:1088    0 days 0:00:13.828
24:12b4    0 days 0:00:13.578
10:110c     0 days 0:00:07.500
11:8d0       0 days 0:00:06.046

Looks like thread# 7 was consuming most CPU time. Let’s look at the stack for that thread. Again, there were lots of information here but, this shows that the thread was executing the code from Application_Start method from global.asax and it was trying to open connection to the SQL server. It got my attention immediately because failure in Application_Start does throw 500 error if not handled gracefully. So was there any error? What database connection was it opening?

0:007> !clrstack
OS Thread Id: 0x4b4 (7)
        Child SP               IP Call Site
00000005b8507dd8 00007fff25140dca [InlinedCallFrame: 00000005b8507dd8] <Module>.SNIReadSyncOverAsync(SNI_ConnWrapper*, SNI_Packet**, Int32)
00000005b8507dd8 00007fff0c4bcb71 [InlinedCallFrame: 00000005b8507dd8] <Module>.SNIReadSyncOverAsync(SNI_ConnWrapper*, SNI_Packet**, Int32)
00000005b8507db0 00007fff0c4bcb71 DomainNeutralILStubClass.IL_STUB_PInvoke(SNI_ConnWrapper*, SNI_Packet**, Int32)
00000005b8507e80 00007fff0c4a350f SNINativeMethodWrapper.SNIReadSyncOverAsync(System.Runtime.InteropServices.SafeHandle, IntPtr ByRef, Int32)
00000005b8507ef0 00007fff0c4a322e System.Data.SqlClient.TdsParserStateObject.ReadSniSyncOverAsync()
00000005b8507fa0 00007fff0c4a3140 System.Data.SqlClient.TdsParserStateObject.TryReadNetworkPacket()
00000005b8507fe0 00007fff0c4a3918 System.Data.SqlClient.TdsParserStateObject.TryPrepareBuffer()
00000005b8508010 00007fff0c4a67a2 System.Data.SqlClient.TdsParserStateObject.TryReadByte(Byte ByRef)
00000005b8508050 00007fff0c4a57d6 System.Data.SqlClient.TdsParser.TryRun(System.Data.SqlClient.RunBehavior, System.Data.SqlClient.SqlCommand, System.Data.SqlClient.SqlDataReader, System.Data.SqlClient.BulkCopySimpleResultSet, System.Data.SqlClient.TdsParserStateObject, Boolean ByRef)
00000005b85081f0 00007fff0c4a53f4 System.Data.SqlClient.TdsParser.Run(System.Data.SqlClient.RunBehavior, System.Data.SqlClient.SqlCommand, System.Data.SqlClient.SqlDataReader, System.Data.SqlClient.BulkCopySimpleResultSet, System.Data.SqlClient.TdsParserStateObject)
00000005b8508250 00007fff0c4a5285 System.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean)
00000005b85082e0 00007fff0c49ff10 System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(System.Data.SqlClient.ServerInfo, System.String, System.Security.SecureString, Boolean, System.Data.ProviderBase.TimeoutTimer, Boolean)
00000005b8508370 00007fff0c49f1eb System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(System.Data.SqlClient.ServerInfo, System.String, System.Security.SecureString, Boolean, System.Data.SqlClient.SqlConnectionString, System.Data.SqlClient.SqlCredential, System.Data.ProviderBase.TimeoutTimer)
00000005b8508510 00007fff0c49ed6d System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(System.Data.ProviderBase.TimeoutTimer, System.Data.SqlClient.SqlConnectionString, System.Data.SqlClient.SqlCredential, System.String, System.Security.SecureString, Boolean)
00000005b85085c0 00007fff0c49e9ce System.Data.SqlClient.SqlInternalConnectionTds..ctor(System.Data.ProviderBase.DbConnectionPoolIdentity, System.Data.SqlClient.SqlConnectionString, System.Data.SqlClient.SqlCredential, System.Object, System.String, System.Security.SecureString, Boolean, System.Data.SqlClient.SqlConnectionString, System.Data.SqlClient.SessionData)
00000005b8508660 00007fff0c49e5fa System.Data.SqlClient.SqlConnectionFactory.CreateConnection(System.Data.Common.DbConnectionOptions, System.Data.Common.DbConnectionPoolKey, System.Object, System.Data.ProviderBase.DbConnectionPool, System.Data.Common.DbConnection, System.Data.Common.DbConnectionOptions)
00000005b8508770 00007fff0c49e0b8 System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(System.Data.ProviderBase.DbConnectionPool, System.Data.Common.DbConnection, System.Data.Common.DbConnectionOptions, System.Data.Common.DbConnectionPoolKey, System.Data.Common.DbConnectionOptions)
00000005b85087e0 00007fff0c49de75 System.Data.ProviderBase.DbConnectionPool.CreateObject(System.Data.Common.DbConnection, System.Data.Common.DbConnectionOptions, System.Data.ProviderBase.DbConnectionInternal)
00000005b85088a0 00007fff0c49dbb7 System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(System.Data.Common.DbConnection, System.Data.Common.DbConnectionOptions, System.Data.ProviderBase.DbConnectionInternal)
00000005b85088f0 00007fff0c49d2ec System.Data.ProviderBase.DbConnectionPool.TryGetConnection(System.Data.Common.DbConnection, UInt32, Boolean, Boolean, System.Data.Common.DbConnectionOptions, System.Data.ProviderBase.DbConnectionInternal ByRef)
00000005b85089d0 00007fff0c49cfe4 System.Data.ProviderBase.DbConnectionPool.TryGetConnection(System.Data.Common.DbConnection, System.Threading.Tasks.TaskCompletionSource`1<System.Data.ProviderBase.DbConnectionInternal>, System.Data.Common.DbConnectionOptions, System.Data.ProviderBase.DbConnectionInternal ByRef)
00000005b8508a60 00007fff0c49cac7 System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(System.Data.Common.DbConnection, System.Threading.Tasks.TaskCompletionSource`1<System.Data.ProviderBase.DbConnectionInternal>, System.Data.Common.DbConnectionOptions, System.Data.ProviderBase.DbConnectionInternal, System.Data.ProviderBase.DbConnectionInternal ByRef)
00000005b8508b20 00007fff0c49c48b System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(System.Data.Common.DbConnection, System.Data.ProviderBase.DbConnectionFactory, System.Threading.Tasks.TaskCompletionSource`1<System.Data.ProviderBase.DbConnectionInternal>, System.Data.Common.DbConnectionOptions)
00000005b8508b80 00007fff0c499ff9 System.Data.SqlClient.SqlConnection.TryOpenInner(System.Threading.Tasks.TaskCompletionSource`1<System.Data.ProviderBase.DbConnectionInternal>)
00000005b8508c20 00007fff0c499eb6 System.Data.SqlClient.SqlConnection.TryOpen(System.Threading.Tasks.TaskCompletionSource`1<System.Data.ProviderBase.DbConnectionInternal>)
00000005b8508c90 00007fff0c4999bf System.Data.SqlClient.SqlConnection.Open()
00000005b8508d00 00007fff0ee34733 System.Web.DataAccess.SqlConnectionHolder.Open(System.Web.HttpContext, Boolean)
00000005b8508d80 00007fff0ee33859 System.Web.DataAccess.SqlConnectionHelper.GetConnection(System.String, Boolean)
00000005b8508e10 00007fff0ef4a3b5 System.Web.Security.SqlRoleProvider.GetAllRoles()
00000005b8508ed0 00007ffec193af3e Sitecore.Data.DataProviders.NullRetryer.Execute[[System.__Canon, mscorlib]](System.Func`1<System.__Canon>, System.Action)
00000005b850b630 00007ffebdb3ab65 InsiteForSitecore.Web.MvcApplication.Application_Start()

Let’s see if there was any exception in this stack. The command for that is !pe. There was only one error. It’s a NullReferenceException and it didn’t seem to create the problem because the code continued after that method call.

0:007> !pe
Exception object: 00000005b9700ea0
Exception type:   System.NullReferenceException
Message:          Object reference not set to an instance of an object.
InnerException:   <none>
StackTrace (generated):
    SP               IP               Function
    00000005B850B568 00007FFEBDB3B19D Microsoft_Practices_ServiceLocation!Microsoft.Practices.ServiceLocation.ServiceLocator.get_Current()+0xd
    00000005B850B570 00007FFEBDB3B056 InsiteForSitecore_Web!InsiteForSitecore.Web.MvcApplication.ConfigureServiceLocation()+0x46

StackTraceString: <none>
HResult: 80004003

But, it seemed the code was trying to open a SQL Connection and that was taking long time to execute. To find the SQL Connection , I decided to dump the stack object using the command !dso and found the System.Data.SqlClient.SqlConnectionString object in the stack. Dumping this object and then dumping the _usersConnectionString object shows me the connectionstring that it was trying to open the core database. Also, the time out value is set to 15 seconds (default value).

0:007> !do 00000006bb9cfab0
Name:        System.Data.SqlClient.SqlConnectionString
MethodTable: 00007fff0c514950
EEClass:     00007fff0c3594f8
Size:        208(0xd0) bytes
File:        C:\Windows\Microsoft.Net\assembly\GAC_64\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll
              MT    Field   Offset                 Type VT     Attr            Value Name
00007fff1bc80bd8  40002cd        8        System.String  0 instance 00000007b99846e0 _usersConnectionString
00007fff1bc84b98  40002ce       10 ...ections.Hashtable  0 instance 00000006bb9cfb80 _parsetable
00007fff0c516670  40002cf       18 ...mon.NameValuePair  0 instance 00000006bb9cfd20 KeyChain
00007fff1bc7f370  40002d0       28       System.Boolean  1 instance                1 HasPasswordKeyword
00007fff1bc7f370  40002d1       29       System.Boolean  1 instance                0 UseOdbcRules
00007fff1bc82458  40002d2       20 ...ity.PermissionSet  0 instance 00000006bb9d0dc8 _permissionset
00007fff1abd8228  40002c9      2d8 ...Expressions.Regex  0   shared           static ConnectionStringValidKeyRegex
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:NotInit  0000000a10219980:NotInit  <<
00007fff1abd8228  40002ca      2e0 ...Expressions.Regex  0   shared           static ConnectionStringValidValueRegex
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:NotInit  0000000a10219980:NotInit  <<
00007fff1abd8228  40002cb      2e8 ...Expressions.Regex  0   shared           static ConnectionStringQuoteValueRegex
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:NotInit  0000000a10219980:NotInit  <<
00007fff1abd8228  40002cc      2f0 ...Expressions.Regex  0   shared           static ConnectionStringQuoteOdbcValueRegex
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:NotInit  0000000a10219980:NotInit  <<
00007fff1bc7f370  40010e7       2a       System.Boolean  1 instance                0 _integratedSecurity
00007fff1bc7f370  40010e8       2b       System.Boolean  1 instance                1 _connectionReset
00007fff1bc7f370  40010e9       bc       System.Boolean  1 instance                0 _contextConnection
00007fff1bc7f370  40010ea       bd       System.Boolean  1 instance                0 _encrypt
00007fff1bc7f370  40010eb       be       System.Boolean  1 instance                0 _trustServerCertificate
00007fff1bc7f370  40010ec       bf       System.Boolean  1 instance                1 _enlist
00007fff1bc7f370  40010ed       c0       System.Boolean  1 instance                0 _mars
00007fff1bc7f370  40010ee       c1       System.Boolean  1 instance                0 _persistSecurityInfo
00007fff1bc7f370  40010ef       c2       System.Boolean  1 instance                1 _pooling
00007fff1bc7f370  40010f0       c3       System.Boolean  1 instance                0 _replication
00007fff1bc7f370  40010f1       c4       System.Boolean  1 instance                0 _userInstance
00007fff1bc7f370  40010f2       c5       System.Boolean  1 instance                0 _multiSubnetFailover
00007fff1bc837c8  40010f3       2c         System.Int32  1 instance               15 _connectTimeout
00007fff1bc837c8  40010f4       98         System.Int32  1 instance                0 _loadBalanceTimeout
00007fff1bc837c8  40010f5       9c         System.Int32  1 instance              100 _maxPoolSize
00007fff1bc837c8  40010f6       a0         System.Int32  1 instance                0 _minPoolSize
00007fff1bc837c8  40010f7       a4         System.Int32  1 instance             8000 _packetSize
00007fff1bc837c8  40010f8       a8         System.Int32  1 instance                1 _connectRetryCount
00007fff1bc837c8  40010f9       ac         System.Int32  1 instance               10 _connectRetryInterval
00007fff0c520418  40010fa       b0         System.Int32  1 instance                0 _applicationIntent
00007fff1bc80bd8  40010fb       30        System.String  0 instance 00000008b87ed558 _applicationName
00007fff1bc80bd8  40010fc       38        System.String  0 instance 00000005b87d1420 _attachDBFileName
00007fff1bc80bd8  40010fd       40        System.String  0 instance 00000005b87d1420 _currentLanguage
00007fff1bc80bd8  40010fe       48        System.String  0 instance 00000006bb9cfcf8 _dataSource
00007fff1bc80bd8  40010ff       50        System.String  0 instance 0000000000000000 _localDBInstance
00007fff1bc80bd8  4001100       58        System.String  0 instance 00000005b87d1420 _failoverPartner
00007fff1bc80bd8  4001101       60        System.String  0 instance 00000006bb9cfe28 _initialCatalog
00007fff1bc80bd8  4001102       68        System.String  0 instance 00000006bb9d01a0 _password
00007fff1bc80bd8  4001103       70        System.String  0 instance 00000006bb9d00e0 _userID
00007fff1bc80bd8  4001104       78        System.String  0 instance 00000005b87d1420 _networkLibrary
00007fff1bc80bd8  4001105       80        System.String  0 instance 0000000000000000 _workstationId
00007fff0c5326a8  4001106       b4         System.Int32  1 instance             2008 _typeSystemVersion
00007fff1bc85590  4001107       88       System.Version  0 instance 00000007b99881f8 _typeSystemAssemblyVersion
00007fff0c52a450  400110a       b8         System.Int32  1 instance                0 _transactionBinding
00007fff1bc80bd8  400110b       90        System.String  0 instance 0000000000000000 _expandedAttachDBFilename
00007fff1bc84b98  40010e5      ef0 ...ections.Hashtable  0   shared           static _sqlClientSynonyms
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:00000007b9988238 0000000a10219980:00000005b889f280 <<
00007fff1bc84b98  40010e6      ef8 ...ections.Hashtable  0   shared           static _netlibMapping
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:0000000000000000 0000000a10219980:0000000000000000 <<
00007fff1bc85590  4001108      f00       System.Version  0   shared           static constTypeSystemAsmVersion10
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:00000007b99881f8 0000000a10219980:00000005b889f240 <<
00007fff1bc85590  4001109      f08       System.Version  0   shared           static constTypeSystemAsmVersion11
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:00000007b9988218 0000000a10219980:00000005b889f260 <<

0:007> !do 00000007b99846e0
Name:        System.String
MethodTable: 00007fff1bc80bd8
EEClass:     00007fff1b611aa8
Size:        266(0x10a) bytes
File:        C:\Windows\Microsoft.Net\assembly\GAC_64\mscorlib\v4.0_4.0.0.0__b77a5c561934e089\mscorlib.dll
String:      Data Source=(local);Initial Catalog=Sitecore_core;Integrated Security=False;User ID=******;Password=******
              MT    Field   Offset                 Type VT     Attr            Value Name
00007fff1bc837c8  40000ab        8         System.Int32  1 instance              120 m_stringLength
00007fff1bc81f38  40000ac        c          System.Char  1 instance               44 m_firstChar
00007fff1bc80bd8  40000ad       18        System.String  0   shared           static Empty
                                 >> Domain:Value  00000005b83561f0:NotInit  0000000a10215940:NotInit  0000000a10219980:NotInit  <<

At this point I know that the longest running thread was trying to open a connection to the core database, the connection timeout was 15 seconds and the thread was running longer than that. There was no timeout error in the stack but, I thought increasing timeout might fix the issue. I increased the timeout and that fixed the issue.

Final Words

I wrote this blog not to tell what I found but to talk about an approach to troubleshoot this kind of problem. I hope this helps others with similar issues.


All about Windbg
SOS.dll (SOS Debugging Extension)

Posted in Debugging, Sitecore | Tagged , , , | Leave a comment

Dynamic Multilist with Search

Recently, in my project, I had a requirement where, I had to show Products in a Carousel. The Content Editor would choose some Products for the Carousel from a “MultiList with Search” field in the Carousel item. The Products are stored in a Bucket. The “MultiList with Search” is very useful in this case because the Products can be search in the control and also, in the Carousel template for the Source of the “MultiList with Search” field, we can give a StartSearchLocation and Filters like below, so that Sitecore will search from that location and apply the filters.


The problem with my requirement was that, the Item Id or the path for the StartSearchLocation could be different for different website. The Products bucket for websites are located under the website node. So, I need a “Multilist with Search” , where the StartSearchLocation can change or in other words, I need to query to find the StartSearchLocation of the Products bucket for the website. Not too difficult to do this. Here is how I did it.

BucketList is the class I needed to extend. It is a FieldType and it is derived from SearchList. The Source property of the FieldTypes is what I had to change. This property is located in MultiListEx which is the parent class of SearchList. The StartSearchLocation now can contain query in addition to ItemId. If it contains the word ‘query:’ that means the StartSearchLocation ItemId has to be found from query. The following code is doing exactly that. If the word ‘query:’ is not present code assumes that StartSearchLocation contains the ItemId and just return the base.Source.

using Sitecore.Buckets.FieldTypes;
namespace Sitecore.SharedControls.Buckets.FieldTypes
    public class DynamicBucketList : BucketList
        public new string Source
            get { return base.Source; }
                if (value.Contains("query:"))
                    var sourceString = value;
                    var queryString = sourceString.Substring(sourceString.IndexOf("query:"),
                            ? sourceString.IndexOf('&') - sourceString.IndexOf("query:")
                            : sourceString.Length - sourceString.IndexOf("query:"));
                    //Find out the item from the query. The query should return only one item.
                    var contextItem = global::Sitecore.Context.ContentDatabase.Items[this.ItemID];
                    var queryItem = contextItem.Axes.SelectSingleItem(queryString.Substring("query:".Length));
                    base.Source = value.Replace(queryString, queryItem.ID.ToString());
                    base.Source = value;

After this I needed to create the new “Dynamic Multilist with Search” FieldType in core database like below.

Dynamic Multilist with Search

and added the following in a config file

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
      <source patch:after="*[@namespace='Sitecore.Buckets.FieldTypes']" mode="on" namespace="Sitecore.SharedControls.Buckets.FieldTypes" assembly="Sitecore.SharedControls" prefix="contentExtension" />

Selected the Field Type like below in the Carousel Template. The StartSearchLocation was a query.
StartSearchLocation=query:./ancestor-or-self::*[@@templatename=’Website’]/Product Repository/Products


That’s it. Now, I can use a query to dynamically select the StartSearchLocation for the Multilist with Search.

Posted in Commerce Connect, Sitecore | Tagged , , , , | 1 Comment