Sitecore Identity Part 3: Connecting to External Identity Provider

Introduction

Sitecore Identity Provider was implemented based on IdentityServer4 framework. IdentityServer4 doesn’t dictate how authentication to be done or what application can use the identity provider. It’s up to the implementer to decide that. In previous blog article, we discussed how a third party application can authenticate using Sitecore Identity Provider. In this blog we will look at the other side of Sitecore Identity. We know that Sitecore Identity authenticates users using the membership provider, but Sitecore Identity can delegate the authentication to other identity provider too. In fact Sitecore Identity comes with inbuilt AzureAd subprovider. If you enable it you should be able to authenticate against the Azure Active Directory. I added the following updated diagram to show how subprovider fits into the architecture.

Sitecore Identity with Subproviders

Configuring Azure Ad Subprovider

Sitecore provided some documentation about how to configure out of the box Azure Ad subprovider. It’s not in very detail and it takes some effort to configure it. I was going to write about it, but I had a problem setting it up end to end. So, I posted this question in Stack Exchange and through that I found this excellent blog post which explains everything in detail. I am skipping that part. I captured the following animation to show you how authentication happens once you setup Azure Ad subprovider.

If you setup the loginpage attribute of the shell site in Sitecore.Owin.Authentication.IdentityServer.config as
$(loginPath)shell/SitecoreIdentityServer/IdS4-AzureAd, Sitecore will skip Sitecore Identity Provider and use Azure Ad provider directly to authenticate.

Custom External Subprovider

Let’s write some code to implement a custom subprovider. I will be utilizing IdentityServer Demo site https://demo.identityserver.io/ to create our subprovider. The reason to use IdentityServer Demo is, it is simple and no configuration is needed on the IdentityServer side. If you go to that site, you will find several grant types. The one we will be using is Implicit grant type and the ClientId for that is ‘implicit’. Our subprovider will be using this ClientId to get access to the IdentityServer identity provider.
Sitecore Identity is a Sitecore Host application. Sitecore Host is a new framework introduced in Sitecore 9.1 that you can use to create Sitecore Services. The benefit of creating services using Sitecore Host is that, you get all the common features of Sitecore Host, like logging, Dependency Injection etc. right out off the bat. The subprovider we will be creating is a Sitecore Host plugin and since Sitecore Host supports dynamic plugin loading, our plugin will be loaded as soon as we drop the plugin the ‘sitecoreruntime’ folder under Sitecore Identity root folder.
You can find the plugin project in the github repo. It’s name is
Sitecore.Plugin.IdentityProvider.Ids4Demo. Here are the important code snippets.


The Sitecore.Plugin.IdentityProvider.Ids4Demo.xml contains the configuration of the subprovider. The <Enabled> property should be true to make the subprovider available to Sitecore Identity. <AuthenticationSchema> is to identify the subprovider and it is used along with the IdentityProvider name in Sitecore to configure which subprovider to be used for different site. <ClientId> contains the Id to be used to connect to IdentityServer demo provider. <DisplayName> is button caption that will be used in the Sitecore Identity login page for this subprovider.<ClaimsTranformations> are used to translate claims from the subprovider to Sitecore Identity claims.

ConfigureSitecore class adds the subprovider in the services collection with appropriate option. In our case the code adds Authority as IdentityServer demo provider and also indicates that it is an external Identity Server to be used for sign in.

Copy the code as shown the following directory structure under the Sitecore Identity root folder.

sitecoreruntime
│ license.xml

└───production
│ Sitecore.Plugin.IdentityProvider.Ids4Demo.dll
│ Sitecore.Plugin.IdentityProvider.Ids4Demo.xml

└───sitecore
└───Sitecore.Plugin.IdentityProvider.Ids4Demo
│ Sitecore.Plugin.manifest

└───Config
Sitecore.Plugin.IdentityProvider.Ids4Demo.xml

Launch Sitecore shell site. As you are redirected to Sitecore Identity site you should see new login button in the site for the new subprovider as shown in the below picture.

Click on the new subprovider button, it should take you to IdentityServer demo provider for authentication. Enter username bob and password bob, you will be signed and redirected to Sitecore Identity provider. It works like below.


Unfortunately, at this point, when website is redirected to Sitecore Identity, it is throwing a server error. The log shows the following error, ‘ The payload was invalid’.

System.Security.Cryptography.CryptographicException: The payload was invalid.
   at Microsoft.AspNetCore.DataProtection.Cng.CbcAuthenticatedEncryptor.DecryptImpl(Byte* pbCiphertext, UInt32 cbCiphertext, Byte* pbAdditionalAuthenticatedData, UInt32 cbAdditionalAuthenticatedData)
   at Microsoft.AspNetCore.DataProtection.Cng.Internal.CngAuthenticatedEncryptorBase.Decrypt(ArraySegment`1 ciphertext, ArraySegment`1 additionalAuthenticatedData)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.UnprotectCore(Byte[] protectedData, Boolean allowOperationsOnRevokedKeys, UnprotectStatus& status)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.DangerousUnprotect(Byte[] protectedData, Boolean ignoreRevocationErrors, Boolean& requiresMigration, Boolean& wasRevoked)
   at Microsoft.AspNetCore.DataProtection.KeyManagement.KeyRingBasedDataProtector.Unprotect(Byte[] protectedData)
   at Microsoft.AspNetCore.DataProtection.DataProtectionCommonExtensions.Unprotect(IDataProtector protector, String protectedData)
   at IdentityServer4.Infrastructure.DistributedCacheStateDataFormatter.Unprotect(String protectedText, String purpose)
   at Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler.ReadPropertiesAndClearState(OpenIdConnectMessage message)
   at Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler.HandleRemoteAuthenticateAsync()

I couldn’t get to the bottom of this problem. I checked the id_token and payload looks valid. So, I opened a support ticket with Sitecore. I have to wait until Sitecore provides me an explanation of why the error is happening for other Identity Providers, but not for Azure Ad. Same problem is happening with Okta too.

I decided to publish this blog before the issue is solved because, I don’t see any problem with the approach. As soon as I get a resolution, I will update the blog. Finger crossed.

References

Update

Sitecore answered the question about the issue I mentioned above and it resolved the issue. In the ConfigureSitecore class ConfigureServices method CallbackPath was missing. After I added options.CallbackPath = “/signin-idsrv”; my custom identity provider started working without any problem. I updated the code in Github also.

Posted in Security, Sitecore, sitecore Identity | Tagged , , , , , , | 17 Comments

Sitecore Identity Part 2: Identity Client and Resource Authorization

Introduction

In the previous blog we discussed fundamentals of OAuth 2.0 and Open Id Connect. We discussed the implementation of these two protocols in Sitecore Identity Server using IdentityServer4 framework. We also discussed, how Sitecore Identity will play the role in Sitecore Micro Service Architecture.
In this blog, we will show, how we can create a Sitecore Identity Client that authenticates against Sitecore Identity Server and how this client can authorize via Sitecore Identity Server to use Sitecore Services. For the purpose of demonstration, I have created a ASP.Net Core MVC client that will use hybrid grant type to authenticate and authorize against Sitecore Identity Server. I also created a sample API service that returns data using Sitecore Layout Service and it is scoped using sitecore.profile.api scope. This sample service can be seen as a sitecore resource.
Following diagram shows how different parts of this example fits into the architecture.

Following animation shows my Sitecore Identity Client in action. This shows

  • Client is authenticated by Sitecore Identity Server and displays the identity information.
  • Renewing token from the Identity Server.
  • Calling Sitecore Service to demonstrate authorizing Sitecore Resource via Sitecore Identity.
  • Logging out.

Sitecore Identity Client in Action

Sitecore Identity Configuration

For an Sitecore Identity Client to be recognized by Sitecore Identity Server, we have to provide some information to it. There are two ways to do this, we can use dependency injection or we can register the client in IdentityServer.xml located in <website root>\sitecore\Sitecore.Plugin.IdentityServer\Config. There is a Clients section in IdentityServer.xml, we can either use the DefaultClient, which is used by Sitecore (Sitecore is a Sitecore Identity Client) or we can add a new client. I chose to add a new Client. Let’s go through that and understand important client configuration. Following fields are most important.

  • ClientId: Uniquely identify a client’s behavior. When a Client use this Id, Sitecore Identity Server use this Id to find out the configuration to be used for the client. Same ClientId can be used by more than one client. For example, instead of creating a new client in the config for my Identity Server client, I could have used DefaultClient, which is used by Sitecore.
  • ClientSecret: Client Secret is used to authenticate client against Sitecore Identity Server. This field is optional.
  • AllowOfflineAccess: Sitecore Identity supports offline_access scope, which is used to renew access token. This field in the configuration determines if offline_access scope is supported for client or not. If not supported, client will not be able to renew the token and will have to authenticate again to get a new token after token expires.
  • RequireConsent: This field determines if Identity Server needs to get consent from the resource owner before the client is authorized to get access to the resources. Typically, it is an intermediate page implemented in the Identity Server to show the user a list of scopes she is giving access to the client. Sitecore Identity Server didn’t implement this page yet. So, if you set this field ‘true’, you will get page not found error.
  • AllowedGrantTypes: This field lists the OAuth grant types allowed for the client. Supported grant types for Sitecore Identity can be found in the previous blog.
  • RedirectUris: This field contains list of URIs where Identity Server will redirect this user after login. The OpenId Connect authentication middleware handles /singin-oidc route and populates the user information before redirecting user to the return url. The
    {AllowedCorsOrigin} is replaced with AllowedCorsOrigins field mentioned below. Sitecore uses /identity/signin  for RedirectUris, not the standard oidc route.
  • PostLogoutRedirectUris: This field is similar to RedirectUris, except that this is the URL where Identity Server redirects the client after logging out. Standard oidc route is /signout-callback-oidc
  • AllowedCorsOrigins: This field lists the domain names allowed to use Identity Server for authentication and authorization. In my case I am running the client website at http://localhost:5002.

Sitecore Identity Client

Now we have to create an Identity Client. For this I haven’t reinvented the wheel. I took the same approach as the IdentityServer samples and added a new sample client for Sitecore Identity Server. My samples can be found here. This is a fork out of original IdentityServer4 samples with Sitecore Client and sample Sitecore API (we will discuss in the next section).
I added the following code in Startup.cs. This configures the client. Lines of code we need to pay attentions are, Authority, ClientId, ClientSecret and scopes. Authority is Sitecore Identity server URL. ClientId and ClientSecret are same as in the Sitecore Identity Configuration above. Also, this client wants to authenticate using OpenId Connect, access Sitecore Profile and Profile API and renew the expired access token without logging in as declared in the scopes.

Following method in the HomController when executes, authentication middleware redirect user to the Authority URL set in the Startup.cs, in this case redirects to Sitecore Identity Server. Once user authenticates, middleware use RedirectUri to collect user information and redirect user to the secure page, where the user, claims and token information are displayed.

There are methods in HomeController to handle Logout and Renewal of token. You can download the code or go to github and explore. I have added the github link at the end of this blog.
Following method is used to make a call to an API which is retrieving data using Sitecore Layout Service API and this API resource is protected by Sitecore Identity Server. The API is like a micro service that the Identity Client is accessing using the access token Identity Server generated on resource owner’s behalf.

Sitecore Sevice API

I created a sample API project which is using Layout Service API to retrieve data for the root item from my Sitecore 9.1 instance. For this I had to install Sitecore JSS Server package. I provided prerequisite for this API project in the Readme of the github repository.
If you look at the Startup code for this project, the APIs from this project can be accessed using Bearer token generated by the Sitecore Identity Server and APIs are available only to clients who are granted sitecore.profile.api scope. This API project only allow client that is coming from http://localhost:5002.

There is only one API controller which is returning simply layout data using Sitecore Layout Service as shown below.

Conclusion

In this blog we discussed how we can register a client for Sitecore Identity Server, how to create a Sitecore Identity Client and how to create a resource that can be authorized by Sitecore Identity Server. All the code created for this blog can be downloaded from github link in the reference section below.

References

Posted in Security, Sitecore, sitecore Identity | Tagged , , , , | 1 Comment

Sitecore Identity Part 1

Introduction

There was a time when applications were simple and so was the application security. Things have changed with rise of internet. Software applications are not confined in a desktop, in a server or in an organization anymore. Today when we think about applications we often think about services. The service based architecture in today’s world made application security a big challenge. An application built today probably has to use multiple services, developed and hosted by different organizations or different teams within the same organization. How can application securely use services, so that proper authentication and authorization can be applied? How does security flows from application to services?

The questions I posed above are not new challenges today due to the rise of internet or service based architecture. We looked for answers for these questions when we started building browser based applications. In the simplest form, we authenticated users using form based authentication, initially custom and later on using something like, Microsoft Membership Provider or Widows Authentication. But, application itself often use other resources. A common example is using a database. In such a case, popular way is to use database connection string and provide authentication information via connection string. What if,we want to authenticate to the database using user’s original credentials (delegated authorization). That requires application to impersonate the user and it’s much harder problem to solve. In windows platform it was solved by using Kerberos authentication protocol. It is much harder to use Kerberos in service based distributed internet applications across domains, where applications/services use variety of platforms. Only common thing in service based distributed internet applications is the HTTP protocol. We needed a security protocol that uses HTTP as the communication protocol. Enter OAuth, an open protocol for authentication and authorization.

why I am not talking about Sitecore Identity Server yet? Because, I don’t want to explain things superficially. Sitecore Identity Server is built on IdentityServer4, which is a framework to build Identity Provider based on OAuth 2.0 and OpenID Connect. If we do not understand the problem in hand, we will not be able to understand why Sitecore has to create Identity Server. What’s the architectural need for future Sitecore Platform to support Authentication and Authorization this way?

Identity Problem

In the following table I tried to capture the Identity problems of modern web applications and technologies used to address the problems.

Form Authentication

The simplest way to authenticate and authorize a user is to use form based authentication, where we take the user credentials and validate it against stored information in the database or other systems like Active Directory. It is not that Form Authentication can not be secured enough, it is just that there are lot limitations when the application needs to integrate with other systems securely. Sitecore, before 9.1, used Form Authentication via Microsoft Membership Provider. In Sitecore 9.1, Sitecore removed security from the core application and created separate Identity Provider. This helps Sitecore to use same Identity Service for securing micro services, architecturally where Sitecore is heading to. You can still use Form Authentication in Sitecore 9.1 if you need to. You need to enable this config, Sitecore.Owin.Authentication.IdentityServer.Disabler.config.example, which disables the Sitecore Identity Server Provider based authentication and reset the login page to old login page.

Security Assertion Markup Language (SAML)

One of most important requirements in web based application is Single Sign-On (SSO). With SSO you can sign in once and authenticate to one or many application without logging in again. Although, it is possible to use Form Authentication with cookies to authenticate to an application once and log in to the application without logging in again as long as the cookies doesn’t expire, it only works within the same domain. You cannot use cookie across domains because of browser’s Same Origin Policy (SOP). So, you cannot SSO to more than one application that has different domain name. 
Security Assertion Markup Language (SAML) is an open standard based on XML based markup language for security assertion. SAML addresses the cross domain SSO by separating Service Provider from Identity Provider. 
Following diagram show the SAML SSO Flow. This diagram shows how SAML authenticates user using Identity Provider and use the same authentication information to log in to applications (Service Providers) across different domain.

SAML SSO Flow

OAuth

SAML addressed the SSO issue by separating Identity Provider from Service Provider. Now we can use one system to authenticate to all applications. But, in the above diagram, consider this scenario, SP1 wants to use service from SP2? How can SP1 authenticate to SP2 on behalf of user without giving SP2 user’s password? This is the Delegated Authorization problem and OAuth addresses this problem.

Let’s look at the following diagram to understand how OAuth addresses the Delegated Authorization problem.  You can log in to StackExchange using your Google account. If you choose to do that, StackExchange will connect to Google to access your Google profile and email. Although, I said log in, which is authentication, in this case authorization to get the access to the user’s Google profile indicates user has authenticated to Google successfully. This diagram shows how that communication works.

We will discover more about OAuth 2.0 communication when we will discuss about OAuth 2.0 code flow. 

OpenID Connect

OAuth 2.0 seems to address all the problem that we wanted to solve. Then, why we need OpenID Connect? To answer this, we need to understand that OAuth 2.0 is an authorization framework, it is not an authentication protocol. People started using OAuth 2.0 for authentication also and the result of that, each solution has become one of its kind. We needed a standard way to do the authentication in OAuth 2.0 authorization server. OpenId Connect is a protocol on the top of OAuth 2.0 to verify identity of end user on a authorization server.

Proof Key for Code Exchange (PKCE)

PKCE pronounce as pixy is a way to securely authorize with the authorization server without using Client Secret. What is Client Secret? In the above diagram when StackExchange connects to Google and ask for the Request Token, first it needs to identify to Google. It will provide a Client Id and Client Secret, which it already provided to Google when it registered the application with Google. Client Secret should be protected by StackExchange, otherwise, any application can claim that it is StackExchange. For the sensitive nature of the Client Secret, it is only used to authorize Web Server application. You wouldn’t use it in Javascript based application or mobile application because it can be seen or intercepted. PKCE protocol helps device or mobile app to address this problem. Since it is not needed to understand PKCE to understand OAuth, we will not explore much about it.

OAuth 2.0 Terminology

Let’s talk about some OAuth 2.0 terminology because we need to know them to understand the OAuth Code Flow (Code Flow is a terminology too).

Resource Owner

Resource Owner is really a user. In the OAuth term it is called Resource Owner because OAuth provides access to some resources to other application on behalf of the owner of the resources. For example, StackExchange is asking access to my Google profile that I own. In this case, I am the Resource Owner. 

Client

Client is an application which is asking access to the resources. In StackExchange example, StackExchange is the Client. 

Authorization Server

The Authorization Server is the system that authorize the user. The server where Client connect and asks for access to some resources. In StackExchange example, Google is the Authorization Server.

Resource Server

Resource Server is the one that contains the resources. In our StackExchange example, the Client is asking to access user’s profile. In this case profile also is stored in Google server. Resource can be something other than profile, for example, it could be documents in a folder in Google Drive. In that case Google Drive is the Resource Server.

Authorization Grant

Authorization Grant is the Resource Owner’s (user) permission to the Client to access the Resource. In the StackExchange example, when user say ‘Yes’ to Google to give access to StackExchange, user grants the permission and Google provides a authorization code to StackExchange, so that, StackExchange can get an Access Token, which it will use to access user’s profile. There are different mechanism to communicate Authorization Grant. We will discuss different grant types later.

Access Token

Client’s ultimate goal is get the Access Token. Once client has the Access Token, it can get to the Resources anytime in the Resource Server, until the Access Token expires.
In OAuth 2.0 an Access Token is just a string type. The Access Token will be validated before a resources is delivered. One way to validate an Access Token is to store the Access Token in the server and every time client sends the Access Token, resource server checks against the stored token. This is why Access Token is sometime called Reference Token. If the Access Token is a JSON Web Token (JWT), it can be both stateless and stateful. The JWT Access Token is signed, thus, it can be validated by the client. This is stateless token because token doesn’t need to be stored in the resource authorization server. JWT Access Token can also be validated against the server using OpenID Connect introspection endpoint and this is stateful Access Token. We will discuss more about Open ID Endpoints later.

Scope

Scope is a string value that is already defined in the authorization server describes granularity of resource access. Client sends the Scope to the authorization server to ask what level of resource access client needs. In the StackExchange example, it needs user’s profile access, so, the Scope can be a string value ‘profile’.  

Consent

The flip side of Scope is Consent. User gets to see the scope of access that client is asking for and grant access to specific scopes. In the StackExchange example, Google will send the profile scopes for the consent to the user and user has to give consent.

Revisit the Authorization

Now, we understand the OAuth 2.0 terminology, let’s revisit the StackExchange example one more time. Few things to notice in the following diagram.

1. StackExchange provides client_id so that Google can identify the client.
2. StackExchange tells Google what grant type (querystring response_type) to use, which is code. This means Google will exchange a code to tell StackExchange that user agreed to give access to the resources.
3. StackExchange tells Google what url (querystring redirect_uri) to use to send the authorization code by providing a callback uri.
4. StackExchange tells Google what resources it needs to access using scope (querystring scope).
5. Once user gives consent, Google sends to authorization code to StackExchange.
6. StackExchange sends the authorization code and the client secret to Google and ask for access token.
7. Google verify the client secret to verify the client and verify the authorization code and sends an access token to StackExchange with access scope included in the token.
8. StackExchange uses the access token to access resources. If StackExchange try to access anything outside to scope included in the access token, Google denies access.

Back Channel and Front Channel

We will talk about different grant types later, but we already discussed one grant type above, it is called Authorization Code grant type. Channel is the way that a grant type communicates with authorization server. If it is a server side communication, like StackExchange web server to Google authorization server, it is Back Channel communication. In the above example when StackExchange is asking for the access token by sending authorization code and client secret, that’s Back Channel communication. If the communication is only through browser, it is a Front Channel communication. In the above example, exchanging the authorization code via StackExchange callback uri is Front Channel communication. The authorization code gets transferred via query string or form post (querystring response_mode). The Back Channel communication more secured than Front Channel communication as it happens between server to server and chance of intercepting the communication is lot less.

Authentication using OpenID Connect

We briefly discussed OpenID Connect. Let’s take a look at the same StackExchange workflow if the authentication is done using OpenID Connect. As discussed before, OpenID is a thin layer on the top of OAuth 2.0, that standardized authentication using OAuth authorization server. To use OpenID Connect, both authorization server and the client has to implement OpenID Connect protocol.
The following diagram shows the Code Flow when OpenID Connect protocol is used. Flow is exactly same as the one we described in the Revisit the Authorization section, except, scope includes openid and get the id_token back. The id_token includes user’s information (we will discuss content of id_token later). Also, authorization server implements standard API endpoint /userinfo, which can be accessed using the access token to retrieve more user information if needed.

In the above discussion, I used StackExchange as an example to describe the OAuth authorization flow, in the real life StackExchange authentication and authorization using google account is lot more than what I described. The concept is not different though. Let’s look at the requests below. I used bold font so that, you can recognize them from our earlier discussion.

Request to Google:
https://accounts.google.com/o/oauth2/auth?client_id=717762328687-p17pldm5fteklla3nplbss3ai9slta0a.apps.googleusercontent.com&scope=profile+email&redirect_uri=https://stackauth.com/auth/oauth2/google&state={"sid":4,"cdl":"https://stackexchange.com/users/login-or-signup/delegated?returnurl=https%3a%2f%2fstackexchange.com%3f_%3d247192781","st":"b332ab522f5a916ec48228026c76ff3f6f249c0392328cab4563a433d0dc405b","ses":"018274bb2cd74c12a71f175403243ff8"}&response_type=codeHTTP/1.1
Callback from Google:
https://stackauth.com/auth/oauth2/google?state={"sid":4,"cdl":"https://stackexchange.com/users/login-or-signup/delegated?returnurl=https%3a%2f%2fstackexchange.com%3f_%3d247192781","st":"b332ab522f5a916ec48228026c76ff3f6f249c0392328cab4563a433d0dc405b","ses":"018274bb2cd74c12a71f175403243ff8"}&code=4/rgBAjiN72soIG0mXXOLC5tN8OOPOaNVGNeJ_M1ajoZ9GvwxxxxxQBUQtd4LP9C3v0DIZQ2-ZhMycPFOLhudH-7A&scope=email+profile+https://www.googleapis.com/auth/userinfo.email+https://www.googleapis.com/auth/userinfo.profileHTTP/1.1
StackExchange Redirects to:
https://meta.stackexchange.com/users/oauth/google?code=
4/rgBAjiN72soIG0mXXOLC5tN8OOPOaNVGNeJ_M1ajoZ9GvwxxxxxQBUQtd4LP9C3v0DIZQ2-ZhMycPFOLhudH-7A&state={"sid":4,"cdl":"https://stackexchange.com/users/login-or-signup/delegated?returnurl=https%3a%2f%2fstackexchange.com%3f_%3d247192781","st":"b332ab522f5a916ec48228026c76ff3f6f249c0392328cab4563a433d0dc405b","ses":"018274bb2cd74c12a71f175403243ff8"}&s=018274bb2cd74c12a71f175403243ff8HTTP/1.1

Things to notice in the second call that, Google changed the scope parameter to provide profile and email API. The third call, which was a redirect to another StackExchange URL, shows the authorization code. It doesn’t look like StackExchange is using OpenID Connect as I don’t see openid in the scope.

OpenID Connect Terminology

OpenID Connect is built on OAuth 2.0, so lot of the terminology discussed in OAuth 2.0 are used in OpenID Connect, but it was created for the purpose of authenticating user in authorization server. 

id_token

The id_token in OpenID Connect returns user’s information. The difference between access token and id_token is, if you want to get information about user using the access token, you have to make another call to some API endpoint. id_token saves that round trip to the server.
The id_token is a JSON Web Token (JWT). It is a JSON decoded in base 64. There are 3 parts in id_token, separated by a period (‘.’). First part is the header and it contains information like, what security algorithm is used. Second part is the payload, which contains information like, user id, token expiration time etc. Third part of the token is signature to validate if token was tampered by anyone. The client has the public key for the private key that generated the token and using that, client can validate the token.
The easiest way to decode an id_token is to use decoder in jwt.io. If you have a Sitecore 9.1 instance, go to Sitecore login screen, it will redirect you to the identity server url. Copy that url and decode it using online url decoder. Below is from my local instance. I decoded it to make it readable. I bolded the part that we already discussed and might interest you. 

https://xp910.identityserver/account/login?returnUrl=/connect/authorize/callback?client_id=Sitecore&response_mode=form_post&response_type=code id_token token&scope=openid sitecore.profile&state=OpenIdConnect.AuthenticationProperties=NIRQvgTO6YlOMQdZkYlKizcVzNV1Felc1SpP4XRYoGtM54aQ3TLipFLsPtnoMjKhbuuNYqHUgefEy4BZkuIjY43NbTPE8NWWrpTTxpQ8P8LMg2o7ZHaSCO8uXRK8A31vO1EoXz1O0RnBuha7GlnN2jjPCfYNuVNl2S4fTiNiMuVBhGPMS7FMPdSMFj0XabYWgCrTCGnxoILALWWksa5cvw&nonce=636797438206701381.MWE4OWJlNzMtZTFhYS00OWViLWE1ZWMtMGFmNzNhOWMzZWE0Mjk2NmRhZDctZDg4Yy00ZDEwLWI2MzgtY2VlMDU0YWQ1NDQx&redirect_uri=http://xp910.sitecore/identity/signin&sc_account_prefix=sitecore\&x-client-SKU=ID_NET451&x-client-ver=5.2.2.0

Now, start fiddler to capture http requests to see what response is coming back from Sitecore Identity Server once you authorize. Mine shown below. Since response_mode is form_post, idententity server is posting a form to the url that was included in the url above. You can take the id_token and copy it in the jwt.io decoder to see the header and payload information. Also, notice that the access token is a stateless JWT access token.

<form method='post' action='http://xp910.sitecore/identity/signin'><input type='hidden' name='code' value='1b6f995dc665faaffaef7db59365b8bae680a800bf7630cf612033d6fdacb512' />
<input type='hidden' name='id_token' value='eyJhbGciOiJSUzI1NiIsImtpZCI6IkE2ODVCMkFBRkU1NkFEMzQ5MDA4Nzg2NzI3NTEzMkM1QUE3ODdFNDkiLCJ0eXAiOiJKV1QiLCJ4NXQiOiJwb1d5cXY1V3JUU1FDSGhuSjFFeXhhcDRma2sifQ.eyJuYmYiOjE1NDQxNDkwNzMsImV4cCI6MTU0NDE1MjY3MywiaXNzIjoiaHR0cHM6Ly94cDkxMC5pZGVudGl0eXNlcnZlciIsImF1ZCI6IlNpdGVjb3JlIiwibm9uY2UiOiI2MzY3OTc0MzgyMDY3MDEzODEuTVdFNE9XSmxOek10WlRGaFlTMDBPV1ZpTFdFMVpXTXRNR0ZtTnpOaE9XTXpaV0UwTWprMk5tUmhaRGN0WkRnNFl5MDBaREV3TFdJMk16Z3RZMlZsTURVMFlXUTFORFF4IiwiaWF0IjoxNTQ0MTQ5MDczLCJhdF9oYXNoIjoiMHlIMW5EWENucS1udE0zUm9WemNzUSIsImNfaGFzaCI6IjZTRy10Zy1WUmJqak5rNlk3aER4TEEiLCJzaWQiOiJiNGRjNmM0ZGY1ZDI4NDQwMjhjYzNhMTdiNTUxMzc3MSIsInN1YiI6IjcwZDhkYWJiMTcwZjQ2MzBhZDRhMzQ1Nzc3Y2M0NGQ4IiwiYXV0aF90aW1lIjoxNTQ0MTQ5MDczLCJpZHAiOiJsb2NhbCIsImFtciI6WyJwd2QiXX0.hb4TgdGIeiqOqgAinEOj7Ih5fVUhb6cqgQN_7gPptvuA3hHwDCavymYvoGM3GAqxAoyNjauOZ0hTa-yrmLB-VpE2AQP9VbEqvGZTk6Y_I5lau08zijQPsrjTgsXyrfbjkzcsqDzfEyQ3igiRbYAkwwd5RxkogyxS-KQehrqBVcKkMZ1gXDqGyeJVmPmTQITN-8QgqgTneFbzjfxEFVEkCxGN6kkUki507FYwRBIFr17XMNMw44ODnBjHogydwlQMvmUcUsa_W8PcgnsHKOmtcgmNESOpJbl-_TmuP0_m3JGqehSSTQD9UU7tKFnAZJo6xJHJ81VKpFdJ2ADMat0zeA' />
<input type='hidden' name='access_token' value='eyJhbGciOiJSUzI1NiIsImtpZCI6IkE2ODVCMkFBRkU1NkFEMzQ5MDA4Nzg2NzI3NTEzMkM1QUE3ODdFNDkiLCJ0eXAiOiJKV1QiLCJ4NXQiOiJwb1d5cXY1V3JUU1FDSGhuSjFFeXhhcDRma2sifQ.eyJuYmYiOjE1NDQxNDkwNzMsImV4cCI6MTU0NDE1MjY3MywiaXNzIjoiaHR0cHM6Ly94cDkxMC5pZGVudGl0eXNlcnZlciIsImF1ZCI6Imh0dHBzOi8veHA5MTAuaWRlbnRpdHlzZXJ2ZXIvcmVzb3VyY2VzIiwiY2xpZW50X2lkIjoiU2l0ZWNvcmUiLCJzdWIiOiI3MGQ4ZGFiYjE3MGY0NjMwYWQ0YTM0NTc3N2NjNDRkOCIsImF1dGhfdGltZSI6MTU0NDE0OTA3MywiaWRwIjoibG9jYWwiLCJzY29wZSI6WyJvcGVuaWQiLCJzaXRlY29yZS5wcm9maWxlIl0sImFtciI6WyJwd2QiXX0.AVcpOYq0OflaPwHKfmgi67Y1ReK9k2kybpyzQxd8J7fIC97Zk0SNUKEEjL5OVQjt9NhkzL7jR37V9TrNAW4X3hnpTn0qTP8VMK8phUdLNt8nEe2YTg0g-Wqq2xbRSENT5VRQdt5L7HaYQVmtOP8A-UbrZ3VUm2jAdxS1gLqJiB3N4WJHvUet3I4gXrNEdmaqQmFwdoe5KpxgDeFRt6nfyRYPAPJMiKbS23G6d1fDXhrBrIRlmYb4JzepNxpvWMf2HQbNrZ3m-oQ7qwbUDNoOpYfvt_Rf7n3EOvSSUyA8z7n9TEyG4mB4rse-91nfyUhbNR46wRThuyOGpj8Ah6YO0g' />
<input type='hidden' name='token_type' value='Bearer' />
<input type='hidden' name='expires_in' value='3600' />
<input type='hidden' name='scope' value='openid sitecore.profile' />
<input type='hidden' name='state' value='OpenIdConnect.AuthenticationProperties=NIRQvgTO6YlOMQdZkYlKizcVzNV1Felc1SpP4XRYoGtM54aQ3TLipFLsPtnoMjKhbuuNYqHUgefEy4BZkuIjY43NbTPE8NWWrpTTxpQ8P8LMg2o7ZHaSCO8uXRK8A31vO1EoXz1O0RnBuha7GlnN2jjPCfYNuVNl2S4fTiNiMuVBhGPMS7FMPdSMFj0XabYWgCrTCGnxoILALWWksa5cvw' />
<input type='hidden' name='session_state' value='MOXMSSjr5T1vZfYiECHizTqOiS_hmqvDuJ-tyDo1C6s.4b49bd60437e81b5ccaf94d2d9acb3ba' />
</form>

Here is the decoded id_token from jwt.io

Endpoints

OpenID Connect standardized the REST EndPoints exposed by the authorization server. Each endpoint a has purpose to accomplish authentication. Following are different endpoints:

Discovery Endpoint
Purpose of Discovery Endpoint is to provide authorization server metadata. Using Discovery Endpoint we can find other endpoints. The Endpoint url is https://xp910.identityserver/.well-known/openid-configuration where https://xp910.identityserver is my Sitecore Identity Server root. You should check the Discovery Endpoint to understand what configurations are supported by the Sitecore authorization server. In fact, what other endpoints are available, can be found in the metadata returned by the Discovery Endpoint. Below is the response from Discovery endpoint in my Sitecore Identity server.

{
  "issuer": "https://xp910.identityserver",
  "jwks_uri": "https://xp910.identityserver/.well-known/openid-configuration/jwks",
  "authorization_endpoint": "https://xp910.identityserver/connect/authorize",
  "token_endpoint": "https://xp910.identityserver/connect/token",
  "userinfo_endpoint": "https://xp910.identityserver/connect/userinfo",
  "end_session_endpoint": "https://xp910.identityserver/connect/endsession",
  "check_session_iframe": "https://xp910.identityserver/connect/checksession",
  "revocation_endpoint": "https://xp910.identityserver/connect/revocation",
  "introspection_endpoint": "https://xp910.identityserver/connect/introspect",
  "frontchannel_logout_supported": true,
  "frontchannel_logout_session_supported": true,
  "backchannel_logout_supported": true,
  "backchannel_logout_session_supported": true,
  "scopes_supported": [
    "openid",
    "profile",
    "email",
    "sitecore.profile",
    "sitecore.profile.api",
    "offline_access"
  ],
  "claims_supported": [
    "sub",
    "name",
    "family_name",
    "given_name",
    "middle_name",
    "nickname",
    "preferred_username",
    "profile",
    "picture",
    "website",
    "gender",
    "birthdate",
    "zoneinfo",
    "locale",
    "updated_at",
    "email",
    "email_verified",
    "role",
    "http://www.sitecore.net/identity/claims/isAdmin",
    "http://www.sitecore.net/identity/claims/originalIssuer"
  ],
  "grant_types_supported": [
    "authorization_code",
    "client_credentials",
    "refresh_token",
    "implicit",
    "password"
  ],
  "response_types_supported": [
    "code",
    "token",
    "id_token",
    "id_token token",
    "code id_token",
    "code token",
    "code id_token token"
  ],
  "response_modes_supported": [
    "form_post",
    "query",
    "fragment"
  ],
  "token_endpoint_auth_methods_supported": [
    "client_secret_basic",
    "client_secret_post",
    "private_key_jwt"
  ],
  "subject_types_supported": [
    "public"
  ],
  "id_token_signing_alg_values_supported": [
    "RS256"
  ],
  "code_challenge_methods_supported": [
    "plain",
    "S256"
  ]
}

Authorize Endpoint
Authorize Endpoint is used specifically to authenticate the user in OpenID Connect. It can return authorization code as well as token. Since this endpoint can return token also, it is used in the hybrid grant type (discussed later) and additional round trip to server is not needed to get token separately. Authorization Endpoint url is
https://xp910.identityserver/connect/authorize.

Token Endpoint
Token Endpoint is used to get access token or id_token from the authorization server. You have to provide some credentials like, Authorization Code, Client Credential etc. to get the token. The Token Endpoint url is 
https://xp910.identityserver/connect/token

Userinfo Endpoint
Purpose of Userinfo Endpoint is to return additional information about the logged in user. You have to pass Access Token to call Userinfor Endpoint. The Userinfo Endpoint url is https://xp910.identityserver/connect/userinfo.

Endsession Endpoint
Purpose of Endsession Endpoint is to end user’s session for selected application. For example, say, there are multiple clients that are authorized by user to the Sitcore Identity Server, using Endsession Endpoint user’s browser session can be ended for all the clients. The Endsession Endpoint url is
https://xp910.identityserver/connect/endsession.

Check Session Iframe Endpoint
Purpose of this endpoint is to check user’s session status using cross-origin communication via HTML5 postmessage API. Identity server accepts the postmessage and return the login status of the user’s session. 
The Check Session Iframe Endpoint url is
https://xp910.identityserver/connect/checksession.

Revocation Endpoint
This endpoint allows revoking the access token and refresh token from the Identity Server. The Revocation Endpoint url is
https://xp910.identityserver/connect/revocation.

Introspection Endpoint
This endpoint is used to validate reference tokens in the server. As discussed before tokens can be stateful or stateless. The JWT tokens are stateless and can be validated using signature, but stateful reference tokens which don’t have signatures can be checked only by making API call to the Introspection Endpoint. The Introspection Endpoint url is https://xp910.identityserver/connect/introspect.

Grant Types

Grant types are different ways to authorize user. Sitecore Identity Server supports following OAuth 2.0 grant types plus Hybrid grant types.

Sitecore Identity Server Grant Types

Authorization Code
We discussed Authorization Code Flow before. This is a grant type to get token  using the Back Channel. Client gets the Authorization Code and exchange that to get id_token or access token. When this grant type is used all tokens come from the token endpoint. 

Client Credentials
This grant type is used for authenticating a client in server to server communication. Client provides it’s Client Id and Client Secret to authenticate via token endpoint.

Password
This grant type uses resource owner’s username and password for authentication via token endpoint. This mostly used in legacy application and it is not recommended. 

Implicit
Implicit grant type is mainly designed to use from browser based application using scripting language like javascript. Implicit flow doesn’t use token endpoint; all tokens are returned from authorize endpoint. In this flow no client authentication is done by the authorization server. Authorization server authenticates the end user (resource owner), gets the consent and redirect the end user back to the client with id token and access token. Client validates the tokens using signature to make sure the token was not altered. Since all communication are done through front channel, this grant type doesn’t allow accessing token using refresh token due to the security concern.

Hybrid
Hybrid authorization is combination of implicit flow and code authorization flow. When this flow is used authorization server uses authorize endpoint via front channel and redirect end user to client with authorization code and id token. Client validates the id token using signature and requests access token using token endpoint via back channel. Since the access token retrieval is done via back channel, authorization server can provide refresh token to be used later to get access token in the offline mode.

Refresh Token
Refresh Token flow was designed to get access token non-interactively when access token is expired. This helps us to avoid asking for client consent every time the access token expires. It is also helpful when client is running a background process  on end user’s behalf  and there is no way to interact with the end user when access token expired. To enable refresh token flow client needs to include offline_access scope.

Claims

Claims is a way to define what end user information will be communicated by OpenID Provider (OP). Claims are communicated via id token or via userinfo endpoint. There are standard Claims defined in OpenID Connect protocol, but additional Claims can be added. Below is the list of Claims available in Siecore Identity server.

Sitecore Identity Server Claims

Scopes

OpenID Connect uses OAuth 2.0 Scopes. It has standardized the Scopes for the protocol. Scopes must include “openid”. Additionally, some optional Scopes can be included, for example, “profile”, “email” etc. For OpenID Connect, scopes can be used to request that specific sets of information be made available as Claim Values. Also, if you want to get the Refresh Token, “offline_access” scope has to be included. Below are Scopes supported in Sitecore Identity server. 

Sitecore Identity Server Scopes

Response Types

Response types defines what response to be sent back from the OpenID Connect Provider (OP). If the request url response_type=code, OP will only return authorization code. In our example above, we found that Sitecore Identity Server url contains response_type=code id_token token. When Sitecore Identity Server gets user consent, it posts all of them to the client redirect url. Following are different Response Types supported by Sitecore Identity Server.

Sitecore Identity Server Response Types

Response Modes

Response Modes define how the responses should be sent to client. Response can be sent as form post, query string or fragment in the redirect URI. If query is used as the mode, care should be taken to make sure length doesn’t exceed url length limit. All Response Modes supported in Sitecore Identity Server.

Sitecore Identity Server Response Modes

Token Validation

Tokens returned by Sitecore Identity server are jwt token. The signature of that can be verified using the public key of the private key that signed the tokens. How to validate the signature?
The discovery endpoint provides the JSON Web Key Set (JWKS) URI endpoint https://xp910.identityserver/.well-known/openid-configuration/jwks.&nbsp;Using the jwt.io libraries the public key can be extracted to verify the signature.
Go to this site https://8gwifi.org/jwkconvertfunctions.jsp to convert JWK to PEM. You need to only use kty, n, e from JWK. Insert the public key in jwt.io
to verify the signature.
I described above method to validate the token signature to tell you how OpenID Connect protocol standardized signature validation. In Sitecore Identity Server, IdentityServer4 framework validates the signature when token is returned to the client.

Role of Sitecore Identity Server in Sitecore Micro Service Architecture

So far we have discussed important OAuth 2.0 and OpenID Connect concepts used in Sitecore Identity server. Let’s discuss why Sitecore created the Sitecore Identity Server and how it fits in Sitecore’s future micro service based architecture. In the following diagram, I tried to explain how Sitecore Identity Provider fits into the future Sitecore micro services architecture. The micro services shown in Sitecore XP are my guess and in reality they will be probably vastly different. The part that I am sure about is the layers above the micro services. Every service will be authorized via Sitecore Identity Provider and it will serve both front channel and back channel communication. Currently, Sitecore Commerce has it’s own Identity Provider, I am guessing Sitecore Identity will serve both Sitecore and Sitecore Commerce.

End of Part 1

This blog post is already too long. I think, it is right time to stop. In this blog post, we discussed about OAuth 2.0 and OpenID Connect concepts. We discussed how those concepts are used in Sitecore Identity Server. We also looked at the need for Sitecore Identity Server from overall Sitecore Micro Service Architecture point of view.
In the next part we will explore how we can use Sitecore Identity Server to authenticate an application that intend to use Sitecore services. We will also explore how Sitecore Identity Server can be connected to external Identity Provider.

References

Posted in sitecore Identity | Tagged , , , , | 1 Comment

Compilation issues with Sitecore Commerce 9 Business Tools SDK, a.k.a Sitecore BizFx SDK

Like Sitecore Commerce 9 Engine SDK, which is used for creating custom plugins, Sitecore Commerce 9 Installation package also comes with Sitecore Commerce Business Tools SDK. This SDK is used to modify the Business Tools. For example, if you want to add a new commerce entity and you want to manage that from the Business Tools, you have to use BizFx SDK.

Now, there are multiple issues to make BizFx SDK working in Sitecore Commerce 9 update-1. Here is the list.

  • You may not have @speak and @sitecore registry in your .npmrc file. Run following commands to add them in the .npmrc file

    npm config set @speak:registry=https://sitecore.myget.org/F/sc-npm-packages/npm/

    npm config set @sitecore:registry=https://sitecore.myget.org/F/sc-npm-packages/npm/

     

  • “@sitecore/bizfx”: “^1.1.9” is missing from package.json. Add that.
  • “@speak/icon-fonts”: “~1.0.2” is missing from package.json. Add that.
  • “@speak/ng-bcl”: “~0.8.0” and “@speak/styling”: “0.9.0-r00078” are missing from Sitecore npm repository:
    For some reason Sitecore has not included these version of Speak npm packages in npm repository. They have included the packages with the Sitecore Commerce 9 installation package in the root directory. You need to install these packages from the file system. Modify “@speak/ng-bcl”: “~0.8.0” and “@speak/styling”: “0.9.0-r00078” as follows in the package.json.

    “@speak/ng-bcl”: “file:../speak-ng-bcl-0.8.0.tgz”,
    “@speak/styling”: “file:../speak-styling-0.9.0-r00078.tgz”,

Once above changes are done, save package.json and run the command ‘npm install‘. This will install all the required packages to the solution. If the BizFx SDK compiles without any error, run the command ‘ng serve‘. This will host the Business Tools in webpack dev server. If you go to http://localhost:4200, business tools will be launched.

Webpack will throw port not available error if SitecoreBizFx site that is installed with Sitecore Commerce 9 is running in IIS. Stop this site and run the command ‘ng serve‘ again.

 

Posted in Sitecore Commerce | Tagged , | Leave a comment

Yet another Sitecore Commerce 9 installation blog post

Update: Issues discussed in this article are on;y applicable to Initial Release of Sitecore Commerce 9. None of these issues occurred in Sitecore Commerce 9 Update 1.

If you are here and reading this blog, you probably have some problem installing Sitecore Experience Commerce 9 in your computer and you have problem because your machine configuration is not exactly what Sitecore Commerce 9 installation script expect it to be. You may have one or many of the following differences.

  • You have more than one instance of SQL Server installed in your machine and none of them are on localhost.
  • You have more than one disk drive and you want to install commerce websites in folder that’s not under C:\inetpub\wwwroot.
  • You have Sitecore Commerce 8.2.1 installed in your machine previously.

In a perfect world, we may have a clean machine with SQL Server 2016 SP1 installed on default localhost and have one C drive with default website folder location. In that case, I wouldn’t be writing this blog :). But, the reality is, in most cases we are not that lucky and the Sitecore Commerce 9 installation document is not that helpful in that situation. In my case, all of the above mentioned points are true. So, what issues I have faced and how I got through them with the help of good folks from Sitecore Community?

        1. Sitecore Commerce 9 installation guide has a section for ‘Host environment requirements’. There are list of softwares here, but most of them will be there in your machine when you installed Sitecore 9 update-1. What you need to install are  following
          • Visual Studio 2017
          • .NET Core 2.0.0 Visual Studio 2017 Tooling
          • .NET Core Windows Server Hosting 2.0.0 (if you have x64 machine, use command line installation to opt out x86 .Net Runtime, DotNetCore.2.0.5-WindowsHosting.exe OPT_NO_X86=1)
        2. In most cases you will be using an administrator account for installation and run the script from Powershell Console in administrator mode. Make sure your Windows logged in account is in SQL Server ‘sysadmin’ role.
        3. If you want to install Sitecore Commerce 9 in location other than C:\inetpub\wwwroot, you should be able to change the Drive, but changing the folder name will fail the installation with this error for the package installation Sitecore Commerce Connect Core 11.0.192.zip
          One or more exceptions occurred while processing the subscribers to the ‘item:saved’ event.
          At this point, if you launch Sitecore and open the indexing manager, you will see all indexes are gone. There is not point of return now, you have to re-install everything from scratch again, starting from XP9.
          If you want to change the Drive you have to make changes in the following files. Replace the environment(‘SystemDrive’) with the drive (e.g. ‘D:’).

          Sitecore.Commerce.2018.01-2.0.254\SIF.Sitecore.Commerce.1.0.1748\Configuration\Commerce\CEConnect\InitializeCommerce.json (4 hits)
          Line 32:     "CommerceOpsPhysicalPath": "[concat(environment('SystemDrive'), concat('\\inetpub\\wwwroot\\', variable('CommerceOps')))]",
          Line 33:     "CommerceShopsPhysicalPath": "[concat(environment('SystemDrive'), concat('\\inetpub\\wwwroot\\', variable('CommerceShops')))]",
          Line 34:     "CommerceAuthoringPhysicalPath": "[concat(environment('SystemDrive'), concat('\\inetpub\\wwwroot\\', variable('CommerceAuthoring')))]",
          Line 35:     "CommerceMinionsPhysicalPath": "[concat(environment('SystemDrive'), concat('\\inetpub\\wwwroot\\', variable('CommerceMinions')))]"
          Sitecore.Commerce.2018.01-2.0.254\SIF.Sitecore.Commerce.1.0.1748\Configuration\Commerce\CommerceEngine\CommerceEngine.Deploy.json (4 hits)
          Line 39:     "CommerceOpsPhysicalPath": "[concat(environment('SystemDrive'), concat('\\inetpub\\wwwroot\\', variable('CommerceOps')))]",
          Line 40:     "CommerceShopsPhysicalPath": "[concat(environment('SystemDrive'), concat('\\inetpub\\wwwroot\\', variable('CommerceShops')))]",
          Line 41:     "CommerceAuthoringPhysicalPath": "[concat(environment('SystemDrive'), concat('\\inetpub\\wwwroot\\', variable('CommerceAuthoring')))]",
          Line 42:     "CommerceMinionsPhysicalPath": "[concat(environment('SystemDrive'), concat('\\inetpub\\wwwroot\\', variable('CommerceMinions')))]",
          Sitecore.Commerce.2018.01-2.0.254\SIF.Sitecore.Commerce.1.0.1748\Configuration\Commerce\CommerceEngine\CommerceEngine.Initialize.json (4 hits)
          Line 38:     "CommerceOpsPhysicalPath": "[concat(environment('SystemDrive'), concat('\\inetpub\\wwwroot\\', variable('CommerceOps')))]",
          Line 39:     "CommerceShopsPhysicalPath": "[concat(environment('SystemDrive'), concat('\\inetpub\\wwwroot\\', variable('CommerceShops')))]",
          Line 40:     "CommerceAuthoringPhysicalPath": "[concat(environment('SystemDrive'), concat('\\inetpub\\wwwroot\\', variable('CommerceAuthoring')))]",
          Line 41:     "CommerceMinionsPhysicalPath": "[concat(environment('SystemDrive'), concat('\\inetpub\\wwwroot\\', variable('CommerceMinions')))]",
          Sitecore.Commerce.2018.01-2.0.254\SIF.Sitecore.Commerce.1.0.1748\Configuration\Commerce\SitecoreBizFx\SitecoreBizFx.json (1 hit)
          Line 33: 		"SitecoreBizFxPhysicalPath": "[concat(environment('SystemDrive'), '\\inetpub\\wwwroot\\SitecoreBizFx')]",
          D:\Deploy\Sitecore.Commerce.2018.01-2.0.254\SIF.Sitecore.Commerce.1.0.1748\Configuration\Commerce\SitecoreIdentityServer\SitecoreIdentityServer.json (1 hit)
          Line 41:     "SitecoreIdentityServerPhysicalPath": "[concat(environment('SystemDrive'), concat('\\inetpub\\wwwroot\\', parameter('SitecoreIdentityServerName')))]",
          
        4. If your SQL Server instance is not a default instance on localhost, the installation will fail with following error
          Get Token From Sitecore.IdentityServer
          Install-SitecoreConfiguration : The remote server returned an error: (500) Internal Server Error.
          

          If this is the case, you need to change the connection string in the json files mentioned in Naveed’s blog post. You might get away with installation by just making the change in \SitecoreIdentityServer\wwwroot\appsettings.json, but it is better to take care of them now than later. Sitecore uses trusted authentication, so don’t change the connection string to use SQL account. Just change ‘localhost’ with your SQL Server instance (machine-name\\instance-name). After making the correction, you can comment out all the tasks before InitializeCommerceEngine in Master_SingleServer.json and run the installation script again.

        5. If you are seeing the following error after running the installation, you might be using one ‘\’ instead of two ‘\\’ in the connection string, like I did and spent hours to find out what’s going on.HTTP Error 502.5 – Process Failure

          Thanks to Kautilya Prasad for helping me with this and actually recreating the issue in his machine.
          This error is little different from wrong connection string. This is a parsing issue in the json config and installation script doesn’t handle the error gracefully. As the effect of that, I was seeing the following error in event log and thought, the issue is something to do with state of .Net Core in my machine. It was bit confusing. So, if you see this error in event log, most probably the issue is related to unhandled error than anything to do with .Net Core installation .

          Application 'MACHINE/WEBROOT/APPHOST/SITECOREIDENTITYSERVER' with physical root 'C:\inetpub\wwwroot\SitecoreIdentityServer\' failed to start process with commandline 'C:\inetpub\wwwroot\SitecoreIdentityServer\Sitecore.IdentityServer.exe ', ErrorCode = '0x80004005 : 0.
          
        6. If you had installed Sitecore Commerce before in your machine and the account CSFndRuntimeUser existed before the installation, it might have a different password than the one used by the installation script to create the SitecoreIdentityServer website. In this case you might see the following error because the App Pool for SitecoreIdentityServer will be stopped for incorrect password.

          Service Unavailable


          HTTP Error 503. The service is unavailable.

        7. If you are installing the Sitecore Commerce 9 in C:\inetpub\wwwroot folder, script might fail to grant permission to CSFndRuntimeUser account to the folder. If that happens, you can resolve the issue by manually granting permission and re-run the script again.
        8. If everything goes well with installation and the Storefront is failing with Sitecore item not found, publishing the site will probably fix the issue.
        9. Finally, if you are not lucky, you might have to remove the Sitecore 9 and Sitecore Commerce 9 once or more. Get the removal scripts from Sitecore SIF-less blog and Sitecore Commerce 9 SIFLess Uninstall and configure them as per your need.
          You may use my script for removing XP9, but it doesn’t use SIF.

Good luck!

Posted in Commercce, Sitecore Commerce | Tagged , , , | Leave a comment

A Powershell script for removing Sitecore 9 instance from local machine

Sitecore 9 installation process is quite different than previous version. The new installation process uses, Sitecore Installation Framework (SIF). You can find installation documents in Sitecore 9 release page. The earlier version of Sitecore installation was easy and it was not hard to remove a Sitecore instance from the computer. If you had used SIM, it was even easier, just clicking a button would do it.

After I installed my first Sitecore 9 instance, I realized that, it installed the websites in C:\inetpub\wwwroot and created all the databases in the C drive. My C drive is smaller SSD drive and I don’t install Sitecore instances in it. It is reserved for only OS. I thought, it will be useful, if I create a script to remove the instance and share it with others. Here is the script.

There seems to have some issues with gist rendering in the blog. If the code is not displayed in the blog, this link will directly take you to Github gist.

Final Thoughts: SIF is extensible and it is probably possible to extend it to add the instance removal functionality. I will look into that as time permits. Feel free to change the script for your use. Make sure you run it carefully. Once the instance is deleted, nothing can be recovered.

Posted in Sitecore | Tagged , | 2 Comments

Sitecore Commerce 9 is Coming Soon

What a great Symposium it was in Las Vegas. So many familiar faces and so much we have learned from the Symposium and the MVP Summit. My focus in last couple months was on Sitecore Commerce and I tried to learn as much as possible from this event about Sitecore Commerce. It’s time to share some of those information with the community. I am allowed to share only unrestricted information but, don’t be disappointed, Sitecore Commerce 9 will be released soon and you should be able to dive into it and explore more. If you are a Sitecore MVP, you will be able to play with a Tech Preview sooner than that. This blog is to give you some idea of what is coming.

RIP Commerce Server: No more Commerce Server!! I think, every one is happy that, we don’t have to deal with Legacy System anymore. This should make the installation and maintenance of the system so much easier. Look at the following Sitecore 8.2.1 architecture diagram. Aren’t you happy to see all the red boxes are gone?

Sitecore Commerce Architecture

Data Migration Tool: As soon the Legacy System is gone, question will be raised about upgrading the old system. How can we move the Catalogs, Profiles, Orders and Inventory data from Commerce Server to Sitecore Commerce 9? Sitecore is working on a migration tool that will help us to migrate data from old system to Sitecore Commerce 9. As mentioned, we should be able to map fields and migrate data easily from Commerce Server to Sitecore Commerce 9.

New Business Management User Experience: Sitecore Commerce Business Tools like, Merchandising Manager, Pricing & Promotion Manager and Customer & Order Manager are not part of Sitecore Dashboard anymore. A new Business User Experience Manager is coming (looks similar to one in Sitecore 8.2.1). The new UI is built using SPEAK 3 component, which is using Angular 4 as the building block. Original SPEAK based UI is gone. The new Business Management Tools are task centric, data driven and extensible. So, if we need to add a new menu or tab in the Business Management Tool, we can write our own plugin and expose the data in the Business User Experience Manager.

Federated Authentication: Sitecore Commerce uses Sitecore’s Authentication model for authenticating and authorizing users. With Sitecore 9, Federated Authentication is now a choice for Sitecore Commerce application. This opens up the possibilities of connecting different third party systems to Sitecore Commerce Securely.

Catalogs and Inventory: The terminology Sitecore uses for product is Sellable Item. The reason for this is, a product may not be a physical product but anything that customer can sell. For example, customer can sell service. In Sitecore 9, Sellable Items are independent of Catalog and they can be used in more than one Catalog. The Categories are part of a Catalog and Sellable Items can belong to more than one Categories. The Inventory is now part of the Sellable Item and inventories belongs a Inventory Set. The Inventory Set is useful to represent multiple warehouses based region or site/sub-site or differentiate inventory based on brick & mortar or online. Following diagram shows the relationship.

Catalog

The schema for Sellable item is based on Schema.org Product Schema. This makes it possible to map the Sellable Item in JSON-LD directly for google product search markup. The Variant Sellable Items are child of original Sellable Item. The schema is basically same and variant properties are overriden properties.

SXA Storefront: Sitecore will be releasing a new Storefront based on SXA. This will include many commerce specific components with capabilities of creating multi tenant ecommerce website. The solution is based on Helix architecture.

ASP.NET  Core 2.0: Sitecore Commerce 9 is built on latest ASP.NET Core 2.0 and the architecture is based on Lightweight Micro Service architecture. This opens up lots of possibility in future to use services in cross platform and take advantages of distributed architecture.

Azure PaaS Support: Since the Commerce Server is not in picture any more, Sitecore Commerce can be deployed in Azure with very little effort. It can take advantage to standard Azure tooling, High Availability, Disaster Recovery etc.

I cannot wait to try Sitecore Commerce 9. There are so much to explore. As I learn more, I will share with the community.

 

Posted in Commercce | Tagged , | 1 Comment

How to debug Sitecore Commerce OData APIs in Fiddler

Sitecore Commerce  APIs are based on OData. When Storefront makes call to Sitecore Commerce Engine, it uses OData clients, which are Commerce Services Proxy generated using T4 templates. If you download Sitecore Commerce SDK, you will find the several Postman Collections, which are json files you can load in Postman tool and fire the OData APIs. Postman is very useful to debug OData APIs but, I find Fiddler is useful for this purpose too. What I find useful in fiddler is, when I am debugging code in Visual Studio or just using the website (storefront), if I would like to see the OData APIs in use, I can see that in Fiddler. Once, I capture the API in Fiddler, I can replay it in Fiddler Composure as many time as I want.

In this blog, I will discuss, how we can setup Sitecore Commerce, so that, we can capture the OData APIs in Fiddler. We will be using Sitecore Storefront, so setup starts from there. The OData API calls are made from ASP.NET server side code. How can we setup the Storefront application, so that, this API calls from the server side code can go through Fiddler. Here is how.

Open up the web.config file for Storefront and add the following in that file

fiddler1

You can find more information in telerik website here

Once that change is made OData calls will be routed through Fiddler, but still it cannot be seen in Fiddler because, base URL for the Commerce Engine site is setup as http://localhost:5000 per Sitecore installation instruction. Fiddler doesn’t capture traffic on the localhost. So, we have to use a different base URL.

First setup a different binding for Commerce Engine site. Open up the site in Internet Information Manager and changed the binding as follows and removed the binding with empty Host Name.

fiddler2

To access the site on that binding you need to add that in the host file. Open up the host file and add 127.0.0.1 commeceauthoring as a new entry. Now, if you go to browser and access http://commerceauthoring:5000/api/$metadata you will see the OData metadata information for Commerce Engine.

Next open up Sitecore.Commerce.Engine.config and Z.Sitecore.Commerce.UX.Shared.config in Storefront and replace localhost with commeceauthoring. Setup is ready now to capture the Sitecore Commerce OData APIs in Fiddler.

Start fiddler and make sure you are capturing traffic from all processes, not only from the browser. Now, if you open the storefront site in browser, traffic will be captured as shown below. This was captured when I went to product detail page.

fiddler3

If you select the request, you can see the request and response on the right.

Request
fiddler4

Response
fiddler5

You can right click on the captured request and go to Replay -> Reissue from Composure to debug the request in Fiddler Composure. You can change request parameters and play with the API.

fiddler6

That’s it. Happy debugging!

Update: When you are done with the debugging make sure to remove the configuration from the web.config. If that configuration is there and you are not running Fiddler while using the site, you will see errors in the application because, there is no way to route the request to Fiddler.

 

Posted in Sitecore Commerce | Tagged , , | Leave a comment

Sitecore Commerce Pricing Feature

Pricing is one of most important and complex features in any eCommerce product. New Sitecore Commerce has introduced a much improved pricing engine than what used to be in Commerce Server. In this article, I will explain Sitecore Commerce pricing features and show you how to work with them in Sitecore Commerce Administration Console. Let’s get familiar with the pricing terms used in Sitecore Commerce and see how to configure them.

Price Book: A Price Book in Sitecore Commerce is the first level construct of organizing the pricing. Say, you want to keep all the pricing for products for a country or a web store in one place, you can create a Price Book for that. A Price Book can be associated with multiple catalogs. If you want to remove the pricing for all products in a catalog, you can just disassociate the catalog from the Price Book.
To create a Price Book, go to the Sitecore Dashboard and click on the Pricing and Promotion button.
Pricing1
This will open up the Price Book list page where a new Price Book can be created by clicking on the ‘Add’ link on the right. I created a Price Book called ‘US Pricing’.
Pricing2
Next, open up the Price Book and associate the Price Book to the Catalogs. I associated it to Habitat_Master catalog.
Pricing3

Price Card: A Price Card is a construct to organize different kinds of pricing for products. Price Card is selected in the product and price gets calculated based on the configuration in the Price Card. Price Card include tags and break pricing to define the price at the currency level. To create a Price Card click on the ‘Add’ link in the Price Card section of the Price Book. I created Price Card called ‘Quarterly Sale’ shown below. I will use this Price Card for defining the product pricing for quarterly sales.
Pricing4

Snapshot: Pricing in Price Card is defined using a Snapshot. A Snapshot has a starting date. That means we can plan and create pricing that will start at some future date. Price Card use the Snapshot that is currently selected and apply the pricing from that Snapshot. Below is a Snapshot I added for the first quarter of 2017.
Pricing5

Tags: Price from Price Card can be applied to particular product or it can be applied to range of products based on tags. In above screenshot you can see I added a tag called ‘q1 computer sale’. Any product with that tag will use the Price Card pricing.

Break Quantity Pricing: In pricing, break quantity is a concept where the pricing of the product varies based on how many of the same item is bought. Once Snapshot is created, you can add currency specific pricing by clicking on the ‘Add’ link in the dropdown on the right. When you add the currency specific pricing, you can create Break Quantity Pricing as shown in the below screenshots.

Once pricing are added you need to request for approval using the dropdown menu and approver needs to approve the price. Only user with Pricer role can create the pricing and user with Pricer Manager role can approve the price.
Pricing6

Pricing7

Pricing8

Pricing10

Once you defined the Pricing, you can go to the Product and select the Pricing Card, like below.
Pricing11

Customer Specific Pricing: Customer specific pricing is important in B2B commerce and heavily used. Often, B2B clients negotiate price with manufacturer or seller and commerce application provides ability to implement the complex pricing. Sitecore Commerce doesn’t support customer specific pricing out of the box at this time. It will be a good feature to add. Although customer specific pricing is much needed in B2B commerce, in my experience, 60 to 70 percent time, our clients want to use already existing pricing logic in ERP. In such situation, we create an integration with ERP to fetch the price real time.

Overall, Sitecore Commerce provides pretty flexible and comprehensive pricing  features. With the new Plugin architecture the pricing framework can be extended for the purpose of any unique requirements.

Posted in Commercce | Tagged , | Leave a comment

Notes from Sitecore Commerce Installation

Finally, I found some time to install Sitecore Commerce. I am excited because, I can play with it and see why everyone in Sitecore Community is talking about it. First of all, installing Sitecore Commerce is quite a complex task, at least for now. It is going to get better that’s what I heard.

First challenge was, where to start. Sitecore has provided Sitecore Commerce 8.2.1 Deployment Guide. It’s pretty comprehensive but, I took a easier path. It’s easier to learn from someone who has done it already and especially, if he has made a video of that. Along with this video Sen Gupta  put together a blog article too.

The video and the document was immensely helpful. But, I did face some issues, mostly, due the configuration of my machine. I am going to describe those issues in this blog and also, talk about how the configure the storefront source code, so that, we can debug the code and learn how Sitecore Commerce architecture works.

Here are the issues I faced.

 Cannot start IISAdmin
I got this problem because ‘Internet Information Services\Web Management Tools\IIS 6 Management Compatibility’ Windows Feature was not turned on.

iisadmin

Commerce Server was unable to create a Registration Marker in Active Directory
I got this error but, it seemed to me that it is normal because my computer is not a domain controller or connected to any active directory. I ignored this message.

Adding the identity ‘domain\myaccount’ to the AzMan role ‘Administrator’…
New-CSWebService : The server could not be contacted
I Got this error but, I was not sure why powershell script is trying to add my login account in Authorization Manager. My login account was not used to run any services. I ignored this. I did manually add the ‘Administrators’ group and ‘CSFndRuntimeUser’ in the Role Assignments in CatalogAdministrator. It can be done by opening up the AzMan.msc console.

Issues with not having SQL instance as the default instance
I had many issues because my SQL Server 2014 is not a default SQL instance. I had to change in the following places
Installing DACPACK: I had to change /TargetServerName:SQLServer\InstanceName Connection string: In the following json files I had to change “Server”: “.” with “Server”: “SQLServer\InstanceName”. Without this change Commerce Authoring bootstrap was throwing error, ‘HTTP Error 502.3 – Bad Gateway The specified CGI application encountered an error and the server terminated the process.’ I had to make that change in the following files

PlugIn.AdventureWorks.CommerceAuthoring-1.0.0.json PlugIn.AdventureWorks.CommerceMinions-1.0.0.json PlugIn.AdventureWorks.CommerceShops-1.0.0.json
PlugIn.Habitat.CommerceAuthoring-1.0.0.json
PlugIn.Habitat.CommerceMinions-1.0.0.json
PlugIn.Habitat.CommerceShops-1.0.0.json CommerceAuthoring\wwwroot\bootstrap\Global.json

Service Url issue
All above json files had “ServiceUrl”: “http://localhost:1004. I had to change that to “ServiceUrl”: “http://csservices:1004, because “ServiceUrl”: “http://localhost:1004 was not accessible. Other way to fix this is to add a iis binding for CommerceAuthoring site with empty host name.

Storefront Solution Setup
I setup the reference storefront solution, so that, I can debug code and understand the architecture. The source code is available here. Download Reference-Storefront-10.0.853.

Storefront\CF\CSF contains the Sitecore Commerce storefront source code
Storefront\CS\CSF contains Commerce Server storefront

Following Changes are needed to compile and deploy the code successfully.

  • Change project properties of TDSCommerceEngine_Master to point to the storefront website
    screenshot-1
  • Change project properties of TDSCommon_Master to point to the storefront website
    screenshot-2
  • In Commerce.Storefront project change the hostName for the storefront in the site node
    screenshot-3
  • Copy all sitecore dlls from Storefront website bin folder to Storefront\Lib\Sitecore in your solution including the following dlls.
    ComponentArt.Web.UI.dll (remove and add reference in the project)
    Microsoft.Extensions.DependencyInjection.Abstractions.dll
    Microsoft.Extensions.DependencyInjection.dll
    Stimulsoft.*.dll
    System.Web.Http.dll (remove and add reference in the project)
    System.Web.WebPages.Deployment.dll (remove and add reference in the project)
    System.Web.WebPages.dll (remove and add reference in the project)
  • Some references from Nuget packages were not picked up. I had uninstall and install those Nuget Packages.
  • Copy GlobalSuppressions.cs from Common project to Commerce.Storefront project

Build and deploy the project and start learning 🙂

Posted in Commercce, Commerce Connect, Sitecore | Tagged , | Leave a comment