Build Product Recommendations in Sitecore OrderCloud using LLM, and Sitecore CDP

What is happening at OpenAI? Sam Altman, the CEO of OpenAI, was fired on Friday by the board members. Over the weekend, there were discussions about bringing him back. The drama is still unfolding, and we have to wait to see what happens in the coming weeks.

This blog is not about OpenAPI, but it’s hard to start without mentioning it because it’s just such a shocking event for the future of Large Language Model (LLM). In a recent discussion with Cambridge audience, Sam Altman was asked whether another breakthrough is needed to achieve Artificial General Intelligence (AGI)? Sam’s answer was ‘yes, another breakthrough is needed’. Understanding how LLM works, we know that it uses language used by humans to find answers to questions. This is a very good article that explains how ChatGPT works. We express our thoughts using language, but is that enough for AI to create new ideas like humans? I don’t think so. Clearly, Sam thinks there are more than LLM needed to achieve AGI. But, there is no doubt LLM is extremely powerful and it can help us finding solutions for many problems. In this blog I will explore how LLM can be used to build product recommendation in Sitecore OrderCloud utilizing LLM and Sitecore CDP. This discussion will be on a proposed solution at the architecture level. Let’s dive into it!

So, why am considering LLM to build product recommendations in an e-commerce system? I think LLM can remove lots of complexity from the current approach of generating product recommendations which is mainly based on Collaborative Filtering, Content Filtering, and Hybrid Filtering. I discussed this previously in my article The Expanding Universe of Software Development. The filtering approach is complex and takes time to generate data for segmentation. Whereas using LLM will be much more real-time approach because it is based on product content and customer’s interest based on closeness of language.

Sitecore OrderCloud doesn’t have a product recommendation engine built in, but you can combine Sitecore Discover with OrderCloud for product recommendation. Sitecore Discover is part of Sitecore composable stack. It is based on same Collaborative Filtering algorithm. It comes with frontend widgets, a Javascript SDK, admin panel for analytics and product management. It works well and it doesn’t need CDP. What we are trying to do is to create a product recommendation solution based on LLM and Sitecore CDP. Not a product like Discover. It can be designed like a product though.

I will explain the working of this solution with an example. A website selling books wants to recommend books based on users’ browsing behavior and previous purchases.

• Send book title and author to CDP
• Read previously browsed and purchased books by the user from CDP
• Send the books and authors’ names to ChatGPT (LLM) API to find out the genres based on sent data
• Save the genres against the visitor in CDP
• Generate a list of books based on the genres provide by ChatGPT
• Show the list in recommended products

Here is an example

I can even ask for recommended books from ChatGPT but storing the genres in CDP and using the genre to generate the recommendations from OrderCloud is better because I can generate the books that are available in the online store, I can decide what to show based on my requirements, and also I can use the genres when next time same user visit the website even before start browsing books.

This solution works fine with ChatGPT and any kind of LLM services that provides API access to their models, but it will not work so well when the online commerce business is based on specialized products. This especially true in B2B commerce. ChatGPT or other LLM services are trained on scrubbing data from internet. We can’t feed the website products and contents ChatGPT. In this kind of situation we need to train LLM of our own using the products and contents used in the website. There are many open source LLMs available today. Two popular ones are Facebook’s Llama 2 and Claude 2 from Anthropic.

So here is the proposed solution

• Set up and train the LLM with the products and contents from the website. LLM will need periodic refresh as new products and contents will be updated.
• Create API interface against the trained LLM to return response to recommendation questions.
• Set up the website to send visitors’ data to Sitecore CDP using Stream APIs.
• Use Sitecore CDP Rest APIs to send order data.
• Use Sitecore CDP Rest APIs to retrieve users’ previously browsed and order products.
• Send users’ previously browsed and ordered product to LLM to learn users profile.
• Save this profile in CDP against users.
• Retrieve user’s profile from CDP using the Rest API.
• Send user’s profile to LLM for recommended products.

Above is an outline of how we can use LLM to build a Product Recommendation Engine. Actual implementation will require to consider many details. Product recommendation can be further improved with users data like their locations, language they speak, age, sex etc. LLM can also help with inventory management and forecasting, sentiment analysis, search and many other things that helps with e-commerce conversion.

What’s New In Sitecore OrderCloud

It was a great week of learning in Sitecore MVP Summit + Sitecore DX + SUGCON NA in Minneapolis from October 2nd to October 6th. There were no announcements of new acquisitions or new products. It looks like Sitecore is fully embracing the composable architecture approach and is now concentrating on refining its existing products. Sitecore already provides the key components required for a Digital Experience Platform (DXP) implementation. The current emphasis is on ensuring these components work together smoothly without compromising the principles of composable architecture.

I was interested to learn what’s happening with Sitecore OrderCloud. In last year’s Symposium and MVP Summit, we learned that Sitecore would work on Project Affinity. I wrote an article about that. The following slide shows what Sitecore’s plan was to build in Project Affinity.

At this year’s MVP Summit and Sitecore Developer Experience (DX), there was no update on Project Affinity. It’s possible that the project was a bit too bold to pursue. However, it doesn’t seem like Sitecore has completely scrapped the ideas from Project Affinity. Rather, it appears their focus has moved towards offering additional resources and enhanced support for developers. Let’s dive into the latest developments.

OrderCloud is Part of the Sitecore Portal

Until now OrderCloud had its own portal. Sitecore has integrated OrderCloud in the Sitecore Portal. You need to work with partner support to add the OrderCloud Portal to your Sitecore Cloud Portal. It will show up in the Apps section.

If you already have have OrderCloud instance in portal.ordercloud.io, you can keep using that.
With the new portal, you have a more granular way to establish API access, the ability to create custom roles, a much enhanced API Console for filtering and sorting, and Index Tools to rebuild products and orders index.

OrderCloud Javascript SDK

What I am most excited about is the release of OrderCloud Javascript SDK. It’s a daunting task to work with Rest API when building a website using a Frontend Framework. As a developer, I want to focus on the feature development. Working with APIs directly unnecessarily adds repeated code to my projects. An SDK also mandates the development team to follow the same pattern for development. OrderCloud Javascript SDK works both on the browser and node.js. This means I can use the same SDK for Client Side Rendering (CSR) and Server Side Rendering (SSR). This is especially useful for building applications using Next.js. The SDK comes with built-in Typescript support, no additional types package is necessary.

React based Headstart Admin

The Headstart Starter Kit is now available in React. The earlier Headstart was based on Angular and it was not very easy to work with. This version of Headstart uses the OrderCloud Javascript SDK. This is only the starter kit for the admin portal; it does not include a buyer portal. While Project Affinity had the ambition to deliver a Universal Commerce Management Backoffice, which hasn’t materialized, the availability of this starter kit for developing an Admin Portal is a step forward. It’s quite rare for a commerce platform not to offer an out-of-the-box Admin Portal, considering such a feature typically doesn’t vary much from one customer to another.
In future releases, Sitecore is planning to add Headstart Buyer Portal, integration with Sitecore Discover for product search and product recommendations, example solution for connecting Sitecore OrderCloud with Content Hub One and XM Cloud.

Delivery Configuration

Sitecore took the approach of using Pub/Sub pattern for delivering data or messages. In this approach, you define a target and subscribe to get the data delivered to the target. For example, you want the order notification to be sent to Sitecore Send so that email notifications can be sent to customers. Delivery targets can be Sitecore’s internal targets (Send, Discover) or external targets (Kafka, HTTP endpoint, Event Hub). This approach can be used for Product Synchronization, Order Synchronization, and more.

New Features

Last year I wrote about Product Collection and mentioned that it was not complete. Sitecore made significant enhancements to the Product Collection feature this year. It seems to be ready for the mainstream use.
The brand new feature introduced this year is Product Bundles or Kits we say sometimes. Product Bundles are unique SKUs created by combining various products. OrderCloud offers a suite of APIs that enable the creation of these bundles, their assignment to catalogs, the application of promotions to them, and the ability to set pricing either for the bundle as a whole or for individual items within it. Functionally, bundles are treated like individual products, meaning they can be searched for and placed into the shopping cart just like any single product.

Acknowledgments
I would like to thank Ashley Wilson, Commerce Product Manager at Sitecore for sharing the PowerPoint slides of her presentation at the MVP Summit. Some images used in this article are taken from her presentation.

How to troubleshoot “dependency failed to start: container sxastarter-cm-1 is unhealthy”

When you are setting up your Fullstack Sitecore XM Cloud environment in your local machine there is a chance that you may see the below error.

dependency failed to start: container sxastarter-cm-1 is unhealthy
Waiting for CM to become available…
Invoke-RestMethod: .\up.ps1:59
Line |
59 | … $status = Invoke-RestMethod “http://localhost:8079/api/http/routers …
| ~~~~~~~~~~~~~
| No connection could be made because the target machine actively refused it.

I am also adding a screenshot of the result after running the .\up.ps1 script to bring up your docker environment.

This error can happen for many reasons. Instead of guessing what went wrong you can troubleshoot the issue using docker commands.

The first thing you should do is inspect the container to see the health check status. Run the below command to do that. Your container name can be different than sxastarter-cm-1.

docker inspect sxastarter-cm-1

Go to the “Health” section of the output and see the messages. In my case, the log showed Internal Server Error.

At this point, you can look at the docker logs by running the below command in the Powershell Window.

docker logs sxastarter-cm-1

In my case log showed the same error as the Health logs above. The health check URL “/healthz/ready” showed HTTP status 500.

To know what exactly caused the 500 error you can browse that URL using the below docker command.

docker exec -it sxastarter-cm-1 curl http://localhost:80/healthz/ready

The above docker command browsed the URL using curl and returned the below result for me.

The output indicates I had a problem with the Sitecore license “Required license is missing: SiteCore.Runtime”. I found that the license I was using had expired.

The above approach can be used to troubleshoot issues with other containers in the XM Cloud.

Funny bit for Sitecore Developers: Error shows SiteCore.Runtime 🙂

So, how did I fix the issue? It was easy for me. The docker inspect command shows me that the license folder for the cm container is mounted to the physical folder of my license file with the mount type “bind”.

That means if I replace my license file with the correct license file, it will be reflected in the container if I restart the container. I replaced the license file, ran .\down.ps1, and then ran .\up.ps1. Voila!

What’s Coming to Sitecore OrderCloud

In my previous blog post, I discussed Sitecore OrderCloud’s philosophy of “Flexibility Over Features” and the product’s strategies around this philosophy. After that Sitecore Symposium happened and we had opportunities to hear Sitecore OrderCloud’s roadmaps and directions from the Sitecore leadership team. Based on that I need to amend my previous blog post. I also gathered some information about the Product Collections functionalities roadmap with the Sitecore OrderCloud team. I will discuss that in this article because some issues I raised in my previous article will be addressed by items included in that roadmap.

Project Affinity

Sitecore is not abandoning the “Flexibility Over Features” philosophy. Sitecore OrderCloud will continue to be part of MACH Alliance and enhance the product based on its core philosophy. The new direction is that there are features and integrations that make sense to add as part of the solutions (not necessarily as part of the core product) because those can be used by customers. And, those can be done without compromising the core philosophy. For example, most eCommerce solutions need to calculate tax and creating tax plug-ins to Avalara, Vertex, etc. makes sense as long as Plug-ins can be customizable and configurable. Full details are not available yet, but I envision that we will be able to choose the tax service in the OrderCloud portal and configure that. In the Calculate an Order integration event a call will be made to the tax plug-in to calculate the tax. The plug-ins will have integration options like webhooks so that we can customize them. In addition to providing out-of-the-box integrations, Sitecore has also decided to provide Storefront templates based on XM Cloud and a Commerce Management Portal for business users. This will significantly make it easy for the partners to recommend OrderCloud as an eCommerce platform to their clients. All these are part of Project Affinity which Sitecore announced in the Symposium. The timeline as announced is the end of 2023.

Sitecore Search

Sitecore Discover although treated as a separate service it is becoming a first-class citizen for OrderCloud-based solutions. It will cover both product search and content search for the Storefront. Discover’s AI-based search will provide experience-based search results for eCommerce solutions. OrderCloud does have an in-built search for products, customers, and orders. I think this search will be used by the Commerce Management Backoffice.

Sitecore Connect

Sitecore announced Sitecore Connect, an integration platform that can be used by partners and customers to build Low-Code/No-Code integrations. Sitecore Connect comes with 1000s of pre-built connectors. These connectors can be integrated with Sitecore OrderCloud in an almost drag-and-drop fashion. It makes sense for Sitecore to build some connectors like payment gateways, tax connectors, etc. in the OrderCloud platform itself because of the sensitive nature of the data. For example, payment gateways require the platform to be compliant with PCI compliance. Also, integration between Sitecore products will also be expected for Sitecore to build. One such example is the integration between Sitecore OrderCloud and Content Hub. But, Sitecore Connect opens up immense possibilities for connecting OrderCloud with diverse platforms. In the B2B business, in most cases, sellers use some kind of ERP for order fulfillment. There are many ERP connectors that will be available in Sitecore Connect. Some examples are Infor, Acumatica, Oracle, SAP, etc. I would imagine that when Sitecore Connect will be released, OrderCloud can be integrated with these ERPs.

New Product Collections Enhancements

In my previous blog, I discussed the limitations of the current Product Collections data model and APIs. Particularly I pointed out that Product Collections can be created only for users and we can’t add eXtended Properties at the item level. I took these to the OrderCloud product team so that I can add the feature requests. What I learned they are ahead of me. They shared the below enhancements from their roadmaps. Bolded enhancements are what I wanted to request.

• ProductCollection entry xp
• ProductCollection readable by Marketplace Owner (MPO)
• “Public” ProductCollection  (viewable by any buyer user in your marketplaces)
• Sharable ProductCollection (viewable by designated buyer users in your marketplace)
• Properties to enable registry-like functionality (QuantityRequested/QuantityPurchased)

The disclaimer from the OrderCloud team is that currently the above enhancements are being considered, but the plan may change and some of them may not be part of the product.

Final Thoughts

I am excited about the future of OrderCloud and the path Sitecore is taking. Especially, including Sitecore Connect in the architecture shows Sitecore’s commitment to Composable Architecture. According to Gartner one of the pillars of composable DXP is Integration. Sitecore Connect will fill up that gap and help OrderCloud to reach diverse platforms.

Flexibility Over Features Philosophy in Sitecore OrderCloud Architecture

Sitecore OrderCloud is a different kind of eCommerce platform. We know that it is a cloud-native, API-first, headless, SaaS platform, but Sitecore claims that they followed a philosophy when they built the platform or when they are thinking about building new features in the platform. Sitecore calls it “Flexibility Over Features“. You can read about this in this article. One of the main reasons for following this philosophy as explained in the article is that B2B Commerce is too complicated and it is not possible to create features that cover all business cases. Instead, Sitecore decided to create an eCommerce engine that’s flexible enough so that all kinds of business scenarios can be built on that. The claim is that building features will make the architecture rigid. Is this true? In this article, I will examine if it is true and if Sitecore is following up with what they are saying.

This is not an article to establish that Sitecore OrderCloud is a superior or inferior eCommerce platform because it follows this particular philosophy. In fact, there is no platform out there that satisfy all B2B Commerce sellers’ need. The decision to choose an eCommerce platform depends on many factors, including architecture, but often architecture is not the only factor.

Most eCommerce platforms in the market come with an administration portal. Sitecore OrderCloud doesn’t have one. There is Seller Admin Headstart, a starter kit open-source solution built on AngularJS, but it is not a full-featured Admin Portal. Not even close. This comes up often when prospective clients look at the OrderCloud platform. OrderCloud provides a portal where the organization’s marketplaces (businesses) can be managed, but it is not built for business users. Portal provides API Console and some enhanced UIs for creating webhooks, API Clients, etc. and it can be understood by developers. Since Sitecore OrderCloud is providing a commerce engine on which clients can implement their features based on business requirements the Admin Portal features will be different for different clients. This is the line of reasoning behind not having a single Admin Portal for the platform.

I will explain how the Sitecore OrderCloud team approaches Flexibility Over Features using a recent enhancement they have added to the platform. They have added Product Collections feature. A Product Collection is a data model which represents a collection of products that you can create via ProductCollection APIs. You can learn more about Product Collections if you read this article. A Product Collection can be used for different purposes, but let me start with some requirements I have seen before in the B2B Commerce implementation that I was part of.

Most eCommerce sellers want Wish List feature and all most all eCommerce platform comes with Wish List feature. But, since the B2B buying process is bulk buying and repeated buying often, clients need more than just Wish List. Some Clients asked us to implement Frequently Purchased Items List from previous orders. Most B2B users want to work with multiple orders. In the process of creating orders they want to save the order so that they can modify or purchase the order later. This is called Saved Order feature. Saved Order is a very common feature in eCommerce platforms. Ecommerce platforms commonly implement the above-mentioned features as separate features with their own workflows and data models that fit the purpose. You can customize Wish List or Saved Order, but you can’t use them for creating something else like Frequently Purchased Items. The benefit of this approach is that you get Wish List and Saved Orders features out of the box with the eCommerce platform. If that works well for your business it’s great. On one occasion, a client’s requirement was to implement Favorites. Favorites is more of a B2C feature where logged-in users can favorite products so that they can find their favorite products from the list. It is close to Wish List except that when the user is visiting a favorited product, we need to show the favorite icon.

Sitecore OrderCloud doesn’t provide features like Wish List or Saved Orders. The approach in OrderCloud is based on Flexibility Over Features. They see all these features as collection of products. It is kind of a minimalistic approach where OrderCloud says, we will support to give you the ability to create and manage Product Collections, implement features using Product Collections that suits your purpose. So, we can use Product Collections to implement Wish List, Saved Orders, Frequently Purchased Items, Favorites, etc. How to do that? Below I tried to explain with some examples.

We will not go into great detail to implement Wish List or Saved Orders, but talk about simple implementation. All these features are collection of products with additional functionalities. Using Product Collections APIs, I can create Product Collections and identify the type of collection with an XP property “CollectionType”. The XP is helping me to customize Product Collections for my implementation. Here is API request for this.

curl --location --request POST 'https://sandboxapi.ordercloud.io/v1/me/productcollections' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <buyer token value>' \
--data-raw '{
    "Name": "Wish List 1",
    "xp": {
        "CollectionType": "Wish List"
    }
}'

The above code will create Wish List product collection. I can use a similar code to create another Wish List or a different product collection like, “Frequently Purchased Items“. After this, we need to add products to the collection. The below code shows how that can be done.

curl --location --request PUT 'https://sandboxapi.ordercloud.io/v1/me/productcollections/<product collection id>/<product id>' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <buyer token value>' \
--data-raw '{
    "xp": {
        "Quantity": 2
    }
}'

The above code adds a product to the product collection, but it doesn’t add the XP. We will talk about the XP part soon when we discuss the current limitation of the Product Collections data model. Using the Product Collections data model and APIs we can implement different kinds of List features as needed. Since OrderCloud doesn’t dictate how Lists should work, it’s completely up to the implementor to decide how to build the features. This is a very powerful way to build solutions, but a lot of restraints require on the platform architecture to stick to this idea and OrderCloud is doing that by considering platform enhancement based Flexibility Over Features philosophy.

I have created a Postman collection for this purpose. You can fork from my collection and work on Product Collections API by visiting the below Postman link.

Let’s talk about the limitations of the current Product Collections data model and APIs. First, Product Collections works at the ‘Me’ level, which is the current storefront user. This works for the lists on B2C Commerce, but on B2B Commerce, lists are often needed at the Buyer level so that all Buyer users can share lists. The Product Collections documentation mentioned this limitation and talk about expanding the feature to the Buyer level in the future. The other limitation I found is that, when adding products to the collection, I can’t add XP at the item level. I can add only the product id. API doesn’t support this.

This is a limitation because implementation using the Product Collections feature will need to save custom data at the collection item level using XP. For example, if I use Product Collections for implementing Saved Orders, I need to save orderline level data like quantity, line notes, etc. We can get around the limitation by storing data outside of OrderCloud, but that will be a lot more complicated to implement than using XP to store that data.

As an architect, I like the idea of Flexibility Over Features because it gives me lots of freedom to create architectures for implementations and reduce friction. My preference for recommending an eCommerce platform for our clients doesn’t depend on just one notion of the architecture. In spite of that, I like the way Sitecore OrderCloud approaches building the platform around this concept. It fits well in Cloud based SaaS architecture.

Sitecore’s Year of XM Cloud

The Sitecore Symposium is in a month, happening in Chicago from 17th Oct. to 20th Oct. Every year we anticipate the big announcements about the platform. We MVPs who closely follow the discussion in different Sitecore forums and Sitecore User Groups, get a sense of what is coming. Last year it was all about Composability. Sitecore acquired a bunch of companies and moved the platform from monolithic architecture to composable architecture. But, composability doesn’t work well if the core platform is not in Cloud and SaaS. To fill up that gap Sitecore quickly moved the core CMS platform to Cloud. It is called XM Cloud which is a Cloud-native Headless CMS platform without losing the Content Authoring experience. This is a year of XM Cloud.

Last year’s focus on the composability of the architecture provided us with opportunities to explore the cloud-native SaaS platform that Sitecore acquired. I wrote several technical articles on OrderCloud, Content Hub, Experience Edge, and JAMStack, and how to approach Frontend First Architecture for CMS applications. This year we haven’t seen much movement on these composable platforms other than building demos by Sitecore using these platforms to showcase how to utilize these platforms to build applications and focusing on moving core Sitecore CMS to the cloud. As MVP, we learned about XM Cloud early on but it is now released in a limited capacity for customers to try. MVPs and Partners have to wait to try their hands at it. So, can’t do much architectural exploration at this time other than what Sitecore is documenting and showing us. A good place to start for everyone is this article.

The architecture and technology considerations for moving XM to the cloud are quite interesting. Sitecore is trying to upgrade the architecture of core CMS to match the current web technology trend but also trying to keep the features and functionalities they built over the years. This tension of not throwing out the good things that Sitecore spent so much time to build but also being competitive with much younger CMS platforms, created an interesting architecture for XM Cloud. For example, the feature like Experience Editor is tightly coupled with the platform and many more features are like that. So to move XM to the cloud requires to deploy everything using containers.

Server-Side Development Experience

Since the XM Cloud is quite different from the SaaS Cloud application, the development experience differs significantly. Thankfully, it is actually a good thing for Sitecore developers because, if you are doing server-side development, it is no different than what you do now except for the deployment process. For example, if you are extending a pipeline, you will do the same way and push your code to GitHub after testing on your local machine. You will go to your Organization’s project environment at https://portal.sitecorecloud.io/ or use the Sitecore Command Line Interface to deploy code. This is usually not the way SaaS application works. In SaaS applications, custom extension codes are kept separate from the core platform code. For example, the extension code for the OrderCloud application is separated into the middleware and the middleware code integrates with the OrderCloud via Webhooks and Integration Events. This architecture has a great benefit when it comes to upgrading the platform because the custom code is separated from the platform code. It will be interesting to see how the XM Cloud upgrades will be done.

Frontend Development Experience

XM Cloud makes the data available in the Experience Edge (a CDN) via GraphQL APIs. So, there is no CD server or web database. Since contents are available to the Edge as headless GraphQL APIs, you are free to use any Frontend Javascript Framework to build the web application. But, the problem with building web applications without considering the XM Cloud server-side development is that you lose the business user-friendly content management through Experience Editor. This additional consideration force you to use JSS and you can’t be just a frontend developer to develop a web application on Sitecore. This would not be the case if Experience Editor experience can be provided on the frontend side. That’s something companies like Uniform and Builder.io are doing. They have created a framework to connect to headless CMS and provide an Experience Editor-like content management experience. The sitecore roadmap shows that they are building SiteBuilder which will be the frontend version of Experience Editor. Since contents are available heedlessly on the Edge building JAMStack application and hosting them on any hosting provider like Vercel, Netlify, etc. has become very easy.

What about SXA?

SXA concerns with server-side component development. Now since things are moving to the Frontend, headless SXA is introduced. In Headless SXA way developers start from SXA to build the web application and then use JSS to complete the Frontend. Again Sitecore is trying to adjust its current technology.

Where are we heading?

Sitecore hasn’t not started as a headless Content Management System. The technology on which Sitecore was built was server-side technology. The advancement of JavaScript frameworks pushed the technology more towards frontend and headless. But, headless has it’s problem too. It’s true that headless system are overly developer centric. I think, Sitecore is going to move more and more content management features to the frontend. The current situation with shift in the architecture is normal in any product’s life cycle in the realm of technology advancement. Like any other product, Sitecore has to catch up with new technologies and at the same time keep selling the product. To be honest architecture doesn’t run the business, business influences architecture.

Sitecore Experience Edge API Architecture and a Postman Collection

Sitecore Experience Edge is the new delivery platform for Sitecore Content Hub and Sitecore XM. As it happens documentation is not all very clear. While setting up the Experience Edge I encountered some issues. In this blog post, I discussed those issues and I discussed some concepts that I think will help others to understand Experience Edge architecture.

I have created a Postman API collection for Experience Edge APIs and shared it in this post. Click on the button below and fork from my publicly shared collection to use the APIs in your Postman. If I modify the collection in the future, you will be able to sync with your version.

Experience Edge APIs

Understanding Experience APIs can be confusing initially. Let’s talk about the purposes of different domains first.

Content Hub Sandbox Domain

This is the domain name for your Content Hub Sandbox. The URL looks like <sanbox name>.stylelabsdemo.com. This is also the domain used for GraphQL Preview API to return GraphQL results of unpublished data. The URL for GraphQL Playground (IDE) is 
https://<sandbox name>.stylelabsdemo.com/api/graphql/preview/ide/ and GraphQL API endpoint is https://<sandbox name>.stylelabsdemo.com/api/graphql/preview/v1

Auth0 or Authentication Server Domain

This service is responsible for providing security for the Experience Edge Administration. We retrieve the JWT authentication token from this domain to use the admin APIs securely. The URL looks like https://one-sc-beta.eu.auth0.com

Audience Domain

The audience URL contains the Tenant Id for your Experience Edge system and identifies the tenant for which the authentication token is going to be issued. This token can be used only for administering the tenant mentioned in the URL. The URL looks like https://delivery.sitecore-beta.cloud/tenant_id. Tenant Id is usually the name of your Content Hub Sandbox. You can also find the Tenant Id in the Content Hub license using the API Content Hub API route /api/status/license.

Experience Edge Domain

Experience Edge is the delivery service that runs on Cloudflare. When we publish content from Content Hub those contents get pushed to Experience Edge and get cached. Experience Edge domain is used to manage the delivery service, manage delivery API Keys, and query data using GraphQL queries. The Admin API endpoint base URL is https://edge-beta.sitecorecloud.io/api/admin/v1,
the API Key endpoint base URL is https://edge-beta.sitecorecloud.io/api/apikey/v1,
GraphQL Playground URL is https://edge-beta.sitecorecloud.io/api/graphql/IDE,
and GraphQL API endpoint URL is https://edge-beta.sitecorecloud.io/api/graphql/v1.

The above mentioned domain names are what provided by Sitecore currently for the Sandbox. It may not be same in future. The production server domain names will be different. Whatever the domain names are these are the four domains and the corresponding URL we should be aware of.

There are Four types of APIs.

Admin API

Admin APIs are used to administer the Experience Edge delivery system. For example, if you want to change the Content Cache Time To Leave (contentCacheTtl) setting, you will use Admin API. The base URL for this is https://edge-beta.sitecorecloud.io/api/admin/v1.

Token API

Token APIs are used to manage API Keys generated for accessing the Experience Edge delivery system. For example, if you want to see what API Keys are there currently available in the system, you need to use the Token API. The base URL for this is https://edge-beta.sitecorecloud.io/api/apikey/v1.

Delivery API

Delivery API is the GraphQL endpoint to query content from the Experience Edge delivery system. You can use the Delivery API to access only the published data from the Content Hub. The URL for GraphQL Playground is https://edge-beta.sitecorecloud.io/api/graphql/IDE, and the URL for GraphQL endpoint is https://edge-beta.sitecorecloud.io/api/graphql/v1.

Preview API

Preview API is used to query unpublished data from Content Hub. The API Key and the URL for Preview API is different from Delivery API. The URL for GraphQL Playground is
https://<sandbox name>.stylelabsdemo.com/api/graphql/preview/ide/ and the URL for GraphQL endpoint is https://<sandbox name>.stylelabsdemo.com/api/graphql/preview/v1.

The following diagram shows different APIs and their URLs.

Location of client_id and client_secret for JWT

To get the JWT token to be used to authenticate Admin API, you need a client_id and client_secret. The documentation about where to find this information is not very clear. This is available in Content Hub OAuth Client. The OAuth Client name is ‘delivery’. Open that client and use the client_id and client_secret from that OAuth Client in Admin Token API.

Summary

In this blog, I discussed how APIs for Sitecore Experience Edge for Content Hub has been structured. I hope this helps you to understand the architecture quickly and helps you get on with your work. I have provided a Postman collection that will help you to quickly start with the APIs in your Experience Edge system.

References

• Experience Edge for Content Hub Documentation
• A developer’s look into Sitecore Experience Edge

Build a Static Website with Sitecore Experience Edge for Content Hub and Next.js

I am back with another installment of building static website with Content Management Service and I am back with another installment of building a static website with Content Management Service and Next.js. This time I am adding Sitecore Experience Edge for Content Hub in the Frontend First Architecture. Sitecore Experience Edge is a Content Delivery Service on the top of Content Hub or Sitecore XM. Experience Edge delivers content through GraphQL endpoints which I have used to generate the static web pages for my photo blog website. For my purpose, it is just another source of my website content. But, Experience Edge is more than just a content source. I have highlighted some important features at the end of this article.

Deployment Architecture

Based on my Frontend First Architecture, I needed to add Experience Edge in the deployment architecture. When I switch my content service to use Experience Edge, it uses GraphQL delivery APIs to get content from Experience Edge CDN and push my static pages to Vercel Edge. The following diagram shows the deployment architecture with Experience Edge included as one of the content delivery platforms.

Deployment Architecture including Experience Edge

Application Architecture

Our application architecture has not changed except that we have a new API Helper for working with Experience Edge. There is no need to discuss about the application architecture again. You can find that in my previous blog article. Below is the application architecture diagram including the new XEdgeApiHelper highlighted.

Application Architecture including Experience Edge Api Helper

It took me very little time to implement the new Api Helper code. In ContentFulApiHelper I have used GraphQL queries to populate data. I used the same approach for XEdgeApiHelper except that the GraphQL schemas are different. So the queries are different and handling data returned by the queries was a little different. I have shared the code in Github so that you can take a look.

Experience Edge Features

I am not going to discuss all Experience Edge features. You can learn about Experience Edge from the

I am not going to discuss all Experience Edge features. You can learn about Experience Edge from the documentation. I am going to talk about how to approach to work with Experience Edge. There is no free tier for Experience Edge or Content Hub. If you are an MVP you can send an email to
mvp-program@sitecore.net to get $50 credit. If the company you work for already has a sandbox that’s a better option for learning. You may read this document to understand how to set up the Content Hub sandbox including Experience Edge.

Currently, there is no UI to interact with Experience Edge. You have to manage it using APIs. I am hoping Sitecore will come up with a CLI at some point. There are two types of APIs, Management APIs (Token and Admin APIs) and Content APIs (Delivery and Preview APIs). The Management APIs are Rest APIs and Content APIs are GraphQL APIs. The following diagram shows how APIs are used.

Source: Sitecore

You will need a client_id and a client_secret from Sitecore to generate the token for authenticating Admin and Token APIs. For using Delivery and Preview APIs, you have to create API Keys in Content Hub.

The way Sitecore is positioning Experience Edge as a Content Delivery platform, I think Experience Edge will be always needed for Content Hub implementation, although the license for Experience Edge is separate from the Content Hub license. You may use just the Content Hub for your implementation but it will not perform as well as if you have Experience Edge. In my first post in this series, I discussed how we can implement with just Content Hub using Javascript Client SDK. Then the question arises, what is the use of Content Hub APIs. I think the Content Hub APIs will be used mainly for automated content creation and management. But when it comes to rendering content, the advantage of using Experience Edge cannot be ignored. The biggest advantage of using Experience Edge is omnichannel content delivery which can be based on devices, location, and many other segments. Rendering content via GraphQL gives the content consumers the ability to decide what content they want. Along with GraphQL, content caching and delivering content from the closest location will provide optimum performance.

That’s all for this article. Please checkout the code in Github. See you in the next article.

References

• How I solved the issues I encountered to build a static website using Next.js and Sitecore Content Hub
• Frontend First Architecture for Decoupled Headless CMS Integration
• Experience Edge Documentation
• Code

Frontend First Architecture for Decoupled Headless CMS Integration

In my last two blog posts, I discussed creating a static website using Nextjs and Sitecore Content Hub as the Headless CMS. I focused on creating an application architecture that can be integrated with Sitecore Content Hub only. A Headless CMS like Content Hub is like any other service in Composable DXP as opposed to traditional CMS. An application built on a traditional CMS has the CMS in the core and other composable services are added around it. There is no separation of the web application from the CMS in traditional CMS. Such is not the case for an application built with Headless CMS. We have a choice to select what CMS we want to use. Therefore the architecture of the application can be built with CMS decoupled from the application. This will give us the flexibility to use any Headless CMS with minimum change in the application. In this blog, I will discuss how I changed the architecture of my application for the previous posts to adapt a decoupled architecture. I used Sitecore Content Hub and ContentFul as Headless CMS. We will establish the fact that with this architecture we can switch between Headless CMS with very little change in the application.

Final Deployment Architecture

Below is the diagram that depicts the final state of our web application deployment architecture. Our application has the ability to switch between Headless CMS. A quick code change and deployment will switch the CMS.

Application Architecture

I built the architecture from the Frontend. If we build the architecture from the backend, i.e. a Headless CMS, it will be an application for that CMS. When we approach the architecture from the Frontend, we know what we need. We need all the React Components and we need data (props) for filling up these components with information. The data will be provided by the Headless CMS APIs. But, for building the schemas for the data, we don’t need to use a CMS immediately. We can build the schema and create fake APIs using something like json-server. I created a JSON file with data I needed and used json-server as if I am calling CMS APIs. This allowed me to build the full Frontend without worrying about any CMS. This way I could focus on the architecture.

To build an architecture that is decoupled from Headless CMS, I needed to use the Inversion of Control (IoC) design principle. There are many Dependency Injection (DI) Framework out there to implement IoC. One of the popular ones is TSyringe, a lightweight DI Framework for Typescript/Javascript. When it comes to using Design Pattern it is easier to work with a Typescript because it is a typed language. I converted my earlier Javascript based Nextjs application to Typescript. Next, I created an interface called IApiHelper for the methods I needed to create props for my React components. My component services used the CMS specific implementation of this interface to get data for the props. The DI container is used to inject the desired API helper to the service class to get the data from json-server or Headless CMS. The below diagram shows the architecture.

I have shared the code in Github.I have three implementations of IApiHelper. For Content Hub, I created ContentHubApiHelper, for ContentFul, I have created ContentFulApiHelper and for Json-server, I have created JasonServerApiHelper. To use a CMS, I need to import the Api Helper implementation of that CMS in GetStaticPropHelper.ts as shown below in the highlighted code.

import "reflect-metadata";
import HomeProps from "../models/HomeProps";
import BlogListProps from "../models/BlogListPorps";
import BlogProps from "../models/BlogProps";
import PageComponentService from "../services/PageComponentService";
import PageLayoutService from "../services/PageLayoutService";
import { container } from "tsyringe";
import { GetStaticPropsContext } from "next";
import { ParsedUrlQuery } from "querystring";
import AboutPorps from "../models/AboutProps";
import ApiHelper from "./ContentFulApiHelper";

container.register("IApiHelper", {
	useClass: ApiHelper,
});

Final Words

The purpose of this blog was to discuss how to build an architecture from the frontend so that we can decouple the web application from Headless CMS. I had to learn a whole new CMS called ContentFul which I enjoyed. ContentFul has fantastic documentation and tutorials. It is free for developers. I highly recommend you to explore ContentFul. I have deployed this web application in Vercel. This link https://photoblog-nextjs-ts.vercel.app/ will take you to the website.

References

• ContentFul
• GraphQL Content API
• Typescript
• getStaticProps not working well with TypeScript
• Json-server
• TSyringe
• How I solved the issues I encountered to build a static website using Next.js and Sitecore Content Hub

Two ways to publish content on-demand from Sitecore Content Hub to a static website

In the last post, I discussed how I created a static website using Next.js and Sitecore Content Hub as a content repository. Web pages in the static website as it says are static. That means when some content changes in the content repository that will not be reflected on the website unless we deploy code. In the solution, I used Next.js Incremental Static Regeneration (ISR) so that static pages will be revalidated after the assigned time and pages will be regenerated. For example, the below code shows that the Blog List page will be regenerated every hour.

  static async getStaticProps() {
      const client=await Helper.getContentHubClient();
      if(client) {
        const mainMenuItems = await Helper.getMainMenuItems(client);
        const footer = await Helper.getFooter(client);
        const intro = await Helper.getPageIntro(client, introName);
        const blogList =  await Helper.getBlogsFromCollection(client, contentCollection, route);
        return {
          props: {
            mainMenuItems: mainMenuItems,
            footer: footer,
            message: intro,
            blogList: blogList
          },
          revalidate: 3600
        }
      }      
  }

The above approach is ok but it doesn’t give any control to content editor. Once the content editor change content, she has to wait until someone accesses the page to trigger the build. The approach I will discuss here will enable the content editor to publish as and when needed.

Publish content to static website using Sitecore Content Hub Action

Vercel lets us trigger a build using Deploy Hooks. Deploy Hooks is an API endpoint with unique id that is associated with the deployment configuration. When someone calls this API, Vercel will start the deployment for the associated configuration. In the build settings in Vercel you will find the Deploy Hooks in the Git section. Here you can add Deploy Hooks.

Deploy Hooks

We need to take this API endpoint and create an action of type API Call in Sitecore Content Hub. Below screenshot shows that action. There is only one purpose of this action. That is to call the Vercel Deploy Hook.

Content Hub Action

The usual way to call an Action in Content Hub is to use a Trigger. A Trigger is associated with some events in Content Hub. When such events happen Trigger calls the associated Action. Example of events are, add/modify/delete of an entity in Content Hub. Below is screenshot of such trigger.

Trigger

Trigger Action

Although this approach works, there is a problem with this approach. The trigger runs every time content gets modified, and the action gets called. The action calls the Deploy Hook to run a build in Vercel. If the content editor changes a lot of content, this approach will deploy the site too many times. That’s not an efficient way to regenerate static pages. Ideally, I would like the content editor to push a button to call the action to publish the changed content when she is ready.

There is no straightforward way to call an Action manually in Sitecore Content Hub. Other than using Trigger, I can use Command API to call the External Action command like below but that will be calling from outside of Content Hub. I want to call the same API from Content Hub Admin site.

Request URL: https://my-ch-sandbox.stylelabs.io/api/commands/external.action/external.action
Request Method: POST

Request Body:
{
    "entity_id": XXXXX,
    "action_id": XXXXX,
    "properties": [],
    "relations": [],
    "action_execution_source": "ExternalAction",
    "extra_data": {
        "culture": "en-US"
    }
}

There are two ways to call the external.action command from within Content Hub that I am aware of. I discussed both methods below.

External Page Component

External Page Component allows us to create a component using external JavaScript libraries. I have used jQuery to create a button on the Content Collection page. I have explained below how it works. Here is the button (External Page Component) on the Content Collection page.

External Page Component

To add an External Page Component on a page, you have to go to Manage -> Pages and select the page where you want to place the component. In my case, I opened the Content Collection page and added External Component on the right header column.

Add External Page Component

To configure the component I clicked on three dots (…) and selected Edit. Here in the Configuration section I entered the control name.

External Component Configuration

In the Template section I entered below HTML code to create the button UI. I used css classes available in Content Hub to create consistent UI. Notice that id of the button is ‘target’. This id is used in the jQuery code to call the action.

<a
		id="target"
		href="#"
		class="btn btn-primary"
		title="Generate static pages"
		aria-label="Generate static pages"
	>
		<i class="m-icon m-icon-lightning-bolt"></i>
		<span class="d-none d-sm-inline-block"
			>Generate static pages</span
		>
</a>

In the Code section, I used below jQuery code to make ajax call to the action on button click event.

$( "#target" ).click(function() {
  var req = '{"entity_id":30487,"action_id":30872,"properties":[],"relations":[],"action_execution_source":"ExternalAction","extra_data":{"culture":"en-US"}}'
  $.ajax({
    url: `${options.api.commands.href}`.replace('{folder}', 'external.action').replace('{command}', 'external.action'),
    contentType: 'application/json',
     headers: {
        'x-auth-token' : '<my api key for auth>'
   },
    type: 'post',
    data : req,
    success: function(data, status, jqXHR)
    {
        console.log('success')
    },
    error: function (jqXHR, status, error)
    {
      console.log('error')
    }
  });
});

This accomplice the task. You may notice that I had to use an entity (entity_id) to make the API call because entity_id is a required parameter. My action though doesn’t need to be associated with an entity. This approach is good for any page but it is a little complex. Also, using External Page Component is a little risky because if Content Hub changes anything related to external libraries, templates, or in the API in the future, the component might break.

Custom External Action Entity Operation

The second approach to call the action manually doesn’t involve any coding. You can add External Action Entity Operation on any Detail Page. A Detail Page is always associated with an entity and the operation use that entity to call the action. If you follow the below animation, you will see how I added the External Action Operation on the Content Collection Details page.

Add External Action Entity Operation

This approach is simple. The only issue with this approach is that the Entity Operation button shows on all Content Collection Details pages. If we can live with that, this is the approach we should take.

Conclusion

In this article, I discussed how we could give control to content editors to publish content from Sitecore Content Hub to a static website. I have used Vercel for deploying and host my static site. The same approach works for other static site hosting services like Netlify, Surge, and others.

References

• How I solved the issues I encountered to build a static website using Next.js and Sitecore Content Hub
• External Page Components
• Entity Operations