Posted on:
Categories: Business;Office 365;SharePoint

​One of the biggest advantages for enterprises using SharePoint is the powerful SharePoint Search engine and, in my opinion, the use of metadata is the biggest strength of SharePoint Search. Although the ability to index content is very helpful for many SharePoint users, as this is a major advantage compared to searching file shares (which are still quite often used), the joining of indexed content and metadata is the real fuel that powers SharePoint Search.

Besides acting as additional filters in a search query, metadata enables SharePoint Search to provide a list of refiners. Refiners allow users to quickly and easily filter large result sets to focus on relevant content. With the help of metadata users can filter their search results manually with just a few clicks. To reduce this manual process, SharePoint Search can pre-sort search results based on a relevancy score that gets calculated by evaluating metadata and user behaviour. In other words: SharePoint Search tries to uplift items in result sets which SharePoint thinks are relevant for the current user.

At first glance this looks like an advantageous feature, but if you have a closer look at how SharePoint Search is handling metadata and is calculating the relevancy score, it becomes obvious that the biggest strength is also the biggest weakness.

SharePoint Search can only utilize metadata in a beneficial manner if metadata is available and maintained thoroughly. If you have ever deployed information architecture and governance rules, you probably know what I'm talking about. Although metadata can be added to a SharePoint library or list very easily, it requires a lot of planning to create a metadata plan that truly reflects a company's requirements. You need to find the right balance between required and optional metadata. With too much required metadata, users get annoyed as they are coerced to enter the data. On the other hand, too much optional metadata has a negative impact on the search results and the relevancy score. One drawback of SharePoint Search is the need to have thorough metadata management.

Is there anything wrong with SharePoint's relevancy score?

Yes. I think the second drawback is the way SharePoint Search is calculating the internal values for the relevancy score. SharePoint search can only create relevancy scores based on all users that initiated a search with the same or similar search term. SharePoint Search evaluates how many users are clicking on each search result item. With this information and the entered search term, SharePoint Search is able to calculate a relevancy score, but this relevancy score is based on all users who searched for the same or a similar term. Although with this algorithm a practical relevancy score could be created, this relevancy score might not be a valid relevancy score for every user as it hasn't been based on individual factors. In other words, this relevancy is an average user relevancy. The more an individual user differs from the average user, the more this relevancy becomes meaningless.

How is Office Graph addressing these issues?

With Office Graph and its front-end application 'Delve', Microsoft is creating a new kind of search engine. Microsoft is advertising Office Graph as the next generation of search and discovery and I believe that's true. But we should take into consideration that Office Graph is still in development and is continuously improving. Office Graph is following a new approach to working with metadata and relevancy scores. Before I continue with the benefits for enterprises, let me explain how Office Graph works.

First of all, it is important to know, that Office Graph is not a replacement for SharePoint Search. In fact, the opposite is true: Office Graph is an addition to SharePoint Search and does not focus on SharePoint exclusively, but on all services that are part of Office 365, including any future services.

Office Graph does not rely on metadata. Instead it creates relationships between data and users and evaluates relationships between users. These relationships are called 'Edges' and are based on actions that users are performing (like viewing or modifying documents). In addition, Office Graph also puts a weight on each of these edges.  For example; the closer a user is to me (like a direct co-worker or one of my team members), the more important my relationship to this user is and the higher the weight for the corresponding edges will be.




How does this help in creating a more significant relevancy?

Instead of evaluating search terms and user clicks, Office Graph relies solely on these relationships. You can see how this is working by simply opening Delve in your Office 365 tenant. Delve is visualizing the results it gets from Office Graph and does not require a user to enter a search term. Instead, it shows all content (which includes documents and data) that the current user has access to and is sorted upon its relevancy – which is calculated based on the weight of the relationships. In other words: Office Graph is calculating the relevancy score based on the relationships (the 'edges') of the current user which is a true relevancy for that particular user. Because relationships are dynamic (they can change often based on the work context of a user), the relevancy score that Office Graph is using is dynamic too. This true relevancy is one of the benefits for enterprises using Office Graph as it will provide more significant search results for every user.

But true relevancy is not only beneficial when thinking of search results. There is an element that is often used in SharePoint intranets to provide individual directories or lists for the current user: a search driven application!

Search driven applications can provide data that is relevant only to the current user. Common examples are 'My documents' lists or 'My project sites' directories. Search driven applications use a preconfigured search query that is sent to SharePoint Search to retrieve data. Most often the results are sorted based on date or time and it is assumed that this kind of sorting provides the most relevant items for the current user. Although timestamps can be a good criteria to create a user-specific relevancy, this is not a true relevancy, because it does not consider the current user's working context. The newest document might not always be the document that matters most. If the information that Office Graph is able to deliver is joined with the result sets that SharePoint Search is providing, the evaluated results would be far more relevant to the current user - because they are using edges to create a true relevancy. User engaging elements that are based on lists or directories created by search driven applications using true relevancy powered by Office Graph are a great benefit that enterprises can expect from using Office Graph.

Is Office Graph tied to Office 365 services or SharePoint Online exclusively?

As previously mentioned, Office Graph is not a replacement for SharePoint Search. It follows a much broader approach by incorporating all services that are part of Office 365. Currently only a few services are included into Office Graph, but as Microsoft is improving Office Graph, more and more Office 365 services will be integrated. And there is more: with Microsoft Graph (formerly known as 'Universal API') it will be possible to include 3rd party systems into Office Graph. This will facilitate the integration of data from external systems into the corporate search results of an enterprise's intranet. Currently, with SharePoint, 3rd party integration is done using the Business Connectivity Service which provides the ability to implement business applications perfectly tailored to the needs of an enterprise. With Office Graph and Microsoft Graph, additional 3rd party systems can be integrated. While the Business Connectivity Service provides a way to integrate data from 3rd party systems, Microsoft Graph will be able to join external search results as an additional mechanism of integration. Although Office Graph and Microsoft Graph are not providing this functionality today, Microsoft is currently working on this feature and it will be released as soon as it's ready. The improved way of integrating data from 3rd party systems is the third benefit for enterprises using Office Graph.

Office Graph is still under development – should enterprises start using it now?

I think it depends on the self-conception of each enterprise. Enterprises following a very traditional way of working might risk overwhelming their employees. For those enterprises it might be helpful to carefully introduce their employees to Delve first. But for enterprises trying to stay up to date with technology, Office Graph and Delve are ready to go. The search results that Delve is delivering today are providing a clear benefit for users. Delve is already a part of Office365 and it is available to every user by clicking on the 'About me' link shown in the user profile's context menu. Many users might have used Delve already.

If enterprises are thinking of remodeling their existing intranet by incorporating modern, user-engaging elements, Office Graph and Delve are providing a great view on the technology that is about to change the way we are searching for data. With the availability of Microsoft Graph (formerly known as 'Universal API'), coding custom applications that use search results from Office Graph will become more future-proof.

Office Graph is providing a first glimpse, but what can we expect in the future? 

It's not easy to predict the future, but if I look into my personal magic crystal ball, I'm expecting search engines to use far more artificial intelligence. Metadata will become less significant and will be replaced by keywords that are automatically created by search engines while scanning content and evaluating relationships. External content will also be included into search results in a seamless manner and will enrich search results. Search engines will continuously evaluate user behaviour and the working context of users in order to provide search results which go beyond today's relevancy.

Last but not least, I'm expecting a massive change in the way we create search queries. Today's search queries will be replaced by artificial intelligence too. Like how we talk to Cortana (or Siri) today, we will search for data by using natural sentences or even voice commands.

I think that there will be enthralling changes and improvements in the way we are using search results in the near future. These changes will also affect the way we are using corporate intranets and the way we are collaborating. Since SharePoint was released in 2000 as 'SharePoint Portal Server 2001' it has dramatically changed the way employees are collaborating in an enterprise environment. In fact, SharePoint is responsible for the massive change in the way employees are working today. Although this process of change isn't finished yet (and probably never will be), it has slowed it's pace to allow for another massive change to take place and this change has already begun ...