Using AI and NLP to eliminate organisational silos

Emmanuel Helbert

February 15, 2024

AI and NLP technologies are behind conversation fingerprinting, a new concept that can optimise distribution of information

One of the greatest challenges within organisations – especially large and multinational entities – is how to circulate information.  Yet, being able to distribute the right information to the right person is central to building competitiveness and can even prove a game-changer.

So, we started with a small team to investigate whether artificial intelligence (AI) and natural language processing (NLP) technologies could be used to help determine and visualise the emergence and propagation of information. Essentially, we wanted to know if we could identify within an organisation where employees were discussing the same topic but had no overlap with each other – a clear indicator of the siloing that can impede information sharing in the workplace.

Respecting privacy laws and maintaining corporate confidentiality were paramount. But this also meant that we needed to devise a method that allowed us to identify topics of conversation without having any knowledge of the content of the conversation.

And that’s how we created this concept of conversation fingerprinting.

Ways to bridge the silos

There are plenty of modes of communications available today, but we believe this new approach of conversation fingerprinting complements those existing tools.

Tools like communities – such as networks of experts – and internal events that encourage the development of cross-functional networks. There are also internal social networks and discussion groups which can simplify the exchange of information.

These tools work because they generate closeness and trust between members and make it possible to tackle cross-functional subjects with a broader range of skills than a single project team might provide. But they are limited by the way they are constituted, which members are included, and which topics are discussed – all of which may remain compartmentalised within the same silo or geographical area.

It is also not easy to graphically represent these collaborative tools. And this aspect of visualising information flow – knowing where information is generated, how it is/is not propagated and who consumes it – is key.

Indeed, visualising is at the heart of graph-based theories such as Social Network Analysis (SNA) and Organisational Network Analysis (ONA). These theories can be used to model and analyse interactions within an organisation, with each employee representing a node in the graph and edges between nodes representing interactions. They are behind consumer and business applications such as web page ranking and purchasing recommendations and can be used to detect communications roadblocks or adjust teams to optimise efficiency.

The upside to these theories is that they can show who is connected to whom and how often, and even estimate the amount of information exchanged. By modelling the communications graph on the organisation graph, it is possible to identify ways to improve productivity by changing the structure of the organisation to match the structure of communications flows.

Still, these graph-based theories remain limited by the fact that they only consider statistical data relating to communications traffic and nodes. That does not account for the semantic content of conversations, on the assumption that the information exchanged is naturally linked to the tasks and missions of each team.

Conversation fingerprinting

By introducing AI and NLP technologies in our study, we learned that we could change this and uncover new perspectives.

So, what is this notion of a conversation fingerprint? Essentially, it’s a unique identifier for information exchanged between two employees, with each piece of information encoded in the form of a fingerprint. In applying the concept, our aim was to assess the similarities between the fingerprints. 

By associating these processes with conversation graphs, it became possible to visualise the branches through which information was propagated, at what speed and with what intensity. 

Better still, it became possible to see similar conversations emerging in parts of the graphs that were not connected to each other. 

The latter is an important aspect of community management and promotion. It enables different entities working on similar subjects to be brought together, thereby encouraging cross-disciplinary interaction and optimising talents and energies.

Where to from here

This research has been very promising. Not only has it validated the relevance and performance of the solution's architecture, but it has also opened up new areas of use – such as intelligent search – making it possible to find specific information buried in a stream of conversation. 

And there’s an environmental benefit to this more frugal AI approach when compared to large language models (LLMs). While Generative AI has made headlines of late, this technology is proving to be a massive energy user. 

Combining AI and NLP technologies offers a less expensive and more controlled alternative, while respecting the confidentiality of corporate and employee data. And that makes conversation fingerprinting even more compelling as a tool to forge high-performing communities that drive competitiveness.