PhD CSCI Seminar: Bazyli Debowski

Improving Natural Language Understanding with Graph Neural Network Based Language Modelling

Bazyli Debowski Advisor: Dr. Fei Song (Contact Fei Song to join the Zoom call at fsong@uoguelph.ca)
Co-Advisor: Dr. Petros Spachos [School of Engineering]
Advisory Committee: Dr. Graham Taylor [School of Engineering]
Advisory Committee: Dr. Luiza Antonie

ABSTRACT:

In this Seminar we present the case for: i) how a Language Model (LM) could be built using a Spatio-Temporal Graph Neural Network (GNN) architecture, and ii) how doing so would form the basis for better language understanding. Specifically, we explore how it is that current state-of-the-art Language Modelling techniques fail to understand the meaning of what is written and we show how incorporating an Ontological Knowledge base could improve understanding. We further demonstrate limitiations to previous approaches of incorporating Ontological Knowledge into Natural Language Processing (NLP) systems, and show how switching to graph based representations and a GNN based LM could overcome these limitiations. Finally, we explore various elements of popular GNN architectures and illustrate how they could be incorporated into a modern Transformer architecture based LM to build a Spatio-Temporal GNN based LM.