Posts by Collection

AutoQG: Web Service Hosting Question Generation Model

T-NER: An All-Round Python Library for Transformer-based Named Entity Recognition

WikiART Face: Face Image Dataset from Portraits

Metric Perspective of Stochastic Optimizers

Published: July 19, 2017

This talk gives a high-level overview for the major classes of stochastic optimizers from the perspective of the model parameter space. The talk covers following algorithms:

• Quasi-Newton Methods: Finite-Difference (SGD-QN, AdaDelta, VSGD), Extended Gauss-Newton (KSD, SMD, HF), LBFGS (LBFGS, RES)
• Natural Gradient Type: Natural Gradient, TONGA
• Root Mean Square Type: AdaGrad, RMSProp, Adam

Relational Knowledge and Language Models

Published: December 01, 2020

This is a talk at an internal research group in CardiffNLP, which focuses on a few recent studies in the area of commonsense & relational knowledge probing of pretrained language models. Following papers are covered in this talk:

• Petroni, et al. “Language models as knowledge bases?” 2019
• Jiang, et al. “How can we know what language models know?” 2019
• Bouraoui, et al. “Inducing relational knowledge from BERT.” 2019

Toward a Better Understanding of Relational Knowledge in Language Models

Published: January 26, 2022

This is an invited talk at NLP Colloquium about relational knowledge representation of language models including the following papers.

• Ushio, et al. “Distilling Relation Embeddings from Pretrained Language Models” 2021
• Ushio, et al. “BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?” 2021

Toward a Better Understanding of Relational Knowledge in Language Models

Published: October 04, 2022

This is an invited talk at UCL NLP Meetup about relational knowledge representation of language models including the following papers.

• Ushio, et al. “Distilling Relation Embeddings from Pretrained Language Models” 2021
• Ushio, et al. “BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?” 2021

TweetNLP: Text Analytic Package for Social Media

Published: November 16, 2023

This is an invited talk at PyData Bristol about tweetnlp, a python library for social media analysis.

Representing Relational Knowledge with Language Models

Published: December 12, 2023

This is the presentation for my PhD viva (thesis is available here)

A Short Introduction of Modern Speech Foundation Models

Published: July 02, 2024

This is an invited talk at 3rd Cardiff NLP Summer Workshop about a short survey of modern speech foundation models.