Improving mental health using machine learning to assist humans in the moderation of forum posts

This work investigates the potential for the application of machine learning and natural language processing technology in an online application designed to help teenagers talk about their mental health issues. Specifically, we investigate whether automatic classification methods can be applied with sufficient accuracy to assist humans in the moderation of posts and replies to an online forum. Using real data from an existing application, we outline the specific problems of lack of data, class imbalance and multiple rejection reasons. We investigate a number of machine learning architectures including a state-of-the-art transfer learning architecture, BERT, which has performed well elsewhere despite limited training data, due to its use of pre-training on a very large general corpus. Evaluating on real data, we demonstrate that further large performance gains can be made through the use of automatic data augmentation techniques (synonym replacement, synonym insertion, random swap and random deletion). Using a combination of data augmentation and transfer learning, performance of the automatic classification rivals human performance at the task, thus demonstrating the feasibility of deploying these techniques in a live system.