“Corpus Construction for People with Mental Disorders and Language Research on Brain Science and Artificial Intelligence”Project
On the morning of January 5, the School of Foreign Languages (SFL) held the project proposal meeting of “Corpus Construction for People with Mental Disorders and Language Research on Brain Science and Artificial Intelligence”, a Major Project of the National Social Science Fund. By conducting research on the brain processing mechanism of speech language on people with mental disorders, the project puts into practice top-notch research findings in the field of linguistic neuro-cognition to construct a corpus for people with mental disorders and to further explore the application of artificial intelligence in the auxiliary diagnosis of mental disorders. The meeting was hosted by Tao Qing, Deputy Dean of SFL, and was attended by Zhang Weigang, Minister of the United Front Work Department of SJTU, Hu Kaibao, Dean of SFL, Gao Yankun, Deputy Director of the Department of Liberal Arts of SJTU, Professor Ding Hongwei, Chief Expert of the project and Executive Deputy Director of the Language, Intelligence and Neuroscience Study Center of SFL, and specialists of the review panel .
First of all, Zhang Weigang delivered a speech in which he highlighted the great significance of Professor Ding Hongwei’s project, as its results should provide useful reference for mental health education and psychological counseling, as well as a scientific basis for government decision-making. He also hoped that Professor Ding and her research group could continue to achieve great accomplishments in theoretical innovation, whilst providing constructive suggestions to the government, contributing to the progress of liberal arts and the development of society. At the same time, he said that the school would strongly support Professor Ding and her team to ensure the development of their project.
Next, Hu Kaibao started the detailed presentation of the project. He explained that the project captured some of today’s matters of general interest: brain science, artificial intelligence and the study of mental disorders. At the same time, the college continued to develop interdisciplinary language research and the research team covered different departments and research centers inside and outside the school. These departments and centers included the School of Foreign Languages, the Department of Computer Science, the Research Center of Language Pathology, Psycholinguistics and Neurobehavioral Development, the Psychology College and Shanghai Mental Health Center. In the end of his speech, Hu Kaibao thanked the present guests and experts for their valuable suggestions on this project.
Following Hu Kaibao’s speech, Gao Yankun went on to point out that Shanghai Jiao Tong University currently ranked top in engineering subjects, but that in recent times art subjects had also made remarkable progress. He expressed his wish for interdisciplinary research to continue to advance in order to achieve a comprehensive development of the university.
Professor Ding Hongwei, Chief Expert of the project, Executive Deputy Director of the Language, Intelligence and Neuroscience Study Center of School of Foreign Languages of SJTU, introduced the three main tasks of the project: corpus construction for people with mental disorders, linguistic study of mental disorders with the help of artificial intelligence and that on the basis of brain science. The research included behavioral neuro-linguistic experiments (perception, eye movement, near-infrared, electroencephalograms, brain imaging, etc.) and the comparison of experimental data of people with mental disorders with that of the general population. In order to build a corpus including speech language, body movements and facial expressions of people with mental disorders, the team also investigated the brain’s language mechanism. This research sets a basis for developing criteria for disease diagnosis and it is expected to contribute to the early detection and treatment of autism, Alzheimer's disease, and mild mental disorders. Furthermore, it will provide some specific indicators for mental disease diagnosis on the basis of speech language research, in relation to brain mechanism and artificial intelligence.
Afterwards, Wu Mengyue, assistant researcher at the Key Laboratory of Intelligent Interaction and Cognitive Engineering of SJTU and SJTU-Al Speech Joint Laboratory, exemplified how artificial intelligence was gradually being applied to diagnostic research. For instance, she presented the DREAM multi-modal fusion behavior perception experiment, aimed at extracting single modal features for data modeling and fusing multi-modal information for classification. This research, which included the analysis of the language features of speech, facial and postural features and physiological signs, was helping distinguish people with mental disorders from the general population and sets a classification of mental disorders in different categories.
Professor Li Chunbo, Deputy Dean of Affiliated Mental Health Center of the Medical College of SJTU, introduced the goal of the newly-established ethics committee of the Mental Health Center: building a systematic, standardized and specialized corpus on schizophrenia, affective disorders, Alzheimer's disease and autism by studying different test models of linguistics and collecting language materials from patients with different mental diseases. Their research was based on the analysis and processing of recorded interviews and talks by the group’s linguists. It is expected that the groups’ findings will be applied to clinical treatment.
The director of Brain Science and Cognitive Science center of the School of Psychology and Cognition at Peking University, Zhou Xiaolin, highlighted the great theoretical significance and practical value of the research, as well as its ambitious goal. He suggested that long-term investigation before and after the onset of the same group should be considered in the selection of samples so as to show more significant differences in data analysis. At the same time, in the selection of sample size, in order to ensure the high quality of data transcription and the large sample data, pre-and post-processing was adopted. This gradually enriched the corpus construction and facilitated transit from assistant screening to assistant diagnosis.
Yang Yufang, former director of the Institute of Psychology of the Chinese Academy of Sciences, affirmed that research tasks were clear and research methods were appropriate. In addition, he congratulated the team on their preparation and capacity to resolve issues and problems without difficulty. Finally, he insisted on strengthening the links between the different research teams of this project, so as to be able to continue studying more kinds of mental disorders, whilst focusing on the changes of behavioral, psychological, and cognitive characteristics of people with mental disorders, such as the pre-sessional memory missing of Alzheimer's patients.
Gu YueDguo, director of the Applied Linguistics Research Lab of the Institute of Linguistics of the Chinese Academy of Social Sciences, suggested that data for the analysis of mental disorders should be carefully selected, clarifying differences between terms, such as speech error, speech apraxia, and speech disorder, and making the differences between temporary and long-term speech barriers clear. Furthermore, he added that the clear distinction between technical terms, such as dysthymic disorder and affective disorder, helped provide rule support for the construction of the corpus.
According to Yang Yiming, dean of the School of Linguistic Sciences and Arts of Jiangsu Normal University, this project made great sense in terms of topic selection and research methods. He said that each sub-topic should be investigated under the major project, and each research team should strengthen cooperation in processes like collecting speech language and facial expression data. He added that the corpus construction process was highly innovative which demonstrated the advantages of this cross-disciplinary research.
Wei Naixing, dean of the School of Foreign Languages at Beihang University, pointed out that the focus of this project was the needs of people with mental disorders. The research covered a wide range of subjects and had clear aims in terms of theoretical innovation, time application and decision-making. In order to solve practical problems and promote the further development of language, psychology and cognitive neuroscience, he encouraged sub-teams to continue to strengthen their cooperation, perseveres and make the most of interdisciplinary work.
This is the fourth major project launched by SFL and funded by the National Social Science Fund in the recent years. The school has taken this project as an opportunity to combine foreign language teaching, artificial intelligence and brain science to promote interdisciplinary research and build first-class foreign language subjects.