Translated by: Wang Min
Date: 2018-07-27
Recently, the 5th Chinese Grammatical Error Diagnosis (CGED) competition was successfully held in Melbourne, Australia, together with the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018). This year’s CGED contest attracted 13 well-known research institutions, including the Chinese Academy of Social Sciences, Alibaba, Peking University, and HIT’s iFLYTEK Language Cognitive Computing Lab (HFL). Through four tasks, HFL was ranked first in terms of total score with three first-place finishes and one second-place finish, thereby securing a resounding win in the competition.
1. Overview of the CGED
The main task of the CGED contest is to identify and correct the grammatical and semantic errors in Chinese sentences written by foreigners through the technique of artificial intelligence. There are four types of errors in those shared tasks, including redundant words, missing words, word selection and word ordering errors. The performance of each team is evaluated from four aspects: error recognition (recognizing whether the sentences are wrong or not), error classification (identifying specific types of errors), error positioning (detecting the error type and its position), and error correction (providing suggestions for the correction of missing words and wrong words). Among them, the fourth item, error correction, is a new addition this year and is more difficult and challenging because the participating teams should not only detect the errors but also suggest improvements. In the task of error correction, HFL received first place, scoring 0.1723 and 0.2527 respectively in “Correction F1” and “Top-3 Correction F1”.
According to Dr. Fu Ruiji, vice dean of Beijing iFLYTEK Research and assistant director of HFL, the error positions are first located through the system, which adopts an end-to-end neural network model combined with pointwise mutual information of vocabularies, and then correct words are suggested on the basis of context and error information.
2. Introduction of the AI system
This award-winning system is built on the improved neural network sequence labeling model BiLSTM-CRF, which is comprised of three parts: the entire model of the bottom layer, the internal fusion of single models, and the external fusion of models. These three parts are detailed as follows:
Firstly, term vectors, statistics and prior grammatical knowledge are integrated, such as the statistical features of Chinese language usage. For instance, in these two Chinese phrases “静静的顿河” and “静静地等待”, the structural auxiliary words “的” and “地” are selected based on the context of the surrounding parts of speech.
Secondly, with the adoption of fusion technology within models, the outputs of multiple BiLSTM single models are weighted and fused, and then the final results are calculated by CRF.
Thirdly, the adoption of external fusion technology, such as model selection and model ordering, exerts the advantages of different models.
On the whole, thanks to the above-mentioned three technologies, HFL’s system gave a standout performance in the contest.
3. Broad application prospects in Chinese Grammatical Error Correction
In the field of grammatical error correction, Chinese is much more difficult than English. There are two main reasons for this difference. For one thing, English is more grammatically standard, with subjects, verbs and objects in each sentence. For another, English corpora tend to be larger; for example, the corpus established by Cambridge University contains tens of millions of sentences. In contrast, Chinese is flexible in grammar, which makes error detection more difficult. Besides, the corpus used in the evaluation contains only 30,000 sentences at present, which needs to be expanded. Therefore, the grammatical correction of English is quite advanced in application nowadays, while the grammatical correction of Chinese is still in its exploratory stage. As seen from the current evaluation results, the technical indicators are still unsatisfactory and are far from the demand of practical application. However, the error detection and correction technologies are expected to have broad application prospects in the future in the areas of both foreign language learning and manuscript proofreading.
Furthermore, with the technology of automated essay scoring developed by HFL through the automated learning of human evaluators’ scoring standards, essays from written tests are able to be assessed by computer programs, concerning various aspects such as handwriting, lexical variation, sentence fluency, style of writing, essay structure, essay topics, etc. Currently, this technology has been used as an auxiliary tool of manual evaluation in many national examinations, covering millions of examinees.
4.Continuous Achievements of HFL
HFL is one of the core R&D teams of iFLYTEK proposed under the project of “iFLYTEK Super Brain”, jointly founded by iFLYTEK AI Research and the Research Center for Social Computing and Information Retrieval of Harbin Institute of Technology ( HIT-SCIR) in 2014. According to the construction plan, the two sides of HFL will carry out long-term and in-depth cooperation in the field of language cognitive computing, especially where related to cutting-edge research on reading comprehension, automated essay scoring, human-like question answering, man-machine conversation, post-speech recognition processing, public opinion calculation, etc. As for the research focus, breakthroughs are expected to be made in key technologies of cognitive intelligence, such as deep semantic understanding, logical decision-making, and autonomous learning, so as to provide support for iFLYTEK AI to technologically leapfrog from “listening and speaking” to “understanding and thinking”. In addition, these technologies are hoped to be applied on a larger scale in the fields of education, finance, human-computer interaction and so on. At present, HFL mainly focuses its research on Chinese scoring, reading comprehension, intelligent judicature, and social computing.
HFL has won first place three times (in July 2017, October 2017, and January 2018) at the Stanford Question Answering Dataset (SQuAD) Challenge, a top-level machine reading comprehension competition. In February 2018, HFL was again awarded first place in solving the reading comprehension task at SemEval 2018. Together with the championship awarded at this year’s CGED competition, all of these achievements have revealed HFL’s position as a leader in the international research of core technologies of natural language processing. All of this progress will continue to contribute to the shift of iFLYTEK AI from “listening and speaking” to “understanding and thinking”.
