Squirrel AI Learning by Yixue Group Invited to the NCME Summit for the Discussion on Evaluation of Online Learning using AI

TORONTO, June 20, 2019 /PRNewswire/ -- The 2019 National Council on Measurement in Education (NCME) Summit was held in Toronto, a picturesque city, lately. About 14,000 education researchers and experts from all over the world gathered together at the Metro Toronto Convention Centre and Fairmont Hotel, where they spent 5 days discussing the important results achieved in education evaluation system research and education evaluation practice in the past one year.

At the noon of April 8th, an important exchange meeting of NCME titled "New Directions in Scoring and Classification Accuracy" was kicked off as scheduled at the hotel. Squirrel AI Learning by Yixue Group, a representative of AI evaluation and quantization in the Chinese education industry, was also invited to this meeting. Yuning Xu, an education researcher at the SRI International and a core member of the project team partnered with Squirrel AI Learning, gave a research report titled "Integrating Expert Review and Diagnostic Classification Models for Online Assessments". He provided detailed information on the core process how Squirrel AI Learning built a cognitive diagnosis and measurement system using the Q-matrix based on DINA model, and showed the concrete results of simulation.

The National Council on Measurement in Education (NCME) is a national education evaluation industry association of the USA. It regularly publishes a guidance document and quality standard for the U.S. education industry. The NCME has a significant influence on the evaluation standards of the U.S. education industry, and its Educational and Psychological Measurement Criteria clearly defines various examination standards, including reliability, validity and fairness. The U.S. Educational Testing Service (ETS), an examination body well-known to Chinese, has designed globally famous linguistic and scholastic assessment tests, such as TOEFL, GRE, GMAT, TOEIC and SAT, based on the education evaluation standards set by the NCME.

The theme of 2019 NCME Summit was "Communicating with the Public about Educational Measurement". Under this theme, "the public" is defined as a wide range of education audience, including common people, parents, educators, education policy makers, the media focused on education and students in different age groups. The communication with students drew particularly high attention from the summit.

Today, as the Internet is subverting and revolutionizing the education industry, individualized education has become highly needed. However, for individualized education, it is essential to accurately measure students' knowledge level and diagnose the problems in the knowledge structure. The accuracy of the traditional Latent Trait Model and Factor Analysis Model can no longer meet the requirements of Internet education. They have been replaced with new education evaluation models such as the cognitive diagnosis model.

Squirrel AI Learning has comprehensively applied its new evaluation model to the domestic K12 education, involving many core subjects, such as math, physics, chemistry, English and Chinese. By designing a perfect knowledge map and precise knowledge points, and accurately drawing the outline of the correlations and interdependence among the knowledge points, Squirrel AI Learning has created an AI evaluation methodology for the entire knowledge system. This scientific methodology has laid a solid foundation for the circular improvement process of "evaluation --> learning --> reevaluation".

In this evaluation method developed by Squirrel AI Learning, the researchers have divided the knowledge map and test content into two core objects: "knowledge point" and "item". The knowledge point describes a basic knowledge unit in the knowledge map. The "item" describes a piece of basic content evaluated, such as a test question. A standardized test contains not only many knowledge points, but also a lot of items, and they are cross-correlated to each other. For instance, a knowledge point is likely to be used in many test questions, or there may be a lot of knowledge points in one test question.

Squirrel AI Learning's evaluation method ultimately aims to reveal the influence of different combinations of knowledge points and items on the accuracy of evaluation, e.g., the influence of simple items (containing only a few knowledge points) and complex items (containing many knowledge points) on the accuracy of evaluation.

At the summit, the researcher Yuning Xu talked about how the project team did evaluation experiments on linear equations, radical signs and parallelograms based on the test content designed by education experts for the corresponding knowledge points, in hopes of building a classified prediction model between knowledge points and items.

After modeling with the cognitive diagnosis DINA model and the dual Q learning matrix, the researchers extracted four "cognitive models" (skill) that have something in common, and built a function model for items and cognitive models, and finally the accuracy of marginal cognitive model classification was completed.

Based on this model, the AI system can build a mathematical model for knowledge points, evaluation items and cognitive models in various fields of knowledge, to organically combine them together to help teaching and evaluation personnel comprehensively understand the correlations between knowledge points and evaluation content and fully describe how proficient students are in mastering knowledge and using skills. This creates favorable conditions for improving the efficiency of "evaluation --> learning --> reevaluation".

Squirrel AI Learning founded a joint AI laboratory with the SRI International two years ago, and the joint AI laboratory is committed to carrying out the work based on the SRI International's unique advantages in AI and educational technology. Currently, the joint lab is making a splash in three key collaborative areas: 1) core adaptive education model and technology; 2) natural language processing and semantic analysis, aimed at realizing the virtual personalized assistance (VPA) function and using dialogue-based interface to diagnose students' error causes and receive feedback on AI from instructors and students. 3) The multimodal integrated behavioral analysis (MIBA) research enables Squirrel AI Learning to understand students' emotional and psychological states and better predict their behavior in a adaptive learning environment, and provide signals for human instructors to take intervention, remedy and support measures, or remind, recommend, and relax students through the system.

At present, Squirrel AI Learning has applied this evaluation method to secondary education evaluation in China, and received positive feedback. Squirrel AI Learning plans to further expand the application range of AI and provide a powerful driving force for the Chinese education industry in an era of AI.

SOURCE Squirrel AI Learning