The Innovation

This patent application concerns automated assessment of examination scripts, particularly English for Speakers of Other Languages (ESOL) examinations. The innovation includes a system comprising a feature analysis module named RASP (robust accurate statistical parsing) which numerically quantifies linguistic features of text to form a feature vector. This vector is used to grade scripts based on discriminative models like SVM or large margin perceptrons, including a novel variant named Timed Aggregate Perceptron (TAP)​​.

Technical Contribution

The application describes the TAP’s training procedure, which is unique in that a timing parameter reduces the update rate based on the progression of the process, the magnitude of the increase in empirical loss, and the balance of the training distributions. This approach provides an approximate solution that prevents overfitting through early stopping. The system can output binary results for pass/fail grading systems and employs preference ranking for relative grading of scripts, focusing on reducing errors in relative grading over absolute grading​​.

The main request in the patent application was amended to replace the term “combining” with “summing” in independent claims 1 and 10, addressing the lack of support objection raised by the Board. The Board’s evaluation focused on whether the claim as a whole defines “mathematical or linguistic steps” used for “grading text scripts” and if these steps are causally linked to a technical effect​​.

The Appellant argued that the differences from document D2 included more features that distinguished their invention. They contended that D2 did not provide for the extraction of linguistic vectors, use of TAP for script grading, nor ranking between different scripts. The Appellant asserted that the problem addressed is not grading scripts per se but providing a computer system that can automatically grade text scripts in a way that correlates well with the grades given by human markers. They suggested that the distinguishing features reflect further technical considerations, proposing that the invention provides a technical contribution in the field of “educational technology”​​.

The Board, however, noted that the method’s steps, including the extraction of linguistic vectors and the training and use of the perceptron, do not provide a technical contribution within the computer itself. It was viewed that the training procedure might constitute a technical contribution but, as a mathematical method, falls within an excluded field and thus is not patentable. The Board concluded that it could not identify any technical problem solved either at the input, in generating the output grade, or by the execution of the claimed process​​.

For the technical effect via “implied technical use,” the Board considered whether the purpose of providing an automated tool for script grading implies a technical problem and whether it is actually solved. The Board expressed doubts about whether this problem is defined well enough to assess if it has been solved in the sense that the system can replace human markers and provide “correct” grades. The Board was satisfied that the system’s outputs correlate well with the training data from human markers but stated that the mere provision of any automated tool does not suffice to solve a technical problem, as per G 1/19. Ultimately, the decisive factor is whether the invention provides a solution to a technical problem, contributing to a field of technology. The field of “educational technology,” as defined by the Appellant, was considered too broad to be deemed entirely technical​​.

Key Findings

• Feature analysis module (RASP) – non-technical
• Numerical quantification of linguistic features – non-technical
• Timed Aggregate Perceptron (TAP) – non-technical
• Preference ranking system in TAP – non-technical
• Extraction of linguistic vectors – non-technical
• Training procedure of perceptron – non-technical (mathematical method)
• Use of perceptron to grade scripts – non-technical

Read the full decision here: Automated Assessment of Examination Scripts – T 0761/20