diff --git a/Koli_2017/Koli_2017_Stanger.tex b/Koli_2017/Koli_2017_Stanger.tex index e084145..1fdfd5d 100644 --- a/Koli_2017/Koli_2017_Stanger.tex +++ b/Koli_2017/Koli_2017_Stanger.tex @@ -130,7 +130,7 @@ Given the relatively static nature of an SQL schema, we feel this verification style of approach is the most appropriate way of constructing an automated grading system for SQL DDL. As an aside, the concept of verifying a schema against its specification sounds like it should be a useful application of formal methods \cite{Spivey.J-1989a-An-introduction}, but work with formal methods and databases has historically focused either on \emph{generating} a valid schema from a specification (e.g., \cite{Vatanawood.W-2004a-Formal,Lukovic.I-2003a-Proceedings,Choppella.V-2006a-Constructing}), or on verifying schema transformation and evolution \cite{Bench-Capon.T-1998a-Report}. A formal methods approach could certainly be used to generate the model answer or rubric for grading an assessment, but this is effectively the inverse of what is needed to grade a student-written schema. -Previous systems focused almost exclusively on SQL \emph{queries} using the \texttt{SELECT} statement (i.e., DML) rather than SQL \emph{schema definitions} (DDL). This focus is unsurprising given that \texttt{SELECT} is probably the most frequently used of any SQL statement, and also because it is relatively complex compared to most other SQL statements. Only a few of systems we reviewed even mentioned schema definition. RDBI \cite{Dietrich.S-1993a-An-educational} supported a DDL, but only its own non-SQL language, while esql \cite{Kearns.R-1997a-A-teaching} simply passed anything that was not a \texttt{SELECT} statement through to the DBMS. \citeauthor{Gong.A-2015a-CS-121-Automation}'s ``CS 121 Automation Tool'' \cite{Gong.A-2015a-CS-121-Automation} focuses primarily on SQL DML statements, but appears to be extensible and could potentially be modified to support SQL DDL statements. +Previous systems focused almost exclusively on SQL \emph{queries} using the \texttt{SELECT} statement (i.e., DML) rather than SQL \emph{schema definitions} (DDL). This focus is unsurprising given that \texttt{SELECT} is probably the most frequently used of any SQL statement, and also because it is relatively complex compared to most other SQL statements. Only a few of the systems we reviewed even mentioned schema definition. RDBI \cite{Dietrich.S-1993a-An-educational} supported a DDL, but only its own non-SQL language, while esql \cite{Kearns.R-1997a-A-teaching} simply passed anything that was not a \texttt{SELECT} statement through to the DBMS. \citeauthor{Gong.A-2015a-CS-121-Automation}'s ``CS 121 Automation Tool'' \cite{Gong.A-2015a-CS-121-Automation} focuses primarily on SQL DML statements, but appears to be extensible and could potentially be modified to support SQL DDL statements. Many previous systems have implemented some form of automated or semi-automated grading, e.g., SQLator \cite{Sadiq.S-2004a-SQLator}, AssesSQL \cite{Prior.J-2004a-Backwash}, ActiveSQL \cite{Russell.G-2004a-Improving,Russell.G-2005a-Online}, SQLify \cite{Dekeyser.S-2007a-Computer}, aSQLg \cite{Kleiner.C-2013a-Automated}, \citeauthor{Gong.A-2015a-CS-121-Automation}'s ``CS 121 Automation Tool'' \cite{Gong.A-2015a-CS-121-Automation}, and XDa-TA\footnote{\url{http://www.cse.iitb.ac.in/infolab/xdata/}} \cite{Bhangdiya.A-2015a-XDa-TA,Chandra.B-2015a-Data,Chandra.B-2016a-Partial}. Some (e.g., SQLator, AssesSQL) provided only correct/incorrect responses, while others (e.g, ActiveSQL, SQLify, aSQLg, XDa-TA) had more sophisticated schemes for assigning partial credit, such as ActiveSQL's accuracy score, SQLify's eight-level scale, and XDa-TA's ability to generate data sets designed to catch common errors. In either case, these systems were often able to automatically mark a significant fraction (e.g., about a third for SQLator \cite{Sadiq.S-2004a-SQLator}) of submitted queries as correct without human intervention, thus reducing marking load for teachers.