GitBucket
4.21.2
Toggle navigation
Snippets
Sign in
Files
Branches
12
Releases
Issues
Pull requests
Labels
Priorities
Milestones
Wiki
Forks
nigel.stanger
/
Publications
Browse code
• Reordered older systems into date order.
DP_2017
1 parent
540b790
commit
545c8b1ffad586c15567f1f042908ed741427c3e
Nigel Stanger
authored
on 24 Jul 2017
Patch
Showing
1 changed file
Koli_2017/notes.txt
Ignore Space
Show notes
View
Koli_2017/notes.txt
Numerous prior systems for automatic grading of student queries, but these are all focused on the DML side of things. • RDBI (Dietrich 1993) supports relational algebra, domain and tuple relational calculus, and SQL (SELECT only). Uses its own internal DBMS. • esql (Kearns et al., 1997) supports CREATE TABLE only as a pass-through to set up or modify a schema. It uses its own internal DBMS. • SQL-Tutor (Mitrovic, 1998) is an intelligent tutoring system that only only supports SELECT. • SQLator (Sadiq et al., 2004) only supports SELECT. • AsseSQL (Prior & Lister, 2004) only supports SELECT. • ActiveSQL (Cumming & Russell, 2005; Russell & Cumming, 2005) only supports SELECT. • SQLify (Dekeyser et al., 2007) only supports SELECT. • aSQLg (Kleiner et al., 2013) only supports SELECT. Works with any back end DBMS. Uses the DBMS as a syntax checker. • (Kenny & Pahl, 2005) queries only. • XData (Bhangdiya et al., 2015, Chandra et al., 2015; Chandra et al., 2016) only supports SELECT. • “CS 121 Automation Tool” (anjoola, cs12x-automate, GitHub, last updated 2015) Automated marking system for SQL that appears be customisable for all kinds of statements? However the code to test CREATE TABLE “Simply executes the CREATE TABLE statements to make sure they have the correct syntax”. It doesn't check against any form of specification. On the pure testing side of things: • SVTe (Farré et al., 2008) tests the “correctness” of a schema, but focuses mainly on consistency of constraints. • Ambler (n.d., http://www.agiledata.org/essays/databaseTesting.html) talks purely about testing the database functionality. There appears to be very little work on automated grading of students’ schema definitions. No-one has used unit testing as a framework for doing so. What use of unit testing there is, is more focused on testing database /applications/ and effective generation of test cases: • (Binnig et al., 2013) Generating test databases (fixtures) for OLTP applications. • (Chays & Shahid, 2008) Generating test inputs for database applications based on analysing the queries used. • (Marcozzi et al, 2012) Generating test inputs for database applications using a formal language (ImperDB). No-one appears to have tried to automatically test whether an extant schema is consistent with its specification. This is almost more of a formal methods thing? These are more focused on /generating/ a conforming schema from the specification (e.g., Vatanawood & Rivepiboon, 2004; Luković et al., 2003), rather than checking that an already existing schema conforms. Also more focused on schema /transformations/ and evolution (e.g., Bench-Capon et al., 1998). ------------------------------------------------------------ NOTE: Checking structure of table Product. TEST: [[ Product ]] + OK +++ PASSED: Table Product exists. TEST: [[ Product.Product_code ]] + OK ... +++ PASSED: Table Product contains all the expected columns. TEST: [[ Product.Product_code: data type is NUMBER | INTEGER ]] + OK ... +++ PASSED: All columns of table Product have data types compatible with the specification. TEST: [[ Product.Product_code precision and scale = 8 (with scale 0) ]] + OK ... +++ PASSED: All columns of table Product have lengths compatible with the specification. TEST: [[ Product.Product_code nullability should be N ]] + OK ... +++ PASSED: All columns of table Product have the expected nullability. TEST: [[ Product PK ]] + OK +++ PASSED: Primary key of table Product exists. TEST: [[ Product PK: Product_code ]] + OK +++ PASSED: Primary key of table Product includes (only) the expected columns. TEST: [[ Product check constraint PRODUCT_STOCK_INVALID ]] + OK ... +++ PASSED: All constraints of table Product that should be are explicitly named. NOTE: Testing constraints of table Product. TEST: [[ Product.Stock_count accepts “0” ]] + OK TEST: [[ Product.Stock_count accepts “99999” ]] + OK TEST: [[ Product.Restock_level accepts “0” ]] - FAILED! Column Product.Restock_level won’t accept legal value 0 [-0.5]. Failed asserting that false is true. TEST: [[ Product.Restock_level accepts “99999” ]] + OK TEST: [[ Product.Minimum_level accepts “0” ]] - FAILED! Column Product.Minimum_level won’t accept legal value 0 [-0.5]. Failed asserting that false is true. TEST: [[ Product.Minimum_level accepts “653” ]] + OK TEST: [[ Product.List_price accepts “0” ]] + OK TEST: [[ Product.List_price accepts “99999.99” ]] + OK --- FAILED: 2 of 8 legal values tested were rejected by a CHECK constraint.
Numerous prior systems for automatic grading of student queries, but these are all focused on the DML side of things. • esql (Kearns et al., 2008) supports CREATE TABLE only as a pass-through to set up or modify a schema. It uses its own internal DBMS. • aSQLg (Kleiner et al., 2013) only supports SELECT. Works with any back end DBMS. Uses the DBMS as a syntax checker. • (Kenny & Pahl, 2005) queries only. • ActiveSQL (Cumming & Russell, 2005; Russell & Cumming, 2005) only supports SELECT. • SQLify (Dekeyser et al., 2007) only supports SELECT. • XData (Bhangdiya et al., 2015, Chandra et al., 2015; Chandra et al., 2016) only supports SELECT. • SQL-Tutor (Mitrovic, 1998) is an intelligent tutoring system that only only supports SELECT. • SQLator (Sadiq et al., 2004) only supports SELECT. • AsseSQL (Prior & Lister, 2004) only supports SELECT. • RDBI (Dietrich et al., 1997) supports relational algebra, domain and tuple relational calculus, and SQL (SELECT only). Uses its own internal DBMS. • “CS 121 Automation Tool” (anjoola, cs12x-automate, GitHub, last updated 2015) Automated marking system for SQL that appears be customisable for all kinds of statements? However the code to test CREATE TABLE “Simply executes the CREATE TABLE statements to make sure they have the correct syntax”. It doesn't check against any form of specification. On the pure testing side of things: • SVTe (Farré et al., 2008) tests the “correctness” of a schema, but focuses mainly on consistency of constraints. • Ambler (n.d., http://www.agiledata.org/essays/databaseTesting.html) talks purely about testing the database functionality. There appears to be very little work on automated grading of students’ schema definitions. No-one has used unit testing as a framework for doing so. What use of unit testing there is, is more focused on testing database /applications/ and effective generation of test cases: • (Binnig et al., 2013) Generating test databases (fixtures) for OLTP applications. • (Chays & Shahid, 2008) Generating test inputs for database applications based on analysing the queries used. • (Marcozzi et al, 2012) Generating test inputs for database applications using a formal language (ImperDB). No-one appears to have tried to automatically test whether an extant schema is consistent with its specification. This is almost more of a formal methods thing? These are more focused on /generating/ a conforming schema from the specification (e.g., Vatanawood & Rivepiboon, 2004; Luković et al., 2003), rather than checking that an already existing schema conforms. Also more focused on schema /transformations/ and evolution (e.g., Bench-Capon et al., 1998). ------------------------------------------------------------ NOTE: Checking structure of table Product. TEST: [[ Product ]] + OK +++ PASSED: Table Product exists. TEST: [[ Product.Product_code ]] + OK ... +++ PASSED: Table Product contains all the expected columns. TEST: [[ Product.Product_code: data type is NUMBER | INTEGER ]] + OK ... +++ PASSED: All columns of table Product have data types compatible with the specification. TEST: [[ Product.Product_code precision and scale = 8 (with scale 0) ]] + OK ... +++ PASSED: All columns of table Product have lengths compatible with the specification. TEST: [[ Product.Product_code nullability should be N ]] + OK ... +++ PASSED: All columns of table Product have the expected nullability. TEST: [[ Product PK ]] + OK +++ PASSED: Primary key of table Product exists. TEST: [[ Product PK: Product_code ]] + OK +++ PASSED: Primary key of table Product includes (only) the expected columns. TEST: [[ Product check constraint PRODUCT_STOCK_INVALID ]] + OK ... +++ PASSED: All constraints of table Product that should be are explicitly named. NOTE: Testing constraints of table Product. TEST: [[ Product.Stock_count accepts “0” ]] + OK TEST: [[ Product.Stock_count accepts “99999” ]] + OK TEST: [[ Product.Restock_level accepts “0” ]] - FAILED! Column Product.Restock_level won’t accept legal value 0 [-0.5]. Failed asserting that false is true. TEST: [[ Product.Restock_level accepts “99999” ]] + OK TEST: [[ Product.Minimum_level accepts “0” ]] - FAILED! Column Product.Minimum_level won’t accept legal value 0 [-0.5]. Failed asserting that false is true. TEST: [[ Product.Minimum_level accepts “653” ]] + OK TEST: [[ Product.List_price accepts “0” ]] + OK TEST: [[ Product.List_price accepts “99999.99” ]] + OK --- FAILED: 2 of 8 legal values tested were rejected by a CHECK constraint.
Show line notes below
GitBucket
4.21.2
GitBucket
4.21.2