<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 12 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:DotumChe;
panose-1:2 11 6 9 0 1 1 1 1 1;}
@font-face
{font-family:"\@DotumChe";
panose-1:2 11 6 9 0 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Arial","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-GB link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><font size=2 face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif"'>Hi everyone,<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif"'>I’m looking at the possibility of getting funding from the latest University HERMES call for the following project. However initial feedback from them says that funding is unlikely unless I have strong evidence of interest from other Schools in the University and beyond.<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif"'>This appears unlikely for this round (deadline May 21<sup>st</sup>), so I am probably looking at the next round (October 2012). On the other hand, to quote our research facilitator David Hawker:<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></font></p><p class=MsoNormal style='margin-left:36.0pt'><font size=2 color="#1f497d" face=Calibri><span style='font-size:11.0pt;color:#1F497D'>There will be a further round of Hermes in October, so if we can make some phone calls and enquiries by then and garner support for your idea, the University sounds as if it would be behind it. Of course if you are able to whip up a frenzy of interest in the next few days we can still go for the current call!<o:p></o:p></span></font></p><p class=MsoNormal style='margin-left:36.0pt'><font size=2 color="#1f497d" face=Calibri><span style='font-size:11.0pt;color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif"'>So, is there a frenzy of interest?<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif"'>Best wishes,<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif"'> Joel<o:p></o:p></span></font></p><p class=MsoNormal><b><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D;font-weight:bold'><o:p> </o:p></span></font></b></p><p class=MsoNormal><b><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D;font-weight:bold'>Automatic generation of random formative and summative multiple choice tests<o:p></o:p></span></font></b></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>What forms of assessment are both efficient and effective in encouraging student engagement?<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>Assessed coursework is a possibility, but has high overheads in terms of marking time, and there is also the issue of plagiarism.<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>Oral presentations and examinations can be an excellent way to measure understanding, but are usually impractical for large classes.<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>Multiple choice tests are very efficient on the marking side (machine marking), but the design of good questions is time-consuming, and there are currently limitations on the variations which can be randomly generated from a particular question type. Some randomization of numbers in questions and of the order of the questions is possible in some systems, but there is currently little scope for randomization of the variable names, function names, language expression, etc.<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>The proposed project involves investigation of the best way to generate large numbers (tens of thousands) of questions from any given prototype, by means of suitable randomization of numbers, variable names, function names, and language used (e.g., for quantifiers in mathematics, symbols can be used, or one can use “for all”, “for each”, “for every”, “there exist(s)”, “there are/is”, “there corresponds”, etc.). Ideally the questions could be output from the software in a form suitable for input into a variety of online systems (e.g. rogo, moodle, …)<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>I already have a few sample prototype questions which can be used to test students’ understanding of the use of mathematical quantifiers. I would need a substantial amount of dedicated time in order to develop software tools of my own and futher question prototypes in order to generate substantial question banks.<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>Although my initial proposal is for mathematics tests, the project could be extended to many other areas.<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>I would envisage the questions generated being used in the following way: having chosen which type of question will appear in a paper-based multiple-choice class test, the students are allowed to practice (formatively) this type of question as much as they like online, with instant feedback. Even if the class test is worth very little credit, it is likely that the students will put in a lot of work on the formative questions to prepare for the test.<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>I believe that a similar formative/summative system has been tried in Engineering with considerable success.<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>I believe that it would be a rather efficient use of staff time to generate prototype questions which can then be used again and again. There would be no need to generate new prototypes every year (though it may be that new prototypes would be introduced from time to time).<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><b><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D;font-weight:bold'>The benefits:<o:p></o:p></span></font></b></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>Increased student engagement.<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>Better levels of student understanding.<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>Essentially unlimited, ready-made assessments of students, with minimal staff input required.<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 color="#1f497d" face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif";color:#1F497D'>These benefits are for us directly, and for anyone else who uses the system. Obviously, the fact that the system originates in Nottingham will raise the profile of Nottingham’s teaching.<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 face=Arial><span style='font-size:11.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></font></p><p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'>-- <o:p></o:p></span></font></p><p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'> Dr J. Feinstein Tel: +44-115-9514968<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'> Pure Mathematics FAX: +44-115-9514951<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'> Nottingham NG7 2RD Web: http://www.maths.nott.ac.uk/~jff<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 face="Courier New"><span style='font-size:10.0pt;font-family:"Courier New"'> United Kingdom Blog: http://explainingmaths.wordpress.com<o:p></o:p></span></font></p><p class=MsoNormal><font size=2 face=Calibri><span style='font-size:11.0pt'><o:p> </o:p></span></font></p></div></body></html>