JS-Input Drawing
&
FormulaResponse Feedback

Chris Chudzicki, 8-17-2016

Drawing in physics

• Helpful to illustrate / understand concepts
• Helpful as step in solving complex problems
   
• Freebody diagrams: multiple choice vs drawing

more authentic
higher cognitive skill

Customresponse & JS-Input

• Customresponse: custom grading through python

• JS-Input: Custom input and display through HTML, Javascript, CSS

<script type="loncapa/python">
<![CDATA[
def check_function(expect, answer):
    #Grading code here
]]>
</script>
<customresponse cfn="check_function">
    <textline />
    <textbox />
    <formulaequationinput />
</customresponse>

• JS-Input: Authoring from scratch vs using existing library

VectorDraw Demo

VectorDraw Code

<problem display_name="Acceleration in Circular Motion">
<script type="loncapa/python">
import python_lib.vectordraw as vectordraw
custom_checks = {'sum_vec': vectordraw.custom.check_sum_vec,}
grader = vectordraw.Grader("Success Message", custom_checks)
cfn = grader.cfn
</script>

<!--Problem Description Here -->

<customresponse cfn="cfn">
  <jsinput gradefn="getInput"
    get_statefn="getState"
    set_statefn="setState"
    initial_state='{}'
    width="800"
    height="500"
    html_file="/static/PROBLEM_FILE.html"
    />
</customresponse>
</problem>

Problem XML File

VectorDraw Details

• Open-source project started by Daniel Seaton while at Davidson College
  Many VectorDraw problems in Davidson Next's AP-support courses
• Further development by OpenCraft and Chris (this summer)

VectorDraw Code

// Setup your vectors
vectors: [
    {name: 'a_t', description: 'Tangential Acceleration'},
    {name: 'a_r', description: 'Radial Acceleration'},
    {name: 'a',   description: 'Acceleration'},
    ],
//Convenient, built-in check functions
expected_result: {
    'a_t': {tail: [-0.25,4.5], angle:180, angle_errmsg: "Is the particle up or slowing down?"},
    'a_r': {tail: [-0.25,4.5], angle:270},
    'a'  : {tail: [-0.25,4.5], tail_tolerance:3 }
    },
//Advanced: all the power of customresponse check functions
custom_checks: [
    {
        check: 'sum_vec',
        vectors: ['a_r', 'a_t', 'a'],
        coeffs: [1, 1, -1],
        expected: {x:0, y:0},
        errmsg: "The acceleration vector drawn is not consistent with its components."
    }
  ]

Problem HTML File

One more VectorDraw ...

Free Body Draw

Drawing Summary

• VectorDraw, open source JS-Input drawing tool
  • arrows, lines, line segments, points
  • convenient and powerful grading!
• FreeBodyDraw good for free body diagrams

• TODO:
  • Get my bug fixes and improvements merged into master
  • UI improvements?
  • Explore randomized vector problems

Interlude: Pretty Customresponse Feedback

import python_lib.cfn_tools as cfn_tools
# define or import old_check_function
pretty_check_function = pretty( old_check_function )

# plus optional settings

Wrong-Answer Feedback for <formularesponse/>

Physics courses (8.01, 8.05) have lots of <formularesponse/> problems

Grading FormulaResponse

<problem>
    <formularesponse type='cs' answer='r*sin(theta)' samples='theta,r@ 0,4:1,9 #3'>
        <!-- cs means case-sensitive -->
        <!-- sample theta in range [0,1]-->
        <!-- sample r     in range [4,9]-->
        <responseparam default='1%' type='tolerance'/>
        <formulaequationinput inline='1'/>
    </formularesponse>
</problem>

1. Numerically evaluate instructor answer
2. Numerically evaluate student answer
3. all eval agree (within tolerance) → mark correct
   any eval disagree → mark incorrect

• Bad variable ranges can cause grading problems
• Most code in file calc.py

FormulaResponse Analyzer

• Table of contents for problems in a course:

• Few, common wrong answers
  → better for feedback
  weight_first_3: what fraction of wrong answers are
  top 3 wrong answers?

• Grading issues:
  • Graded Correct ... should be 1
    How many equivalence groups (according to me) did edX mark as correct
  • Graded Inconsistent:  ... should be zero
    How many equivalence groups (according to me) did edX mark some submissions correct, some wrong

FormulaResponse Analyzer

• Inconsistent Grading

• Multiple Correct Answers

Automated feedback?

 <formularesponse answer="m*c^2" samples="...">
    <formulaequationinput inline="1"/>
    <hintgroup>
        <formulahint answer="m*c^3" samples="..." name="cubed"/>
        <hintpart on="cubed"><text>Some Hint</text></hintpart>
    </hintgroup>
</formularesponse>

• Coding feedback for individual wrong answers is a lot of effort
• not all problems have just a few common wrong answers

Can we provide useful feedback automatically?

Checking Dimensions

• Dimensions = "Length", "Mass", "Time", "Length * Time^-2"
• Wrong answers often have wrong dimension

• Implemented as a hint function that borrows edX formularesponse grading function, but plugs in dimension-jul quantities instead of pure numbers

What Next?

• Specific, targeted wrong-answer feedback:
  • continue adding feedback to 8.01
  • AB Experiment with feedback?
    attempts till correct, competition time, retention within problem
• Automated Feedback
  Augmenting formularesponse with standardized hint_fn seems powerful ...
  • ​allow multiple formularesponse correct answers (on purpose)
  • sample with complex numbers, e.g., enforce
  • Check submission?
  • Check if submission is valid vector expression

Thanks!

https://github.com/ChristopherChudzicki/

• edX-author-tools
  make hint_fn and check_fn pretty
  check dimensions for formularesponse
• edX-FormulaResponseAnalyzer
• jsinput-VectorDraw*
• jsinput-FreeBodyDraw

*or use OpenCraft version