AIRR Brain Dump

Overview

Engrade Integration

Acceptance Criteria

  • Integrate Engrade Assessments into existing products
  • Engrade has established UIs
  • Engrade exposes everything via RESTful web services
  • MHE owns Engrade, so they will accommodate our every whim
  • We can have a single experience across all apps

How hard can that be?

Engrade UI Components

Customers

  • Everyday Math 4
  • Reading Wonders (Replacing Cognero)
  • Open Court Reading
  • Music

Teacher-Facing Tests & Quizzes

  • Test creation and question pool management
  • Class assignments
  • Schedule tests
  • Per-test submission results
  • Access to preset test banks and question banks

How Many UIs?

Teacher-Facing Gradebook

  • Grid view
  • All students in the class
  • All assignments for the year
  • All scores for every student and every assignment
  • Drill downs for test details and standards coverage

How Many UIs?

Student-Facing Test & Quizzes

  • Grid view
  • All students in the class
  • All assignments for the year
  • All scores for every student and every assignment
  • Drill downs for test details and standards coverage

How Many UIs?

Customer Requirements

  • Wanted the Engrade UIs to launch in same window
  • Wanted to customize the look & feel
  • Some customers do not want to expose Gradebook

How Can We Embed?

  • Engrade is not componentized
  • (Some) customers would not accept a popup experience
  • Not too many options...

iFrame

  • Bad user experience
  • "seamless" attribute is the worst supported specification
  • Third-party cookie problems

Seamless iFrame

  • Eliminate scrollbars - iFrame resizes according to content size
  • CSS Injection (L&F and hiding functionality)
  • Keep alive callbacks
  • Needs to work across domains (iframe restrictions)

CSS Injection

  • Engrade had to add ids and classes to items we need to hide
  • Parent page includes CSS
mhe-engrade iframe .content h2 {
    color: #4444cc;
}

Third Party Cookies

  • https://goo.gl/O9Vjhi

Project

  • svn://srcrep.cdiapps.com/ConnectED/trunk/mhe-engrade
  • Exposed to developers via sharedwebassets-mhe
  • Documentation - http://srcrep.cdiapps.com/sites/mhe-engrade/

Data Sync

Push

Data Sync

Data

Engrade Coordination

  • Engrade UIs are powered by Engrade data
  • Engrade knows absolutely nothing about ConnectED
  • It is not data sync, it is data push
  • Proactive Push or Reactive Push?

Data Push Highlights

  • The push begins with a class (docs)
  • Class ->  School -> District (no subdistricts)
  • Class -> Teacher -> Students
  • Calling User

Data Push Lowlights

  • One call to sync a class can result in more than 50 individual HTTP requests to Engrade
  • ConnectED does not have good support to track whether push is needed
  • Push typically takes more than 10 seconds
  • Engrade has different constraints on data that are opaque to us
  • Many edge cases for failure
  • No collaboration

Authentication

Engrade SSO

  • ConnectED shim generates Engrade SSO endpoints
GET /connected/shim.do?uri=/engrade/sso/classes/M7XC4N7NOYHM297GMJXDOXL4BQ 

{
    "student_quizzes": "https://www.engradesandbox.com/user/sso-login.php
        ?uid=10000000005981833
        ×tamp=1433514066
        &nonce=413
        &token=MCwCmg
        &next=class%2Fquizzes/index-stu.php
            ?hideframe=1
            &stuid=C8YZR9Z99ZWSO82S6427D3NKBQ
            &clid=5000007032229",

    "gradebook": ...,

    "teacher_quizzes": ...
}

Custom UI

Customer Request

  • Engrade UIs are too complicated or aesthetically dissimilar from desired interfaces

Engrade Public API Access

  • Engrade APIs return data specific to the authenticated user
  • Engrade does not have the ability to authenticate on behalf of a user (aka user spoofing, runas, etc.)
  • Our only option at this time is to authenticate as the user using an Engrade password we store in cleartext in our database

Better Solution (maybe)

  • user-login
    • apikey
    • usr - username
    • pwd - password
    • remember - (Optional) remember password? (1 = Yes, 0 = No)
    • runas - username to associate with session

Reporting

Acceptance Criteria

  • Reading Wonders wants to provide data-driven recommendations to students
  • Teachers want to see students grouped by their proficiency
  • View student's response at the question level
  • iPad friendly
  • Later: support for other applications

How is Proficiency Measured?

  • Difficult to quantify proficiency vs. performance
  • Proficiency is science, performance is 3rd grade math
  • Proficiency has meaning, performance has none

MHE: The Learning Science Company

  • Educational Data Mining, Learning Science, Learning Analytics, Item Response Theory, Bayesian Knowledge Trace, Performance Factors Analysis
  • Dr. Ryan Baker is consulting with MHE
  • Internal whitepaper: https://goo.gl/l8IvZR
  • Who has a higher proficiency?

Alice

Bob

☒ ☒ ☒ ☒ ☒ ☒ ☑ ☑ ☑ ☑ ☑

☑ ☑ ☒ ☑ ☑ ☒ ☑ ☒ ☑ ☒ ☒

t →

Data Sources

In Priority Order

  • Engrade Assessments
  • Teacher Observations (Rubric)
  • Interactive Activities (Games, PBL)

In Delivery Order

  • Engrade Assessments
  • Interactive Activities (Games, PBL)
  • Teacher Observations (Rubric)

Architecture

Architecture

https://goo.gl/paCjmY

viz-report

  • AngularJS Component
  • Designed to be embedded into any application
  • Currently AngularJS 1.3
  • Using ui-router

  • svn://srcrep.cdiapps.com/ConnectED/trunk/viz-app

viz-report-service

  • Not started - AIRR effort
  • REST endpoints specific to needs of viz-report

report-service

  • Not started - LEAP/AIRR effort
  • General queries of LEAP data
  • PII information hydration

Reporting UI

Explorable

  • The entire UI experience is designed to be touchable
  • Instant feedback to users
  • Visual transitions and data animations
  • Using D3 and Angular
  • Meant to be a showcase to demonstrate what Columbus is capable of implementing

Documentation

  • CA Submission (compromised navigation, no working components, performance data)
    • http://goo.gl/HRSC7l
  • Full UX documentation 
    • https://goo.gl/Gfrad6
  • Screen Mockups
    • https://goo.gl/519nKc

Pseudo LTI

Reporting Data Needs

  • Reporting component will have to make LTI-like callbacks to embedding application for some data
    • Class rostering data (student names -> id)
    • Mastery threshold settings
    • Recommendations by taxon
    • Lesson and resource preview
    • Configuration data (may be specified declaratively)
  • Nice to have: actually implement as an LTI Provider

Learning Events

Acceptance Criteria

  • Set of data structures to represent student interactions with the system
  • Require events necessary to support the needs of the viz-report component

Specifications

Required Events

  • activity started
  • activity ended
  • item graded
    • question on assessment
    • challenge in a game
    • observation on a rubric
  • https://goo.gl/loEI9z

Formalized Specification

  • Began with xAPI, switched to Caliper
  • https://goo.gl/0SwVZI
  • Analytics team is the owner of schemas
  • 95% complete (need to define how to store outcome results)

Data Generator

Generator Tool

  • Developed by Chris in Node.js

  • svn://srcrep.cdiapps.com/ConnectED/trunk/viz-tools/generator

  • Simulates a year's worth of Reading Wonders assessment events for a single student

  • Extreme data sets when specifying number of districts, schools and classes

  • Mastery heuristics

  • Needs more work to match final Caliper specification

PII

Acceptance Criteria

  • No personal identifying information will be included on a learning event

Identified PII

  • districtID
  • schoolID
  • classID
  • instructorID
  • studentID

Remaining Tasks

  • Affirm with Andy Bloom (CPO) that our externalized ids will provide sufficient obscurity to satisfy PII

Learning Event Factory

Learning Event System

https://goo.gl/B6UB7e

Environmental

  • MHE Github: https://goo.gl/okuPUy
  • New JVM (Java 8, Tomcat 6)
  • Limited memory
  • Spring Boot, Groovy, Spring Data
  • Couchbase

Q3 2015

  • Integration with Reading Wonders and VSM games
  • New microservice to return all standards associated with any asset
  • Assist RW in context generation
  • Performance testing
  • Final product delivered in Q3

SensorJS

Acceptance Criteria

  • Provide third-parties with a Javascript library to make it simple for them to report their result data
  • This result data will be converted to a viable learning event

SensorJS

  • Small JS utility that will generate learning event fragments and send them to ConnectED event factory
  • MHE GitHub
    • https://goo.gl/gluPrC
  • Demo
    • https://goo.gl/OLBSQa

California Submission

Acceptance Criteria

  • Provide a working UI for California reviewers to understand how data-driven instruction will be supported in the forthcoming Reading Wonders product

Mock UI

Q2 2015 Deliverable

  • Delivered a working version of the reporting UI
  • Embedded in Reading Wonders 2.0
  • Sample data provided by RW AD
  • http://goo.gl/HRSC7l

Mock Components

  • A separate project created to house all components
  • Designed for reuse, AngularJS components

  • svn://srcrep.cdiapps.com/ConnectED/trunk/viz-ui

  • Documentation

    • ​http://goo.gl/5xgWMw

Sample Data

  • AD used a Google Spreadsheet to collaboratively enter data
  • https://goo.gl/9tTdCm
  • Zayeem created an add-on to export data from spreadsheet into a JSON object plugged directly into the Mock UI
Made with Slides.com