How to Create Thousands of Versions of Summative Exams

One of the difficulties of moving to a true mastery system is the logistics of multiple versions of a test.  How many different versions can you manage on a day to day basis?  When Aaron Sams and I first started implementing the Flipped-Mastery model, we had three paper versions of every test.  Students who struggled either memorized each test version or in some cases other students had taken cell phone pictures of the test and were cheating.   We were using Moodle as our learning management system.  We discovered it had the ability to administer tests, so we wondered if there was a way to solve the management of many different versions.  One day as I was reading posts on the Moodle forum, I stumbled upon a solution to my dilemma. When I realized what I had found, I literally did a dance in my living room.

Though we were using Moodle, what we did can also be done with other Learning Management Systems.  Quia and Fishtree gave me demos of their products and I have been impressed with how easy they are to use and how robust their analytics are.  I am also aware that Blackboard has this feature though I have not personally reviewed it.  I imagine that almost all learning management systems have this feature, but I am not certain so check with your system administrator.  So, instead of walking you through step by step with each system, I want to give you the big picture of how to write thousands of versions of a test which covers specific objectives.

Step 1:  Create a list of discrete objectives.  This is simply a list of what it is you want assessed.  This is good teaching and forms the basis for your questions to come.  Below is the list of objectives for our unit on Gas Laws. 

Step 2:  Create a bank of questions for each objective.  So for each objective listed, we usually wrote between eight and ten questions.

Step 3:  Let the Learning Management System/Web Tool randomly select questions.  This is where the magic happens.  Once you have all of your banks of questions written, you have the computer randomly select questions from each bank.  So if there are eight questions that assess objective one, then have it pick one question.  And for objective two, you might have it choose two questions out of your bank of twelve.  Repeat until you have “built” your test.

Step 4:  Students take the exam.  Each student will get their own exam and it will be different.  Some will have the same questions, but they will be generated randomly.

Notes:   

• This requires access to computers for your students.  We had eight computers in each classroom.  And since students were not taking the exams at the same time, this was sufficient.
• We came to realize that we didn’t need to have long tests.  A good exam can take a shorter period of time.
• This a lot of work.  Writing good, meaningful questions that assess the same objective is hard.  We spent many hours creating questions but it was well worth the time spent.

What do you think? Do you have any strategies for mastery exams?  I would love to hear from other Flipped-Mastery teachers out there.

Portions of this blog post will be included in an upcoming article for Educational Leadership that I co-wrote with Aaron Sams.  Learn more about Educational Leadership by clicking HERE

Follow-up:  I created this video to explain how to do this using Moodle (an old version)

 Photo by John Schnobrich on Unsplash