DIFFERENTIATED OBSERVATION 2021
LISA KAROSAS
Assignment Submission
As presented in the "Submission Link" area of the LMS Setup discussion, the evidence required by students to demonstrate completion of each weekly Adaptive Practice set took the form of an image produced by the online program once ten questions were answered correctly.
​
An actual example of this image submitted by a student during the second semester is provided here:
Students aim to answer 10 questions correctly and are given multiple opportunities to do so. After a few tries, a consistently incorrectly answered question is skipped and is counted in the score as a 10% deduction.
_PNG.png)
This value reflects the total number of questions answered correctly despite the number of times it was attempted.
This value reflects the total number of questions answered correctly, consecutively.
This value reflects the total time spent to complete the practice set.
The date each practice set was completed is recorded.
The program considers all of the listed data to produce a "skill level" for each practice set completed.
The practice set title appears at the top of the graphical summary.
Each graphical summary provides a wealth of data regarding each student's understanding of a specific concept. Among this collected data, the actual "Skill Level" indicator included in the graphic was the most informative when I reviewed student work and the "Practice Goal" percentage was the least informative. This was due, in part, to critical information that's missing from the summary, the number of times a student attempted the entire practice set. For example, this student's Atomic Number adaptive practice indicates 140% was earned which isn't possible unless the practice set was accessed and completed multiple times. Despite the 140% and 7-question correct answer streak, I'm able to note the "proficient" skill level.
.png)
It's important to note that the "mastery" designation is not reserved for perfect attempts. It truly reflects all data relevant to the attempt. Consider both "mastery" data sets provided, below. Mastery is implied by the percentage, number correct, best streak, and time spent on the Volume-Volume Stoichiometry practice set. However, the student who completed the Activity Series practice set had a lower best streak and took a longer amount of time to achieve the required ten correct. Still, this performance is exemplary and the program justly classifies it as such.
_PNG.png)
_PNG.png)
Finally, given the nature of the integration of this technology into our LMS, it was very important for me to observe the date on which each student completed the practice item to ensure they weren't handing in the graphical data for a practice item they had completed earlier in the semester or year.
LIMITATIONS & ROSTERING ON CK12.ORG
The content provider, CK12.org, equips teachers with the ability to roster students within their website as a means of easily assigning any textbook content, active explorations, practice, or assessment content on their site. Rostering also enables teachers to quickly and easily view collective class progress on concepts using conventional heat map spreadsheets. Individual student data can also be accessed from these spreadsheets.
In addition to providing students with ease and simplicity where access, completion, and submission were considered, I chose to have students submit this graphical data because I was unsure about the legalities of rostering students on the CK12.org website in accordance with FERPA guidelines as it pertains to preserving student privacy. With so much going on at the start of this unique 2020-2021 academic year, I decided to research that option at a later date.
​
It wasn't until the start of the second semester that I contacted Rick Tkach for advice and approval concerning the rostering option. He assured me that I could require students to add themselves to a group I created on the CK12.org website for now and that at the start of next year, we could discuss the possibility of using Clever technology to potentially circumvent this step to ensure our students are provided safe access to the content.
​
During a condensed-schedule live session on March 16th, I instructed students to add themselves to the group I created. Due to the fact that the highest number of students is disengaged during live session at this time of year, not all the students who have registered accounts participated in the exercise. For this reason, I was unable to completely transition my data consolidation efforts for the remainder of this school year. Of the 107 students on my classlist, 67 added to my group. This was enough students to further explore the options and troubleshoot the details of how to accomplish this task in the future.
TECHNICAL CONSIDERATIONS TO COLLECTING ROSTERED DATA ON CK12.ORG
In order to review consolidated student/group data on the CK12.org website, several technical requirements must be satsified:
​
-
Practice items must be "assigned" to the CK12.org group.
-
A specific, "assigned" practice set link must be published in the LMS.
-
Students must click an orange "Turn In" button above the graphical data produced by their attempt.
.png)
The first two conditions are easily satisfied by the teacher. Though not as quick as copying an address bar link, this process was not difficult.
​
Once an Adaptive Practice set is identified, the teacher merely clicks the "Assign to Class" button at the top of the page and chooses the group that has been created. In doing so, the teacher must also set a "due date". In my pilot, I set the "due date" as the final day of school, June 4th, to compliment the student-choice feature of my design and integration. Additionally, many of my students have remarked on the utility of CK12 for study in their math courses and to review in preparation for the biology Keystone test they may need to retake. It's entirely logical to assume students might choose to revisit some chemistry concepts for their own practice and study in advance of the semester test during the final week of school.
In evaluating whether to use this feature in the future, I'm concerned about the potential for students to be confused by the dual-nature of the single assignment. In the LMS, there will be a scheduled assignment and a due date which will be factored into their academic performance in the course. In the CK12 website, there will be a scheduled assignment and a due date which will NOT be factored into their academic performance in the course. During my pilot period, I didn't receive any inquiries about this. However, 40% of my class list wasn't engaged enough to even know the assignment list and due dates on CK12.org exist because they didn't participate on the day we created the group. Many of my students demonstrated understanding of how to execute WEEKLY PRACTICE at the beginning of the year but, admittedly, a small group of 10-15 students repeatedly submitted graphical data that was either incomplete or representative of practice items not on the approved list in the LMS. Effective implementation would require constant messaging and issuing reminders when students begin in the course.
To capture the link which will collect and consolidate student data, the "REPORTS" function from the group screen must be accessed. The "REPORTS" automatically open in a new browser tab. The "REPORTS" screen shows individual student performance just like in a conventional gradebook. When a student's name is clicked, the individual graphical summaries for each attempt by each student is produced for review.
To generate the practice set link that will produce data in this spreadsheet, the practice item title must be clicked and, on the right side panel that appears, the link title must be right-clicked to copy the link to share in the LMS.
Perhaps the biggest obstacle to reliably, repeatably collecting consolidated data on students' adaptive practice performance in the CK12.org platform is the requirement for students to remember to click the "Turn In" button on the final screen. Currently, I don't have any data suggesting the extent of the challenge this presents because the students participating in this pilot are among the most engaged and responsible on my classlist. I also don't have any ideas for helping students create this routine when they complete Weekly Practice. The use of a more universal LMS would eliminate most of these concerns!
However, without attempting to roster the data in this way, a potentially valuable perspective is lost. Using only snapshots submitted to the LMS, I have only an anectodal idea of how many students chose to complete a particular practice set. With this spreadsheet, I can clearly and easily identify the most popular practice sets chosen by the students in my classes.
When used in tandem with common assessment data, it could help explain unexpected positive or negative results. On some occasions such as compressed-schedule days, asynchronous days, and substitute days, I have assigned students specific practice to complete. In this situation, the immediate feedback helps me make instructional decisions related to pace.