Difference between revisions of "Analysis:Student affect and interaction behavior in ASSISTments"
Pinventado (talk | contribs) (Add anchor to external resources section) |
Pinventado (talk | contribs) (Added skill level section and added anchors to each section) |
||
| Line 1: | Line 1: | ||
{{Infobox_analysis | {{Infobox_analysis | ||
|author=[[User:Pinventado|Paul Inventado]] | |author=[[User:Pinventado|Paul Inventado]], Peter Scupelli | ||
|dataset=[[Data:ASSISTments_2012-2013_problem-student_level|ASSISTments problem-student-level data | |dataset=[[Data:ASSISTments_2012-2013_problem-student_level|ASSISTments problem-student-level data]], [[Data:ASSISTments 2012-2013_problem_level|ASSISTments problem-level data]] | ||
}} | }} | ||
| Line 7: | Line 7: | ||
=Student learning strategies= | =Student learning strategies= | ||
<div id="learningstrategies"></div> | |||
Online learning systems that host student exercises and activities sometimes provide feedback in the form of hints or additional explanations. Feedback can help students figure out the answer, but can also be used for gaming the system, which hurts learning <ref>Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., and Koedinger, K. (2008). [http://www.editlib.org/p/24328/ Why students engage in “gaming the system” behavior in interactive learning environments]. Journal of Interactive Learning Research, 19(2):185-224.</ref>. Gaming is taking advantage of system feedback to get the answer without actually learning such as continuously requesting for hints or trying out all possible answers. | Online learning systems that host student exercises and activities sometimes provide feedback in the form of hints or additional explanations. Feedback can help students figure out the answer, but can also be used for gaming the system, which hurts learning <ref>Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., and Koedinger, K. (2008). [http://www.editlib.org/p/24328/ Why students engage in “gaming the system” behavior in interactive learning environments]. Journal of Interactive Learning Research, 19(2):185-224.</ref>. Gaming is taking advantage of system feedback to get the answer without actually learning such as continuously requesting for hints or trying out all possible answers. | ||
| Line 15: | Line 16: | ||
==External resources== | ==External resources== | ||
<div id="externalresources"> | <div id="externalresources"></div> | ||
An analysis of ASSISTments' [[Data:ASSISTments_2012-2013_problem_level | problem level data]] showed a correlation between the ''has_text_color'' feature, and boredom and gaming. Majority of problems having ''text color'' contained references to problems on the textbook, but did not contain the actual content. This means that when students answered these problems, they had to switch between looking up the problem in their book and using the ASSISTments interface. | An analysis of ASSISTments' [[Data:ASSISTments_2012-2013_problem_level | problem level data]] showed a correlation between the ''has_text_color'' feature, and boredom and gaming. Majority of problems having ''text color'' contained references to problems on the textbook, but did not contain the actual content. This means that when students answered these problems, they had to switch between looking up the problem in their book and using the ASSISTments interface. | ||
According to Sweller (2004<ref>Sweller, J. (2004). [http://link.springer.com/article/10.1023/B:TRUC.0000021808.72598.4d Instructional design consequences of an analogy between evolution by natural selection and human cognitive architecture]. Instructional science, 32(1-2), 9-31.</ref>), unnecessary processing of information imposes a cognitive load that interferes with learning. High cognitive load impairs performance, which could increase the difficulty of a learning task. When students experience too much difficulty or get stuck in trying to solve a problem, they are likely to disengage from the activity<ref>D’Mello, S., & Graesser, A. (2012). [http://www.sciencedirect.com/science/article/pii/S0959475211000806 Dynamics of affective states during complex learning]. Learning and Instruction, 22(2), 145-157.</ref>. Resulting disengagement behavior can explain the boredom and gaming behavior seen in the data. | According to Sweller (2004<ref name="Sweller2004">Sweller, J. (2004). [http://link.springer.com/article/10.1023/B:TRUC.0000021808.72598.4d Instructional design consequences of an analogy between evolution by natural selection and human cognitive architecture]. Instructional science, 32(1-2), 9-31.</ref>), unnecessary processing of information imposes a cognitive load that interferes with learning. High cognitive load impairs performance, which could increase the difficulty of a learning task. When students experience too much difficulty or get stuck in trying to solve a problem, they are likely to disengage from the activity<ref name="DMello2012">D’Mello, S., & Graesser, A. (2012). [http://www.sciencedirect.com/science/article/pii/S0959475211000806 Dynamics of affective states during complex learning]. Learning and Instruction, 22(2), 145-157.</ref>. Resulting disengagement behavior can explain the boredom and gaming behavior seen in the data. | ||
=Over-practice and student affect= | =Over-practice and student affect= | ||
<div id="overpractice"></div> | |||
Frustration is an emotion often experienced by students while learning. It is unclear whether it hinders learning or pushes students to exert more effort to learn (cf. Artino & Jones 2012<ref>Artino, A.R. and Jones, K.D. (2012). [http://www.sciencedirect.com/science/article/pii/S1096751612000073 Exploring the complex relations between achievement emotions and self-regulated learning behaviors in online learning]. The Internet and Higher Education 15, 3, 170-175. </ref>; Patrick et al. 1993<ref>Patrick, B.C., Skinner, E.A. and Connell, J.P. (1993). [http://psycnet.apa.org/journals/psp/65/4/781/ What motivates children's behavior and emotion? Joint effects of perceived control and autonomy in the academic domain]. Journal of Personality and social Psychology 65, 4, 781-791.</ref>; Pekrun et al. 2002<ref>Pekrun,R., Goetz, T., Titz, W. and Perry, R.P. (2002). [http://www.tandfonline.com/doi/abs/10.1207/S15326985EP3702_4 Academic emotions in students' self-regulated learning and achievement: A program of qualitative and quantitative research]. Educational psychologist 37, 2 (2002), Taylor & Francis, Philadelphia, PA, 91-105.</ref>). The role of frustration in learning with [[ASSISTments]] was investigated using both [[Data:ASSISTments_2012-2013_problem-student_level | problem-student-level data]] and [[Data:ASSISTments_2012-2013_problem_level | problem-level data]] that described student behavior while answering a series of questions in the system. | Frustration is an emotion often experienced by students while learning. It is unclear whether it hinders learning or pushes students to exert more effort to learn (cf. Artino & Jones 2012<ref>Artino, A.R. and Jones, K.D. (2012). [http://www.sciencedirect.com/science/article/pii/S1096751612000073 Exploring the complex relations between achievement emotions and self-regulated learning behaviors in online learning]. The Internet and Higher Education 15, 3, 170-175. </ref>; Patrick et al. 1993<ref>Patrick, B.C., Skinner, E.A. and Connell, J.P. (1993). [http://psycnet.apa.org/journals/psp/65/4/781/ What motivates children's behavior and emotion? Joint effects of perceived control and autonomy in the academic domain]. Journal of Personality and social Psychology 65, 4, 781-791.</ref>; Pekrun et al. 2002<ref>Pekrun,R., Goetz, T., Titz, W. and Perry, R.P. (2002). [http://www.tandfonline.com/doi/abs/10.1207/S15326985EP3702_4 Academic emotions in students' self-regulated learning and achievement: A program of qualitative and quantitative research]. Educational psychologist 37, 2 (2002), Taylor & Francis, Philadelphia, PA, 91-105.</ref>). The role of frustration in learning with [[ASSISTments]] was investigated using both [[Data:ASSISTments_2012-2013_problem-student_level | problem-student-level data]] and [[Data:ASSISTments_2012-2013_problem_level | problem-level data]] that described student behavior while answering a series of questions in the system. | ||
| Line 26: | Line 28: | ||
The problem level instances were investigated further by referring to the student-level data, which described students' low level actions while answering the problem (e.g., order of hint requests and answer attempts, answer correctness, time duration between actions). Students who were frustrated even when they got the answer correctly, were observed to have spent a significant amount of time answering the exercise and consistently getting the answers correctly. This could mean that students were frustrated at the type and sequence of problems they were asked to solve rather than the problem itself. | The problem level instances were investigated further by referring to the student-level data, which described students' low level actions while answering the problem (e.g., order of hint requests and answer attempts, answer correctness, time duration between actions). Students who were frustrated even when they got the answer correctly, were observed to have spent a significant amount of time answering the exercise and consistently getting the answers correctly. This could mean that students were frustrated at the type and sequence of problems they were asked to solve rather than the problem itself. | ||
=Skill level= | |||
<div id="skilllevel"></div> | |||
An analysis of [[Data:ASSISTments_2012-2013_problem-student_level | student behavior data]] showed that students reacted differently to the same problem. Students who were skilled in a certain topic rarely used hints and could quickly answer the problem. Students who were less skilled on the topic, often requested for hints, made multiple attempts to solve the problem, and spent more time before answering it correctly. Students skilled in one topic struggled with other topics, which they might not have learned as much. Without prior knowledge, students will have difficulty answering a problem<ref name="Sweller2004"/>. | |||
Data showed that students were often bored or disengaged from the problem when they did not have enough skills to solve it (i.e., evidenced by answer correctness and number of hint requests). When students experience too much difficulty or get stuck in trying to solve a problem, they are likely to disengage from the activity<ref name="DMello2012"/>. | |||
=References= | =References= | ||
Revision as of 09:57, 8 July 2015
| Student affect and interaction behavior in ASSISTments | |
| Author(s) | Paul Inventado, Peter Scupelli |
|---|---|
| Data set(s) used | ASSISTments problem-student-level data, ASSISTments problem-level data |
| Usage in design patterns | %TITLE% %TITLE% %TITLE% %TITLE% %TITLE% |
Multiple studies were conducted on data collected from the ASSISTments system between September, 2012 to September, 2013. The goal of the analysis was to uncover relationships between content, student behavior, and student affect.
Student learning strategies
Online learning systems that host student exercises and activities sometimes provide feedback in the form of hints or additional explanations. Feedback can help students figure out the answer, but can also be used for gaming the system, which hurts learning [1]. Gaming is taking advantage of system feedback to get the answer without actually learning such as continuously requesting for hints or trying out all possible answers.
Such behavior was seen in student data collected from ASSISTments (i.e., quickly requesting for hints to get the answer). However, further analysis showed that some students seemed to game the system to get the answer for one problem, but no longer displayed such behavior in a similar problem. It is possible that some students actually want to display all the hints not just to get the answer, but learn how to arrive to that answer.
Problem content and student affect
Relationships between content delivery design and student affect were observed. These results could be used to guide preferable design decisions and designs to avoid.
External resources
An analysis of ASSISTments' problem level data showed a correlation between the has_text_color feature, and boredom and gaming. Majority of problems having text color contained references to problems on the textbook, but did not contain the actual content. This means that when students answered these problems, they had to switch between looking up the problem in their book and using the ASSISTments interface.
According to Sweller (2004[2]), unnecessary processing of information imposes a cognitive load that interferes with learning. High cognitive load impairs performance, which could increase the difficulty of a learning task. When students experience too much difficulty or get stuck in trying to solve a problem, they are likely to disengage from the activity[3]. Resulting disengagement behavior can explain the boredom and gaming behavior seen in the data.
Over-practice and student affect
Frustration is an emotion often experienced by students while learning. It is unclear whether it hinders learning or pushes students to exert more effort to learn (cf. Artino & Jones 2012[4]; Patrick et al. 1993[5]; Pekrun et al. 2002[6]). The role of frustration in learning with ASSISTments was investigated using both problem-student-level data and problem-level data that described student behavior while answering a series of questions in the system.
First, problems that likely led to student frustration were identified by ranking their average frustration confidence values. The most frustrating problems were investigated manually and revealed that they were usually difficult to understand and their hints were confusing. However, the data also showed that students were frustrated even when they answered questions correctly.
The problem level instances were investigated further by referring to the student-level data, which described students' low level actions while answering the problem (e.g., order of hint requests and answer attempts, answer correctness, time duration between actions). Students who were frustrated even when they got the answer correctly, were observed to have spent a significant amount of time answering the exercise and consistently getting the answers correctly. This could mean that students were frustrated at the type and sequence of problems they were asked to solve rather than the problem itself.
Skill level
An analysis of student behavior data showed that students reacted differently to the same problem. Students who were skilled in a certain topic rarely used hints and could quickly answer the problem. Students who were less skilled on the topic, often requested for hints, made multiple attempts to solve the problem, and spent more time before answering it correctly. Students skilled in one topic struggled with other topics, which they might not have learned as much. Without prior knowledge, students will have difficulty answering a problem[2].
Data showed that students were often bored or disengaged from the problem when they did not have enough skills to solve it (i.e., evidenced by answer correctness and number of hint requests). When students experience too much difficulty or get stuck in trying to solve a problem, they are likely to disengage from the activity[3].
References
- ↑ Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., and Koedinger, K. (2008). Why students engage in “gaming the system” behavior in interactive learning environments. Journal of Interactive Learning Research, 19(2):185-224.
- ↑ 2.0 2.1 Sweller, J. (2004). Instructional design consequences of an analogy between evolution by natural selection and human cognitive architecture. Instructional science, 32(1-2), 9-31.
- ↑ 3.0 3.1 D’Mello, S., & Graesser, A. (2012). Dynamics of affective states during complex learning. Learning and Instruction, 22(2), 145-157.
- ↑ Artino, A.R. and Jones, K.D. (2012). Exploring the complex relations between achievement emotions and self-regulated learning behaviors in online learning. The Internet and Higher Education 15, 3, 170-175.
- ↑ Patrick, B.C., Skinner, E.A. and Connell, J.P. (1993). What motivates children's behavior and emotion? Joint effects of perceived control and autonomy in the academic domain. Journal of Personality and social Psychology 65, 4, 781-791.
- ↑ Pekrun,R., Goetz, T., Titz, W. and Perry, R.P. (2002). Academic emotions in students' self-regulated learning and achievement: A program of qualitative and quantitative research. Educational psychologist 37, 2 (2002), Taylor & Francis, Philadelphia, PA, 91-105.