“Abraham Lincoln’s Crossroads”:  Online Exhibit Evaluation, Beth A. Twiss-Garrity and M. Merritt Haine, The University of the Arts, March 31, 2006

Introduction

In December 2005, Night Kitchen Interactive and the National Constitution Center produced “Abraham Lincoln’s Crossroads,” an online exhibit designed to support the Center’s first traveling exhibit, Lincoln: The Constitution and the Civil War.  The exhibit, planned for upper level students and adults, allows online visitors to experience firsthand Lincoln’s fateful choices on issues such as the Civil War, the abolition of slavery, and civil liberties. After reviewing each major decision point, the visitor selects which course of action Lincoln should take – receiving feedback from Lincoln on whether or not he actually made that choice himself. To engage and encourage the visitor, the exhibit utilizes rich media that enhances the narrative structure of the exhibit, namely a talking, animated Lincoln.  A more traditional, accessible HTML version is also available for those who prefer it.

The learning objectives for the online exhibit are that students will learn:

• Facts:  The basic chronology of events leading up to and during the Civil War as experienced by President Lincoln, e.g. running for office, supplying Fort Sumter, issuing the Emancipation Proclamation.
• Cognitive Processes and Metacognition:  The decision-making process as reflected in the actions of historical figures, e.g. problem definition, choice evaluation, outcomes.
• Agency: That historical events, like the outbreak of the Civil War and the end of slavery, were not pre-determined.

These learnings support Pennsylvania academic standards by discussing the political contributions of apolitical leaders, demonstrating how continuity and change influence history, and by illustrating conflict and cooperation among groups involving domestic instability ace relations and military conflicts.

Research Goals

This evaluation implemented by the Museum Communication Program of The University of the Arts was requested by Night Kitchen to evaluate the effectiveness of online rich media and the interactive narrative format for transferring knowledge and holding student interest.  The research questions were:

• How do rich media virtual characters change the experience of online learning?
  • Is factual information better conveyed through reading information or having facts embedded in a rich media narrative?
  • Which form of delivery best sustains student attention?
• What are the psycho-social characteristics of learning with virtual characters?
  • What kinds of learning are enhanced through audience identification with virtual characters, their voices, and their stories?
• What is the role of narrative and dramatic reenactment in teaching history?

Methodology

In February 2006, “Abraham Lincoln’s Crossroads” was evaluated by fifty-three 11th graders at a suburban Philadelphia middle-class high school. Half (N=28) of the students were enrolled in an AP level history course that had studied the Civil War in December 2005.  The other half of the students (N=25) were in the Basic level of Social Studies and had not studied the Civil War since 8th grade.    Likewise, half of the students in the whole sample used the rich media version (N=28) and half used the low broadband version (N=25).  One Basic class used both, though only did the survey for the HTML version.

All students followed the same protocol.  They took a pre- and post-test with Survey Monkey.  In between tests, they had time to explore the on-line exhibit on their own and through directed discovery.  The surveys included multiple choice questions and Likert scale statements as well as two open-ended questions on the post-test.  Following the total survey period of about a half hour, a class discussion was held covering the main points.

The survey attempted to answer five main questions relating to cognitive and affective learning:

• Did students gain any new knowledge about the Civil War and Lincoln?
• Were psycho-social characteristics at play in the learning, due to the use of an animated Lincoln?
• What was the role of narrative in learning?
• Could students see that history is not pre-ordained but results from a series of decisions made by real people in the past?
• Did the use of multiple points of view and an ability to make choices themselves help students understand the content?

Results were tabulated to determine change between the pre- and post-test answers.  In addition, data was studied to see if there were differences between the AP and Basic students, and between those using the rich media versus HTML versions.

General Observations

Students in the AP classes, as might be expected, were more diligent in sticking to task.  They, however, were more reticent to give verbal feedback to the evaluator.  Because the students who used the HTML version did not have to wear headphones to hear audio, they were able to communicate with each other during the exhibit viewing experience, making the online exhibit experience more social.

Summary of results

Overall the students were positive about the idea of using an online exhibit with interactivity to learn about history. As many students said, “it was better than a book,” indicating that story-telling and the use of real historical figures intrigued them.  When asked to disagree or agree with the statement that they “enjoyed learning” from the on-line exhibit, all four classes agreed or strongly agreed. 

Chart I:  Was use of on-line exhibit enjoyable?

Students generally seemed to enjoy the chance to make decisions at the choice points, sometime cheering or moaning based upon the “correctness” of their answers. 

In general, based upon observation and class discussion, the rich media version also was better received than the HTML version, though it was not seen as perfect.  Students enjoyed the chance to make choices as Lincoln did.  While some students thought even more points-of-view from which to choose might be a good addition (42%), in discussion students disagreed on the amount of reading required to reach the choice point.  Some students wanted the ability to skip reading and go right to the choice point, while others argued that would defeat the learning that could be accomplished from the exhibit text.  For those who used the rich media version, they found a “talking head” attractive at getting their attention; holding power was diminished, however, by the slowness of the voice.  One student even claimed he would “rather watch a public TV show.”  

But did the students learn anything new about the Civil War or Lincoln?  Based upon the open ended questions and class discussions, the students learned more about Fort Sumter, the use of advisors, and the decision making process. They wanted to learn even more about decision making, biographical information on Lincoln, and the actual fighting of the War (see Chart II and III). Despite acknowledgment of the use of advisors, they found Lincoln to be decisive, not thoughtful as the exhibit creators had intended (56% for decisive versus 37% for thoughtful).

When these answers are studied more closely, the results are more positive for the students in the AP classes than in the Basic classes.  These students both were more diligent in use of the online exhibit and, according to the teacher, as students in general. These students also have more recently studied the Civil War.  Positive answers, however, are even more prominent for those students at either level who used the rich media version of the exhibit, indicating that the animated character may have influenced the ability to understand or retain new information.   In addition, the answers to the open-ended questions about what was learned and what else they would like to teach, the students who were in the Advanced Placement classes and the other students who used the rich media version had similar answers (see Charts IV and V).  These results indicate that for students with either less motivation or less prior knowledge, the use of the animated character is especially influential in the learning process.

Overall the online exhibit experience was successful in heightening the understanding that history was not pre-ordained.   Students did not think war was inevitable, they saw that decisions around the War were controversy and that Lincoln took advice to make his decisions  (see Chart IV).  Students in the Basic classes especially got this idea, but over half of all students commented that the use of advisors and the lack of pre-ordained decision was a new learning.  The use of the Flash versus the HTML version did not seem to make a difference.   The relevance of studying the Civil War was heightened for some AP students.

The version did make some difference in how students felt about the advisors and choice points  (see Chart V). The students who used the rich media version identified themselves more often as a member of Lincoln’s cabinet, whereas those who used the HTML version more often felt they were Lincoln’s friend.  Despite that difference by version, all other results were more determined by level of student class and/or recent Civil War study than by the version used. 

Students in the AP classes understood that the advisors were real historical people and that Lincoln used their advice in making decisions  (see Chart V). These students, however, cared less if their decisions matched the choices made by Lincoln, though in examining their choices they did usually make the “right” choice.   The students in the Basic classes who had not studied the Civil War in three years did not necessarily think the advisors were not real people nor that Lincoln used his advisors’ opinions in making his decision.  While these students also usually made the correct choices, they nonetheless reported they were much more disappointed if their choices did not match historical reality.

Conclusions

The use of an online exhibit can make learning more enjoyable for high school students of all academic levels, especially when students have a chance to make decisions based upon a variety of points-of-view.  While it does not necessarily increase their cognitive learning of facts, it vastly improves their understanding of historical process and interpretation.  Students learn that history is not pre-ordained, but that real people in the past weighed various points-of-view and facts to make a decision that had lasting consequences.  Students also generally see the use of an animated character and rich media as positive.  It not only makes the learning more fun but it also enhances cognitive and affective learning.  Due to their media-saturated short attention spans, students request even more interactivity and animation to enliven the study of history, making this online exhibit an exciting step in the right direction.  Narrative and dramatic reenactment in teaching history is effective. 

Additional Charts

Chart II:  What is one new thing you learned from “Abraham Lincoln’s Crossroads”?

	Total responses	Comparison of AP and Basic		Comparison of Rich media and HTML
		AP	Basic	Rich media	HTML
Took advice/used advisors	11	9	2	5	6
Emancipation Proclamation	3	3	0	1	2
Fort Sumter	9	7	2	6	3
Empathetic	1	1	0	1	0
Decision making	13	9	4	8	5
Biographical	2	2	0	1	1
Nothing	2	1	1	1	1
Reasons for not running against Douglas	1	1	0	0	1
Battle of Fort Knox	1	0	1	0	1
Risk taker	2	0	2	0	2
African Americans’ ability to fight	1	0	1	1	0
Freed the slaves	1	0	1	1	0
Good lawyer	2	0	2	2	0

Chart III:  “Abraham Lincoln’s Crossroads” left me wanting to learn more about the following:

	Total responses	Comparison of AP and Basic		Comparison of Rich media and HTML
		AP	Basic	Rich Media	HTML
Civil War	5	3	2	4	1
Battles	5	5	0	4	1
Relationships with advisors	6	6	0	2	4
Decisions	8	7	1	4	4
Draft riots	4	3	1	1	3
Fort Sumter	2	2	0	1	1
Biographical	11	5	6	6	5
Emancipation Proclamation	5	5	0	0	5
Societal thoughts	1	1	0	0	1
Nothing	4	0	4	2	2
Politics	2	0	2	0	2
Nation post-Lincoln	1	0	1	1	0

Chart IV:  Was war inevitable?

	AP Rich media	Basic Rich Media	AP HTML	Basic HTML
A war was the only way to end slavery. With the exception of the AP Rich media class, more than 50% of the students in the other classes disagreed with this statement, indicating that there were more possible outcomes in the conflict over slavery.
Strongly disagree	6%	14%	12%	18%
Disagree	6%	43%	44%	26%
Agree	25%	29%	38%	27%
Strongly Agree	62%	14%	6%	18%
The decisions that Lincoln made during his presidency were not controversial. The fact that the three-quarters of the students AP classes and the Basic Rich media class disagree with this statement indicates that understood Lincoln probably had choices – some which he made would be popular and some not.
Strongly disagree	69%	0%	69%	27%
Disagree	12%	71%	31%	18%
Agree	6%	29%	0%	27%
Strongly Agree	12%	0%	0%	27%
Lincoln took advice from his advisors. More than 50% of the students in all classes are in agreement; if there was room for Lincoln to take advice on issues, then outcomes probably were not pre-determined.
Strongly disagree	0%	0%	0%	0%
Disagree	19%	43%	6%	27%
Agree	56%	14%	62%	55%
Strongly Agree	25%	43%	31%	18%

Chart V:  Lincoln and Advisors

	AP Rich media	Basic Rich Media	AP HTML	Basic HTML
Lincoln took advice from his advisors. After looking at the online exhibit, more than 50% of the students in all classes recognized Lincoln consulted advisors.
Strongly disagree	0%	0%	0%	0%
Disagree	19%	43%	6%	27%
Agree	56%	14%	62%	55%
Strongly Agree	25%	43%	31%	18%
The advisors did not feel like real people. By disagreeing with the statement, the AP students especially realized that the online exhibit referenced real historical people.
Strongly disagree	38%	14%	44%	9%
Disagree	38%	43%	31%	55%
Agree	25%	43%	12%	9%
Strongly Agree	0%	0%	12%	27%
I was disappointed when my answers did not match Lincoln’s. As evidenced by observation, the students in the Basic classes cared much more at an emotional level if their answers matched Lincoln’s.
Strongly disagree	44%	0%	19%	27%
Disagree	50%	29%	38%	18%
Agree	6%	29%	25%	36%
Strongly Agree	0%	43%	19%	18%
How did Lincoln make decisions? The AP students learned that Lincoln did use advisors and there was a greater gain among students in Basic classes who used the Rich media version.
By himself	25%	28.6%	25%	27.3 %
Based on public opinion	0%	28.6%	0%	36.4%
By listening to advisors	75%	42.9%	75%	36.4%
Did you feel you were? The students who used the rich media version were more apt to feel as if they were a member of Lincoln’s cabinet helping to make decisions; the students who used the HTML version were more apt to feel like a close personal friend. One can speculate on which type of person would hold more sway over Lincoln, those obviously those in the cabinet could take political action.
A close friend of Lincoln’s	25%	14.3%	56.2%	45.5%
A member of his cabinet	37.5%	42.9%	25%	36.4%
Lincoln himself	12.5%	28.6%	12.5%	9.1%
An uninvolved observer	25%	14.3%	6.2%	9.1%