“To a child, there is no difference between [written, IV] words and pictures — they are one and the same.” [1] This quote by Paul Martin Lester really stuck by me: I’ve always made more sense of images than written words, even though I’m an active reader. I can imagine this might even be more the case for young children who “do not know the difference between a line drawing and a letter.” [2] Therefore I’m well aware of the importance of visual storytelling and would very much like to contribute to it.

When growing up you’ve probably come across a lot of storybooks. Parents and teachers have read them to you and as you grew older you’ve might even read them aloud to others or just enjoyed them by yourself. These books were based on a story that consisted of printed text and illustrations. However since the 1990s storybooks have become more and more interactive. Through technical advancement other forms of storybooks became widely available, such as e-books. In 1994 the first e-books “Just Grandma & Me” and “P. B. Bear’s Birthday Party” could be viewed on your computer through the use of a cd-rom.[3] This opened up a world of possibilities for interactive storytelling.

In this paper I’ll take a look at the developments and features used in interactive storytelling for children. I’m very interested in creating these kinds of stories myself; therefore I’d like to find out which features help in understanding the story and which ones should be used with caution. How can we make the most of interactive storytelling?

This paper is divided up in two parts. First, I’ll summarize the most important features, based on explorative reading on the subject. And second, I’ll take a look at how these features are used in two of the more popular storybooks according to Submarine, “pioneers in the Dutch (new) media world”. [4]

How to build an interactive story?

I believe the starting point in creating high quality storybooks for children, or any audience really, is a good and engaging storyline. When developing the story it’s important that the children can relate to it. As mentioned by Koshi Dhingra on creating the television series Blue’s Clues, “constructivist theories of learning science point to the importance of personal relevance and prior conceptions if students are to construct their own understandings (Osborne SiFreyberg, 1985; Bodner, 1986; among others).” [5] She describes the importance of combining existing knowledge with the curiosity to learn new things and placing them in a world that is recognizable to kids. Add interactivity to that mix and you can create an even more personal experience, which helps the understanding even more.

Someone who has been doing research on e-books for the past 30 years is Adriana Bus at the Leiden University. She’s published many interesting articles about interactive storytelling for learning purposes. Bus noticed that in the earlier days of e-books the use of interactivity was actually distracting children and even obstructing the story. She states that the combination of sound and image works really well for understanding a story since “…humans process visual and auditory information in separate channels”. However it is very important that these two support the story and each other for creating the best understanding. An example would be the use of text-highlighting words in sync with the narrated audio, to have words pronounced when clicked on or have a dictionary description appear when a word is unknown.[6]

“Research has shown that task switching is especially difficult for young children whose executive functions are immature…”[7] Therefore it’s really important to help focus their attention. Features such as games that are not part of the story can interfere with the story understanding. [8] Animating the most important elements can help children follow the storyline. “Motion in particular seems to direct children’s attention to important details of animated illustrations….”. However using “…purely ‘decorative’ or incidental animations…” such as the movement of background elements, are quite distracting from the storyline. [9]

Bus also states that providing options that you can switch on or off can make a story more accessible and enjoyable for different levels of readers. An example she mentions is the “…option for turning off background music and narration separately”, since more experienced learners could get distracted by additional information. [10] The on/ off switch would also work for the text-highlighting, pronunciation and dictionary features. Creating different options can offer a more enjoyable experience for different age groups. This way a storybook can be read multiple times and still be engaging whilst growing up. I think that would benefit a lot of the existing storybooks, as we will see in the following examples.

Interactive examples

I’ve read a lot about which features work and which ones should be used with caution. Now I’m going to put it to the test with two of the best interactive stories according to the Submarine Channel. [11]

millie-the-snail

Still from Millie the snail

The first story I’ve read was Millie the snail by the German Honig Studios. It won many prizes such as the Red Dot Design Award 2015 and the Cinekid Best Kids Concept. It’s a beautifully designed adventure in a cutout style about a tiny snail looking for it’s place in the world. It’s a calmly narrated story with enchanting background music. The narration can be turned on or off. There’s also a written story you can turn on or off, however this is quite a lot of text on a relatively small screen that does not move along with the narration.

In each shot there are two or three interactive elements. Some of these elements do not support the story and are a bit distracting, like apples that fall out of a tree or dandelion fluff that blows away when clicked. Each element has a sound effect when clicked, which are sometimes a bit loud and overrules the narration. Some of these elements however do support the storyline, for example the chameleon that blends in with different backgrounds you can select by swiping them whilst this trait is being described. There are three or four mini games evenly divided throughout the story. I don’t think they’re necessary but they are in line with the story. All in all it’s an enjoyable storybook that definitely benefits from the interactive elements.

wuwu-co2Still from Wuwu & Co

The second story I’ve read was Wuwu & Co from the Nordic company Step in books. Winner of Best Nordic Children’s game and many other awards, this app can be used in two ways. If you lay down your smartphone or tablet the story is being narrated and visible in a big static text block. However if you tilt your device upwards you can take a look around the imaginative world through augmented reality. In this world you can help finish the story by resolving a problem. One of my favorite moments was when you have to wake up a characters family members (see picture above) by saying something out loud. The audio then gets repeated by the creatures in funny voices. This way you’re actually becoming part of the story. The sketchy styled illustrations are mostly black and white. Color is used to indicate important areas or characters that are interactive or help guide the story. I think that really helps focus on the story since it’s quite a complex little world. I do miss a pause, next and previous button to help navigate the story. Guidance from a parent would be helpful, especially since there are multiple stories in one world. These could be mixed up very easily if rushed through the story. However I think Wuwu & co is a great storybook that really engages children in the story world through interactivity. Personally I think the style is a bit too scary for children, but that’s something I’d have to look into some more.

Concluding

I’m really excited about what I’ve read and seen so far. I can’t wait to take a look at other interactive storybooks and do some more research on the subject. I’m especially interested in more of Adriana Bus’ work. I’d love to see if there’s a possibility to combine her research and knowledge in a storybook of my own design. The features I would add to this would be:

  • On/off switches for audio elements such as narration, background music and sound effects.
  • On/off switches for visual elements such as: text highlighting that’s in sync with the narration, dictionary and pronunciation on click.
  • Clean and simple navigation by tilting a device (Wuwu) or swiping/clicking (Millie).
  • Focus attention by using colors, movement or sounds.
  • Really simple mini games that support the narrative such as searching for an element to continue the story.
  • Have the reader add a personal touch to the story in some way like the audio playback in Wuwu.

Two or three interactive elements per shot are more than enough to keep kids entertained and focused on the story. I think the most important thing is to keep the interactive elements simple and let the story be leading in the overall design. Interactive or not, it’s still a storybook.

[1] Lester P. M. (2006) Syntactic Theory of Visual Communication http://paulmartinlester.info/writings/viscomtheory.html [Accessed 4 November 2016].

[2] Lester P. M. (2006) Syntactic Theory of Visual Communication http://paulmartinlester.info/writings/viscomtheory.html [Accessed 4 November 2016].

[3] Bus A., Zsofia K. Takacs, Cornelia A.T. Kegel (2014) Affordances and Limitations of Electronic Storybooks for Young. Leiden University, Leiden, The Netherlands.

[4] Dicieanu M (2016) Top 5 Favorite Apps For Kids (under 6) http://www.submarinechannel.com/top5s/top-5-favorite-apps-kids-6/ [Accessed 4 November 2016].

[5] Dhingra K., Wilder A. (2001) Science on Television: Case Study of the Development of “Bugs”onBlue’s Clues. City University of New York, New York, U.S.A.

[6] Bus A., Zsofia K. Takacs, Cornelia A.T. Kegel (2014) Affordances and Limitations of Electronic Storybooks for Young. Leiden University, Leiden, The Netherlands.

[7] Bus A., Zsofia K. Takacs, Cornelia A.T. Kegel (2014) Affordances and Limitations of Electronic Storybooks for Young. Leiden University, Leiden, The Netherlands.

[8] Guernsey L., Levine M. H. (2016) Getting smarter about e-books for children.

[9] Bus A., Zsofia K. Takacs, Cornelia A.T. Kegel (2014) Affordances and Limitations of Electronic Storybooks for Young. Leiden University, Leiden, The Netherlands.

[10] Bus A., Zsofia K. Takacs, Cornelia A.T. Kegel (2014) Affordances and Limitations of Electronic Storybooks for Young. Leiden University, Leiden, The Netherlands.

[11] Dicieanu M (2016) Top 5 Favorite Apps For Kids (under 6) http://www.submarinechannel.com/top5s/top-5-favorite-apps-kids-6/ [Accessed 4 November 2016].