SIX YEARS OF design experiments using concept mapping - at
the beginning and at the end of each of 23 learning projects (the paper as it was really presented in the session)
Mauri Åhlberg and Vuokko
Ahoranta,
Email: mauri.ahlberg@helsinki.fi
Homepage: http://www.helsinki.fi/people/mauri.ahlberg
Abstract. Results of six year long period of field
testing of concept mapping are
described. There has been published very few long term series of concept
mapping experiments. Most publications report very short experiments. In our
research there were two three years long experimental periods. They were
partial replications. During each period the pupils were from grade levels 4 –
6 (10 – 12 years old). Two intact school classes were used. There were 20
pupils in both classes. Concept mapping clearly revealed that pupils learnt
during each learning project. Number of relevant concepts and number of
relevant propositions were used as indicators of meaningful learning. At both
times there were statistically significant differences between those indicators
at the beginning and at the end of learning projects. Pupils were happy to see
by themselves that they really learned during learning projects. It seemed to
increase their self-esteem and happiness with teaching-studying-learning
process. For teacher this was constructivism at its best. She could easily and
clearly monitor and promote her pupils metalearning and metathinking. Qualitative research results of the research
project include teacher’s observations, her tentative theory of benefits and
drawbacks of concept mapping after six years experimenting. In the preliminary
quantitative analysis reliability was estimated, and it was statistically
tested how much 1) shared teaching-studying-learning time, 2) sex and 3)
earlier school achievement levels account for variation of the two selected
indicators of meaningful learning.
1
Introduction
There are very few long term field experiments with concept mapping.
Most published articles concern only relatively short term experiments. We have
studies from 1997 to 2003 and even after that with design experiments in which
there is concept mapping at the beginning and at the end of the learning
project. Sometimes even in the middle of it. For most of the teachers, concept
mapping is an innovation. The second
author of this paper heard about concept mapping only a couple of days before
she decided to join research project and learnt to use concept maps. She had no
previous conceptions or theory of concept mapping. According to our observations, concept
mapping is often used only for a short experimenting period. This means that we cannot know about how things would develop in long-term use. We
have not seen any tentative theories of classroom teachers after they have used
concept mapping. But after six years of experimenting that kind of tentative
theory was constructed by an experimenting classroom teacher. Neither we have
seen any research on how much previous school achievement,
teaching-studying-learning time and sex account for variation of indicators of
meaningful learning.
There are different conceptions of what is concept mapping and
meaningful learning and its indicators. We have experimented with a version of
concept mapping developed by Ahlberg (1993 – 2005). He calls it an improved
method of concept mapping. He clearly admits that he builds on the work done by
Novak and his research groups in
We have following research
problems in mind:
1) After six years of
design experiments with concept mapping, what kind of theory teacher has
constructed about benefits and drawbacks of concept mapping?
2) How reliable is long
term concept mapping?
3) How much does following
variables account for variation of selected indicators of meaningful learning:
a) the shared teaching-studying-learning time, b) prior school achievement
level and c) sex of pupils?
2 Methods
Research design and subjects
There were two three years long periods of experimenting, together six
years of experimenting resulting totally 23 learning projects and over 46 pupil
made concept maps. Together 1395
individual concept maps have been collected and analyzed. For more
intensive study we have selected 9 (3+3+3) pupils of each threeyear period.
They were partial replicates to each other in many senses. Intensively we have
studied 414 concept maps.
|
School years |
Grade levels |
Age |
Number of learning
projects per school year |
|
1997 – 2000 |
4 - 6 |
10 - 13 |
5 |
|
2000 - 2003 |
4 - 6 |
10 - 13 |
3 |
The first school class had of 20 pupils. They started at the beginning
of their fourth grade, when they were 10 years old. The last concept mapping
experiments were done at the end of their sixth grade when they were 12-13
years old. During the school year 1997 – 1998 they had five learning projects
were individual concept maps were collected. The next school year 1998 – 1999
they had also five learning projects were concept mapping was used in the
similar way. During the third school year pupils had four learning projects in
which concept mapping was used at the beginning and at the end of the learning
projects. In the second intact school class there were also 20 pupils. During
three school years 2000 – 2003, they had three learning projects per year,
totally nine learning projects. Pupils constructed individual concept maps at
the beginning and at the end of each learning project. Individual Vee
heuristics were also constructed in 20 of the 23 learning projects, but we do
not analyze that data here. Examples of collected data are presented in the
figures 1 – 3.
School itself is an ordinary municipal comprehensive school in
Examples of collected data are presented in the figures 1 – 3.
Figure 1. The first concept map of the male pupil (code:
208) in the beginning of the design experiment of Atmosphere. The sum of
relevant concepts is 4. The sum of relevant propositions is three. ‘Atmosphere’
is the most central concept
Figure 2. The
last concept map of the male pupil (code: 208) in the end of the design
experiment of Atmosphere. The sum of relevant concepts is 11. The sum of
relevant propositions is 12. The most central concept is ‘phenomena’, because
it has more links (4 links) with other concepts than any other concept.
Figure 3. An
example of pupil (code 208) constructed Vee-heuristic from the design
experiment of Atmosphere.
Atmosphere: Transforming pupil writing /essay
or short answers) into a concept map
The
teacher-made short- answer questions/tasks were as follows: 1) Name the gases
the air is made of. 2) Explain how the wind is formed. 3) What do you know
about air pressure? 4) How the atmosphere is important for humankind? 5) How
rainbow is created? 6) How Northern lights are created? 7) What are uses of air
for humans? 8) What causes air pollution and how it can be prevented? 9) How is
thunderstorm created? 10) How is burning linked to earlier questions and their
answers? Its content validity is high, because these short questions correspond
the local curriculum, its main themes and big ideas.
We
transformed the answers of the pupils into concept maps. As an example the
answers of the pupil #208 are as follows:
1) Name the gases the air is made of.
Pupil #208 answers: “Oxygen, carbon dioxide, and hydrogen.”
2) 2) Explain how the wind is formed.
Pupil #208 answers: “Wind is created when war air is replaced by cool air.”
3) What do you know about air pressure? Pupil
#208 answers: “Air pressure can be measured in the small space.”
4) How the atmosphere is important for humankind?
Pupil #208 answers: “Life on earth needs gases. Atmosphere gives people
oxygen. It protects people from ultraviolet radiation.”
5) How rainbow is created? Pupil #208 answers: “Rainbow
has many colours. It is created when sun light goes through water drop. Rainbow is created when there are at the same
time both sunshine and rain.”
6) How Northern lights are created?
Pupil #208 answers: “Northern light can be seen in atmosphere. They are
formed when dust particles from Sun collide in the atmosphere. Northern lights
are seen in the Winter nights in the sky.”
7) What are uses of air for humans?
Pupil #208 answers: “Life on earth needs gases.”
8) What causes air pollution and how it can be prevented? Pupil #208
answers: “Gases cause pollution, which can be diminished by reducing
traffic. Gases cause pollution which can be diminished by filtering all pipes
of factories, which burn oil and peat.”
9) How is thunderstorm created? Pupil
#208 answers: “Thunderstorms occur in the hot day. They are created when ice
crystals touch each other in the clouds.”
10) How is burning linked to earlier
questions and their answers? “Clouds may be linked to lightning,
which is often directed to
In the
Figure 4 there is the concept map, which the inquiring teacher constructed
transforming each of the pupil #208’s short-answer test’s propositions into
concepts and propositions in the concept map. The appropriate hierarchy is
there, because the teacher as an adult created it. But on the other hand the
pupil himself created also appropriate hierarchies. There are 31 relevant
concepts and 32 relevant propositions. From the pupil’s propositions we can
easily see that he does not name Nitrogen as one of atmospheric gases. This is
a fatal mistake, because the most of the air is Nitrogen. Also in Biology
Nitrogen cycle is very important in ecosystems and production of amino acids
and proteins in organisms. A clear misconception is that there could be “dust
particles from Sun” and when these collide in the atmosphere Northern lights
are created. These misconceptions were discussed and corrected in the classroom
after the short answer tests were scored. When a pupil herself/himself
constructs a concept map s/he can only take into the concept map what is in
her/his metacognition. However all pupils were able to provide much more
information when explicitly asked in the teacher-made short-answer test. These
two methods to gather knowledge about what and how pupils learn and think are
complementary. Both provide useful knowledge for both pupils themselves and for
teachers.
Figure
4. An example of a concept map which the teacher
constructed transforming proposition by proposition pupil #208’s answers in the
teacher-made short-answer test of ‘Atmosphere’. The number of relevant concepts
is 31 and the number of relevant propositions is 30.
2
Results
Answer to
the first research problem: After six years of
design experiments with concept mapping, what kind of theory teacher has
constructed about benefits and drawbacks of concept mapping?
At the beginning of
the research project the teacher had no idea, no conception or tentative theory
of concept mapping. What is presented below, she has constructed since 1997.
The teacher’s tentative theory consists
of following propositions:
1. Concept
mapping is an excellent way of finding out what pupils have in their
metacognition about the theme of a learning project.
2. For
pupils it is important to know at the beginning of the learning project what do
they know about the theme of the learning project.
3. Teacher
can plan and implement her teaching according to what pupils know at the
beginning of the learning project.
4. Teacher
can group pupils in co-operative learning based on results of first concept
maps in the way that it facilitates knowledge building in the classroom.
5. Teacher
understands deeply, broadly and more clearly what kind of learner each of her
pupils is. Concept mapping is fastest and the most practical way I know to get
an overview of what pupils know both individually and as a group.
6. Concept
mapping is a good method for both teachers and pupils to avoid rote learning
and to learn and think meaningfully, and it promotes collaborative knowledge
building according to my observations.
7. Concept
maps are constructed individually in my classroom. Each pupil takes full
responsibility of her own learning and thinking. There are times when knowledge
is built collaboratively, and there are many kinds of collaborative learning,
but in the end concept maps are individual products.
8. When
pupils compare their concepts maps from the beginning of the learning project
and from the end of the learning project they see clearly how much they have
learnt. This supports their positive self image as a learner. They feel happy
and empowered. At the same time teacher becomes more empowered.
9. Concept
mapping is a concrete way to demonstrate pupils how much they know and learn
about themes that are studied at school.
10. Concept
mapping is a good tool for assessment of learning.
11. Misconceptions
are easily revealed by concept mapping.
12. Constructing
concept maps is often a flow experience for pupils. The have deep positive
feelings of their learning and their concept maps. They see constructivism in
practice: Increase of concepts and propositions in their thinking,
understanding and learning
13. Concept
mapping promotes meaningful learning. School learning is not any more rote
learning, but understanding broader wholes, its parts, and how they are
connected, and contexts.
14. Concept
mapping facilitates other kinds of written composition task.
15. Teacher
can monitor and promote better learning of individual pupils.
16. Constructing
concept maps is hard and energy demanding work. There is an optimal amount in
use of concept mapping. If it is used too often pupils become bored. The
situation is the same with all other methods of monitoring and promoting
learning
17. Concept
mapping is a method of learning to learn, which can be used any time when
needed.
18. Constructing
concept maps is hard work. Usually pupils do not ask themselves to construct
more concept maps. People mostly try to avoid hard intellectual work. But they
do understand importance of concept mapping from time to time.
19.
Most pupils learn quickly how to construct good concept maps. Some pupils need
more help and instructions.
20. Some
good pupils try to include all they know into their concept maps. Then teacher
has to teach her to select only the most relevant items. It is probably very
educative experience for those otherwise advanced pupils.
21. If
a pupil has very little or no prior knowledge of the theme, she may feel
herself helpless. On the other hand when learning project proceeds and she
clearly sees how much she has learnt then it does not matter that she did not
know it all at the beginning
22. Individual
concept maps can be complemented by constructing also from time to time
collaboratively or co-operatively constructed concept maps. It seems to promote
shared understanding and learning.
Answer to
the second research problem: How reliable is long
term concept mapping?
During the first research period of three
years (1997 – 2000) a total reliability estimate can be calculated from sums of
relevant concepts and relevant propositions.
The resulting Cronbach’s aplha is 0.88 if calculated from raw scores,
but from standardised scores the same estimate is 0.96. There is a very big
variation in the raw scores. The lowest sum of relevant concepts is only 62
and the highest one 210. This is why standardised score is probably the best
estimate.
During the second research period of three years
(2000 – 2003) a total reliability
estimate can be calculated from sums of relevant concepts and relevant
propositions. The resulting
Cronbach’s aplha is 0.93 if calculated from raw scores, but from standardised
scores the same estimate is 0.99. There is a very big variation in the raw
scores. The lowest sum of relevant concepts is only 49 and the highest one
395. Because of the huge variation of raw scores, standardised scores are
probably the best estimates.
The
reliability estimates are very high. If calculated separately for relevant
concepts on the other hand and relevant propositions on the other hand the
reliability estimates are a little bit lower (Tables 1. and 2.)
|
Learning projects during 1997 – 2000 |
Reliability estimates at the beginning of the
learning projects |
Reliability estimates at the end of the
learning projects |
|
Reliability of sums of relevant concepts
calculated from pupil’s concept maps. |
a
= 0.75 (a
= 0.72) |
a
= 0.87 (a = 0.89) |
|
Reliability of sums of relevant propositions
calculated from pupil’s concept maps. |
a
= 0.76 (a
= 0.73) |
a
= 0.85 (a
= 0.87) |
Table 1.
Reliability estimates separately for sums of relevant concepts and relevant
propositions at the beginning and at the end of the learning projects during
the three first years of research (1997 – 2000) The estimates are calculated
first from raw scores. In parentheses are the estimates calculated from
standardised scores.
|
Learning projects during 2000 – 2003 |
Reliability estimates at the beginning of the
learning projects |
Reliability estimates at the end of the
learning projects |
|
Reliability of sums of relevant concepts
calculated from pupil’s concept maps. |
a
= 0.87 (a = 0.88) |
a
= 0.90 (a = 0.93) |
|
Reliability of sums of relevant propositions
calculated from pupil’s concept maps. |
a
= 0.88 (a = 0.89) |
a
= 0.91 (a = 0.94) |
Table
2.
Reliability estimates separately for sums of relevant concepts and relevant
propositions at the beginning and at the end of the learning projects during
the second period of the research (2000 – 2003) The estimates are calculated
first from raw scores. In parentheses are the estimates calculated from
standardised scores.
Answer to
the third research problem: How much does
following variables account for variation of selected indicators of meaningful
learning: a) the shared teaching-studying-learning time, b) prior school achievement
level and c) sex of pupils?
How much
teaching-studying learning time accounts for variation of meaningful learning
was calculated as follows.
During
years 1997 – 2000 there was a statistically significant
increase in sums of relevant concepts from the beginning of learning projects (M = 85.44, SD = 17.10) to the end of learning projects (M =
145.44, SD = 44.93); paired samples t-test was used: t (8) = -5.732, p =
0.000. The eta squared statistic (.80) indicated a very large effect size. It
means that 80 % of difference of sums of relevant concepts are explained
statistically by the shared teaching-studying –learning time: from the first
concept map to the last concept map of each learning project.
During
the years 2000 – 2003 there was also a statistically
significant increase in sums of relevant propositions from the beginning of
learning projects (M =
83.33, SD = 17.20) to the end of
learning projects (M = 148.67, SD = 53.10), t (8) = -4.555, p = 0.002. The
eta squared statistic (.72) indicated a very large effect size. It means
that 72 % of variation of sums of relevant propositions are explained
statistically by the shared teaching-studying –learning time: from the first
concept map to the last concept map of each learning project.
b) How much
prior school achievement accounts for variation of meaningful learning was
calculated as follows. According to prior school achievement pupils were
selected and categorized into three groups, three pupils in each group: the
three pupils of low achievement (Group 1), the three pupils of average
achievement (Group 2) and the three
pupils of high achievement (Group 3). There were nine intensively studied
pupils. The one-way analysis of variance was conducted to explore the impact of
prior school achievement on variation of indicators of meaningful learning
(sums of relevant concepts and sums of relevant propositions). There were no statistically significant
differences found during the first research period of 1997 – 2000. It means
that prior school achievement do not statistically account for variation of
meaningful learning measured by concept mapping.
But during
the second research period (2000 – 2003) prior school achievement accounted for
variation of indicators of meaningful learning.
There were statistically significant differences in all four analyses:
Sums of relevant concepts at the beginning of the learning projects (F (2, 8) =
9.701, p = 0,013) and at the end of learning projects (F (2, 8) = 6.123, p =
0,036); and sums of relevant propositions at the beginning of the learning
projects (F (2, 8) = 9.412, p = 0,014)
and at the end of learning projects (F (2, 8) = 8.283, p = 0,019). The
effect size estimates (eta squared) varied from 0.67 – 0.76. These figures
indicated very large effect sizes.
There is no
good explanation why in the first three year period prior school achievement
did nor account for meaningful learning, but during the second three years
period it accounted for meaningful earning. This shows only that these kinds of
analyses ought to be done separately in each group. The main point is that all
pupils learn as indicated by concept mapping regardless of prior learning.
c) The
one-way analysis of variance was conducted to explore the impact of sex of
pupils on variation of indicators of meaningful learning (sums of relevant
concepts and sums of relevant propositions).
There were no statistically significant differences found during either
the first research period of 1997 – 2000 or the second research period 2000 -
2003. It means that sex of pupils do not statistically account for variation of
meaningful learning measured by concept mapping. This means that concept
mapping suits as well to both sexes. Both girls and boys learn meaningfully by
concept mapping.
3
Discussion
There are very few long term
research projects of any innovation. Concept mapping is not an exception. We
have described some of the main results of the six years long research and
development project where individual concept mapping was used both at the
beginning and at the end of each 23 learning projects. Also individual Vee
heuristics were used, but hat data is not presented in this paper. We found
that teacher developed a tentative theory, what does concept mapping mean in
everyday classroom use at grade levels 4 – 6 (pupils ages 10 – 12 years). We
found also that concept mapping is a sensitive tool to show also statistically
significantly that there occurs plenty of meaningful learning between the
beginning and the end of learning projects. That shared teaching-studying-learning
time accounted very much of the variation of the selected indicators of
meaningful learning, from 72 – 80 per cent. Prior school achievement did not
account for variation of meaningful learning in the first three research years
(1997 – 2000). But with the second group of pupils (2000 – 2003) it accounted
considerably. However there is no general regularity here. Sex of pupils did
not account for variation of indicators of meaningful learning. This is good
news, because it shows that both females and males can use concept maps to
promote meaningful learning as indicated by concept maps at the beginning and
at the end of each 23 learning projects.
This paper is based on the version of concept mapping which has been
developed by Ahlberg (1993 – 2005) and the empirical data presented in
Ahoranta’s (2004) doctoral dissertation.
4
References
Ahlberg, M. (1993). Concept maps, Vee diagrams
and Rhetorical Argumentation (RA) Analysis: Three educational theory-based
tools to facilitate meaningful learning. Paper presented at The Third International Seminar on
Misconceptions in Science and Mathematics. August 1- 5, 1993.
Ahlberg,
M. (1998). Education for sustainability, good environment and good life. In
Åhlberg, M. & Leal Filho, W. (Eds.)
Environmental Education for Sustainability: Good Environment, Good Life. Frankfurt am Main: Peter Lang, 25 – 43.
Ahlberg,
M. (2001). Concept mapping as a research method , www.metodix.com/showres.dll/en/metodit/methods/metodiartikkelit/kasitekartta_tutkimusmenetelmana/
Ahlberg, M. (2002). Translator’s postscript:
Twenty years research on theory of integrating education, improved concept maps
and Vee heuristics in
Ahlberg,
M., Aanismaa, P. & Dillon, P. 2005. Education
for sustainable living: Integrating theory, practice, design and development.
Accepted to be published in Scandinavian Journal of Educational Research
39(2).
Ahlberg,
M. & Ahoranta, V. (2002). Two improved educational theory based tools to
monitor and promote quality of geographical education and learning.
International Research in Geographical and Environmental Education 11(2), 119 –
137..
Ahlberg, M., Turja, L. & Robinson, J.
(2003). Educational research and development to promote sustainable development
in the city of
Ahoranta, V. (2004). Oppimisen laatu
peruskoulun vuosiluokilla 4-6 yleisdidaktiikan näkökulmasta käsitekarttojen ja
Vee-heuristiikkojen avulla tutkittuna [Quality
of learning monitored and promoted by concept mapping and Vee mapping in school
grades 4 – 6]. Doctoral dissertation.