Strategy analysis of probability learning 1

Strategy analysis of probability learning

Balzs Aczl Psychology Institute, University of Etvs Lornd Budapest, Hungary

MA Thesis (Psychology Course)

2005/2006 Spring Term

Supervisor: Mihly Racsmny, PhD.

Strategy analysis of probability learning 2

Contents
I. II. II.1. Introduction The origins of probabilism in psychological research Egon Brunswik Brunswiks Lens model Brunswick on learning Multiple cue probability learning The expansion of Brunswick works 4 5 5 6 8 10 11 12 14 15 17 20 20 23 24 26 29 29 30 31 32 34

II.1.1. II.1.2. II.1.3. II.1.4. II.2. III. III.1. III.2. IV. IV.1. IV.2. IV.3. IV.4. V. V.1.

Estes and the probability matching New interest in probability learning Probabilistic associative learning Probabilistic classification learning Methodological considerations Base-rate neglect Strategy analysis of PCL The question of consciousness Analysing the test results The Dynamical approach Dynamic models of cognition Static and dynamic models of learning

V.1.1. V.2. VI.

Dynamical analyses Experiments

VI.1. Methods

Strategy analysis of probability learning 3 VI.1.1. Participant VI.1.2. Materials VI.1.3. Apparatus VI.1.4. Procedure VI.1.5. Data collection VII. Results VII.1. Experiment 1 VII.1.1. Hit rate VII.1.2. Rolling regression VII.1.3. State-space model VII.1.4. Test phase and explicit measures VII.2. Experiment 2 VII.2.1. Hit rate VII.2.2. Rolling regression VII.2.3. State-space model VII.2.4. Implicit, explicit measures VII.3. Summary of results VIII. Discussion VIII.1. Rationality under uncertainty VIII.2. The power of statistical learning VIII.3. From duality to multiple systems IX. Conclusion References Appendix 34 34 37 37 38 39 39 39 40 43 43 45 46 47 47 48 49 51 52 53 55 58 60

Strategy analysis of probability learning 4

God has afforded us only the twilight of probability; suitable, I presume, to that state of mediocrity and probationership he has been pleased to place us in here. John Locke, 1690

I. Introduction All of us spend our lives learning, yet our goal to understand its basic processes is still unaccomplished. Learning theory is a critical field of our psychological investigation because most human behaviour involves some form of learning (Robertson, 1970). Learning may be defined as a process in which behavioural patters are changed as the result of experience (Kelso, 1997). Learning is an adaptive process of our cognition to predict the future on the basis of past experience. In an uncertain world this prediction can rely only on probabilistic relations about the environment (Lagnado, Newell, Kahan, & Shanks, in press). Therefore to understand how people learn probabilistic information from experience is a fundamental question of human behaviour. In this explorative study I concentrate on four basic aspects of probability learning. First of all, during my discussion I rely on Brunswiks theoretical framework about the probabilistic nature of human psychology. Secondly, I wish to provide a critical review of the methodology used in the previous studies of probability learning. Thirdly, I am motivated to examine the role of conscious and unconscious processes behind the applied decisional strategies. Finally, I consider learning to be a dynamic process and find the dynamical approach and methodology to be relevant for the investigation. In the experiment I demonstrate how these aspects of probability, consciousness and dynamic processes play essential rule in probability learning. The research question intends to explore what decisional strategies we use in experiment-based probability learning situations. The modified experimental task is applicable to explore the used decisional strategies. The employed methodology provides useful tools to the field to investigate simultaneous processes underlying learning behaviour. The results may yield direct implications to the understanding of the well-known suboptimalities of human learning and decision making. The whole work may raise new questions about the interaction of the association- and rule-based processes behind human learning.
I greatly acknowledge Dnes Tths collaboration in the planning, execution and analysis of the experiments reported. Special thanks for the careful revisions to Tams Makny and for the linguistical check of the manuscript to James Wason from Cambridge.

Strategy analysis of probability learning 5 II. The origins of probabilism in psychological research Probabilism in psychological research started on its way with the argument of Egon Brunswick (e.g., 1939) to describe the relations between organisms and their environments in probabilistic terms. Brunswiks scientific work was consequently provocative. He believed that psychology needs a revolutionary turn to understand behaviour with regard of its function and in terms of probabilistic relations. However the impact of his work did not live up to his original aim for a change in the mainstream of the field, after all it is becoming apparent that his thoughts were basically right. In this chapter I wish to describe some of the views and works of Egon Brunswik, because his conceptions on perception, learning and experimental design serve as a theoretical background to the methodologies discussed in this work. I also give a short description about the contribution of William Estes who started an initial research to model probability learning in mathematical terms. Contemporary conceptions in probability learning studies originate directly from these two approaches.

II.1.

Egon Brunswik Brunswik, although born in Budapest, began his career in psychology as the first

assistant of Karl Bhler in Vienna (Doherty & Kurz, 1996). Under the auspices of Bhler, Brunswiks views turned against the popular psycho-physical parallelism, and this attitude determined his later theories. He started to state it in Vienna and continued in Berkeley that both the incoming perception and the outgoing behaviour have a rather ambiguous nature. In his view the probable partial causes and probable partial effects of the behaviour should be under focus when we wish to understand the great compatibility between organism and environment (Hammond, 2001). Brunswick intended to say in an evolutionary view that in natural environments survival is possible only if the organism is able to establish compensatory balance in the face of comparative chaos within the physical environment (Brunswik, 1943, p. 257). In that physics-envy time of psychology, his concept of probable behaviour in a somewhat unpredictable environment was in sharp contrast with the mainstream thinking that sought for stability in laws and research on behaviour. With his Lens model which intended to describe this compensatory balance of the organism in inherent uncertainty within the environment and within the person, he went against the dominating determinism of his time. Indeed, Brunswiks

Strategy analysis of probability learning 6 belief was that the probability character of the causal (partial cause-and-effect) relationship in the environment calls for a fundamental, all-inclusive shift in our methodological ideology regarding psychology (Brunswik, p. 261). These harsh words according to Hammond, Brunswiks student and follower established a distance between Brunswiks views and those of Hull, Lewin and many generations of future experimental psychologists that is still hard to overcome (Hammond, p. 56).

II.1.1.

Brunswiks Lens Model In the domain of perceptual constancy Brunswiks first research was conducted with

Lajos Kardos (Brunswik & Kardos, 1929) on Bhlers duplicity principle. This principle opposed the view that context has only a modifying effect on perception that comes into counts only after the object, instead, Bhler and his students stated that context is always present and never subordinate in perception (Brunswick, 1937). This reconceptualisation of context served as a crystallisation point for his ideas that led to his early lens analogy and his later generalised lens model (Cooksey, 2001). The analogy of the doubly convex lens is a heuristic tool what he conceived as a composite picture of the functional unit of behaviour, or the unit of achievement (Brunswik, 1952, p. 19-20). This tool was meant to help the researcher in structuring the investigation of organismic achievement. The explanation of the model contains another metaphor, the intuitive statistician, which depicts the perceptual system, being equipped with latent capacities capable for basic statistical functioning in the uncertainty of the environment (Brunswick, 1956, p. 80). The cues in the environment are only probabilistically related to the objective of the individual. In that sense, the decision maker has only probabilistic information about the environment and also about how to utilize these perceived cues. During judgemental processes the decision maker relies on these environmental cues to attain his/her goal. Not all of the cues may have equal relevance for predicting the outcome of the decision, but according to the view all of them go into account (cf. Brunswiks views on the context as additional mediating data (1937)). The decision maker uses his/her memory of cue-outcome correlations from previous experiences. In Brunswiks view, this statistical processing occurs involuntarily. Further principle to understand the Lens model is his principle of parallel concepts (Doherty & Kurz, 1996). This principle states that the perceived environment and the cognitive system should be described by the same type of

Strategy analysis of probability learning 7 constructs. These thoughts gave the basic theory for the construction of the Lens model. Brunswik (1952) thought that using correlation statistics these constructs become measurable.

Figure 1. Illustration of the Lens model. The objective weights (w) between the environment (YE) and the cues (X) and the subjective weights (j) between the cues and the judgement (YS) are parallel concepts. The functional achievement (ra) is the correlation between the persons judgement and the ecological criterion value. (based on Cooksey, 1996)

Thus, as Figure 1. depicts, ecological validity is defined as the correlation between the values of the distal criterion of the environment and the perceived cues; the cue utilization validity is defined as the correlation between the values of the perceived cues and the individuals judgements; finally, the achievement is measured by the correlation of the values of the distal criterion of the environment and the individuals judgements. Achievement, as the most general measurement of the model, reflects Brunswiks broadest descriptive term, the probabilistic functionalism. In this terminology, achievement is the degree to which the orgamism successfully attains its goals (Doherty & Kurz, 1996). This conception makes apparent distinction from most of the other traditions in psychology. Investigators of human behaviour almost exclusively define the correctness of human performance in comparison to some kind of normative models (mostly adapted from statistics or logic). Unlike the appliers of these coherence standards of assessments (e.g., investigators of heuristics and biases), those of using the Lens model look for

Strategy analysis of probability learning 8 correspondence in performance. In their words, successful performance (i.e. for achievement) [is] the degree to which a persons responses agree with, or correspond with, the environmental events (Doherty & Kurz, p. 123). The biggest contribution of the model is that it provided a general tool for those investigations of cognition where the focus is on any kind of judgemental processes under uncertainty. Brunswik had only a few demonstrations using the Lens model (e.g., 1952), but after his death, the approach, starting from Hammonds clinically oriented research initials (Hammond, 1955), evolved into the Social Judgement Theory, which reached many areas of investigation on judgement and decision making including educational decision making (e.g., Cooksey, 1988), medical decision making (e.g., Wigton, 1988), accounting (e.g. Krogstad, Ettenson, & Shanteau, 1984), or risk judgement (e.g., Earle & Cvetkovich, 1988). The other expansion of Brunswiks Lens model had enriching effect on the theories of learning, setting off the studies of multiple cue probability learning (described in Section II.1.3.).

II.1.2.

Brunswick on learning Brunswik dedicated two research studies to the investigation of probability learning. The

first work, the Probability as a determiner of rat behaviour (1939) was executed in 1936 and 1937, as his first experiment in the United States. This paper presents not only an excellent reflection of Brunswiks main attitude towards the psychology of his time, but the first experiment on probability learning in the literature as well. The design of the experiment followed the standard design of his age, the T-maze. The T-maze was a usual experimental setup of behaviourism, where the animal (usually a rat) was places in a two armed T-form maze. Food reward was exclusively conditioned to one side and never to the other side, thus training the animal to learn the expected behaviour. Brunswiks innovation was that he has altered the predictability of the sides for the running series. In every run there was food on only one side, but the location of the rewarded side was not consistent in each group. Brunswik calibrated the predictabilities of certainty for the groups following 100:0, 75:25, 67:33 ratios. In each case, the generally profitable side was counted as correct choice, the generally unprofitable side as error. In this sense, in the exceptional cases some of the successful choices were errors and some of the unsuccessful choices were correct responses. Further to study the effect of ambiguity, after 4 days of training Brunswik gave reversal trainings to the animals. Now the profitable and unprofitable sides were exchanged, however, keeping the previously set probabilities the same for

Strategy analysis of probability learning 9 all groups. In order to study more about the effect, he added repeated reversal training to the 100:0 group. In the following six consecutive days these animals had the same type of training except the directions were reversed in each day (Brunswik, 1939). The results in general showed that discrimination increases with the increase of the difference of reward probability on the two sides. The description of the design here served not as an introduction the results of the study, but to present the innovative conception in it. Leaving Thorndikes ideas alone on the probabilistic nature of the environment (Thorndike, 1932), most of the behaviourist studies steadily stuck to deterministic reinforcements in their designs. It was Brunswiks explicit goal with strong argument to reform psychology to accept the genuine uncertainty in observation and judgement into the model. It is worth to take notice of his argument that can be called passionate in scientific context. The title already hides an ironic paradox: probability as a determiner. Further, the first two sentence concisely sums up his overall conception stating In the natural environment of a living being, cues, means or pathways to a goal are usually neither absolutely reliable nor absolutely wrong. In most cases there is, objectively speaking, no perfect certainty that this or that will, or will not, lead to a certain end, but only a higher or lesser degree of probability. (Brunswik, 1939, p. 195). Brunswiks other study on probability learning was conducted with Hans Herma in 1951 titled as Probability learning of perceptual cues in the establishment of the weight illusion. This work applied probability learning to Brunswiks genuine interest, the perception. In a brief summary, in this perceptual experiment the participants had to lift heavy and light weight objects simultaneously in both hands. The objects were painted in two colours. Each participant was told that after some presentations of weights, he/she has to tell in a snap judgement which of the two objects appeared heavier at the first moment of the lifting. (Brunswik & Herma, 1951). Because of the successive weight contrast effect the participants underestimated the relatively heavier weight in the subsequent trial and vica-versa for the light weights. The estimated weight contrast was under focus, since, in the balanced weights test trials, the one presented on the side with generally lesser frequency of heavy objects is judged as the heavier of the pair (p. 174), thus showing the effect of probability learning. Thus, the location and the colour served as cues and the estimated weights as the dependant variable. The uncertainty of the environment was represented in the probability by which the multiple cues predicted the objects. The results showed that the contrast responses, after an early maximum, declined under continued reinforcement. This paradoxical

Strategy analysis of probability learning 10 result was speculated to be a special characteristic of probabilistic learning (Brunswik & Herma, 1951.) The interest of this work is not about its design or results, but about the questions that it aims to study of how rapidly the organism adapts the probabilistic structure of the environment; or if probability learning reaches a final level at which gets stabilized (Bjorkman, 2001). For our current research and understanding of probability learning the relevance of these questions is still unexpired. Although, learning was not closely related to Brunswiks original interest, in these works he presented probability learning and multi-cue probability learning experiments for the first time. The implications of this concept are detectable best in the various areas of the multiple cue probability learning paradigm.

II.1.3.

Multiple cue probability learning In a typical multiple cue probability learning situation (MCPL) participants make

judgements based on cues probabilistically associated to feedbacks in a series of trials. The aim of this experimental design is to model the organisms attempt to learn the relationship of the variables of the environment and to model its function of predicting the efficiency of behaviour. The probabilistic reinforcement of cues by feedbacks represents the conception of general uncertainty of real life environments and its perception as well. The description is in strike contrast with many of the traditional learning models where the degree of learning is tested and defined by the number of correct retrieval of items or of associations previously presented in deterministic pairing. The degree of learning in probability learning studies is assessed by the percentage of the correct decisions made on the basis of the previous experience on the task. The merit of this model is not just for its contribution to the learning theories, but also for its essential involvement in most of the cognitive processes ranging from perception, and categorization to decision making. In this section I wish to provide an insight to the evolvement of the research of MCPL to elucidate what attempts have led to the concepts and experimental trials of today.

Strategy analysis of probability learning 11 II.1.4. The expansion of Brunswick works Although, the debate goes on, if Brunswik and Herma used the term probability learning rightfully in their experiment published in 1951 (described in Section II.1.1.), or a present-day reader would make distinction calling it partial reinforcement (Bjorkman, 2001), some readers of his article understood the originality in the concept and started a series of experiments that triggered off an emerging of questions for the coming five decades. The first work after Brunswiks 1955 death applying multiple and single cues in probability learning was conducted by Smedslund (1955) in Norway. In his inquiry into the origins of perception assumed perception to be established by a process of multiple-probability learning, meaning that in learning people utilise complex configurations of ambiguous and probabilistic cues of the environment. One of his two experiments was to explore the possibility of utilising the probability learning procedure as a diagnostic tool in clinical psychology. He found the method to be slow and inefficient (Holoworth, 1999). Extensive research program was initiated only from 1964 by Hammond, Brunswiks follower, and his students in the United States (e.g., Hammond, Hursch, & Todd, 1964), they began to analyze the components of clinical inference explicitly in the framework of Brunswiks Lens model (sketched earlier in Section II.1.1.). The other main propagator of the approach was Bjrkman, who started a long research project in Sweden (e.g., Bjrkman, 1965, 1987) as well as his student Brehmer, who published 77 articles in the topic counting between 1972 and 1988 (Holoworth, 1999). Until the late 1980s a big proportion of the studies documented substantial learning effects, however not reaching optimal level (cf. Hammonds results), while Brehmer expressed his pessimistic conclusion, stating: When we learn from outcomes, it may, in fact, be almost impossible to discover that one really does not know anything. This is especially true when the concepts are very complex in the sense that each instance contains many dimensions. In this case, there are too many ways of explaining why a certain outcome occurred, and to explain away failures of predicting the correct outcome. Because of this, the need to change may not be apparent to us, and we may fail to learn that our rule is invalid, not only for particular cases but for the general case also. (Brehmer, 1980, p. 228-229). It seems that the interest in the MCPL approach dwindled considerably after the mid1980s, but in fact, the question only has merged into a parallel research tradition of probability learning, marked by the name of Estes.

Strategy analysis of probability learning 12 II.2. Estes and the phenomenon of probability matching Another theoretical origin of probability learning research can be found in the early articles of William Estes. In contrast to Brunswiks case, here the thought of probabilistic relations came from the inside circles of behaviourists. In the time of Estes in the 1930-40s, after the resultless attempts of the big search for the global theories of learning, the field was looking for a new direction. The orientation of that time was to suppose that all psychological phenomena could be understood in terms of some version of associationism, meaning that situational stimuli (Ss) control all behavioural responses (Rs). Estes wanted to keep this paradigm while involving probability into learning. He, as a good follower of Skinner, his mentor, in his 1950 article found a way to execute the task. First, he defined the response classes for the organism in a given situation. The advantage of having a closed set of responses is that the organisms behavioural state in a given situation can be fully characterized in terms of probabilities of its emitting each of the N response classes (Bower, 1994). In this way, learning can be defined as the increase of the probability of the correct responses alongside decreasing the competing alternative responses in a given situation. This concept made Estes to be able to construct differential equations to predict quantitative data of behavioural response. This statistical theory of learning brought about the mathematical models flourishing for the coming 25 years in the field. After describing some finite differential equations in trial-by-trial changes in learning (Estes & Burke, 1953) Estes, the former rat runner, started his first probability learning experiments with his students in the mid-1950s. Adapting the experimental situations from the earlier studies of Brunswick, the subjects had to predict which of the two possible outcomes will occur on each trial in the given situation (e.g., a light would appear on the left or the right). The events occurred in random sequence and only the base rate became available to help subjects to predict which event will show up in that trial. Thus the fixed probability was independent of the history of the outcomes and of the behaviour of the subject. The optimal strategy to maximize the expected utility would be to always choose the option which appeared with probability greater than one half. The striking feature of the results of the following experiments was that subjects matched the underlying probabilities of the two outcomes along their decisions. For instance the experimenter has programmed the lights to flash randomly, but the red light would flash 70% on the time and the blue 30% of the time. During the course of the experiment the participants most often will predict the red light roughly 70% of the time and the blue light roughly 30% of the time.

Strategy analysis of probability learning 13 This strategy is suboptimal, since they will predict correctly only 58% of the time (0.7x0.7+0.3x0.3=0.58), while predicting always the more likely light would bring 70% hit rate (1x0.7+0x0.3=0.70). The illogical sense of the results inconveniently surprised the researchers of the field. It is worth here to cite Kenneth Arrow economists comment from the time: We have here an experimental situation which is essentially of an economic nature in the sense of seeking to achieve a maximum of expected reward, and yet the individual does not in fact, at any point, even in a limit, reach the optimal behavior. I suggest that this result points out strongly the importance of learning theory, not only in the greater understanding of the dynamics of economic behavior, but even in suggesting that equilibria maybe be different from those that we have predicted in our usual theory. (Arrow, 1958, p. 14). Bush & Monstellers (1955) stochastic learning theory, as much as Estess (1957) probability matching theorem, did predict this kind of behaviour in linear difference equations. The researchers enthusiasm began to wane about the forms equations of the learning curves during the following decades. Estess colleague, Roger Shepard mentions three reasons explaining this turndown (1992). First, Estes and his fellows of that time based their theories on the assumption that the associative bonds come about through temporal contiguity between events, however with the cognitive approach it turned out that the predictive significance of events is a better explanative basis for this kind of learning (Rescorla & Wagner, 1972). Secondly, they realized only lately that what can be learned must be genetically internalized and thus the experiments have to set to be ecologically valid (Gibson, 1979). Finally, these mathematical derivations could not be employed expectedly for more complex classification tasks and with the availability of stored program computers the search for few general principles was waned and what started was a seemingly endless patching together of details, as doc heuristics explicitly engineered to accomplish various practical tasks(Shepard, p.420). Even if the equations of learning curves did not remain a central research topic after the 1950s, the previous robust findings that people use probability matching instead of normative optimal strategies in binary prediction tasks still startled researchers for further studies. Searching for explanation, a vast number of experiments tried to vary the parameters of the task in the coming decades. In general they found that matching decreased with size of the reward (Brackbill, Kappy, & Srarr, 1962; Siegel & Goldstein 1959) and with the number of trials (Edwards 1961). Shanks, Tunney, & McCarthy (2002), supporting rational choice theory showed that three factors

Strategy analysis of probability learning 14 contribute to reaching optimal response strategy. These factors are (1) large financial incentives; (2) meaningful and regular feedback; (3) extensive training. Rewards: Although, Friedman and Massaro (1998) failed to find evidence that monetary payoff effects performance, we can find documentation of this in the literature (Siegel & Goldstein, 1959; Vulkan, 2000). In general, even under monetary payoff, the asymptotic levels of responding rarely exceeded 95% correct choice (Vulkan, 2000). Although, Shanks et al. (2002) paid almost 40 in average to his participants, he could show only a slight positive effect on performance, yet the magnitude of payoff did not show correlation with the performance. Number of trials: Restle (1961) showed that that sequence effect may disappear after 1000 trials, Goodie and Fantino (1999) found gradual transition towards optimal responding over 1600 trials. Considering these data we should conclude that humans are slow learners, rather than rational strategists. These findings even seems to be inadaptable for real life, since, as Fantino (1998) noted [l]ife rarely offers 1,600 trials (p. 213). Feedback: At least from Thorndike (1898) we know that one is more likely to choose an option in the future, if he/she receives a positive feedback. Nevertheless, previous research has found that outcome feedback is quite limited in its usefulness, particularly in comparison to cognitive feedback (Balzer, Doherty, & OConner, 1989). Cognitive feedback refers to information about the relations between responses and outcomes (functional validity information), or a summary of the relations between responses and outcomes (task information). Shanks et al. (2002) found that individuals may be differentially sensitive to the motivating properties of feedback. Despite of these observations, we can tell in summary that previous researches paid little attention to the role of feedback and its effects on asymptotic levels of performance is far from being explained.

III.

New interest in probability learning Although from Brunswick up to the late 1980s 280 journal articles, book chapters,

doctoral dissertations and technical reports were published on MCPL, research on this approach of probability learning dwindled after the mid-1970s. The renewed interest in this topic started on its way again only in the late 1980s. This was the time when personal computers became available for psychologist experiments demanding complex computations, thus allowing developing models of

Strategy analysis of probability learning 15 connectionist networks. This was the movement that helped Brunswikian probabilism get adopted eventually into the classical schools of human learning studies.

III.1.

Probabilistic associative learning With the appearance of the cognitive approach animal and human learning research

started to follow separate routes. The researchers of animal learning continued to focus on elementary learning, while interest in human research shifted from learning to memory and from the classical models to the models of artificial intelligence. With the development of the connectionist networks after the mid-1980s the methodological apparatus became available to study the elementary aspects of human learning within the domain of cognitive psychology. Apart from some exceptions (e.g., Dickinson & Shanks, 1985) no studies have attempted directly to bridge results of animal experiments to human learning before Gluck and Bower (1988a). In their seminal papers (1988a,b), they developed an adaptive connectionist network to test the RescorlaWagner model of associative learning (Rescorla & Wagner, 1972) in human category learning. Gluck and Bower were looking for a learning model that involves probabilistic relation in its formulation. The Rescorla-Wagner model is based on Rescorlas previous demonstration (1968) that in the case of animal associative learning the level of the probability of a CS will vary the probability of the US in the presence of the CS compared to the US probability in the absence of the CS (Gluck & Bower, 1988a). Gluck and Bower found that the Rescorla-Wagner rule for association formulation can be regarded as a special case of the least-mean-squares (LMS) learning rule which was used in training the adaptive connectionist networks of that time. Attempting to evaluate the LMS rule as a component of human learning, Gluck and Bower (1988a,b) conducted a series of experiments to explore the accuracy of the model to probabilistic classification learning situations. They adapted the experimental task from Medins medical classification task (e.g., Medin, Altom, Edelson, & Freko, 1982). In this task in each trial, participants pretending medical diagnosticians - met one or more of the four symptoms of hypothetical patients in medical charts. They had to classify each patient as having one of the two fictitious diseases. After each trial they received feedback about the correct diagnosis. The combinations of the four cues (symptoms), unknown to the participants, were imperfectly, probabilistically associated with the feedbacks (diagnoses) (Figure 2.) following the scheme of the multiple-cue probability learning studies (e.g., Castellan, 1977). During the training, participants learnt the relationship of the

Strategy analysis of probability learning 16 symptom patterns with the diseases and at the end of the experiment they were asked directly to estimate the conditional probabilities of each symptom to the diseases.

Figure 2. Cue-outcome relations in the probabilistic association task (Gluck & Bower, 1988a). Similarly to Brunswiks Lens model (Figure 1.) objective weights (w) represent the relation between the environment and the cues.

The aim of the application of this design was to receive distinguishable predictions of their adaptive network from the three competing models of category learning (exemplar-, featurefrequency-, prototype model). The best fitting model could shed light on the question of to what extent we use similarity and base-rate (category probability) information in probabilistic categorization (Estes, Campbell, Hatsonpoulous, & Hurwitz, 1989). The exemplar (or context) model assumes that the learner stores all exemplar of each category and a new instance get categorised on the basis of its relative similarity to the stored exemplars (e.g., Nosofsky, Kruschke, & McKinley, 1992); the feature-frequency model presumes that the learner stores relative

frequencies of occurrence of cues within the categories and then classifies an instance according to the relative likelihood of its particular pattern of features arising from each of the categories (Gluck, Bower, 1988b) (e.g., Reeds, 1972); the prototype model assumes that the learner abstracts an average description of each category and then the new instance gets classified according to its similarity to this prototype (e.g., Matsuka, 2004). The adaptive network was an error-driven one-layer LMS network. To generate differential predictions of the LMS model and the alternative models, Gluck and Bower (1988a) had to unbalance the overall frequencies of the two diseases. This way, one of the diseases occurred more often than the other one. The results from the learning phase showed that the baserate information (the overall frequencies of the two diseases) were reflected in performance, as a

Strategy analysis of probability learning 17 form of probability match, however, when the participants were given explicit test trials at the end, they showed substantial base-rate neglect. As a result, it was found here and in several other experiments (e.g., Estes et al., 1989; Gluck & Bower, 1988b) that simple network model has stronger predictive value for these results than the alternative models. Recently, out of the 12 current models of category learning, the COVIS (competition between verbal and implicit systems) model is assumed as the best describer of probabilistic classification (Ashby, AlfonsoReese, Turken, & Waldron, 1998; Kri, 2003).

III.2.

Probabilistic classification learning More recently, four different kinds of category learning tasks are generally used: rule-

based tasks, information integration tasks, prototype distortion tasks, and the weather prediction task (Ashby & Maddox, 2005). The weather prediction task (WP) is a version of probabilistic classification learning task, developed by Knowlton, Squire and Glucks (1994). This test follows the structure of Gluck & Bowers (1988a) construction except participants play a weather forecaster. On the basis of one of the 14 combinations of four tarot cards (binary cues), participants had to predict rainy or sunny weather (binary outcome) (Figure 3.).

Figure 3. Probabilistic Classification Learning task. In this task people have to guess the weather on the basis of the presented combination of cards with geometric signs on them. (adapted from Aczel & Gonci, 2005)

Strategy analysis of probability learning 18 Outcomes associated to the patterns appeared with fixed probabilities, but in random distribution. Thus, the WP is substantially analogous to Gluck and Bowers (1988a) medical diagnosis task and to the MCPL tasks in a brunswikian sense, since the participants have to make judgements on the basis of the experienced relation between multiple-cues and a distal object, just as the Lens model describes. The standard analysis of the test proceeds by averaging correct responses in blocks of 10 trials and measuring the deviation of these mean values from chance level. A response is meant to be correct on a particular trial, if the outcome selected was more frequently associated with the given pattern than the other one in the course of the whole experiment (Knowlton & Squire, 1996). During the last decade, this task became extensively used in cognitive neuroscience (e.g., Eldridge, Masterman, & Knowlton, 2002; Knowlton et al., 1994, 1996; Poldrack, Clark, Par-Blageov, Shohamy, Creso Moyano et al., 2001; Reber, Knowlton, & Squire 1996; Reber & Squire, 1999). The exciting result that has started this interest in clinical research was Knowlton et al.s (1994) finding with an initiative trial of this task. They compared the performance amnesiacs with normal control on the WP task. The result was that the two groups performed equally well during the first 50 trials of learning, however, in the extended part of the training, the amnesiacs performance decreased relative to the healthy group (Knowlton et al., 1994). The impairment of the declarative memory in amnesiacs is often coupled with the finding of relatively sound nondeclarative learning (Milner, Corkin, & Teuber, 1968; Warrington & Weiskrantz, 1968). On the basis of this neurological observation, Knowlton et al. (1994) interpreted the results as the performance of the two groups being processed by non-declarative learning systems in the first part of the task, whereas in the late training the control people began memorising the test, what the amnesiacs could not (Knowlton et al., 1994; see also Gluck, Oliver, & Myers, 1996). In contrast, patients with Parkinson disease show learning deficit in the first 50 trials of the test which continues throughout the training (Knowlton et al., 1996). A learning patter similar to the amnesiacs was found with Alzheimer patients who were in the early stages of the disease. In both cases the anterograde amnesic symptoms is connected to neurodegenerative processes in the medial temporal lobes (Eldridge et al., 2002). Szabolcs Kri and his colleagues (Kri et al., 2000) examined schizophrenics with the WP task who, as well known, have abnormalities in executive function and explicit memory. The results showed normal performance for schizophrenics comparing to control. These results suggested that the WP task is processed by non-declarative neural systems.

Strategy analysis of probability learning 19 The key cortical area activations found generally correlating with probabilistic classification learning were the occipital cortex and the right nucleus caudatus (Kri, 2003). It is in correspondence with the observation that people with impaired basal ganglia have difficulties in implicit learning tasks. The correct responses correlate positively with caudate and prefrontal activation, however, the role of the prefrontal and parietal cortices in this task is not yet understood (Fera, Weickert, Goldberg, Tessitore, Hariri et al., 2005). The abnormal functioning of the basal ganglia is well documented with Tourette patients (e.g., Peterson, Leckman, Duncan, Ketzles, Riddle et al., 1994). In a clinical study, the WP task was used with children with Tourette syndrome (Kri, Szlobodnyik, Benedek, Janka, & Gdoros, 2002). The children exhibited impaired learning in the WP, however, in an explicit transfer version of the test they showed normal learning (Kri et al., 2002). Further study showed that transcranial direct current stimulation of the left prefrontal cortex could improve implicit learning in WP task in healthy people (Kincses, Antal, Nitsche, Bartafi, & Paulus, 2003). Interestingly, while healthy people show activation in the striatum with no activation in the MTL during learning in implicit motor sequence task, people with obsessive-compulsive disorder exhibited no activation in the striatum and activation in the MTL (Moody, Bookheimer, Vanek, & Knowlton, 2004). Poldrack, et al. (2001) intended to study this interaction of the basal ganglia and the medial temporal lobe (MTL) during probabilistic classification learning. The appealing finding showed that during the initial part of the WP task the MTL was active, while the caudate inactive, but very shortly the MTL became deactivated (presumably inhibited), whereas the caudate nucleus became activated. Poldrack et al. (2001) interpreted the results as the first substantive evidence of competition between memory systems. They supposed that in the initial part of the task the two systems (explicit vs. implicit) may compete and as it turns out that the task demands implicit processing the MTL becomes inhibited (Poldrack & Rodriguez, 2004). This result supports the view that the two systems are not dissociated imperviously, but are in constant interaction and are encoded by common factors (e.g., McDonald, Devan, & Hong, 2004; Turk-Browne, Yi, & Chun, 2006). The probabilistic associative learning (Gluck & Bower, 1988a,b) and the probabilistic classification learning (Knowlton et al., 1994) tasks were developed for examining concrete computational and clinical analysis, but their main contribution to the field is that they provide experimental and analysing methodology for probability learning studies. These initial examinations resulted sufficient data and experience to be able to reconsider the basic procedures and methodologies for further studies.

Strategy analysis of probability learning 20 IV. Methodological considerations It became apparent very early that probabilistic experiments bring about confusing results, if they are not measured with special attention. This section highlights the three main problematic points of the field. The base-rate neglect is a phenomenon that intrigued not just the researchers of learning, but it seems that this field has to most elaborated explanations to the issue. Strategy analysis of probability learning has no long history in the literature, but bears special interest for this explorative study. All of the methodological problems are entangled by our lack insight to the questions of consciousness in the processes.

IV.1.

Base-rate neglect Gluck and Bowers (1988a,b) observation that although the participants reflected the

experienced probabilities in their decisions in the training phase, they did not consider the base rate information in their decisions in the test part calls for special attention. This result may seem to be uninterpretable, still it fits well with the literature of the base rate fallacy. The dominating research on judgement and decision making in the 1970s and 1980s was concerned with the heuristics and biases paradigm (Koehler, 1993). This approach was developed by Daniel Kahneman and Amos Tversky (1972) who elaborate the attractive theory stating that peoples intuitive judgements about probabilistic events are made via erroneous error-prone heuristics. In their 1973 seminal paper, presenting empirical support to the view, they concluded that by [representativeness] heuristics, people predict the outcome that appears most representative of the evidence. Consequently, intuitive predictions are insensitive to the reliability of the evidence or to the prior probability of the outcome, in violation of the logic of statistical prediction. [] It is shown that [] people erroneously predict rare events and extreme values if these happen to be representative. (Kahneman & Tversky, 1973. p. 237). As evidence in support of this theory mounted, base rate fallacy (Bar-Hiller, 1980) became a favourably used example of the heuristics and bias paradigm. The results have implicated a common view about human judgement as genuinely biased and generally poor (e.g., Lopes, 1991). However, for the early 1990s, the preliminary converging evidence became apparent that the general base rate fallacy has been overstated previously, the base rates are not uniformly ignored (Koehler, 1994, 1996). The question arises: if base rates are not always ignored, when are they likely to be used? One can find

Strategy analysis of probability learning 21 two main streams of researchers answering the question. One emphasizes that relative frequencies can be better represented than single event probabilities (e.g., Gigerenzer, 1991; Hoffrage, Gigerenzer, Krauss, & Martignon, 2002), the other one seeks the answer in the structure of the tests (e.g., Koehler, 1996; Spellman, 1993). Gigerenzer and colleagues (e.g., 1995) forcefully argue that the dominating theory of heuristics is flawed, because representations in terms of natural frequencies better facilitate the usage of the probability (or frequency) than the given conditional probabilities (e.g., Hoffrage et al., 2002). This conception comes from the empirical works generated by ecological views (e.g., Gigerenzer, 1996). Natural frequencies originate from natural sampling (Gigerenzer & Hoffrage, 1995). Natural sampling is an automatic way of encountering statistical information from the natural environment (Hoffrage et al., 2002). Giving a concrete example for these concepts: Natural frequencies: Out of each 1000 patients, 40 are infected. Out of 40 infected patients, 30 will test positive. Out of 960 uninfected patients, 120 will also test positive. Normalized frequencies: Out of each 1000 patients, 40 are infected. Out of 1000 infected patients, 750 will test positive. Out of 1000 uninfected patients, 125 will also test positive. (Hoffrage et al, p. 346). Thus, probability is the normalised value of the natural frequency for one hundred. The authors point to this fact as a reason of the fallacy, since the computation is simpler if natural frequencies are provided rather then normalised frequencies or probabilities are given (Gigerenzer & Hoffrage, 1995). Koehler (1996) advocates the view that the fallacy is brought about by the way in which base rate summary statistics are provided in typical base rare tasks. He claims that people, given opportunity for implicit base-rate learning, will be more sensitive to probabilities and will show higher use of base rates in final judgements. It was published in previous studies that when base rates were directly experienced, through trial-by-trial feedback, they seemed to be used more accurately on judgements (Lindeman, Van Den Brink, & Hoogstraten, 1988; Manis, Dovalina, Avis, & Cardoze, 1980; Medin & Edelson, 1988), in contrast to the method of presenting mere summary statistics. Directly experiencing base rates is found to be helpful e.g. for auditors learning financial statement errors (Butt, 1988), or for physicians learning the relationship of base

Strategy analysis of probability learning 22 rates and diagnostic information (Christensen-Szalanski & Beach, 1982). Koehler (1996) assumes that directly experienced base rates may be better relied on implicit rather than explicit learning systems and that is why it is better remembered, or more easily accessed than information that is learned explicitly. He renders a reason for this stating: the implicit learning experience comes in the form of trial-by-trial learning, the information at each trial may be encoded as a separate "trace". In this way, multiple traces develop, and the information associated with these traces may be cognitively available. This contrasts with the explicit learning of a single summary statistic which does not produce multiple traces and which has been associated with less accurate judgments (Koehler, 1996). Other explanations support the view that personally experienced information is more vivid or salient, thus more available (Brekke & Borgida, 1988); or people are more trusting of self-generated base rates (Ungar & Sever, 1989). This conception is consistent with many observations from category learning literature, where the probability matching strategy indicates that people learn the experienced base rates and use them in their decisions (however not optimally), meanwhile, in the explicit test phase they show base rate fallacy (e.g., Estes, et al. 1989; Gluck & Bower, 1988a,b; Medin & Edelson, 1988). Holyoak and Spellman (1993) considered the phenomenon and suggested two components being behind the base rate usage. Acquisition (1), which, in a trial-by-trial format, is processed implicitly and quite accurately (perhaps based on learning conditional probabilities); and access (2), which (depending on the type of test) may be under explicit and conscious control. Consequently, when both acquisition and access part of the test tap the implicit system, people will apply better on base rates than one of the phases are not implicit (Spellman, 1993). In sum, we can conclude that the base rate literature holds substantial relevance to the study of probabilistic categorization. For a comprehensive understanding of the issue, we must consider the above described aspects both in experiment construction and data interpretation.

IV.2.

Strategy analysis of PCL In probabilistic classification learning people acquire information about cue-outcome

relations, therefore the category learning literature regards the PCL technically as an informationintegration task (Ashby & Maddox, 2005), however, little is known about, how people integrate the observed cues. As will be argued below, a variety of different strategies are all about equally effective to solve the task.

Strategy analysis of probability learning 23 The hypothesis is reasonable that the evolved animals and humans have to be optimal in categorization decisions (Ashby & Maddox, 1992). According to Thorndikes law of effect (1898), the probability of successful trials will increase with time. Still, robust deviations from this law are observed from the 1950s, one popular instantiation is the probability matching first studied by Estes (1950) (see in Section II.2.). Ashby and Gott (1988) examined the performance on many human categorization tasks comparing it to an optimal classifier (a hypothetical device maximizing reward, e.g., Morrison, 1990). The overall data showed that human classification cannot be described by optimality. Decision bound theory (Ashby & Townsend, 1986) attributed two inherent suboptimalities to human (and all other organisms) decision making processes. Both suboptimalities are the by-producs of the neural system (Maddox & Bohil, 1998). The perceptual noise comes from the spontaneous activity within the central nervous system, further, the optimality of the cognitive system is also limited by the observers memory (criterial noise). Deviation from optimality is observable in the strategy level of decision making as well. Suboptimal strategies are also often found in probability learning literature (Vulkan, 2000). The known deviations from optimality in human decision making still seek for a proper explanation, however, one can find at least three distinct effects attributed to the phenomena in the previous works. One set of explanations is classified as payoff variability effects (Busemeyer & Townsend, 1993; Haruvy & Erev, 2001). This theory claims that the increase in pay-off variability move choice behaviour towards random choice (Erev & Barron, 2005). The second set of explanations is classified as underweighting of rare events (Barron & Erev, 2003). In these situations people tend to rely on the typical outcomes that have the best pay-off. The third set of explanations involves loss aversion (Kahneman & Tversky, 1979). This counterproductive action is observed in the stock markets showing that people tend to avoid any loss. But, in probabilistic cases, the choice of the less probable alternative decreases the overall chance of maximal profit (e.g., Gneezy & Potters, 1997). These three cognitive strategies may seem to be reasonable from a point of view, but their negative by-product is the deviation from maximization of gain. Recently it became apparent that the WP task is solvable by a range of different strategies. Gluck, Shohamy and Myers (2002) presented techniques of post-hoc analyses by which they deduced that participants may use at least three different strategies. Post-experiment questionnaires uncovered that most the participants believed to use one of the following strategies: (1) optimal multi-cue strategy, in which they respond according to the outcome probability of each combination of the presented four cues; (2) one-cue strategy, in which they respond on the basis of

Strategy analysis of probability learning 24 their focus on the presence or absence of only one cue; and (3) singleton strategy, in which they focus and learn only about the patterns where there is only one cue present. Only the first strategy is optimal in a normative sense, but all of these strategies can raise the level of performance above chance level. Bases on these reports, Gluck et al. (2002) developed a strategy analysis method for the WP task describing ideal judgement profiles for each of these strategies. This method identified the applied individual learning strategies in two follow up experiments. The results showed that 90% of the participants in experiment 1 and 80% of the participants in experiment 2 fitted the singleton criterion, however, when they break the task into 50-trial blocks a gradual shift was observable towards multi-cue strategies (Gluck et al, 2002). Controversially, there was little correspondence between the explicit self-reports and the actually used strategies of the individuals. Recently Lagnado and colleagues (in press) introduced an alternative strategy to Glucks multi-cue strategy. The multi-match strategy supposes that people match the underlying probabilities of the two outcomes by their predictions. As we will see later, the adoption of these simple heuristics represents only a few of the plausible options that may be involved in probabilistic decision making situations. The fact that good performance can be achieved by explicit memorisation of heuristic strategies (e.g., one-cue strategy, or singleton strategy) makes the implicit nature of the test quite questionable. As it is argued in this study, beside finer strategy analysing methods, a better alternative of the WP task might prevent the method from being susceptible to indentifiability problems.

IV.3.

The question of consciousness Most of the previous studies of probabilistic classification learning assumed that the

decision makers lack self-insight into the judgemental policies underlying these judgements, therefore regarded the test as a pure implicit learning test (e.g., Evans, Clibbens, Cattini, Harris, & Dennis, 2003; Gluck et al. 2002; Wigton, 1996; York, Doherty, & Kamouri, 1987). This view was also supported by those who emphasized the implicit nature of the experience-based learning tasks (e.g., Spellman, 1993). Although the question is important, the relation between peoples learning performance and their knowledge has received less attention. The thought that learning can be based on separate conscious and non-conscious systems is detectable in some of the early theories of learning (e.g., Tolman, 1932), systematic research has not started on its way until the late 1960s (Reber, 1967, 1969). Rebers concept of

Strategy analysis of probability learning 25 implicit learning concerns that people acquire information from the environment without intending to do so (Cleeremans, Destrebecqz, & Boyer, 1998). This thesis made it possible to study unconscious phenomena in cognitive psychology without relying on any psychoanalytic conception. In a later paper, Reber (1992) taking an evolutionist standpoint argued that consciousness is a novel phenomena evolving after many higher perceptual and cognitive processes. He stated four hypotheses about implicit mechanisms (1) it is in robust relation with psychological and neurological effects; (2) it is independent of the IQ; (3) it is independent of age; and (4) it has little variance among populations. Others derive from this view that implicit learning acquire little effort, is often accurate and even optimised comparing to the explicit ways learning (e.g., Holyoak & Spellman, 1993). The PCL task is one of the few tests that can be used to reveal the rightfulness of these assumptions. A large body of research has documented apparent dissociation with the WP task and took the separation of the two learning systems as evidence (Ashby, Ell, & Waldron, 2003; Knowlton et al., 1996; Reber & Squire, 1999; Squire, 1994). The two systems were also demonstrated in neuropsychological studies arguing that the two systems are differentially impaired in the certain clinical cases (Ashby et al., 2003; Knowlton et al., 1996; Poldrack et al., 2001). Davis Shanks is one of the few researchers who questions the need of using any implicit concept in the explanation. In a voluminous paper with a colleague he (Shanks & St. John, 1994) reviewed the implicit learning literature according to their sensitivity criterion. This criterion claims that for terming a learning behaviour to be implicit, one has to rule out the insensitivity of the explicit test. Shanks & St. John (1994) argue that the explicit tests are possibly insensitive to measure the explicit processes that occurred during learning for two reasons. They suspect that the retrospective questionnaires can distort the validity of the assessment because of the memory constraint and the possible interferences. Taking this criterion serious, they did not find any previous studies where implicit learning was satisfyingly demonstrated. More than a decade later he and his colleagues pointed to other methodological shortcomings of the field (Lagnado et al., in press). First of all, they emphasized the need to distinguish between someones insight into the task (task knowledge), and someones insight into his/her own judgemental processes (self insight). In the case of the WP task, it refers to the difference between the learners knowledge of the cue-outcome relation, and to how to use this knowledge to predict the outcome. Lagnado et al. (in press) conjecture that the two might be

Strategy analysis of probability learning 26 separate, although, the previous researches ran faultfully the two together. Thus, the proclaimed dissociations could have referred to the dissociation between insight and learning, task knowledge and learning, or both (Lagnado et al., in press). Further problem of explicit testing was added to the list by the reasonable claim that verbalisation difficulties of probabilistic inferences might be a natural obstacle of a valid report. Recently, Lagnado et al. (in press) conducted a series experiments to demonstrate these claims. They used strongly and weakly predictive cards on the basic WP task. In Experiment 1, to measure the knowledge and insight, participants were given explicit questions after each block of 50 trials. One of the two different sets of questions asked the participants to rate on a continuous scale the probability of the outcome (rainy vs. sunny weather) in the case of each individual card (measuring task knowledge). The other question tested the participants of how much they had relied on each card in making their decisions (self-insight). This rating was also registered in a similarly continuous scale. The results indicated strong correspondence of the performance and both task knowledge and self-insight, being accurate in all. In Experiment 2 they asked similar explicit question from the participants on the screen of the same WP task. The difference was that the questions - of how much they relied on each card were presented to the participants after each trial. The results revealed that from early on in the task people rated strong cards more important than weak ones. The authors concluded that participants developed insight into their cue usage relatively early in the task. In Experiment 3 they tested whether the explicit questions after each trial directed conscious attention to the task, thus biasing the characteristics of the learning performance. In the final results there were no changes in any measurement of the third experiment comparing to the second one. These findings strongly supported the authors doubt that the WP is purely an implicit task. Lagnado and colleagues (in press) argument gains credence from these findings, but the appealing results supporting the existence of an implicit way of learning as well as the lack of the consensus in analysing the test results let us without satisfactory answer to the question.

IV.4.

Analysing the test results The standard analysis of the PCL task follows a simple procedure. It computes a mean

percentage of the correct responses for the whole task by averaging across both trials and participants. The aim of this analysis was mostly to develop a categorization models (e.g., Gluck

Strategy analysis of probability learning 27 & Bower, 1988a,b), or to study clinical groups (e.g., Knowlton et al., 1994, 1996). While the test became popular in these research fields, this analysing method does not tell much about the process of probability learning. The testing is insensitive about measuring both individual differences and dynamic processes along the test. Probabilistic classification learning tasks were developed from the multiple-cue probability learning paradigm, thus, although unacknowledged, its basic structure reflects the Brunswikian Lens model (for details see Section II.1.1.). This framework is applicable for every judgemental process where the decision maker has to rely on environmental cues. The original theory was based on the view that people construct internal cognitive models that reflect the probabilistic properties of the environment (Doherty & Kurz, 1996; Lagnado et al., in press). The central tenet of this approach was to analyse individual judgemental processes prior to computing group averages. Individual judgemental policies can be examined by considering the relation of the given cues and the patterns of judgement (for overview see Cooksey, 1996). More specifically, computing multiple regression analysis for the judgements and the cue values across all the trials of the task, we can measure the cue-utilization weights from the resultant beta-coefficients. Simply put, it shows that weights that the individuals have given to each cue during their judgements. Having the judges policy models, it can be compared to the actual structure of the environment. This is achieved by computing parallel multiple linear regression for the environmental cues. Here the beta coefficients will be the objective cue weights which are the same for all participants exposed to the same task. This way, the judgment policies (by their cue utilization weights) will be comparable with the objective weights. The analysis reveals eventually how the individuals learnt the task environment. The analysing method can be applied in a parallel way for the assessment of the explicit judgements as well, measuring the task knowledge and the self-insight. Brunswiks Lens model and its later developments provided a useful framework for studying judgemental processes, but its shortcomings hinder us from receiving a detailed picture of the dynamics of the process both from the aspect of the environment and the decision maker. Regading the WP task, the way of averaging performance across all trials not just ignores the possibility that one can vary his/her subjective weights over the trials, but overlooks the fact that the individual cannot obtain a representative picture about the probabilistic structure early on in the task. In fact, the observed probability of an outcome changes from trial to trial and reaches the final (given) value only after meeting the last feedback (so it is actually never

Strategy analysis of probability learning 28 measured). Consider the structure of the following Figure 4. It depicts the binary decisional tree in PCL. Going downwards from the top of the tree, one can observe the percentage of the normative expectancy of one of the feedbacks in each step.

Figure 4. Binary decisional tree in WP. Going downwards from the top of the tree, the numbers on the diagonal lines indicate if the correct response was set to be sun (1) or rain (0). The numbers (x) inside the rectangles represent the percentage of the normative expectancy of sun feedback in each given step (for rain the values are 100-x).

In practice the outcome patterns are (quasi-)randomly distributed (with fixed overall probabilities) in the WP experiments. It can be read from the illustration (Figure 4.) that regarding solely the final percentage values each participant might observe different probabilities on the previous trials. This makes the group averaging technique at least quite robust, if not inadequate.

Strategy analysis of probability learning 29 Furthermore, on the 14 trials, first presenting a particular pattern, participants have no previous knowledge about the outcomes at all, so their responses cannot be a result of learning. It is also plausible that the people have no perfect memory for all the observed stimuli, thus recency may have effect on the decisions. People in experience-based learning situations have to update their impressions according to the newly sampled outcomes (Hogarth & Einhorn, 1992). Recently presented outcomes may have greater weight than earlier ones (e.g., Hertwig, Barron, Weber, & Erev, 2004). Moreover, individual memory constraints may vary regarding the number of considered samples in decisions (e.g., Jones, Love, & Maddox, in press). In view of these facts the conclusion is inevitable that a more sensitive analysing method is required for a precise description of probability learning and a coherent framework for modelling the dynamics of the process.

V.

The Dynamical approach One more sensitive methodology for this analysis is provided by the dynamical

approach. Dynamical approach to cognitive science rejects the idea that cognition is the operation of a special mental computer located in the brain, rather it provides a framework for understanding cognitive processes within cognition as a complex natural system (van Gelder & Port, 1995). One of the two main tenets of the Dynamical Hypothesis (van Gelder, 1998) which distinguishes it from the traditional views of cognition is its primary focus on the processes in real time. Contrary to the computational aspects, the main aim of the approach is to describe behaviour in its temporal course. Instead of the input-output relation, its concern is about the changing of the overall system in time. In contrast to the computer storage analogy, followers of the dynamic view reinforce the common psychological view that we are not passive recipients of information rather that we actively manipulate, reconstruct and bias it (MacLeod, Uttl, & Ohta, 2005). The other key aspect of the approach is an emphasis on total state. Total state refers to the conjunction of all aspects of the system at a given point in time (Beer, 2000; Bosse, 2005).

V.1.

Dynamic Models of Cognition The approach appeared to be expedient for decision making theories. Busemeyer and

Townsend (1993) were one of the first ones publishing a useable dynamic model for high-level cognitive processing. The study provided a new aspect for understanding the relation of decisional

Strategy analysis of probability learning 30 models. They classified the decision making models according to two attributes: deterministic versus probabilistic and static versus dynamic (1993). This dynamic-cognitive decision field theory in contrast to the earlier dominating deterministic or static theories successfully accounts for many time-varying aspects of the phenomenon and can offer a more detailed processoriented explanation of motivational and cognitive mechanisms of decision making (Port, 2000). Before we exult over finding a revolutionary new alternative for the traditional theories of cognitive science, we should notice not only the long-established presence of its components, but the considerable limitations of its current applicability as well. The general framework was developed to study the biological and cognitive systems in their evolution, but unfortunately in many fields of this science there is not much realistic prospect for its empirical testing (French & Thomas, 2001). In most cases of cognition processing transitions are extremely rapid, the variables are exceedingly numerous and hardly detectable. Except for a few carefully constrained simple cases, we have insufficient amounts of data and inadequate method of computation compared to what these analyses would require (Port, 2000).

V.1.1.

Static and Dynamic Models of Learning Still, one of the areas where dynamic analyses may provide stronger descriptive and

predictive power in modelling cognitive processes is the modelling of learning processes. As discussed before, learning is a process in which we change the predictions about our environment on the basis of new experiences. This process is an evolving product of changing factors along the course of time (Smith et al., 2004). In typical experiments learning curves can be documented by recording responses (decisions) in multiple trial tasks. Multiple trial tasks provide us with continuous sampling of the changing phases of the process. In these studies (usually with binary responses) stimuli can be associated deterministically or probabilistically with reinforcement. Following Busemeyer and Townsends (1993) categorization of decision theories, here I propose an outlining classification of human learning models according to the deterministic versus probabilistic and static versus dynamic attributes, as in Table 1.

Strategy analysis of probability learning 31 Table 1 Categorization of Learning Models

Category Deterministic

Static Classical/ Operant Conditioning RescorlaWagner model

Dynamic Skill Learning Dynamic Probability Learning model

Probabilistic

Note. The matrix depicts and hypothetical categorization of learning models according to the deterministic-probabilistic and static-dynamic axes.

A model is static when it regards and measures learning not in its course of process as dynamic ones do , but as a property of the system in a certain point in time. Therefore traditional conditioning theories are in this block, but most standard tests also belong here. Regarding the other attribute, a model is deterministic, if the stimulus is always, or never associated with the response, while in probabilistic theories the stimuli are reinforced according to varying distribution where. In this sense the Rescorla-Wagner model (1972) is static, because it does not account on the changing factors from trial to trial. Considering these aspects the study of probability learning requires techniques of a dynamic learning model.

V.2.

Dynamical analyses Two techniques have been published recently to monitor learning behaviour over time.

The rolling regression analysis (or sequential least square technique) was introduced to illuminate individual behavioural differences in price forecasting (Kelley & Friedman, 2002). This method computes series of regressions by a moving window, generating trial-by-trial estimates about the individuals responsiveness to the observed cues. The learning curve is then compared with the curve that an ideal learner would show on the same task. The method regards the trial by trial information that the participant has actually observed. Thus, ideal learners strategies are defined according to the current state of knowledge of the ideal observer in each trial. This technique makes it possible to examine decisional attitudes individually along the course of the experiment and provides a tool to compare learning performance with an ideal learner of various strategies (e.g. Kitzis, Kelley, Berg, Massaro, & Friedman, 1998; Lagnado et al., in press). The other,

Strategy analysis of probability learning 32 explicitly dynamic statistical analysis of learning is the state-space model paradigm (Smith et al. 2004). This model computes the probability of correct responses for each state of the learning process by maximum likelihood applying expectation maximization algorithms. Knowing the learning curve and the confidence intervals permits us to identify the first trial on the curve where individuals performs better than chance level (Smith et al., 2004). This technique gives a precise definition of learning and a coherent statistical framework for learning studies with binary responses.

VI.

Experiments The present explorative study aims to analyse what decisional strategies are applied in

probabilistic learning situations. The question was examined with some modification on the standard PCL task. In addition, two novel statistical analyses rolling regression and state-space model were applied on the data. First of all, I used a new version of the usual PCL scheme (see Section III.1., III.2.). As it was reviewed in Section IV.2., a variety of different strategies are all about equally effective to solve the WP task. The ambiguity springs from the perceptual design of the four cues. In this task the four cues are four geometric forms of which combinations are the basis of judgements. As Gluck et al. (2002) demonstrated the application of the singleton strategy (in which the focus is only on the patterns where there is only one cue present) can also lead the participant to good performance than the multi-cue strategy (where all the cue combinations are considered). For example, if the participant associates every single triangle with rain and guesses in the other trials randomly, he may reach a good score on the whole experiment, however, no probability learning occurred. To prevent this complication I used a different PCL task than the WP. The experimental setup was adapted from Shohamy and colleagues (Shohamy et al., 2004) who constructed a design, which kept the basic structure of the WP, but used less detached cues. As described in details below, the participants had to make guesses on the basis of the features (cues) of a toy figure (hat, moustache, etc.) (Figure 6.). Another modification was that the sequence of the trials and feedbacks were set to be identical for each participant. As written in Section IV.2., the random distribution of the patterns (with fixed overall probability) limits the possibilities of individual comparison. With fixed pathways of the patterns in the decisional tree the order of the stimuli and outcomes became identical for all participants. This made the data usable for an aggregated trial-by-trial evaluation within the group and between the individuals.

Strategy analysis of probability learning 33 To measure the performance differences after different feedback pathways, different pathways belonged to some of the final probabilities. Figure 5. demonstrates that more pathways lead to the final value of 33.3, 66.7, 25, 75, 16.6, 83.3, and 50 of probabilities. In the case of some the pathways with common final point, one answer was more probably correct in the beginning, but later on the alternative answer became dominant, while in other cases one answer was dominant in the whole course of trials. The fixed pathway method also permits us to compare the learning differences at the same final point after different paths.

Figure 5. Feedback pathways. The 14 pathways following the arrows represent the feedback sequences of the 14 patterns. Numbers (x) in the rectangles indicate the percentage of the normative expectancy of vanilla feedback in each given step (for chocolate the values are 100-x)1.

Constructed by Dnes Tth.

Strategy analysis of probability learning 34 In practice the final value that represents the normative overall probability of a pattern can be measured only after the last feedback (about previous practice see Section IV.4.). Therefore, I inserted one more trial of each pattern (14) with no feedback after their last presenting. These extra trials provide us data to measure learning at the final point. In Section IV.1. we could see that experienced based learning is assumed to based on rather implicit then explicit mechanisms. The ignorance of this led previous studies to confusion about the explanation of learning processes along this task (see in Section IV.3.). A large body of researchers emphasized that the misinterpretation of data originates from the malpractice that the experimental learning was tested by explicit assessments. To avoid this, I added an implicit test phase following the learning phase. This phase was identical to the learning phase except, the participants did not get feedback after the trials, but were told that they receive their result at the end of the session. The advantage of this extra part is to test the subjects in a similar situation to the learning phase, and since there is not feedback, the observed probability remains unchanged, thus we can count mean scores from the sequences of test trials. To measure participants explicit task knowledge, the third part of the experiment was an overt test of what probability value they associate to each pattern. The explicit test had to follow the other tests not to interfere with them.

VI.1. Methods VI.1.1. Participants Fourty-five undergraduate students from the Psychology Institute of ELTE, Budapest participated in the present study (mean age = 23.46 years; SD = 3.77 years). There were 15 males and 30 females. The participants were divided into two groups: baseline and dual-task groups. Twenty-eight participant were in the baseline group (n = 28; 6 males and 22 females with a mean age of 22.00 years and a SD of 2.87 years). The dual-task group consisted of 17 participants (n = 17; 9 males and 8 females with a mean age of 25.88 years and a SD of 3.92 years). The participants received course credits.

VI.1.2. Materials The A modified version of the PCL I task (as introduced by Shohamy et al., 2004) was used in this study. In this version participants are told that they are selling ice cream in an ice

Strategy analysis of probability learning 35 cream shop and that customers will come in to buy vanilla or chocolate ice cream cones. Each time a customer visits, they have to try to guess whether the customer wants vanilla or chocolate. In this experiment all stimuli consisted of a MrPotatoHead toys figure (Figure 6.). The toy was created using a MrPotatoHead set (Silly Suitcase versions) (Playschool/Hasbro Inc., Pawtucket, RI, USA; item 2279). The toy figure was changeable by adding nose, ears, glasses etc. to the basic face. 14 different stimuli were created using the set. The figures were photographed using the same yellow background and black ground for all stimuli, changing only the 4 cues. Then the colour digital photographs were edited using Adobe Photoshop software to ensure that the image size (637 pixels high, 796 pixels wide) and memory (35Kb) were identical for all stimuli.

Tallt ! Figure 6. Stimulus (left side) and feedback (right side) examples from the experiment. When the stimulus appeared on the computer screen, the participants had to press the corresponding key to choose chocolate or vanilla. After each trial, participants received feedback indicating incorrect response, or correct response with a coin added to the image of the tip jar. The cues which changed were 1 = brown moustache, 2 = red hat, 3 = blue glasses, 4 = yellow arm. Fourteen stimuli were created using these four cues following the scheme of Gluck et al. (2002) as shown in Table 2. Each pattern was encoded to a numeric four-digit pattern. The binary numbers indicate whether the cue was present (1) on not (0) in a given pattern. The number of combinations was 14 out of the 16 possible combinations of the 4 cues. The patterns with all

Strategy analysis of probability learning 36 cues present (1111) and with no cue present (0000) were never used (following the scheme of the previous works since Knowlton et al., 1994). Table 2

Note. Each card could be present (1) or absent (0) for each pattern. The all-present (1111) and allabsent (0000) patterns were never used. The overall probability of vanilla outcome for all patterns is 50%.

Additionally, two digital photographs of vanilla and chocolate ice creams and an image of the tip jar that showed the extra tips were used on the screens. All the materials were presented on a computer screen with identical black background. Using the 14 patterns of stimuli, 214 trials were constructed for the learning phase and 70 for the test phase. For the explicit test 14 PowerPoint slides were created using the 14 patterns. During the learning phase the two feedbacks (vanilla, chocolate) were equally probable, but each pattern was independently associated to one of the feedbacks according to the scheme of Table 2. 200 of the 214 learning trials followed the pathways of Figure 5., the remaining 14 trials were not associated with feedback.

Strategy analysis of probability learning 37 VI.1.3. Apparatus The entire procedure was run on desktop personal computers using a software program written in Presentation (version 9.81). Two keys were used on the keyboards on the sides of the feedbacks for response and the Enter key to proceed to the next trial.

VI.1.4. Procedure The experiments were conducted in group sessions in a university lab. Participants were seated in front of the computers running the experimental software. Each of the participants read the general instructions after accepting the ethical consent form appearing on the screen (Appendix 1.). Before they read the instructions (Appendix 2.), the experimenter explained the task for the group. In the learning phase, they had to pretend that they are selling ice cream in an ice cream shop where customers will come in to buy vanilla or chocolate ice cream cones. Each time a customer visits, they had to try to guess whether the customer (a MrPotatoHead figure) wants vanilla or chocolate. When the stimulus appeared on the computer screen, they had to press the corresponding key to choose chocolate or vanilla. After each trial, participants received feedback about the correctness of their guess. If they guessed correctly, they earned a (symbolic) tip. To guess they had to use the two keys appointed to correspond to the two outcomes. The task was to collect as many tips as possible, however, it was emphasized to guess intuitively and not search for any logic in the task. After reading the instruction, the participants had an opportunity to raise questions before starting the test. On each of the 214 learning phase trials, the toy appeared on the screen with 300 ms delay after pressing the key. The toy remained on screen for 8000 ms. If the participants pressed one of the corresponding keys, the toy figure remained on the screen and one of the feedbacks appeared (for 2500 ms) indicating incorrect response, or correct response, in this later case a coin was added to the image of the tip jar after 700 ms. If the participant failed to press any key during the 8000 ms seconds, he/she was warning script (Appendix 2.) appeared for 3000 ms to respond faster. To proceed to the next trial, the participants had to press the Enter key, then the next stimulus appeared. The test phase followed the learning phase with a slightly different description. Here the instruction was the same, but the participants were informed that no feedback will appear, however, the eared tips will be counted and its sum will be shown at the end of the session. This

Strategy analysis of probability learning 38 part of the training consisted of 70 trials. Each pattern appeared five times in random distribution, but in fixed order for all participants. Other aspects of this phase were identical to the learning phase. After the test phase the participants were presented with each of the 14 patterns and they were asked verbally to estimate the percentage value by which the given pattern was associated with vanilla feedback during the task. There was another group in the experiment receiving the same 3 tasks and instructions, but this group received a secondary task added to the learning phase. The burdening task was to count backward by two from a given three digit odd number. They had to count out laud so that the experimenter could ensure that they follow the instruction. Each participant received a different three digit odd number between 801 and 999. During the test phase and the explicit phase they did not have dual task, the procedure was identical to the basic group.

VI.1.5. Data collection During each trial the software recorded the given pattern, participants response and the actual outcome. Since the actual feedback was previously defined, the optimal response does not necessarily match the actual feedback. Previous researchers following Knowlton et al. (1994; 1996) defined a response to be optimal, if it was the same as which was more often associated with the given feedback during the whole course of the experiment. For this reason two patterns, being equally often associated with each outcome were usually not considered in the analysis. In accordance with the aim of this study, I did not select the responses in this phase of the experiment, and involved all data to the analysis. Another practice in the prior studies (e.g., Gluck et al., 2002; Poldrack et al., 2001; Shohamy et al., 2004) was to exclude participants from the analysis who failed to reach the set criterion of 60% correct responses by the last block. For a thorough examination of the learning process and strategies I did not follow this practice and involved every participants data into the analysis, as long as the participant followed the instructions.

VII. Results VII.1. Experiment 1 In this experiment there were three successive phases: learning phase, test phase and explicit questionnaire. The results of the learning phase was analysed by hit rate analysis and two dynamical

Strategy analysis of probability learning 39 methods. The test phase and the explicit questionnaire were analysed by linear regression models and correlation analyses. VII.1.1. Hit Rate A fine-grained analysis of individual learning is possible, if we rate the actual predictions according to more strategies. Previous studies (Gluck et al., 2002; Jones et al., in press) assumed that more suboptimal strategies reach similar performance levels. The optimal multi-cue strategy was the pattern learner, who decides on the basis of all seen cues and patterns. The suboptimal strategies were: the singleton learner, who learns only about the patterns where there is only one cue present; and the single-cue learner, who responds on the basis of the presence, or absence of only one cue. Furthermore it is plausible for any strategy that the participant has memory constraint and makes decision on the basis of the recently met trials. Each strategy had a recency-variant, where the same strategy was constructed on the basis of the five last seen trials. Therefore, each individual prediction was evaluated by 12 strategies. The analysis was conducted in four blocks of (53, 53, 54, 54) trials. A 2x6x4 repeated measures ANOVA with memory constraint (with or without recency), strategy (multi-cue, singleton, single-cue1-4) and block (1-4) as within-subject factors was applied. A main effect of block was revealed block F(3, 81) = 0.568, p < 0.01, and a linear trend showing significant improvement across blocks F(1, 27) = 15.818, p < 0.01 despite of the severe decline in the fourth block (Figure 7.).

Figure 7. Learning performance measure by mean proportions of correct predictions. The decline in the final block can be the result of participants observing and overweighting the atypical outcomes appearing in that block.

Strategy analysis of probability learning 40 Strategy also had a significant main effect F(1.697, 45.816) = 16.489, p < 0.01. Planned contrast tests showed that participants reached significantly better performance by singleton strategy F(1,29) = 16.424, p < 0.001, multi-cue strategy F(1,27) = 22.192, p < 0.001 and cue4 single-cue strategy F(1,27) = 16.687, p < 0.001 then by any of the other single-cue strategies. Interaction was found between strategy and block F(7.793, 210.414) = 7.512, p < 0.01, and a three-way interaction was found between memory X strategy X block F(5.720, 154,446) = 4.717, p < 0.01. Figure 8. shows the percentage of participants for whom multi-cue, single-cue, and singleton strategies provided the best fit along the four blocks. The diagram depicts how the applied strategies play role in task solving. The results convincingly support the view that aggregated data analyses cannot provide sufficient information about how people solve the PCL task.
Cumulative percentage of participants (%) 50,00 40,00 30,00 20,00 10,00 0,00 1 2 3 4
multi-cue

1 2 3 4
singleton

1 2 3 4
cue4

Strategies by block

Figure 8. Best fit model for each of the four training blocks. The bars show the percentage of the participants for whom multi-cue, singleton and cue4 strategies provided the best hit rate.

VII.1.2. Rolling regression This technique was introduced by Kelley and Friedman (2002) for measuring learning performance in economic forecasting, it provides a dynamic model for both actual and ideal learners by their trial-by-trial comparison. This method computes series of regressions by a moving window, generating trial-by-trial estimates about the individuals responsiveness to the

Strategy analysis of probability learning 41 observed cues. In this study the regression analysis was computed with expanding windows where the starting point was fixed to the first trial and the window size expanded from 30 to 214 trial length. Moving window was not needed here since recency was involved to some of the strategies. Logistic regression is applicable for both individual learning profiles and an ideal learner of each examined strategy2. Thus, the estimates of the ideal learners show the best possible estimates of the objective cue weights along each trial of the test. The parameters of the constructed learning profiles were compared with the regression profiles of the ideal learners across all windows (the number of parameters was 4x185 by the 4 cue and 185 windows). Since all participants were exposed to the same stimulus distribution, each of them was compared with the same ideal learner. The comparison was performed by rank correlation analysis (Spearmans rank correlation rs) between the regression weights of the participant and the ideal strategy learner (e.g., Figure 9.). Further a mean rank correlation coefficient was averaged for each block.

Figure 9. Rolling regression. Learning curves of a participant (C.S.) in cue1 (dashed line) and cue4 (dotted line) strategies compared to an ideal learner (continuous line). The curves demonstrate that this person underestimated cue1, while followed cue4 along all trials. (adapted from Tth & Aczl, 2006)
2

Because of some deterministic relation in the task, the logistic regression did not give reliable results for cue2 and cue3 strategies, therefore no more analysis was performed for these strategies.

Strategy analysis of probability learning 42 A 2x3x4 repeated measures ANOVA with memory constraint (without/ with recency), strategy (multi-cue, single-cue1,4) and block (1-4) as within-subject factors revealed that the block main effect was significant F(1.529, 41.279) = 8.716, p < 0.05. The gradual learning of weights is reflected in the significant linear improvement across blocks F(1, 27) = 10.080, p < 0.01. It proves that participants improved on the task from block to block. I found no main effect of recency F(1, 27) = 0.246, n.s., which means that involving all the previous trials to the strategy did not bring better performance for the participants than the strategies with 5-trials recency. The strategies had no main effect F(1.223, 33.023) = 3.137, n.s., but the strategy-block interaction was significant F(3.358, 90.671) = 4.253, p < 0.05. The explanation of this interaction is that unlike in the case of cue4 and pattern learner strategies, in cue1 strategy the rank correlation coefficients did not increase with the blocks F(1,16) = 0.311, n.s. (Figure 10.).
Block Main Effect 0,50 Estimated marginal means 0,40 0,30 0,20 0,10 0,00 -0,10 -0,20 -0,30 Blocks Block1 Block2 Block3 Block4

Figure 10. Estimates means by blocks in rolling regression. Continuous line represents multi-cue strategy, dashed line represents cue4 strategy, dotted line indicates cue1 strategy. Cue1 strategy did not improve by blocks.

Another way of analyzing strategy learning becomes possible by utilizing the confidence intervals of the computed regression parameters. In this sense, we can define an applied strategy of

Strategy analysis of probability learning 43 a participant in each regression window, if the confidence intervals in regression weights regarding all the four cues overlap the ideal learners confidence intervals of weights. Adding the numbers of these windows in the whole sequence a new measurement for strategy following (strategy fit) was derived. A 2x3 repeated measures of ANOVA with memory constraint (without/ with recency) and strategy (multi-cue, single-cue1,4) fit numbers as within-subject factors. The results demonstrated that memory F(1, 27) = 15.332, p < 0.01 and strategy F(2, 54) = 19.566, p < 0.01 both had significant main effects. Furthermore, the strategy and memory interaction was significant F(1.383, 37.342) = 6.569, p < 0.01. The planned contrast comparison revealed that strategies with recency brought higher scores F(1,44) = 22.970, p < 0.001 and the multi-cue learning had the highest fit estimate F(1,44) = 73.695, p < 0.001 compared to the other strategies.

VII.1.3. State-space model This model (SSM) is another explicitly dynamical statistical analysis of learning (Smith et al., 2004). This method computes the probability of correct responses for each trial of the learning process by maximum likelihood applying expectation maximization algorithms. In the SSM analysis a response is counted to be correct in the given trial, if it equals to the choice of the ideal learner of the examined strategy. Unlike other standard analyses where the criterion of learning is based on performance in the whole experiment, here learning is counted, if the probability of the correct response is higher than chance level (0.5) with 95% confidence, in other words, if the confidence interval of the SSM learning curve is above 0.5. Summing the numbers of these learning trials in each block we receive an SSM learning fit number for each block by strategies. A repeated measures of ANOVA was computer on these fit numbers, where within subjects were memory constraint (without/ with recency), strategy (multi-cue, singleton, singlecue1-4) and block (1-4). The results showed significant main effect for memory F(1, 27) = 8.157, p < 0.01, strategy F(2.087, 56.357) = 15.450, p < 0.01 and block F(2.240, 60,475) = 21.356, p < 0.01. Significant interactions were found between memory and strategy F(2.795, 75.465) = 4.162, p < 0.05, block and strategy F(5.941, 160.413) = 8.370, p < 0.01. The interaction was also significant for memory, stragery and block F(6.374, 172.106) = 4.887, p < 0.01.

VII.1.4. Test phase and explicit measures

Strategy analysis of probability learning 44 After the learning phase participants were presented with the same ice cream seller task except they were informed that no feedback will appear, however, the eared tips will be counted and its sum will be shown at the end of the session. This part of the training consisted of 70 trials. Each pattern appeared five times in random distribution, but in fixed order for all participants. Other aspects of this phase were identical to the learning phase. The advantage of this extra part is to test the participants in a similar situation to the learning phase, and since there is not feedback, the observed probability remains unchanged, thus aggregated data may provide new information about the applied strategies. Here we can test how many of the participants maximize (always choose the outcome that is most probable) or follow probability match (matched the underlying probabilities along their decisions). The individual data were compared with two ideal response profiles. An ideal maximizer would always predict vanilla for a pattern in which that outcome appeared with >50% during the course of the learning phase, and would never predict vanilla for a pattern in which that overall appearance of vanilla was <50%. An ideal probability match follower would distribute his/her predictions for each pattern as the given pattern was associated with outcomes in the learning phase. Statistically speaking the question is whether the order of the mean prediction values of the patterns follow a linear trend or rather are closer to a dichotomous function (with 0 and 1 values). The least sum of squares (LSS) of the residuals for both profiles can give a fit estimate for both profiles. The mean of the LLS of the participants (n=27) for maximizer profile was 0,899, while for probability match profile it was 0,160. In the case of 24 (89%) participants the probability match value was stronger and only 3 participants had stronger LLS for the maximizer profile. These results tell that people follow the probability match strategy much stronger than the maximizer one. Figure 11. demonstrates the mean values of the groups for each pattern comparing to the ideal probability match and the maximizer profile. Here only those were involved to this averaging whose overall performance reached at least 55% (n = 13). The patterns are in increasing order according to their original association with vanilla. The curve follows ideal probability matching values and does not follow the optimal maximizer strategy.

Strategy analysis of probability learning 45

Percentage of vanilla response (%) ddd

100,00 90,00 80,00 70,00 60,00 50,00 40,00 30,00 20,00 10,00 0,00 3 1 5 7 11 2 6 Patterns 9 4 13 14 10 8 12

Figure 11. Percentages of vanilla responses of the ideal and the stronger learners for each pattern. The dashed line represents the ideal maximizer, the dotted line represents the ideal probability matcher, and the continuous line shows the mean percentage values of those who whose overall learning performance reached at least 55% (n = 13).

In the final, explicit task participants gave estimated percentage values about the association of vanilla outcome with each pattern during the task. Out of the 26 received score-lists 9 (35%) had significant positive correlation with the original values and 1 showed significant negative correlation. In comparison of the explicit values and the values of the test phase 7 (28%) positive and 1 negative significant correlations were found out of the 25 available pairs of lists.

VII.2. Experiment 2 In experiment 2 the group included 17 university students, 9 males, 8 females (mean age: 25,88, SD: 3,92). In the test phase the participants were presented with the same task as in experiment 1, except they received a burdening secondary task. Each of them had to count out loud backward starting from different three digit odd numbers by two in group sessions. The test phase and the explicit questionnaire was the same for the two groups.

Strategy analysis of probability learning 46 VII.2.1. Hit rate A repeated measures ANOVA with memory constraint (without/ with recency), strategy (multi-cue, singleton, single-cue1-4) and block (1-4) as within-subject factors was applied for the hit rates. The results showed a reliable main effect of block F(3, 48) = 3.839, p < 0.05. The significance of the linearity of the block effect F(1, 15) = 7.963, p < 0.05 indicate that learning gradually improved in this group. Memory F(1, 16) = 18.135, p < 0.01 and strategy F(2.448, 39.174) = 6.186, p < 0.01 also had a main effect. Paired comparison revealed that singleton F(1,16) = 6.652, p < 0.05 and multi-cue strategies F(1,16) = 5.031, p < 0.05 with memory constraint both were more effective than the single-cue strategies. It is of special interest that the group as between-subject showed no significance F(1,43) = 0.004, n.s.. This allows us to analyse the two groups together. The repeated measures of ANOVA showed significant effect of block F(3, 129) = 16.973, p < 0.01 with linearity F(1, 43) = 17.251, p < 0.01. Memory F(1, 43) = 13.361, p < 0.01 and strategy F(2.030, 87.290) = 19.286, p < 0.01 presented a very similar significant main effect for the two groups together. The paired comparison showed that the singleton strategy F(1,44) = 35.996, p < 0.001 and the multi-cue strategy F(1,44) = 31.605, p < 0.001 resulted best performance (Figure 12.).
Learning Performance 75,00 Mean proportion correct 70,00 65,00 60,00 55,00 50,00 Block1 Block2 Blocks Block3 Block4

Figure 12. Learning performance measure by mean proportions of correct predictions. The dashed curve represent the singleton strategy, the continuous line represent the multi-cue strategy.

Strategy analysis of probability learning 47 VII.2.2. Rolling regression Applying the same repeated measures of ANOVA for the regression weights of this group revealed that the block main effect was significant F(1.737, 27.797) = 4.097, p < 0.05 with significant linear improvement across blocks F(1, 16) = 2.877, p < 0.05, indicating learning. Further memory main effect was found to be significant F(1, 16) = 7.654, p < 0.05. The analysis of the strategy fit numbers brought reliability to recency F(1, 16) = 12.007, p < 0.05, and strategy F(2, 32) = 16.902, p < 0.05 main effects, while their interaction was also significant F(2, 32) = 0.552, p < 0.01. Comparing the two groups, the test of between-subjects presented no significant effect of group F(1,43) = 1.214, n.s.. It shows similar result as the hit rate analysis. Analysing the two groups together, I found reliable block main effect F(1.635, 71.946) = 6.274, p < 0.01 where the linear improvement was also significant across blocks F(1, 44) = 15.732, p < 0.01. Strategy had F(1.306, 57.445) = 6.204, p < 0,05, but memory had not F(1, 44) = 0.875, n.s. main effect. The between subjects test failed to fine significant difference between the two groups on the basis of strategy fit values F(1, 43) = 57.572, p < 0.01. A joined analysis indicated memory F(1, 44) = 22.970, p < 0.01 and strategy F(2, 88) = 27.482, p < 0.01 main effect, further the interaction of strategy and memory F(1.406, 61.844) = 6.774, p < 0.01 was also significant. A paired comparison explained that memory constrained multiple-cue strategy had the best (significant) fit F(1,44) = 6.981, p < 0.05.

VII.2.3. State-space model A repeated measures of ANOVA was computer on these fit numbers, where within subjects were memory constraint (without/ with recency), strategy (multi-cue, singleton, singlecue1-4) and block (1-4) similarly to Experiment 1. Here block main effect was not significantly F(2.102, 33.633) = 2.529, n.s. (Figure 13.), while memory F(1, 16) = 10.916, p < 0.01 and strategy F(2.934, 46.941) = 3.635, p < 0.05 main effects show significant indices.

Strategy analysis of probability learning 48

Block Main Effect 14,00 Estimated marginal means 12,00 10,00 8,00 6,00 4,00 2,00 0,00 Block1 Block2 Blocks Block3 Block4

Figure 13. Estimated means by blocks in the state-space model. The result that block main effect was not found may come from the severe decline by the fourth block.

Comparing the two groups again, no significant difference was found between the two groups in a between-participants test of variances F(1, 44) = 0.043, n.s.. The paired comparison revealed that the multi-cue strategy always had better fit than other strategies F(1, 44) = 23.269, p < 0.01.

VII.2.4. Implicit, explicit measures The mean of the LLS of the participants (n=15) for maximizer profile was 1.016, while for probability match profile it was 0,134. In the case of 14 (93%) participants the probability match value was stronger and only 1 participant had stronger LLS for the maximizer profile. Similarly to experiment 1 these results also tell that people follow the probability match strategy much stronger than the maximizer one. Figure 14. depicts the probability matching behaviour compared to the ideal probability matcher. Here the two groups were joined, but only those were involved to this averaging whose overall performance reached at least 55% (n = 19). The patterns are in increasing order according to their original association with vanilla. The curve follows ideal

Strategy analysis of probability learning 49 probability matching values and shows deviation at around 50% where the uncertainty is the highest.

Probability Match 100,00 Percentage of vanilla responses 90,00 80,00 70,00 60,00 50,00 40,00 30,00 20,00 10,00 0,00 3 1 5 7 11 2 6 9 Patterns 4 13 14 10 8 12

Figure 14. Percentages of vanilla responses of the ideal and the stronger learners for each pattern. The continuous line represents the ideal probability matcher, the dashed line shows the mean percentage values of those who whose overall learning performance reached at least 55% (n = 19).

In the explicit task out of the 16 received score-lists none of them reached a reliable correlation level. Furthermore, in comparison to the values of the test phase no significant correlation was found. These results demonstrate that although the group of Experiment 2 showed similar matching behaviour, could not reflect its knowledge into explicit reports.

VII.3. Summary of results In this explorative study the data were collected from a learning phase, a test phase and an explicit report of probability learning. The results of a baseline group were contrasted with the results of a dual-task group. Summing up the findings, I can conclude that each applied assessment showed gradual learning during the task. The degree of pattern learning was found to be a leading strategy in all methods, which is much higher than previous studies would predict. Interestingly

Strategy analysis of probability learning 50 cue4 strategy was also highly effective, while the appearance frequency of the cue4 and the probability of its outcome does not differ from those of cue1. Regarding other aspects cue4 strategy we can find a reason for its strong advantage. Patterns containing cue4 is more frequently associated to chocolate (6 out of 7), whereas cue1 was less predictively associated vanilla through its patterns (5 out of 7). The higher predictivity power of cue4 (Figure 15.) can explain the better effectivity of cue4 strategy. The main implication of these results is that even in the case of cue learning strategy, participants consider the patterns, therefore people are able to follow multiple strategies in the same time.
1 0,9 0,8
Predictive power

0,7 0,6 0,5 0,4 0,3 0,2 0,1 0 1 15 29 43 57 71 85 99 113 127 141 155 169 183 197
Trials

cue1 cue2 cue3 cue4

Figure 15. Predictivity distribution of the four cues. The lines indicate the percentage of the normative expectancy of vanilla feedback in each given step (for chocolate the values are 1-x). In accord with the results, Cue4 and Cue1 show greater deviation from the uncertainty line of 0.5.

The coherence of the three analyzing methods is supported by the fact that with the exception of the singleton strategy the power rankings of the strategies are the same in all cases. Singleton strategy showed a good fit to the participants responses according to the hit rate analysis, whereas in the SSM analysis it was found to play a marginal role. This difference may be accounted for the differing concept of learning of the two methods. The SSM analysis is in fact very sensitive to the order of the correct responses, since the probability of the hits is expected to

Strategy analysis of probability learning 51 increase in time, as we consider a learning process to be dynamical. In contrast, in the case of hit rate analysis only the number of correct responses is measured in a given block, but the sequence of correct and incorrect responses does not receive attention. It is to be emphasized that the assumption of memory constraint does not reveal significant differences in the case of pattern learning and cue4 learning. However, the assumption of memory constraint receives support from other results showing that the performance in the fourth block decreases compared to the third block. This decline is observed in other studies as well (e.g., Fera et al., 2005; Knowlton et al. 1996), however, with less significant degree. In our case this effect may come from the feature of our fixed sequences of patterns where the atypical outcomes occurred mostly in later appearances of the patterns (see Figure 5.), which is true consequently for cue4 as well. Therefore the decline of performance in the final block can be the result of participants observing and overweighting the changings of the outcome rates. On the basis of these results it is to be considered that, if the order of consistent and inconsistent outcomes has an influence on the learning performance, then the application of random sequences may lead to differences in individual comparisons. It is of special interest that there was no significant difference between the performance of the two groups, the heavy secondary task had no burdening effect on performance neither on the learning phase, nor on the test phase. Since the dual task was supposed to burden the central executive, the performance in this task has to depend on another processing system. The results of the test phase support the claim that people tend to match the experienced probability distributions along their decisions. The number of maximizers was minimal. In this sense we can assume that people show a good learning ability in probability tasks, however, this is not reflected in optimal decisions. The explicit test phase presented that despite no observable difference was found between the two groups during the tasks, it seems that the secondary task hindered the participants to be able to report the observed probability distributions in an explicit way.

VIII. Discussion The analysis of the test phase confirmed the previous findings that people tend to use probability match strategies in experience-based decisional situations. This strategy is considered to be suboptimal and counter-intuitive (e.g., Vulkan, 2000). In this chapter I consider the question of optimality in probability learning behaviour. I argue that human decisional strategies are

Strategy analysis of probability learning 52 perhaps not that suboptimal and learning abilities are not that unpowerful as it is often assumed. At the end of this chapter I discuss the role of interactive multiple memory systems behind probability learning.

VIII.1. Rationality under uncertainty Early theories of rationality assumed that the laws of human thinking and inference are equivalent to the laws of probability and logic (Peterson & Beach, 1967). They followed old philosophical views supposing that logic is the physics of thoughts (Chase Hertwig, & Gigerenzer, 1998). This conception still determines our everyday beliefs that human behaviour should be reduced to some basic laws similar to those of physics. In the last 30 years a substantial amount of psychological research challenged this view. Proponents of the heuristic-and-biases approach argued that it is the exception rather than the rule that human inference and decision making follow classical norms. According to this theory, human thinking is systematically biased. We apply not logical inferences, but heuristics that rely on simple characteristics of the environment. The observed suboptimal strategies in probabilistic situations led some to the conclusion that our mind was not created to work by the rules of probability (Gould, 1992). The idea of bounded rationality originates from Herbert Simon (cf., Simon, 1990) who proposed that human behaviour can be described only by reference to environmental and cognitive constraints. These constraints limit the available information and the complexity of information processing (e.g., Simon, 1990). Some researchers believe that human cognition is basically adaptive and bounded rationality is a constrained optimality (e.g., Anderson, 1990). In this sense, under the given limited information and capacity, human cognition is optimal. Other researchers deny that humans are fundamentally irrational (e.g., Gigerenzer, 1996). Rather they claim that from unrepresentative designs of the environment we cannot conclude general statements. They have shown that when people are tested in representative samples the violation of rational norms disappears (e.g., Juslin, 1994). They argue that classical definitions of rationality disregard the content and context of probability problems and have unrealistic demands on complexity and computation of the mind (Chase et al., 1998). Followers of the view of ecological rationality go further and claim that fast a frugal heuristics can perform almost as well as normative algorithms (Cosmides & Tooby, 1997). In ecologically appropriate situations these heuristics draw smart inferences even on limited information. People presented with natural frequencies outperform those who receive the same

Strategy analysis of probability learning 53 problem in probabilistic terms. Percentages are relatively recent inventions, therefore they are not a natural way of representing numerical information. Three of the aforementioned approaches provide a logic to interpret the inherent suboptimalities in human behaviour. The heuristic-and-biases approach argues directly that human inference departs from logical norms. According to the bounded rationality theory, cognition is optimal within limited information and capacity. Followers of the ecological rationality consider the problems to be inadequate unless we examine them in ancestral environments. One can easily notice the substantial relevance of Brunswiks conceptions to the issue. Genuine uncertainty in the relations of the environment and genuine uncertainty in the judgements of the observer accord with Simons external (environmental) and internal (mental) constraints. The Lens model gives a framework to understand the organisms functioning amongst these imperfect relations. Brunswiks functionalism, similarly to the ecological rationality emphasizes the contextual validity of the situation. His theories were grounded on the notion that humans are intuitive statisticians (1955), but the question of how good statisticians we are did not get elaborated in his works. Present review and the results of this study would suggest that the cognitive system is able to subtract the statistical features of the environment from the massive amount of sensory input. From phenomena, such as probability match it seems that we acquire complex information from the environment, however, we use it not in accordance with our normative expectations. It would indicate that learning ability is not as constrained as it was previously assumed. The robust observation of the capacity limitation of short-term memory suggests that learning has to have another way of processing.

VIII.2. The power of statistical learning The recent decade has witnessed an increasing interest in the understanding the actual learning mechanisms involved during infancy. Most of the studies examined if language learning is a special ability, or it is based on a general domain. The rapidity, efficiency and ease with which infants master language have prompted the view flourish that it cannot occur without a languagespecific learning device (e.g., Pinker, 1997). Recently, a counter-argument is getting stronger that the ability to exploit complex statistical information is not exclusive to language learning. An initial report of statistical learning demonstrated that 8-month-old children can learn subtle statistical relationships across nonsense syllable sequences (Saffran, Aslin, & Newport,

Strategy analysis of probability learning 54 1996). In another research, 8-month-olds were found to detect transitional probabilities in nonlinguistic tone sequences (Saffran, Johnson, Aslin, & Newport, 1999). These reports received considerable attention, because the results can indicate that statistical learning is not a purely linguistic mechanism, but may be based on a general purpose learning device (Kirkham, Slemmer, & Johnson, 2002). Research projects started to explore whether statistical learning is powerful enough to avoid the postulation of abstract rules or whether this special learning ability is limited to auditory information. Following the first demonstrations, several findings reported supporting evidence. In a study adults showed the same learning performance for tone sequences that they have showed for syllable sequences (Saffran et al., 1999). Further examinations found statistical learning in lexical stress acquisition (Christiansen, Allen, & Seidenberg, 1998; Christiansen & Curtin, 1999) and in multiple-cue integration in language acquisition (Christiansen, Conway, & Curtin, in press). Connectionist simulations (Christiansen & Chater, 1999) also suggest that statistical learning devices are sufficiently powerful in contrast with other claims that language acquisition can only be explained by algebraic rules (Marcus, 2001; Marcus, Vijayan, Rao, & Vishton, 1999). Researches in the visual modality substituted syllables to object shapes in the experiment (e.g., Fiser & Aslin, 2002), or created complex visual scenes from the spatial arrangement of shapes (e.g., Fiser & Aslin, 2001) and found the same statistical learning mechanisms used. Others familiarised 2, 5, and 8 month-olds with series of 6 discrete colour shapes of which order was defined by statistical regularities (Kirkham et al., 2002). The infants showed preference to the novel sequences that violated the transitional probability which they observed in a previous training. It was particularly interesting that there was no significant difference among the age groups. Using a probabilistic version of the serial reaction time task (Nissen & Bullemer, 1987) Hunt and Aslin (2001) found that learners are capable to exploit the relative predictability of the sequence units from more than one source of statistical information in the same time. Statistical learning can also operate over musical tones (Creel, Newport, & Aslin, 2004). Interestingly, cotton-top tamarin monkeys could acquire the transitional probabilities exposed to the same artificial-language streams as infants (Hauser, Newport, & Aslin, 2001; Newport, Hauser, Spaepen, & Aslin, 2004) showing cross-species commonalities in mechanisms of word segmentation. Nevertheless, there were some differences between children and monkeys, this result suggests that statistical learning is not unique to humans.

Strategy analysis of probability learning 55 At present our knowledge is still little concerning what mechanisms accomplish such computations in statistical learning. However, researchers have some cues and assumptions about its nature. Most of them believe that these mechanisms may be rather primitive, operating associatively, however outside of potential contributors, such as reinforcement (Kirkham et al., 2002). In several regards, statistical learning corresponds with what we know about implicit learning. It is often advocated that statistical learning is automatic, occurs without intent or awareness (e.g., Turk-Browne, Jung, & Scholl, 2005). Little direct study was devoted for the question of automaticity in statistical learning, still there are plausible reasons supporting this view. First of all, in these experiments participants received no particular orienting instruction to learn anything about the stimulus sequences (e.g., Saffran, Newport, Aslin, Tunick, & Barrueco, 1997), and reported to be completely unaware of the underlying statistical structure (e.g., Fiser & Aslin, 2002), while showing learning along their judgements. These experimental circumstances and the facil and robust learning motivated researchers to label statistical learning to be incidental (Saffran et al., 1997), spontaneous (Fiser & Aslin, 2001), byproduct of mere exposure (Saffran et al., 1999, p. 30) and automatic (Turk-Browne et al., 2005). The fact that human infants and nonhuman primates perform similarly to adults also support the view that statistical learning is based on implicit systems. Irrespective of their denomination, there seems to be no particular reason to suppose that probability learning and statistical learning are different phenomena, instead both refer to the exploitation of probabilistic relations from the environment. Furthermore, their descriptions categorise them to be alike a way of implicit learning.

VIII.3. From duality to multiple systems Since the early priming experiences (e.g., Milner, 1968) it became evident that memory processes cannot depend on one unique system. As we could see above (Section III.2.2.), the probabilistic classification learning task was used in several experiences emphasizing dissociation of neural systems underlying category learning. Originating from these observations several theories started to propagate duality in human cognition. This dichotomous conception advocates strongly that the two systems are independent and control different domains of behaviour. The systems are often called declarative/non-declarative (Squire & Zola-Morgan, 1988), or explicit/implicit (Graf & Schacter, 1985), mostly relating to the presence or absence of

Strategy analysis of probability learning 56 consciousness. Lately the neutral terms of System 1 and System 2 were introduced (Stanovich & West, 2000) to leave open the question of relation to consciousness. Studies of the recent years showed that the strong independence hypotheses went too far (e.g., Hartley & Burgess, 2005; McDonald, Devan, & Hong, 2004; Poldrack et al., 2001; TurkBowne, Yi, & Chun, 2006). It became progressively difficult to interpret the results along this dichotomy, a new taxonomy was needed to fit the data in it. While from the beginning the concept of declarative system was apparent, the non-declarative expression was used only as an umbrella term (Squire & Zola-Morgan, 1988), it referred to several less clear subsystems. The first detailed taxonomy of biological and memory systems (Squire, 1987) distributed the nondeclarative system into four sub-systems (procedural, priming, classical conditioning, nonassociative learning). This conception supports multiplicity of subsystems, in a sense that memory systems of the brain operate in parallel to support behaviour (Squire, 2004, p. 174). Today it seems that for the explanation of the converging new evidence we need a better framework than the multiple parallel memory systems (MPMS) theory (White & McDonald, 2002). Increasing number of behavioural and neurobiological researchers claim that separating brain regions by different memory functions without knowing much about their relations is not sufficient enough for a comprehensive understanding. Interactive memory systems theory (IMST) (McDonald, Devan, & Hong, 2004) emphasizes interactions between memory systems. The few studies done to date examining interacting relations found competitive (e.g., Mizouri, Yeshenko, Gill, & Davis, 2004), cooperative (e.g., Hartley & Burgess, 2005) and compensational (e.g., Voermans, Petersson, Daudey, Weber, van Spaendonck et al., 2004) relations between the various learning and memory systems. The basis of these interactions is hold to be the shared representations (Turk-Bowne et al., 2006) and common access to the same information (McDonald et al., 2004) between the simultaneously processing parallel systems. The independent retrieval hypothesis assumes that the difference is made not by the encoding, but only by the retrieval between the systems (Turk-Bowne et al., 2006). Our knowledge is still little about when and how these systems interact, but the argument seems to be plausible that in some tasks, cooperation is possible because the parallel systems support compatible behaviours, whereas in other tasks they drive conflicting responses and must therefore compete to control behaviour (Hartley & Bugress, p. 170). Concentrating on our issue, it becomes untenable to carry on the notion that probabilistic classification depends only on one system, instead, a growing body of evidence supports that

Strategy analysis of probability learning 57 category learning uses many, or maybe all of the major memory systems (Ashby & OBrien, 2005). Working memory is found to play a mediating role in rule-based category learning (Waldron & Ashby, 2001). Indirect evidence supports that category learning depends on episodic and semantic memory as well. The case that amnesiacs are impaired in the PCL task only after the first 50 trials (Knowlton et al., 1994) is explained that it takes about 50 trials to memorise single cues for healthy people, while amnesiacs cannot use this episodic information for heuristic singleton strategies (Gluck et al., 2002). Procedural memory system is involved in all kind of information integration tasks (e.g., Ashby, Ell, & Waldron, 2003). Some neuroimaging data supports the position that the perceptual representation memory system mediates performance in prototype distortion tasks (e.g., Seger, Poldrack, Prabhakaran, Zhao, Glover et al., 2000). Each of the systems has capabilities that others does not, therefore it is mere reasonable to assume rather simultaneous complimentary relations, then severe independence. The results of the present study do not oppose these conceptions, instead, they hold strong relevance to the issue. The application of multiple strategies could not be reasonable in a single system, or in an independent systems framework. The surprising finding that the secondary explicit task did not show any effect of interference on any measures of performance except the final explicit task, while the other group performed well on that task suggests that the two systems can work simultaneously. This could give an alternative explanation to Lagnado et al.s (in press) results assuming that the presence of explicit insight into the task does not rule out the possibility that the procedural system played a role in processing. In this discussion I contended three closely related aspects of probability learning. Various approaches of rationality studies claim that humans often follow suboptimal strategies in uncertain environments. The reason of this shortcoming as they argue is the constrained capacity of our information processing. I argued that probability matching behaviour would not be possible to occur without sufficient ability to subtract the statistical features of the environment. The argument that our nervous system is capable to execute this demanding processing is supported by the recent developments of statistical learning studies. These studies demonstrated that complex statistical computations can get operated automatically. Yet the conclusion would be erroneous to state that probability learning is solely processed by the implicit system. For a comprehensive understanding of probability learning it is essential first to understand the competing and cooperative relations between the multiple systems of memory.

Strategy analysis of probability learning 58 IX. Conclusion The goal of this study was not hypothesis testing, but an exploration of what strategies are applied in probability learning behaviour. In psychological sense learning can be measured only by its reflection in behaviour and decision making. Measuring decisional strategies is a complex problem since one has to have knowledge of the basis and the goal of the given decisional behaviour to be able to assess it. Furthermore, these aspects may change from trial to trial and from individual to individual. Strategy analysis of probability learning requires special consideration of all of these problematic points. Therefore I conducted an experimental demonstration of these aspects of probability learning and presented a methodology and analysis that tried to provide a finer-grained picture about the applied strategies in experimental-based learning situation. Lacking a standard definition of probability learning, I had to highlight the emerging concepts that led to the present theoretical and experimental constructions. The basic theoretical conception was found at Brunswik, who described judgemental behaviour in his Lens model which deals with probabilistic terms. This model fertilized the thoughts in psychology with the notion of probabilistic relations in perception and behaviour. This approach and Estess work to model probability matching by mathematical curves produced testing methods that allowed later researchers conducting explorative studies in the domain of probability learning. It became apparent very early that probabilistic experiments bring about confusing results and the used methods and the applied analyses need to be reconsidered. Being critical with the previous standards of procedures and assessments, the task was partly reconstructed and new statistical methods were employed in this work. By these dynamical analyses it became feasible to examine decisional attitudes individually along the course of the experiments and to define learning in each trial according to several possible strategies. First of all, the experiment showed that these newly applied methods are in accord with the standard assessment of the test (hit rate), but can provide further details about the behaviour. One of the main results of this dynamical analysis demonstrated that participants are able to follow multiple strategies in the same time. The other surprising finding was the lack of significant difference in any measurements between the baseline and the explicit dual-task groups. This result supports the view that the implicit processing system is involved in probability learning, but the explicit processes cannot be excluded from a comprehensive model.

Strategy analysis of probability learning 59 Analysing the applied strategies in probability learning, most of the previous studies differentiated them from optimal behaviour and stated them to be poorly effective. The reason of this suboptimality was mostly originated from the constrained cognitive capacity for the demanding tasks in uncertain environments. In the discussion I argued that the repeatedly demonstrated probability learning behaviour provides evidence that we are able to acquire sufficient information from a complex environment, however, we apply them not in accordance with our normative expectations. The interesting point is that most of the animals do not match the underlying probability distributions, but follow an optimal strategy (Vulkan, 2000). Some researchers hypothesized that the difference is that humans search for causal relations in all events (Gazzaniga, 1995). They say that the existence of an interpreter in our mind that tries to find relations of the causal effects and to make sense of the information may hinder us from the optimal strategies. The potential benefits of this tendency can also lead to over-thinking and to nonoptimal behaviour (Kareev, 1995). I argue that it is to be considered whether the evolutionary development of our cognitive capacities may have caused shortcomings in our behavioural patterns or perhaps our goals have deviated from those of animals. The strategy to match and reflect the behaviour of the surrounding may have brought more benefits for social beings in the evolutionary past. In this sense, to learn and follow the rules of the environment can be more essential for humans than bare profit maximization. In this regard for a valid comparison of human and animal decisional patterns one has to involve the goal and function of behaviour into the model (Makny, 2006). Taking these aspects as a whole we may suppose that human decisional strategies in probability learning situations are not suboptimal, but reflect a special ability to subtract the statistical features of the environment and use them according to a different goal from animals. This explorative study brought some new aspects to probability learning research both in methodology and interpretation. I concentrated on the optimal and conscious nature of the applied strategies. The results emphasize that both explicit and implicit systems may contribute to coherent decisional strategies under uncertainty. Further questions remain to be elaborated of how the two main memory systems interact, compete, or cooperate in these processes; what is the basis of individual differences in probability learning; does deterministic learning differ from probabilistic learning, or can we understand them in one model; does the way we model our environment have an effect on our executive behaviour; or does performance on probability learning tasks predict aptitude in other domains of decision making.

Strategy analysis of probability learning 60 References Aczl, B., & Gnci, D. (2005). Unity or fractionality of implicit learning: A methodological aspect. In Bara, B. G., Barsalou, L., & Bucciarelli, M. (Eds.), Proceedings of the CogSci2005 - 27th Annual Meeting of the Cognitive Science Society (pp. 51-56). Mahwah, NJ: Lawrence Erlbaum. Anderson, J. R. (1990). The Adaptive Character of Thought. Hillsdale, NJ: Lawrence Erlbaum. Arrow K. J. (1958). Utilities, attitudes, choices: A review note. Econometrica, 26, 123. [cited in Vulkan, 2000] Ashby, E. G., & Townsend, J. T. (1986). Varieties of perceptual independence. Psychological Review, 93, 154-179. Ashby, F. G., & Gott, R. E. (1988). Decision rules in the perception and categorization of multidimensional stimuli. Journal of Experimental Psychology: Learning Memory and Cognition, 14, 33-53. Ashby, F. G., & Maddox, W. T. (1992). Complex decision rules in categorization: Contrasting novice and experienced performance. Journal of Experimental Psychology: Human Perception and Performance, 18, 50-71. Ashby, F. G., & Maddox, W. T. (2005). Human category learning. Annual Review of Psychology, 56, 149178. Ashby, F. G., & O'Brien, J. B. (2005). Category learning and multiple memory systems. Trends in Cognitive Science, 2, 83-89. Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998). A neuropsychological theory of multiple systems in category learning. Psychological Review, 105, 442-481. Ashby, F. G., Ell, S. W., & Waldron, E. M. (2003). Procedural learning in perceptual categorization. Memory & Cognition, 31, 1114-1125. Balzer W. K., Doherty, M. E., & OConnor, R. (1989). Effects of cognitive feedback on performance. Psychological Bulletin, 106, 410433. Bar-Hillel, M. (1983). The base rate fallacy controversy. In: R. W. Scholz (Ed.), Decision making under uncertainty (pp. 39-61). Elsevier Science. Barron, G., & Erev, I. (2003). Small feedback-based decisions and their limited correspondence to description-based decisions. Journal of Behavioral Decision Making, 16, 215233.

Strategy analysis of probability learning 61 Beer, R. (2000). Dynamical approaches to cognitive science. Trends in Cognitive Sciences, 4, 91-99. Bjrkman, M. (1965). Studies in predictive behavior: Explorations into predictive judgments based on functional learning and defined by estimation, categorization, and choice. Scandinavian Journal of Psychology, 6, 129-156. [concise in Holzworth, 1999] Bjrkman, M. (1987). A note on cue probability learning: What conditioning data reveal about cue contrast. Scandinavian Journal of Psychology, 28, 226-232. [concise in Holzworth, 1999] Bjrkman, M. (2001). Probability learning, or partial reinforcement? In K. R. Hammond, & T. R. Stewart (Eds.), The essential Brunswik: Beginnings, explications, applications (pp. 225238.). New York: Oxford University Press. Bosse, T. (2005). Analysis of the dynamics of cognitive processes. Ph.D. Thesis, Vrije Universiteit Amsterdam. Bower, G. (1994). A turning point in mathematical learning theory. Psychological Review, 101, 290-300. Brehmer, B. (1980). "In one word: Not from experience." Acta Psychologica 45, 223-241. Brekke, N., & Borgida, E. (1988). Expert psychological testimony in rape trials: A social cognitive analysis. Journal of Personality and Social Psychology, 55, 372-386. Brunswik, E. (1937). Psychology as a science of objective relations. Philosophy of Science, 4, 227260. [Reprinted in Hammond, K. R., & Stewart, T. R., 2001] Brunswik, E. (1939). Probability as a determiner of rat behavior. Journal of Experimental Psychology, 25, 175197. [Reprinted in Hammond, K. R., & Stewart, T. R., 2001] Brunswik, E. (1943). Organismic achievement and environmental probability. Psychological Review, 50, 255272. [Reprinted in Hammond, K. R., & Stewart, T. R., 2001] Brunswik, E. (1952). The conceptual framework of psychology. In International encyclopedia of unified science (Vol. 1, pp. IV-102.). Chicago: University of Chicago Press. Brunswik, E. (1956). Historical and thematic relations of psychology to other sciences. The Scientific Monthly, 83, 151161. [Reprinted in Hammond, K. R., & Stewart, T. R., 2001] Brunswik, E., & Herma, H. (1951). Probability learning of perceptual cues in the establishment of a weight illusion. Journal of Experimental Psychology, 41, 281-290. [Reprinted in Hammond, K. R., & Stewart, T. R., 2001] Brunswik, E., & Kardos, L. (1929). Das Duplizittsprinzip in der Theorie der

Farbenwahrnehmung. Zeitschrift fr Psychologie, 111, 307320. Busemeyer, J. R., & Townsend, J. T. (1993). Decision Field Theory: A dynamic cognition approach to decision making. Psychological Review, 100, 432-459.

Strategy analysis of probability learning 62 Bush, R., & Mosteller, F. (1955). Stochastic models for learning. New York: Wiley. Castellan, N. J. (1977). Decision making with multiple probabilistic cues. In N. J. Castellan, D. P. Pisoni, & G. R. Ports (Eds.), Cognitive theory (Vol. 2). Hillsdale, NJ: Erlbaum. Chase, V. M., Hertwig, R., & Gigerenzer, G. (1998). Visions of rationality. Trends in Cognitive Sciences, 2, 206-214. Christensen-Szalanski, J. J. J., & Beach, L. R. (1982) Experience and the base-rate fallacy. Organizational Behavior and Human Performance 29, 270-278. Christiansen, M. H., & Chater, N. (1999). Toward a connectionist model of recursion in human linguistic performance. Cognitive Science, 23, 157-205. Christiansen, M. H., & Curtin, S. L. (1999). The power of statistical learning: No need for algebraic rules. Proceedings of the Twenty first Annual Conference of the Cognitive Science Society. Mahwah, NJ: Lawrence Erlbaum Associates. Christiansen, M. H., Allen, J., & Seidenberg, M. S. (1998). Learning to segment using multiple cues: A connectionist model. Language and Cognitive Processes, 13, 221-268. Christiansen, M. H., Conway, C. M., & Curtin, S. (in press). Multiple-cue integration in language acquisition: A connectionist model of speech segmentation and rule-like behavior. In J. W. Minett & W. S.-Y. Wang (Eds.), Language acquisition, change and emergence: Essay in evolutionary linguistics. Hong Kong: City University of Hong Kong Press. Available from: http://cnl.psych.cornell.edu/papers/ACE-chapter.pdf Cleeremans, A., Destrebecqz, A., & Boyer, M. (1998). Implicit learning: News from the front. Trends in Cognitive Sciences, 2, 406-416. Cooksey, R. W. (1988). Social judgment theory in education: Current and potential applications. In B. Brehmer, & C. R. B. Joyce (Eds.), Human judgment: The SJT view (pp. 273316). Amsterdam: North-Holland Elsevier. Cooksey, R. W. (1996). The methodology of social judgement theory. Thinking and Reasoning, 2, 141173. Cooksey, R. W. (2001) Conceptual Framework of Psychology: Then and Now. In K. R. Hammond, & T. R. Stewart (Eds.), The essential Brunswik: Beginnings, explications, applications (pp. 225-238.). New York: Oxford University Press. Creel, S. C., Newport, E. L., & Aslin, R. N. (2004). Distant melodies: Statistical learning of nonadjacent dependencies in tone sequences. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30, 1119-1130.

Strategy analysis of probability learning 63 Dickinson, A., & Shanks, D. (1985). Animal conditioning and human causality judgment. In L. Nilsson & T. Archer (Eds.), Perspectives on learning and memory. Hillsdale, NJ: Erlbaum. Doherty, M. E., & Kurz, E. M. (1996). Social judgement theory. Thinking and Reasoning, 2, 109-140. Earle, T. C., & Cvetkovich, G. (1988). Risk judgment, risk communication, and conflict management. In B. Brehmer, & C. R. B. Joyce (Eds.), Human judgment: The SJT view (pp. 361400). Amsterdam: North-Holland Elsevier. Edwards, W. (1961). Probability Learning in 1000 trials. Journal of Experimental Psychology, 4, 385394. Eldridge, L. L., Masterman, D., & Knowlton, B. J. (2002). Intact implicit habit learning in Alzheimers disease. Behavioral Neuroscience, 116, 722726. Erev, I., & Barron, G. (2005). On adaptation, maximization, and reinforcement learning among cognitive strategies. Psychological Review, 112, 912-931. Estes, W. (1957). Theory of learning with constant, variable or contingent probabilities of reinforcement. Psychometrika, 22. 113-132. Estes, W. K., & Burke, C. J. (1953). A theory of stimulus variability in learning. Psychological Review, 60, 276-286. Estes, W. K., Campbell, J. A., Hatsopoulis, N., & Hurwitz, J. B. (1989). Base-rate effects in category learning: A comparison of parallel network and memory storage-retrieval models. Journal of Experimental Psychology: Learning, Memory, &Cognition, 15, 556-571. Evans, J. St. B. T., Clibbens, J., Cattani, A., Harris, A., & Dennis, I. (2003). Explicit and implicit processes in multicue judgment. Memory & Cognition, 31, 608-618. Fantino, E. (1998). Behavior analysis and decision making. Journal of the Experimental Analysis of Behavior, 69, 355-364. Fera, F., Weickert, T. W., Goldberg, T. E., Tessitore, A., Hariri, A., et al. (2005). Neural mechanisms underlying probabilistic category learning in normal aging. The Journal of Neuroscience, 25, 11340-11348. Fiser, J., & Aslin, R. N. (2001). Unsupervised statistical learning of higher-order spatial structures from visual scenes. Psychological Science, 12, 499-504. Fiser, J., & Aslin, R. N. (2002). Statistical learning of higher-order temporal structure from visual shape sequences. Journal of Experimental Psychology: Learning Memory & Cognition, 28, 458-467.

Strategy analysis of probability learning 64 French, R. M., & Thomas, E. (2001). The Dynamical Hypothesis in Cognitive Science: A review essay of Mind As Motion. Minds and Machines, 11, 101-111. Friedman, D., & Massaro, D. W. (1998). Understanding variability in binary and continuous choice. Psychonomic Bulletin & Review, 5, 370389. Gazzaniga, M. (1995). Principles of human brain organization derived from split-brain studies. Neuron, 14, 217-228. Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton-Mifflin. Gigerenzer, G. (1996). The psychology of good judgment: Frequency formats and simple algorithms. Journal of Medical Decision Making, 16, 273-280. Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review, 102, 684-704. Gigerenzer, G., Hoffrage, U., & Kleinblting, H. (1991). Probabilistic mental models: a Brunswikian theory of confidence. Psychological Review, 98, 506-528. Gluck, M. A., & Bower, G. H. (1988a). From conditioning to category learning: An adaptive network model. Journal of Experimental Psychology: General, 117, 227247. Gluck, M. A., & Bower, G. H. (1988b). Evaluating an adaptive network model of human learning. Cognitive Psychology, 27, 166-195. Gluck, M. A., Oliver, L. M., & Myers, C. E. (1996). Late-trainingamnesic deficits in probabilistic category learning: A neurocomputational analysis. Learning Memory, 3, 326240. Gluck, M., Shohamy, D., & Myers, C. (2002). How do people solve the Weather Prediction task? Individual variability in strategies for probabilistic category learning. Learning & Memory, 9, 408 418. Gneezy, U., & Potters, J. (1997). An experiment on risk taking and evaluation periods. The Quarterly Journal of Economics, 112, 631645. Goodie, A. S., & Fantino, E. (1999). What does and does not alleviate base-rate neglect under direct experience. Journal of Behavioral Decision Making, 12, 307-335. Gould, S. J. (1992). Bully for brontosaurus: Further reflections in natural history. London: Penguin Books. [cited by Chase et al., 1998] Graf, P., Squire, L. R., & Mandler, G. (1984). The information that amnesic patients do not forget. Journal of Experimental Psychology: Learning, Memory and Cognition, 10, 164178. [cited by Squire, 2004]

Strategy analysis of probability learning 65 Hammond, K. R. (1955). Probabilistic functioning and the clinical method. Psychological Review, 62, 255-262. Hammond, K. R., & T. R., Stewart (2001). The essential Brunswik: Beginnings, explications, applications. New York: Oxford University Press. Hammond, K. R., (2001). On center stage. In K. R. Hammond, & T. R. Stewart (Eds.), The essential Brunswik: Beginnings, explications, applications (pp. 55-66.). New York: Oxford University Press. Hammond, K. R., Hursch, C. J., & Todd, F. J. (1964). Analyzing the components of clinical inference. Psychological Review, 71, 438-456. Hartley, T., & Burgess, N. (2005). Complementary memory systems: competition, cooperation and compensation. Trends in Neuroscience, 28, 169170. Haruvy, E., & Erev, I. (2001). On the application and interpretation of learning models. In R. Zwick, & A. Rapoport (Eds.), Advances in experimental business research (pp. 285300). Norwell, MA: Kluwer Academic. [cited by Erev & Baron, 2005] Hauser, M., Newport, E. L., & Aslin, R. N. (2001). Segmentation of the speech stream in a nonhuman primate: Statistical learning in cotton-top tamarins. Cognition, 78, B41-B52. Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004). Decisions from experience and the effect of rare events in risky choices. Psychological Science, 15, 534-539. Hoffrage, U., Gigerenzer, G., Krauss, S., & Martignon, L. (2002). Representation facilitates reasoning: What natural frequencies are and what they are not. Cognition, 84, 343-352. Hogarth, R. M., & Einhorn, H. J. (1992). Order effects in belief updating: The belief-adjustment model Cognitive Psychology, 24, 155. Holyoak, K. J., & Spellman, B. A. (1993). Thinking. Annual Review of Psychology 44, 265-315. Holzworth, R. J. (1999). Annotated Bibliography of Cue Probability Learning Studies. Retrieved April 14, 2006, from http://www.albany.edu/cpr/brunswik/resources/mcplbib.doc Hunt, R. H., & Aslin, R. N. (2001). Statistical learning in a serial reaction time task: Simultaneous extraction of multiple statistics. Journal of Experimental Psychology: General, 130, 658680. Jones, M., Love, B. C., & Maddox, W. T. (in press). Recency as a window to generalization: Separating decisional and perceptual sequential effects in category learning. Journal of Experimental Psychology: Learning, Memory, and Cognition.

Strategy analysis of probability learning 66 Juslin, P. (1994). The overconfidence phenomenon as a consequence of informal experimenterguided selection of almanac items. Organizational Behavior and Human Decision Processes, 57, 226246. Kahneman D., & Tversky, A. (1972). Subjective probability: A judgment of representativeness. Cognitive Psychology 3, 430-454. Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80, 237251. Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263291. Kareev, Y. (1995). Through a narrow window: Working memory capacity and the detection of covariation. Cognition, 56, 263-269. Kelley, H., & Friedman, D. (2002). Learning to forecast price. Economic Enquiry, 40, 556 573. Kelso, J. A. S. (1997). Dynamic Patterns. Cambridge, MA: The MIT Press. Kri, Sz. (2003). The cognitive neuroscience of category learning. Brain Research Reviews, 43, 85109. Kri, Sz., Kelemen, O., Szekeres, G., Bagczky, N., Erdlyi, R., et al. (2000). Schizophrenics know more than they can tell: Probabilistic classification learning in schizophrenia. Psychological Medicin 30, 149155. Kri, Sz., Szlobodnyik, C., Benedek, G., Janka, Z., & Gdoros, J. (2002). Probabilistic classification learning in Tourette syndrome. Neuropsychologia, 40, 13561362. Kincses, T. Z., Antal, A., Nitsche, M. A., Brtfai, O. & Paulus, W. (2003). Facilitation of probabilistic classification learning by transcranial direct current stimulation of the prefrontal cortex in the human Neuropsychologia, 42, 113-117. Kirkham, N. Z., Slemmer, J. A., & Johnson, S. P. (2002). Visual statistical learning in infancy: evidence. of a domain general learning mechanism. Cognition, 83, B35-B42. Kitzis S., Kelley, H., Berg, D., Massaro, D., & Friedman, D. (1998). Broadening the tests of learning models. Journal of Mathematical Psychology, 42, 327-55. Knowlton, B., Mangels, J., & Squire, L. (1996). A neostriatal habit learning system in humans. Science, 273, 1399-1402. Knowlton, B., Squire, L., & Gluck, M. (1994). Probabilistic classification leaning in amnesia. Learning & Memory, 1, 106120. Koehler, J. J. (1993). The Base Rate Fallacy Myth. Psycoloquy, 4, 49. Koehler, J. J. (1994). Base Rates and the "illusion illusion". Psycoloquy: 5, 9.

Strategy analysis of probability learning 67 Koehler, J. J. (1996). The Base rate fallacy reconsidered: Normative, descriptive and methodological challenges. Behavioral and Brain Sciences, 19, 1-53. Krogstad, J. L., Ettenson, R. T., & Shanteau, J. (1984). Context and experience in auditors materiality judgments. Auditing: A Journal of Practice & Theory, 4, 54-73. Lagnado, D. A., Newell, B. R., Kahan, S., & Shanks, D. R. (in press). Insight and strategy in multiple cue learning. Journal of Experimental Psychology: General. Lindeman, S. T., Van Den Brink, W. P. & Hoogstraten, J. (1988). Effect of feedback on base-rate utilization. Perceptual and Motor Skills, 67, 343-350. Lopes, L. L. (1991). The rhetoric of irrationality. Theory and Psychology, 1, 65-82. MacLeod, C. M., Uttl, B., & Ohta, N. (2005). Dynamic cognitive processes in broad perspective. In N. Ohta, C. M. MacLeod, & B. Uttl (Eds.), Dynamic cognitive processes (pp. 1-9). Tokyo: Springer-Verlag. Maddox, W. T., & Bohil, C. J. (1998). Overestimation of base-rate differences in complex perceptual categories. Perception & Psychophysics, 60, 575-592. Makany, T. (2006). Optimality and cognition: Shortest paths and smartest brains [Electronic version]. Periodicals of Implicit Cognition, 4, 1-3. Retrieved April 18, 2006, from http://www.implab.org/library/pic4.pdf Manis, M., Dovalina, I., Avis, N. E., & Cardoze, S. (1980). Base rates can affect individual predictions. Journal of Personality and Social Psychology, 38, 231-248. Marcus, G. F. (2000). The algebric mind: Reflections on connectionism and cognitive science. Cambridge, MA: MIT Press. Marcus, G. F., Vijayan, S., Rao, S. B., & Vishton, P. M. (1999). Rule learning in seven month-old infants. Science, 283, 7780. McDonald, R. J., Devan, B. D., & Hong, N. S. (2004). Multiple memory systems: The power of interactions. Neurobiology of Learning and Memory, 82, 333-346. McDonald, R. J., Devan, B. D., & Hong, N. S. (2004). Multiple memory systems: the power of interactions. Neurobiology of Learning and Memory, 82, 333-346. Medin, D. L., & Edelson, S. M. (1988). Problem structure and the use of base-rate information from experience. Journal of Experimental Psychology: General, 117, 68-85. Medin, D. L., Altom, M. W., Edelson, S. M., & Freko, D. (1982). Correlated symptoms and simulated medical classification. Journal of Experimental Psychology: Learning, Memory, and Cognition, 8, 37-50.

Strategy analysis of probability learning 68 Milner, B., Corkin, S., & Teuber, H. L. (1968). Further analysis of the hippocampal amnesic syndrome: l4 year follow-up study of H.M. Neuropsychologia, 6, 215234. Mizumori, S. J. Y., Yeshenko, O., Gill, K. M., & Davis, D. M. (2004). Parallel processing across neural systems: Implications for a multiple memory systems hypothesis. Neurobiology of Learning and Memory, 82, 278-298. Moody, T. D., Bookheimer, S. Y., Vanek, Z., & Knowlton, B. J. (2004). An implicit learning task activates medial temporal lobe in patients with Parkinsons disease. Behavioral Neuroscience, 118, 438442. Morrison, D. E (1990). Multivariate statistical methods. New York: McGraw Hill. [cited in Ashby & Maddox, 1998] Newport, E. L., Hauser, M. D., Spaepen, G., & Aslin, R. N. (2004). Learning at a distance: II. Statistical learning of non-adjacent dependencies in a non-human primate. Cognitive Psychology, 49, 85-117. Nissen, M. J., & Bullemer, P. (1987). Attentional requirements of learning: Evidence from performance measures. Cognitive Psychology, 19, 1-32. Nosofsky, R. M., Kruschke, J. K., & McKinley, S. C. (1992). Combining exemplar-based category representations and connectionist learning rules. Journal of Experimental Psychology: Learning, Memory, & Cognition, 18, 211-233. Peterson, B. S., Leckman, J., Scahill, L., Naftolin, F., Keefe, D., et al. (1994). Steroid hormones and Tourette's syndrome: Early experience with antiandrogen therapy. Journal of Clinical Psychopharmacology, 14, 131-135. Peterson, C. R., & Beach, L. R. (1967). Man as an intuitive statistician. Psychological Bulletin, 68, 29-46. Pinker, S. (1997). How the Mind Works. New York: WW Norton & Company. Poldrack, R. A., & Rodriguez, P. (2004). How do memory systems interact? Evidence from human classification learning. Neurobiology of Learning and Memory, 82, 324-332. Poldrack, R. A., Clark, J., Par-Blagoev, J., Shohamy, D., Creso Moyano, J., et al. (2001). Interactive memory systems in the human brain. Nature, 414, 546550. Port, R. F. (2000). Dynamical Systems Hypothesis in Cognitive Science. draft entry for Encyclopedia of Cognitive Science. MacMillan. London: Amy Lockyer. Retrieved April 16, 2006, from http://www.cs.indiana.edu/~port/pap/ency.dec.htm Reber, A. S. (1967). Implicit learning of artificial grammars. Journal of Verbal Learning and Verbal Behavior, 6, 855-863.

Strategy analysis of probability learning 69 Reber, A. S. (1969). Transfer of syntactic structure in synthetic languages. Journal of Experimental Psychology, 81, 115-119. Reber, A. S. (1992). The cognitive unconscious: an evolutionary perspective. Consciousness Cognition 1, 93-133. Reber, P. J., & Squire, L. R. (1999). Intact learning of artificial grammars and intact category learning by patients with Parkinsons disease. Behavioral Neuroscience, 113, 235242. Reber, P. J., Knowlton, B. J., & Squire, L. R. (1996). Dissociable properties of memory systems: differences in the flexibility of declarative and nondeclarative knowledge. Behavioral Neuroscience, 110, 861871. Reed, S. K. (1972). Pattern recognition and categorization. Cognitive Psychology, 3, 382-407. Rescorla, R. A. (1968). Probability of shock in the presence and absence of CS in fear conditioning. Journal of Comparative and Physiological Psychology, 66, 1-5. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non-reinforcement. In A. H. Black, & W. F. Prokasy (Eds.), Classical conditioning: II. Current research and theory. New York: AppletonCentury-Crofts. [cited by Gluck & Bower, 1988a] Restle, F. (1961). Psychology of judgment and choice. New York: Wiley. Robertson, T. S. (1970). Consumer behavior. Glenview, IL: Scott, Foresman and Company. Saffran, J. R., Aslin, R. N., & Newport, E. L. (1996). Statistical learning by 8-month-old infants. Science, 274, 19261928. Saffran, J. R., Johnson, E. K., Aslin, R. N., & Newport, E. L. (1999). Statistical learning of tone sequences by human infants and adults. Cognition, 70, 2752. Saffran, J. R., Newport, E. L., Aslin, R. N., Tunick, R. A., & Barrueco, S. (1997). Incidental language learning: Listening (and learning) out of the corner of your ear. Psychological Science, 8, 101-105. Seger, C. A., Poldrack, R. A., Prabhakaran, V., Zhao, M., Glover, et al. (2000). Hemispheric asymmetries and individual differ-ences in visual concept learning as measured by functional MRI. Neuropsychologia, 38, 13161324. Shanks, D. R., & St. John, M. F. (1994). Characteristics of dissociable human learning systems. Behavioral and Brain Sciences, 17, 367-447. Shanks, D. R., Tunney, R. J., & McCarthy, J. D. (2002). A Re-examination of probability matching and rational choice. Journal of Behavioral Decision Making, 15, 233-250.

Strategy analysis of probability learning 70 Shepard, R. (1992). The advent and continuing influence of mathematical learning theory: Comment on Estes and Burke. Journal of Experimental Psychology: General, 121, 419-421. Shohamy, D., Myers, C. E., Grossman, S., Sage, J., Gluck, M. A. et al. (2004). Cortico-striatal contributions to feedback-based learning: Converging data from neuroimaging and neuropsychology. Brain, 127, 851859. Siegel, S., & Goldstein, D. A. (1959). Decision making behaviour in a two-choice uncertain outcome situation. Journal of Experimental Psychology, 57, 37-42. Simon, H. A. (1990). Invariants of human behavior. Annual Review of Psychology, 41, 119. Smedslund, J. (1955). Multiple-probability learning: An inquiry into the origins of perception. Oslo, Norway: Oslo University Press. [concise in Holzworth, 1999] Smith, A. C., Frank, L. M., Wirth, S., Yanike, M., Hu, D. et al. (2004). Dynamic analysis of learning in behavioral experiments. Journal of Neuroscience, 24, 447461. Spellman, B. A. (1993). Implicit learning of base rates. Psycoloquy, 4, 61. Spellman, B. A. (1996). The implicit use of base rates in experiential and ecologically valid tasks. Behavioral and Brain Sciences, 19, 38. Squire, L. R. (1987). Memory and brain. New York: Oxford University Press. [cited by Squire, 2004] Squire, L. R. (1994). Declarative and nondeclarative memory: Multiple brain systems supporting learning and memory. In D. Schacter, & E. Tulving (Eds.) Memory Systems 1994 (pp 203231). Cambridge, MA: MIT press. Squire, L. R. (2004). Memory systems of the brain: A brief history and current perspective. Neurobiology of Learning and Memory, 82, 171-177. Squire, L. R., & Zola-Morgan, S. (1988). Trends in Neurosciences, 11, 170175. Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. Psychological Monographs, 2. Thorndike, E. L. (1932). The Fundamentals of Learning. New York: Teachers College Press. Tolman, E. C. (1932). Purposive behavior in animals and men. New York: Century. [cited by Hothersall, D. (1990) History of Psychology, 2nd. Ed., New York: McGraw Hill.] Toth, D., & Aczel, B. (2006). Analyzing Dynamics of Probability Learning. manuscript. Turk-Browne, N. B., Junge, J. A., & Scholl, B. J. (2005). The automaticity of visual statistical learning. Journal of Experimental Psychology: General, 134, 552-564.

Strategy analysis of probability learning 71 Turk-Browne, N. B., Yi, D. J., & Chun, M. M. (2006). Linking implicit and explicit memory: Common encoding factors and shared representations. Neuron, 49, 917-927. Turk-Browne, N. B., Yi, D. J., & Chun, M. M. (2006). Linking implicit and explicit memory: Common encoding factors and shared representations. Neuron, 49, 917-927. Ungar, S., & Sev'er, A. (1989). 'Say it ain't so, Ben': Attributions for a fallen hero. Social Psychology Quarterly, 52, 207-212. van Gelder, T. (1998). The dynamical hypothesis in cognitive science. Behavioral and Brain Sciences, 21, 615665. van Gelder, T., & Port, R. F. (1995). It's about time: An overview of the dynamic approach to cognition. In R. Port, & T. van Gelder (Eds.), Mind as Motion: Explorations in the Dynamics of Cognition (pp. 1-44). Cambridge, MA: MIT Press. Voermans, N. C., Petersson, K. M., Daudey, L., Weber, B., van Spaendonck, K. P. et al. (2004). Interaction between the human hippocampus and the caudate nucleus during route recognition. Neuron, 43, 427-435. Vulkan, N. (2000). An economists perspective on probability matching. Journal of Economic Surveys, 14, 101118. Waldron, E. M., & Ashby, F. G. (2001). The effects of concurrent task interference on category learning: evidence for multiple category learning systems. Psychonomical Bulletin and Review, 8, 168-76. Warrington, E. K., & Weiskrantz, L. (1968). A new method of testing long-term retention with special reference to amnesic patients. Nature, 217, 972974. West, R. F., & Stanovich, K. E. (2003). Is probability matching smart? Associations between probabilistic choices and cognitive ability. Memory & Cognition, 31, 243-251. White, N. M., & McDonald, R. J. (2002). Multiple parallel memory systems in the brain of the rat. Neurobiology of Learning and Memory, 77, 125184. Wigton, R. S. (1988). Use of linear models to analyze physicians' decisions. Journal of Medical Decision Making 8, 241-252.

Strategy analysis of probability learning Appendix 1. Ethical statement

NYILATKOZAT

Kijelentem, hogy Tth Dnes s Aczl Balzs kognitv pszicholgiai vizsglatban szeminriumi kreditszerzs cljbl veszek rszt. A vizsglat jellegrl annak megkezdse eltt kielgt tjkoztatst kaptam. Tudomsul veszem, hogy az azonostsomra alkalmas szemlyi adataimat a vizsglat vezetje bizalmasan kezeli, azokba a vizsglat lebonyoltsban s feldolgozsban rszt vev szemlyeken kvl msoknak nem enged betekintst. Hozzjrulok ahhoz, hogy a vizsglat sorn a rlam felvett, szemlyem azonostsra nem alkalmas adatok ms kutatk szmra is hozzfrhetk legyenek. Fenntartom a jogot arra, hogy a vizsglat sorn annak folytatstl brmikor elllhassak. Ilyen esetben a rlam addig felvett adatokat trlni kell. Tudomsul veszem, hogy a vizsglati adatok kutatsi, s nem diagnosztikai clokat szolglnak, ilyen jelleg szakvlemnyre a vizsglatok elvgzst kvetn ignyt nem tmasztok.

IGEN (Enter)

NEM (Esc)

Strategy analysis of probability learning Appendix 2.

Instructions from the experimental program. The instructions were presented on screen at the beginning of the task session.

"Szervusz! Ebben a jtkban fagylaltrus vagy. Vevkkel fogsz tallkozni, akik vanlia- vagy csokifagyit krnek. Minden alkalommal, ha egy vev megrkezik, prbld megtippelni, hogy vanlit vagy csokit fog-e krni, s nyomd meg az ennek megfelel gombot: BAL SHIFT = CSOKI JOBB SHIFT = VANLIA

Ha eltallod a vev kvnsgt, akkor jutalmul kapsz 10 petk borravalt. Prblj annyi pnzt sszegyjteni, amennyit csak tudsz! Sok szerencst! "

Lejrt az idd. Prblj meg gyorsabban vlaszolni!