The Abstraction and Reasoning Corpus (ARC) is a challenging program induction dataset that was recently proposed by Chollet (2019). Here, we report the first set of results collected from a behavioral study of humans solving a subset of tasks from ARC (40 out of 1000). Although this subset of tasks contains considerable variation, our results showed that humans were able to infer the underlying program and generate the correct test output for a novel test input example, with an average of 80% of tasks solved per participant, and with 65% of tasks being solved by more than 80% of participants. Additionally, we find interesting patterns of behavioral consistency and variability within the action sequences during the generation process, the natural language descriptions to describe the transformations for each task, and the errors people made. Our findings suggest that people can quickly and reliably determine the relevant features and properties of a task to compose a correct solution. Future modeling work could incorporate these findings, potentially by connecting the natural language descriptions we collected here to the underlying semantics of ARC.

Categories carve up the world in a structured way, allowing people to inductively reason about the properties of novel ex- emplars. Children are still in the process of learning category structure, and often fail to leverage the inductive power of these representations to their advantage. For example, young chil- dren generally fail to recognize the value of sampling diverse exemplars to support category-wide generalization. This study investigates whether teaching children the structure within a natural category increases diversity-based inductive reasoning. In an informal science learning environment, we presented 259 children aged 5 to 8 years with exemplars of the three main types of birds: raptors, songbirds, and waterbirds. After a short dialogue pointing out the various within-type similarities and between-type differences, children?s diversity-based inductive reasoning did not significantly improve, despite them evidenc- ing a better understanding of the category?s structure. Instead, children tended to avoid sampling waterbirds, the least typical cluster of birds. These patterns suggest that children?s neglect of sample diversity is unlikely to be solely due to their relative ignorance of category structure.

A number of recent computational models treat concept learning as a form of probabilistic rule induction in a space of language-like, compositional concepts. Inference in such mod- els frequently requires repeatedly sampling from a (infinite) distribution over possible concept rules and comparing their relative likelihood in light of current data or evidence. However, we argue that most existing algorithms for top-down sampling are inefficient and cognitively implausible accounts of human hypothesis generation. As a result, we propose an alternative, Instance Driven Generator (IDG), that constructs bottom-up hypotheses directly out of encountered positive instances of a concept. Using a novel rule induction task based on the children?s game Zendo, we compare these ?bottom- up? and ?top-down? approaches to inference. We find that the bottom-up IDG model accounts better for human infer- ences and results in a computationally more tractable inference mechanism for concept learning models based on a probabilis- tic language of thought.

Although a common assumption in models of perceptual dis- crimination, most models of categorization do not explicitly account for uncertainty in stimulus measurement. Such un- certainty may arise from inherent perceptual noise or noise in stimulus measurement (e.g., a medical test that gives variable results). In this paper we explore how people decide to gather information from various stimulus properties when each sam- ple or measurement is noisy. Participant?s goal is to correctly classify the given item. Across two experiments we find sup- port for the idea that people take category structure into ac- count when selecting information for a classification decision. In addition, we find some evidence that people are also sensi- tive to their own perceptual uncertainty.

Collecting (or "sampling") information that one expects to be useful is a powerful way to facilitate learning. However, little is known about how people decide which information is worth learning about, particularly during the course of learning. We describe several alternative models of how people might decide to collect a piece of information inspired by "active learning" research in machine learning. We additionally provide a theoretical analysis demonstrating the situations under which these models are empirically distinguishable, and report a novel empirical study that exploits these insights. Our model-based analysis of participant?s information gathering decisions reveal that people prefer to select items which resolve the uncertainty between two possibilities at a time rather than items that have high uncertainty across all relevant possibilities simultaneously. The results appear to challenge standard normative models of the value of information as well as accounts which argue that people are hamstrung by confirmation bias.

The latest generation of neural networks has made major performance advances in object categorization from raw images. In particular, deep convolutional neural networks currently outperform alternative approaches on standard benchmarks by wide margins and achieve human-like accuracy on some tasks. These engineering successes present an opportunity to explore long-standing questions about the nature of human concepts by putting psychological theories to test at an unprecedented scale. This paper evaluates deep convolutional net- works trained for classification on their ability to predict category typicality ? a variable of paramount importance in the psychology of concepts ? from the raw pixels of naturalistic images of objects. We find that these models have substantial predictive power, unlike simpler features computed from the same massive dataset, showing how typicality might emerge as a byproduct of a complex model trained to maximize classification performance.

People often make repeated decisions from experience. In such scenarios, persistent biases of choice can develop, most notably the "hot stove effect" (Denrell & March, 2001) in which a prospect that is mistakenly believed to be negative is avoided and thus belief-correcting information is never obtained. In the existing literature, the hot stove effect is generally thought of as developing through interaction with a single, stochastic prospect. Here, we show how a similar bias can develop due to people's tendency to selectively attend to a subset of features during categorization. We first explore the bias through model simulation, then report on an experiment in which we find evidence of a decisional bias linked to selective attention. Finally, we use these computational models to design novel interventions to "de-bias" decision-makers, some of which may have practical application.

People can test hypotheses through either selection or reception. In a selection task, the learner actively chooses observations to test their beliefs, while in reception tasks data is passively encountered. People routinely use both forms of testing in everyday life, but the critical psychological differences between selection and reception learning remain poorly understood. One hypothesis is that selection learning improves learning performance by enhancing generic cognitive processes related to motivation, attention, and engagement. Alternatively, we suggest that differences between these two learning modes derives from a hypothesis-dependent sampling bias that is introduced when a person collects data to test their own individual hypothesis. Drawing on influential models of sequential hypothesis testing behavior, we show that such a bias 1) can lead to the collection of data that facilitates learning compared to reception learning, and 2) can be more effective than observing the selections of another person. We then report a novel experiment based on a popular category learning paradigm that compares reception and selection learning. We additionally compare selection learners to a set of ``yoked" participants who viewed the exact same sequence of observations under reception conditions. The results revealed systematic differences in performance that depended on the learner's role in collecting information and the abstract structure of the problem.

What effect does labeling an object as a member of a familiar category have on memory for that object? Recent studies suggest that recognition memory can be negatively impacted by categorizing objects during encoding. This paper examines the "representational shift hypothesis" which argues that categorizing an object impairs recognition memory by altering the trace of the encoded memory to be more similar to the category prototype. Previous evidence for this idea comes from experiments in which a basic-level category labeling task was compared to a non-category labeling incidental encoding task, usually a preference judgment (e.g., "Do you like this item?"). In two experiments, we examine alternative tasks that attempt to control for processing demands and the degree to which category information is explicitly recruited at the time of study. Contrary to the predictions of the representational shift hypothesis, we find no evidence that memory is selectively impaired by category labeling. Overall, the pattern of results across both studies appears consistent with well-established variables known to influence memory such as encoding specificity and distinctiveness effects.

Amazon Mechanical Turk (AMT) is an online crowdsourcing service where anonymous online workers complete web-based tasks for small sums of money. The service has attracted attention from experimental psychologists interested in gathering human subject data more efficiently. However, relative to traditional laboratory studies, many aspects of the testing environment are not under the experimenter's control. In this paper, we attempt to empirically evaluate the fidelity of the AMT system for use in cognitive behavioral experiments. These types of experiment differ from simple surveys in that they require multiple trials, sustained attention from participants, and millisecond accuracy for response recording and stimulus presentation. We replicate a diverse body of tasks from experimental psychology including the Stroop, Switching, Flanker, Simon, Posner Cuing, attentional blink, subliminal priming, and category learning tasks using participants recruited using AMT. While most of replications were qualitatively successful and validated the approach of collecting data anonymously online using a web-browser, for others the alignment between laboratory results and online results showed more of a disparity. A number of important lessons were encountered in the process of conducting these replications that should be of value to other researchers.

Studies of human category learning typically focus on situations where explicit category labels accompany each example (supervised learning) or on situations were people must infer category structure entirely from the distribution of unlabeled examples (unsupervised learning). However, real-world category learning likely involves a mixture of both types of learning (semi-supervised learning). Surprisingly, a number of recent findings suggest that people have difficulty learning in semi-supervised tasks. To further explore this issue, we devised a category learning task in which the distribution of labeled and unlabeled items suggested alternative organizations of a category. This design allowed us to determine whether learners combined information from both types of episodes via their patterns of generalization at test. In contrast with the prediction of many models, we find little evidence that unlabeled items influenced categorization behavior when labeled items were also present.

Self-directed information sampling - the ability to collect information that one expects to be useful - has been shown to improve the efficiency of concept acquisition for both human and machine learners. However, little is known about how people decide which information is worth learning more about. In this study, we examine self-directed learning in a relatively complex rule learning task that gave participants the ability to "design and test" stimuli they wanted to learn about. On a subset of trials we recorded participants' uncertainty about how to classify the item they had just designed. Analyses of these uncertainty judgments show that people prefer gathering information about items that help refine one rule at a time (i.e., those that fall close to a pairwise category "margin") rather than items that have the highest overall uncertainty across all relevant hypotheses or rules. Our results give new insight into how people gather information to test currently entertained hypotheses in complex problem solving tasks.

Recent functional magnetic resonance imaging (fMRI) studies have found that distinct neural systems may mediate perceptual category learning under incidental and intentional learning conditions. The present study was designed to replicate and extend previous investigations of these effects using an event-related design. In addition, the design is aimed at decoupling the influence of stimulus-encoding processes from the contribution of implicit and explicit learning on the recruitment of alternative neural systems. Consistent with previous reports, following incidental learning in a dot-pattern classification task, participants show decreased neural activity in occipital visual cortex (extrastriate region V3, BA 19) in response to novel exemplars of a studied category compared to members of a foil category, but do not show this decreased neural activity following explicit learning. Crucially, however, our results show that this pattern can be modulated by aspects of the stimulus-encoding instructions provided at the time of study. In particular, when participants in an implicit learning condition were encouraged to evaluate the overall shape and configuration of the stimuli during study, we failed to find the pattern of brain activity that has been taken to be a signature of implicit learning, suggesting that activity in this area does not uniquely reflect implicit memory for perceptual categories.

In this paper, we evaluate the "representational shift" hypothesis which argues that the act of explicitly labeling an object as a member of a familiar semantic category alters the trace of the encoded memory in the direction of the category prototype. The typical procedure for such experiments has been to compare category labeling to a non-categorization encoding task such as a preference judgement. In a series of experiments, we examine alternative comparison tasks that attempt to control the depth of encoding and the degree to which category information is explicitly recruited at the time of study. The results appear most consistent with a depth of processing (Craik & Lockhart, 1972) (Exp. 1) or distinctiveness (Exp. 2) explanation for the pattern of memory effects found in previous studies.

Numerous proposal have been put forward concerning the nature of human category representations, ranging from rules to exemplars to prototypes. However, it is unlikely that a single, ?xed form of representation is suf?cient to account for the ?exibility of human categories. In this chapter, we describe an alternative to these ?xed-representation accounts based on the principle of adaptive clustering. The speci?c model we consider, SUSTAIN, represents categories in terms of feature bundles called clusters which are adaptively recruited in response to task demands. In some cases, SUSTAIN acts like an exemplar model, storing each category instance, while in others it appears more like a prototype model extracting only the central tendency of a number of items. In addition, selective attention in the model allows it to mimic many of the behaviors associated with rule-based systems. We review a variety of evidence in support of the clustering principle, including studies of the relationship between categorization and recognition memory, changes in unsupervised category learning abilities across development, and the in?uence of category learning on perceptual discrimination. In each case, we show how the nature of human category representations are best accounted for using an adaptive clustering scheme. SUSTAIN is just one example of a system that casts category learning in terms of adaptive clustering and future directions for the approach are discussed.

Recent research has argued that categorization is strongly tied to language processing. For example, language (in the form of verbal category labels) has been shown to influence the perceptual discriminations of color (Winawer et al., 2007). However, does this imply that categorical perception is essentially verbally-mediated perception? The present studies extend recent findings in our lab showing that categorical perception can occur even in the absence of overt labels. In particular, we evaluate the degree to which certain interference task (verbal, spatial) reduce the effect of learned categorical perception for complex visual stimuli (faces). Contrary to previous findings with color categories, our results show that a verbal interference task does not disrupt learned categorical perception effects for faces. Our results are interpreted in light of the ongoing debate about the role of language in categorization. In particular, we suggest that at least a sub-set of categorical perception effect may be effectively "language-free."

Laboratory studies of human category learning tend to emphasize passive learning situations by limiting participants' control over what information they experience on every trial. In this paper, we explore the impact that active data selection has on category learning. In our experiment, participants attempted to learn standard rule-based (RB) and information-integration (II) categories under either entirely passive (observational) conditions, or by actively selecting and querying the labels associated with particular stimuli. Our primary aim was to characterize the information sampling strategy that participants adopted in the task and to examine how the passive/active learning distinction interacted with the structure of the categories. We found that participants acquired categories faster when they were able to select and query category members on their own. Furthermore, this advantage depended on learners actually making the decisions about which stimuli to query themselves rather than simply the statistics of the experienced exemplars. Model based analyses explain this effect in terms of the number of active hypotheses under consideration which is assumed to be higher in the active learning condition due to the greater engagement of the learner in the task

Effective learning often involves actively querying the environment for information that disambiguates potential hypotheses. However, the space of observations available in any situation can vary greatly in potential "informativeness." In this report, we study participants' ability to gauge the information value of potential observations in a cognitive search task based on the children's game Battleship. Participants selected observations to disambiguate between a large number of potential game configurations subject to information-collection costs and penalties for making errors in a test phase. An "ideal-learner" model is developed to quantify the utility of possible observations in terms of the expected gain in points from knowing the outcome of that observation. The model was used as a tool for measuring search efficiency, and for classifying various types of information collection decisions. We find that participants are generally effective at maximizing gain relative to their current state of knowledge and the constraints of the task. In addition, search behavior shifts between an slower, but more efficient "exploitive" mode of local search and a faster, less efficient pattern of "exploration."

We live in a world consisting of concrete experiences, yet we appear to form abstractions that transcend the details of our experiences. In this contribution, we argue that the abstract nature of our thought is overstated and that our representations are inherently bound to the examples we experience during learning. We present three lines of related research to support this general point. The ?rst line of research suggests that there are no separate learning systems for acquiring mental rules and storing exceptions to these rules. Instead, both items types share a common representational substrate that is grounded in experienced training examples. The second line of research suggests that representations of abstract concepts, such as same and different that can range over an unbounded set of stimulus properties, are rooted in experienced examples coupled with analogical processes. Finally, we consider how people perform in dynamic decision tasks in which short? and long?term rewards are in opposition. Rather than invoking explicit reasoning processes and planning, people's performance is best explained by reinforcement learning procedures that update estimates of action values in a reactive, trial-by-trial fashion. All three lines of research implicate mechanisms of thought that are capable of broad generalization, yet inherently local in terms of the procedures used for updating mental representations and planning future actions. We end by considering the benefits of designing systems that operate according to these principles.

We asked 169 participants to spontaneously categorize nine sets of items. A category structure was assumed to be more intuitive if a large number of participants consistently produced the same classification. Our results provide a rich empirical framework for examining models of unsupervised categorization?and illustrate the corresponding profound modeling challenge. We provide a preliminary examination comparing two models of unsupervised categorization: SUSTAIN (Love, Medin, & Gureckis, 2004) and the simplicity model (Pothos & Chater, 2002).

A novel study is presented that explores the effect that learning internally organized categories has on the ability to subsequently discriminate category members. The results demonstrate the classic categorical perception effect whereby discrimination of stimuli that belong to different categories is improved following training, while the ability to discriminate stimuli belonging to the same category is reduced. We further report a new within-category perceptual effect whereby category members that share the same category label but fall into different sub-clusters within that category are better discriminated than items that share the same category and cluster. The results show that learners are sensitive to multiple sources structure beyond simply the labels provided during supervised training. A computational model is presented to account for the results whereby multiple levels of encoding (i.e., at the item-, cluster-, and category- level) may simultaneously contribute to perception.

Mental localization efforts tend to stress the where more than the what. We argue that the proper targets for localization are well-specified cognitive models. We make this case by relating an existing cognitive model of category learning to a learning circuit involving the hippocampus, perirhinal, and prefrontal cortices. Results from groups varying in function along this circuit (e.g., infants, amnesics, and older adults) are successfully simulated by reducing the model's ability to form new clusters in response to surprising events, such as an error in supervised learning or an unfamiliar stimulus in unsupervised learning. Clusters in the model are akin to conjunctive codes that are rooted in an episodic experience (the surprising event) yet can develop to resemble abstract codes as they are updated by subsequent experiences. Thus, the model holds that the line separating episodic and semantic information can become blurred. Dissociations (categorization vs. recognition) are explained in terms of cluster recruitment demands.

Mental localization efforts tend to stress the where more than the what. We argue that the proper targets for localization are well-specified cognitive models. We make this case by relating an existing cognitive model of category learning to a learning circuit involving the hippocampus, perirhinal, and prefrontal cortex. Results from groups varying in function along this circuit (e.g., infants, amnesics, older adults) are successfully simulated by reducing the model's ability to form new clusters in response to surprising events, such as an error in supervised learning or an unfamiliar stimulus in unsupervised learning. Reported task dissociations (e.g., categorization vs. recognition) are explained in terms of cluster recruitment demands.

Computational models of infant categorization often fail to elaborate the transitional mechanisms that allow infants to achieve adult performance. In this paper, we apply a successful connectionist model of adult category learning to developmental data. The Supervised and Unsupervised Stratified Adaptive Increment Network (SUSTAIN) model is able to account for the emergence of infants' sensitivity to correlated attributes (e.g., has wings and can fly). SUSTAIN offers two complimentary explanations of the developmental trend. One explanation centers on memory storage limitations, whereas the other focuses on limitations in perceptual systems. Both explanations parallel published findings concerning the cognitive and sensory limitations of infants. SUSTAIN's simulations suggest that conceptual development follows a continuous and smooth trajectory, despite qualitative changes in behavior, and that the mechanisms that underlie infant and adult categorization may not differ significantly.

SUSTAIN (Supervised and Unsupervised STratified Adaptive Incremental Network) is a model of how humans learn categories from examples. SUSTAIN initially assumes a simple category structure. If simple solutions prove inadequate and SUSTAIN is confronted with a surprising event (e.g., it is told that a bat is a mammal instead of a bird), SUSTAIN recruits an additional cluster to represent the surprising event. Newly recruited clusters are available to explain future events and can themselves evolve into prototypes/attractors/rules. SUSTAIN's discovery of category substructure is affected not only by the structure of the world but by the nature of the learning task and the learner's goals. SUSTAIN successfully extends category learning models to studies of inference learning, unsupervised learning, category construction, and contexts in which identification learning is faster than classification learning.

SUSTAIN (Supervised and Unsupervised STratified Adaptive Incremental Network) is a network model of human category learning. SUSTAIN initially assumes a simple category structure. If simple solutions prove inadequate and SUSTAIN is confronted with a surprising event (e.g., it is told that a bat is a mammal instead of a bird), SUSTAIN recruits an additional cluster to represent the suprising event. Newly recruited clusters are available to explain future events and can themselves evolve into prototypes/attractors/rules. SUSTAIN has expanded the scope of findings that models of human category learning can address. This paper extends SUSTAIN so that it can be used to account for both supervised and unsupervised learning data through a common mechanism. A modified recruitment mechanism is introduced that creates new conceptual clusters in response to surprising events during learning. The new formulation of the model is called uSUSTAIN for "unifed SUSTAIN." The implications of using a unified recruitment method for both supervised and unsupervised learning is discussed.

SUSTAIN (Supervised and Unsupervised STratified Adaptive Incremental Network) is a network model of human category learning. SUSTAIN initially assumes a simple category structure. If simple solutions prove inadequate and SUSTAIN is confronted with a surprising event (e.g., it is told that a bat is a mammal instead of a bird), SUSTAIN recruits an additional cluster to represent the surprising event. Newly recruited clusters are available to explain future events and can themselves evolve into prototypes/attractors/rules. SUSTAIN has expanded the scope of findings that models of human category learning can address. This paper extends SUSTAIN to account for both supervised and unsupervised learning data through a common mechanism. The modified model, uSUSTAIN (unified SUSTAIN), is successfully applied to human learning data drawn from Love (2002) that compares unsupervised and supervised learning performance.

How do people learn and organize examples in the absence of a teacher? This paper explores this question through a examination of human data and computational modeling results. The SUSTAIN (Supervised and Unsupervised STratified Incremental Network) model successfully fits human learning data drawn from two published studies. The first study examines how correlations between features can facilitate unsupervised learning. The second set of studies examines the role that similarity and attention play in unsupervised category construction (i.e., sorting) tasks. Importantly, SUSTAIN suggests two novel behavioral predictions that are confirmed.

SUSTAIN (Supervised and Unsupervised STratified Adaptive Incremental Network) is a network model of human category learning. This paper extends SUSTAIN so that it can be used to model unsupervised learning data. A modified recruitment mechanism is introduced that creates new conceptual clusters in response to surprising events during learning. Two seemingly contradictory unsupervised learning data sets are modeled using this new recruitment method. In addition, the feasibility of using a unified recruitment method for both supervised and unsupervised learning is discussed.

Suspense is a cognitive and affective state that is often experienced in theanticipation of information and contributes to the enjoyment and consump-tion of entertainment such as movies or sports. Ely, Frankel, and Kamenica(2015) proposed a formal definition of suspense which relies upon predictionsabout future belief updates. In order to empirically evaluate this theory, wedesigned a task based on the casino card game Blackjack where a varietyof suspense dynamics can be experimentally induced. Our behavioral dataconfirmed the explanatory power of this theory. We further compared thisformulation with other heuristic models inspired by studies in other do-mains such as narratives and found that most heuristic models cannot wellaccount for the specific temporal dynamics of suspense across wide range ofgame variants. We additionally propose a way to test whether experienc-ing greater levels of suspense motivates more game-playing. In summary,this work is an initial attempt to link formal models of information anduncertainty with affective cognitive states and motivation.

The study of question asking in humans and machines has gained attention in recent years. A key aspect of question ask- ing is the ability to select good (informative) questions from a provided set. Machines?in particular neural networks? generally struggle with two important aspects of question ask- ing, namely to learn from the answer to their selected ques- tion and to flexibly adjust their questioning to new goals. In the present paper, we show that people are sensitive to both of these aspects and describe a unified Bayesian account of ques- tion asking that is capable of similar ingenuity. In the first ex- periment, we predict people?s judgments when adjusting their question-asking towards a particular goal. In the second ex- periment, we predict people?s judgments when deciding what follow-up question to ask. An alternative model based on su- perficial features, such as the existence of certain key words in the questions, was not able to capture these judgments to a reasonable degree.

Suspense is an affective state that contributes to our enjoy- ment of experiences such as movies and sports. Ely, Frankel, and Kamenica (2015) proposed a formal definition of suspense which depends on the variance of subjective future beliefs about an outcome of interest (e.g., winning a game). In order to evaluate this theory, we designed a task based on the card game Blackjack where a variety of suspense dynamics can be experimentally induced. By presenting participants with iden- tical sequences of information (i.e., card draws), but manip- ulating contextual knowledge (i.e., their understanding of the rules of the game) we were able to show that self-reported sus- pense follows the predictions of the model. Follow-up model comparison further showed an advantage for the ?suspense as variance of future beliefs? account over a number of alterna- tive definitions of suspense, including some that depend only on current uncertainty (not the future). This paper is an initial attempt to link aspects of formal models of information and uncertainty with affective cognitive states.

Intervening on causal systems can illuminate their underlying structures. Past work has shown that, relative to adults, young children often make intervention decisions that confirm sin- gle hypotheses rather than those that discriminate alternative hypotheses. Here, we investigated how the ability to make in- formative intervention decisions changes across development. Ninety participants between the ages of 7 and 25 completed 40 different puzzles in which they had to intervene on vari- ous causal systems to determine their underlying structures. We found that the use of discriminatory strategies increased through adolescence and plateaued into adulthood. Our results identify a clear developmental trend in causal reasoning, and highlight the need to expand research on causal learning mech- anisms in adolescence.

This paper reports three experiments testing whether active control of study leads to enhanced learning in 5- to 10-year-old children. Children played a simple memory game on a touchscreen tablet. In Experiments 1 and 2, the goal was to study and remember as many as possible from set of 64 everyday objects. In Experiment 3 the goal was to learn the French names for the same objects. For half of the materials presented, participants could decide the order and pacing of study (Active condition). For the other half of the materials, they passively observed the study decisions of a previous participant?s active study phase (Yoked condition). Across all three experiments, we found that memory was more accurate for objects studied in the active condition as compared to the yoked condition. However, the advantage for active learning was relatively small among 5-year-olds and increased with children?s age, becoming comparable to adults? by age 8. Our results suggest that the ability to effectively control study emerges and develops during early childhood and leads to lasting memory benefits over a week delay.

What is the best way of discovering the underlying structure of a causal system composed of multiple variables? One prominent idea is that learners should manipulate each candidate variable in isolation to avoid confounds (known as the ``Control of Variables'' or CV strategy). Here, we demonstrate that CV is not always the most efficient method for learning. Using an optimal learner model which aims to minimize the average number of tests, we show that when a causal system is sparse (when the outcome of interest has few or even just one actual cause among the candidate variables) it is more efficient to test multiple variables at once. Across a series of behavioral experiments, we then show that people are sensitive to causal sparsity. When interacting with a non-sparse causal system system (high proportion of causes among variables), they use a CV strategy, changing one variable at a time. When interacting with a sparse system (low proportion of causes among variables) they are more likely to test multiple variables at once. However, we also consistently find that some people use a CV strategy even when a system is sparse. This runs counter to prior work, which showed that the CV principle can be challenging to learn.

When encountering unfamiliar physical objects, children and adults often perform structured interrogatory actions such as grasping and prodding, so revealing latent physical properties such as masses and textures. However, the processes driving and supporting these curious behaviors are still largely mys- terious. In this paper, we develop and train an agent able to actively uncover latent physical properties such as the mass and force of objects in a simulated physical ?micro-world?. Concretely, we used a simulation-based-inference framework to quantify the physical information produced by observation and interaction with the evolving dynamic environment. We used model-free reinforcement learning algorithm to train an agent to implement general strategies for revealing latent phys- ical properties. We compare the behaviors of this agent to the human behaviors observed in a similar task.

People ask questions in order to efficiently learn about the world. But do people ask good questions? In this work, we designed an intuitive, game- based task that allowed people to ask natural language questions to resolve their uncertainty. Question quality was measured through Bayesian ideal- observer models that considered large spaces of possible game states. During free-form question generation, participants asked a creative variety of useful and goal-directed questions, yet they rarely asked the best questions as identified by the Bayesian ideal-observers (Experiment 1). In subsequent experiments, participants strongly preferred the best questions when evaluating questions that they did not generate themselves (Experiments 2 & 3). On the one hand, our results show that people can accurately evaluate question quality, even when the set of questions is diverse and an ideal-observer analysis has large computational requirements. On the other hand, people have a limited ability to synthesize maximally-informative questions from scratch, suggesting a bottleneck in the question asking process.

Acting effectively in the world requires learning and control-ling dynamic systems, that is, systems involving feedback relations among continuous variables that vary in real time. Weintroduce a novel class of dynamic control environments usingthe continuous Ornstein-Uhlenbeck process united with causal Markov graphs that allow us to systematically test people?sability to learn and control various dynamic systems as they change in real time. We find that people?s control is robust to changing goals, and exhibits heterogeneity of performance in different environments that matches closely with complexity defined by our optimal model. These results suggest peopleare capable learners of dynamic systems, able to leverage a rich representation of their environment to accomplish their goals.

Many aspects of our physical environment are hidden. For example, it is hard to estimate how heavy an object is from visual observation alone. In this paper we examine how people actively ?experiment? within the physical world to discover such latent properties. In the first part of the paper, we develop a novel framework for the quantitative analysis of the information produced by physical interactions. We then describe two experiments that present participants with moving objects in ?microworlds? that operate according to continuous spatiotemporal dynamics similar to everyday physics (i.e., forces of gravity, friction, etc.). Participants were asked to in- teract with objects in the microworlds in order to identify their masses, or the forces of attrac- tion/repulsion that governed their movement. Using our modeling framework, we find that learners who freely interacted with the physical system selectively produced evidence that re- vealed the physical property consistent with their inquiry goal. As a result, their inferences were more accurate than for passive observers and, in some contexts, for yoked participants who watched video replays of an active learner?s interactions. We characterize active learners? actions into a range of micro-experiment strategies and discuss how these might be learned or gen- eralized from past experience. The technical contribution of this work is the development of a novel analytic framework and methodology for the study of interactively learning about the physical world. Its empirical contribution is the demonstration of sophisticated goal directed human active learning in a naturalistic context.

Learning usually improves the accuracy of beliefs through the accumulation of experience. But are there limits to learning that prevent us from accurately understanding our world? In this paper we investigate the concept of a "learning trap" - the formation of a stable false belief even with extensive experience. Our review highlights how these traps develop though the interaction of learning and decision making in unknown environments. We further document a particularly pernicious learning trap driven by selective attention, a mechanism often assumed to faciliate learning in complex environments. Using computer simulation we demonstrate the key attributes of the agent and environment that lead to this new type of learning trap. Then, in a series of experiments we present evidence that people robustly fall into this trap, even in the presence of various interventions predicted to meliorate it. These results highlight a fundamental limit to learning and adaptive behavior that impacts individuals, organizations, animals, and machines.

The ability to act on the world with the goal of gaining information is what makes humans such an adaptable and intelligent species. Perhaps the most successful and influential account of such abilities is the Optimal Experiment Design (OED) hypothesis, which argues that humans intuitively perform experiments on the world similar to the way an effective scientist plans an experiment. The widespread application of this theory within many areas of psychology calls for a critical evaluation of the theory?s core claims. Despite many successes, we argue that the OED hypothesis remains lacking as a theory of human inquiry and that research in the area often fails to confront some of the most interesting and important questions. In this critical review, we raise and discuss nine open questions about the psychology of human inquiry.

A hallmark of human intelligence is the ability to ask rich, creative, and revealing questions. Here we introduce a cognitive model capable of constructing humanlike questions. Our approach treats questions as formal programs that, when executed on the state of the world, output an answer. The model specifies a probability distribution over a complex, compositional space of programs, favoring concise programs that help the agent learn in the current context. We evaluate our approach by modeling the types of open-ended questions generated by humans who were attempting to learn about an ambiguous situation in a game. We find that our model predicts what questions people will ask, and can creatively produce novel questions that were not present in the training set. In addition, we compare a number of model variants, finding that both question informativeness and complexity are important for producing human-like questions.

What is the best way of figuring out the structure of a causal system composed of multiple variables? One prominent idea is that learners should manipulate each candidate variable in isolation to avoid confounds (known as the ?Control of Vari- ables? strategy). Here, we demonstrate that this strategy is not always the most efficient method for learning. Using an opti- mal learner model which aims to minimize the number of tests, we show that when a causal system is sparse, that is, when the outcome of interest has few or even just one actual cause among the candidate variables, it is more efficient to test mul- tiple variables at once. In a series of behavioral experiments, we then show that people are sensitive to causal sparsity when planning causal experiments.

This study explores developmental changes in the ability to ask informative questions. We hypothesized an intrinsic link between the ability to update beliefs in light of evidence and the ability to ask informative questions. Four- to ten-year-old children played an iPad game asking them to identify a hidden bug. Learners could either ask about individual bugs, or make a series of feature queries (e.g., "Does the hidden bug have antenna") that could more efficiently narrow the hypothesis space. Critically the task display either helped children inte- grate evidence with the hypothesis space or required them to perform this operation themselves. Although we found that helping children update their beliefs improved some aspects of their active inquiry behavior, children required to update their own beliefs asked questions that were more context-sensitive and thus informative. The results show how making a task more difficult may actually improve children?s active inquiry skills, thus illustrating a type of desirable difficulty.

The ability to ask questions during learning is a key aspect of human cognition. While recent research has suggested com- mon principles underlying human and machine ?active learning,? the existing literature has focused on relatively simple types of queries. In this paper, we study how humans construct rich and sophisticated natural language queries to search for information in a large yet computationally tractable hypothesis space. In Experiment 1, participants were allowed to ask any question they liked in natural language. In Experiment 2, par- ticipants were asked to evaluate questions that they did not generate themselves. While people rarely asked the most informa- tive questions in Experiment 1, they strongly preferred more informative questions in Experiment 2, as predicted by an ideal Bayesian analysis. Our results show that rigorous information- based accounts of human question asking are more widely applicable than previously studied, explaining preferences across a diverse set of natural language questions.

This paper reports an experiment testing whether volitional control over the presentation of stimuli leads to enhanced recognition memory in 6- to 8-year-old children. Children were presented with a simple memory game on an iPad. During the study phase, for half of the materials children could decide the order and pacing of stimuli presentation (active condition). For the other half of the materials, children observed the study choices of another child (yoked condition). We found that recognition performance was better for the objects studied in the active condition as compared to the yoked condition. Furthermore, we found that the memory advantages of active learning persisted over a one-week delay between study and test. Our results support pedagogical approaches that emphasize self-guided learning and show that even young children benefit from being able to control how they learn.

This study explores developmental changes in the ability to ask informative questions. We hypothesized an intrinsic link between the ability to update beliefs in light of evidence and the ability to ask informative questions. Four- to ten-year-old children played an iPad game asking them to identify a hidden bug. Learners could either ask about individual bugs, or make a series of feature queries (e.g., ?Does the hidden bug have antenna??) that could more efficiently narrow the hypothesis space. Critically the task display either helped children inte- grate evidence with the hypothesis space or required them to perform this operation themselves. Although we found that helping children update their beliefs improved some aspects of their active inquiry behavior, children required to update their own beliefs asked questions that were more context-sensitive and thus informative. The results show how making a task more difficult may actually improve children?s active inquiry skills, thus illustrating a type of desirable difficulty.

Despite widespread consensus among educators that active learning leads to better outcomes than comparatively passive forms of instruction, it is often unclear why these benefits arise. In this article we review research showing that the opportunity to control the information experienced while learning leads to improved memory relative to situations where control is absent. By integrating findings from a wide range of experimental paradigms, we identify a set of distinct mechanisms that mediate these effects, including the formation of distinctive sensorimotor associations, elaborative encoding due to goal-directed exploration, improved coordination of selective attention and encoding, adaptive selection of material based on existing memory, and metacognitive monitoring. Examining these mechanisms provides new insights into the effects of active learning, including how different forms of active control lead to improved outcomes relative to more traditional, passive instruction.

Collecting (or "sampling") information that one expects to be useful is a powerful way to facilitate learning. However, little is known about how people decide which information is worth learning about, particularly during the course of learning. We describe several alternative models of how people might decide to collect a piece of information inspired by "active learning" research in machine learning. We additionally provide a theoretical analysis demonstrating the situations under which these models are empirically distinguishable, and report a novel empirical study that exploits these insights. Our model-based analysis of participant?s information gathering decisions reveal that people prefer to select items which resolve the uncertainty between two possibilities at a time rather than items that have high uncertainty across all relevant possibilities simultaneously. The results appear to challenge standard normative models of the value of information as well as accounts which argue that people are hamstrung by confirmation bias.

People often make decisions with the goal of gaining information which can help reduce their uncertainty. However, recent work has suggested that people sometimes do not select the most diagnostic information queries available to them. A critical aspect of information search decisions is evaluating how obtaining a piece of information will alter a learner?s beliefs (e.g., a piece of information that is redundant with what is already known is useless). This suggests a close relationship between information seeking decisions on one hand, and belief updating on the other. This paper explores the deeper relationship between these two constructs in a causal intervention learning task. We find that patterns in belief updating biases are predictive of decision making patterns in tasks where people must make interventions learn about the structure of a causal system.

How do people choose interventions to learn about causal systems? Here, we considered two possibilities. First, we test an information sampling model, information gain, which values interventions that can discriminate between a learner?s hypotheses (i.e. possible causal structures). We compare this discriminatory model to a positive testing strategy that instead aims to confirm individual hypotheses. Experiment 1 shows that individual behavior is described best by a mixture of these two alternatives. In Experiment 2 we find that people are able to adaptively alter their behavior and adopt the discriminatory model more often after experiencing that the confirmatory strategy leads to a subjective performance decrement. In Experiment 3, time pressure leads to the opposite effect of inducing a change towards the simpler positive testing strategy. These findings suggest that there is no single strategy that describes how intervention decisions are made. Instead, people select strategies in an adaptive fashion that trades off their expected performance and cognitive effort.

People can test hypotheses through either selection or reception. In a selection task, the learner actively chooses observations to test their beliefs, while in reception tasks data is passively encountered. People routinely use both forms of testing in everyday life, but the critical psychological differences between selection and reception learning remain poorly understood. One hypothesis is that selection learning improves learning performance by enhancing generic cognitive processes related to motivation, attention, and engagement. Alternatively, we suggest that differences between these two learning modes derives from a hypothesis-dependent sampling bias that is introduced when a person collects data to test their own individual hypothesis. Drawing on influential models of sequential hypothesis testing behavior, we show that such a bias 1) can lead to the collection of data that facilitates learning compared to reception learning, and 2) can be more effective than observing the selections of another person. We then report a novel experiment based on a popular category learning paradigm that compares reception and selection learning. We additionally compare selection learners to a set of ``yoked" participants who viewed the exact same sequence of observations under reception conditions. The results revealed systematic differences in performance that depended on the learner's role in collecting information and the abstract structure of the problem.

The potential information gained from asking a question and one?s uncertainty about the answer to that question are not always the same. For example, given a coin that one believes to be fair, the uncertainty a person has about the outcome of flipping that coin is high, but either outcome is unlikely to make them believe that the coin is biased (i.e., the ?information gain? of that observation is low). In the present paper we show that people use a simple form of predictive uncertainty to guide their information sampling decisions, a strategy which is often equivalent to maximizing information gain, but is less efficient in environments where potential queries vary in their reliability. We conclude that a potentially powerful driver of human information gathering may be the inability to predict what will happen as a result of an action or query.

How do people choose interventions to learn about a causal system? Here, we tested two possibilities: an optimal infor- mation sampling strategy which aims to discriminate between multiple hypotheses, and a second strategy that aims to confirm individual hypotheses. We show in Experiment 1 that individual behavior is best fit using a mixture of these two options. In a second experiment, we find that people are able to adaptively alter the strategies they use in response to their expected payoff in a particular task environment.

Self-directed learning is often associated with better long-term memory retention; however, the mechanisms that underlie this advantage remain poorly understood. This series of experiments was designed to ?deconstruct? the notion of self-directed learning, in order to better identify the factors most responsible for these improvements to memory. In particular, we isolated the memory advantage that comes from controlling the content of study episodes from the advantage that comes from controlling the timing of those episodes. Across four experiments, self-directed learning significantly enhanced recognition memory, relative to passive observation. However, the advantage for self-directed learning was found to be present even under extremely minimal conditions of volitional control (simply pressing a button when a participant was ready to advance to the next item). Our results suggest that improvements to memory following self-directed encoding may be related to the ability to coordinate stimulus presentation with the learner?s current preparatory or attentional state, and they highlight the need to consider the range of cognitive control processes involved in and influenced by self-directed study.

An important type of decision making concerns how people choose to gather information which reduces their uncertainty about the world. For example, when learning about a novel piece of technology, like a smartphone, people often actively intervene on various aspects in order to better understand the function of the system. Interventions allow us to tell apart causal structures that are indistinguishable through observation, but only if the right variables are intervened on. Normative models of decision making developed in the machine learning literature specify a process of comparing hypotheses to identify those interventions that will allow a learner to distinguish between them. An experiment that asked subjects to decide between two causal hypotheses found that while they often chose useful interventions, they frequently perform interventions whose expected effects were typical of one causal structure but that did not always allow the two structures to be distinguished. We interpret this tendency as a type of positive-test-strategy with a preference for outcomes that are representative of a single causal structure.

Self-directed information sampling - the ability to collect information that one expects to be useful - has been shown to improve the efficiency of concept acquisition for both human and machine learners. However, little is known about how people decide which information is worth learning more about. In this study, we examine self-directed learning in a relatively complex rule learning task that gave participants the ability to "design and test" stimuli they wanted to learn about. On a subset of trials we recorded participants' uncertainty about how to classify the item they had just designed. Analyses of these uncertainty judgments show that people prefer gathering information about items that help refine one rule at a time (i.e., those that fall close to a pairwise category "margin") rather than items that have the highest overall uncertainty across all relevant hypotheses or rules. Our results give new insight into how people gather information to test currently entertained hypotheses in complex problem solving tasks.

A critical aspect of human cognition is the ability to actively query the environment for information. One important (but often overlooked) factor in the decision to gather information is the cost associated with accessing different sources of information. Using a simple sequential information search task, we explore the degree to which human learners are sensitive to variations in the amount of utility related to different potential observations. Across two experiments we find greater support for the idea that people gather information to reduce their uncer- tainty about the current state of the environment (a "disinterested", or cost-insenstive, sampling strategy). Implications for theories of rational information collection are discussed.

A widely advocated idea in education is that people learn better when the flow of experience is under their control (i.e., learning is self-directed). However, the reasons why volitional control might result in superior acquisition, and the limits to such advantages, remain poorly understood. We review this issue from both a cognitive and computational perspective. On the cognitive side, self-directed learning allows individuals to focus effort on useful information they do not yet possess, can expose information that is inaccessible via passive observation, and may enhance the encoding and retention of materials. On the computational side, "active learning" algorithms that optimize learning by selecting their own learning experiences is an emerging research topic in machine learning. This review argues that recent advances in these related fields may offer a fresh theoretical perspective on how people gather information to support their own learning.

We investigated how subjects sample information in order to improve performance in a visuomotor estimation task. Subjects were rewarded for touching a hidden circular target based on visual cues to the target?s location. The cues were 'dots' drawn from a Gaussian distribution centered on the middle of the target. Subjects could sample as many cues as they wished, but the potential reward for hitting the target decreased by a fixed amount for each additional cue requested. The subjects' objective was to balance the benefits of increased information against the costs incurred in acquiring it. We compared human performance to ideal and found that subjects sampled more cues than dictated by the optimal stopping rule that tries to maximize expected gain. We contrast our results with recent reports in the literature that subjects typically under-sample.

Laboratory studies of human category learning tend to emphasize passive learning situations by limiting participants' control over what information they experience on every trial. In this paper, we explore the impact that active data selection has on category learning. In our experiment, participants attempted to learn standard rule-based (RB) and information-integration (II) categories under either entirely passive (observational) conditions, or by actively selecting and querying the labels associated with particular stimuli. Our primary aim was to characterize the information sampling strategy that participants adopted in the task and to examine how the passive/active learning distinction interacted with the structure of the categories. We found that participants acquired categories faster when they were able to select and query category members on their own. Furthermore, this advantage depended on learners actually making the decisions about which stimuli to query themselves rather than simply the statistics of the experienced exemplars. Model based analyses explain this effect in terms of the number of active hypotheses under consideration which is assumed to be higher in the active learning condition due to the greater engagement of the learner in the task

Effective learning often involves actively querying the environment for information that disambiguates potential hypotheses. However, the space of observations available in any situation can vary greatly in potential "informativeness." In this report, we study participants' ability to gauge the information value of potential observations in a cognitive search task based on the children's game Battleship. Participants selected observations to disambiguate between a large number of potential game configurations subject to information-collection costs and penalties for making errors in a test phase. An "ideal-learner" model is developed to quantify the utility of possible observations in terms of the expected gain in points from knowing the outcome of that observation. The model was used as a tool for measuring search efficiency, and for classifying various types of information collection decisions. We find that participants are generally effective at maximizing gain relative to their current state of knowledge and the constraints of the task. In addition, search behavior shifts between an slower, but more efficient "exploitive" mode of local search and a faster, less efficient pattern of "exploration."

Recent work has demonstrated robust learning traps during learning from experience ? decision-making biases that persist due to the choice-contingent nature of outcome feedback. In two experiments, we investigate the effect of outcome valence on learning trap development. Participants chose to approach or avoid category exemplars associated with rewards or losses, and, to maximize reward, must learn a categorization rule based on two stimulus dimensions. We replicate previous findings showing that when outcome feedback was contingent upon approaching exemplars, people frequently fell into the trap of using an incomplete categorization rule based on only a single dimension, which was suboptimal for long-term reward. Notably, learning trap development was attenuated in an environment with frequent loss outcomes, even when participants received explicit information about the base rates of gains and losses. The implications of these findings for theoretical models and future research are discussed.

From navigation in unfamiliar environments to career plan- ning, people typically first sample information before com- mitting to a plan. However, most studies find that people adopt myopic strategies when sampling information. Here we challenge those findings by investigating whether contingency planning is a driver of information sampling. To this aim, we developed a novel navigation task that is a shortest path find- ing problem under uncertainty of bridge closures. Participants (n = 109) were allowed to sample information on bridge sta- tuses prior to committing to a path. We developed a computa- tional model in which the agent samples information based on the cost of switching to a contingency plan. We find that this model fits human behavior well and is qualitatively similar to the approximated optimal solution. Together, this suggests that humans use contingency planning as a driver of information sampling.

This paper asks how strategies of information sampling are affected by a learner's goal. Based on a theoretical analysis and two behavioral experiments, we show that learning goals have a crucial impact on the decision of when to stop sampling. This decision, in turn, affects the statistical properties (e.g. average values, or standard deviations) of the data collected under different goals. Specifically, we find that sampling with the goal of making a binary choice can introduce a correlation between the average value of a sample and its size (the number of values sampled). Across multiple rounds of sampling, this has the potential of biasing learners? inferences about the underlying process that generated the samples, specifically if learners ignore sample size when making these inferences. We find that people are indeed biased in this way and make different inferences about the same data-generating process when sampling with different learning goals. These findings highlight yet another danger of inferring general patterns from samples of evidence the learner had a hand in collecting. [This paper ended up just as a preprint]

Teaching is an intuitive social activity that requires reason- ing about and influencing the mind of others. A good teacher forms a belief about the knowledge of their student, asks clar- ifying questions, and gives instructions or explanations to try to induce a target concept in the student?s mind. We propose Partially Observable Markov Decision Processes (POMDPs) as a model of intuitive human teaching. According to this ac- count, teachers make pedagogical decisions with uncertainty about the knowledge state of their student. In two behavioral experiments, human participants were tasked with balancing assessments (asking questions) and instructions to help teach a student to build a tower of colored blocks. Human behavior in the task was compared to the performance of a computerized teaching algorithm optimized to solve the equivalent POMDP. Our results show that humans favor asking questions and estab- lishing common ground during teaching even at an economic cost and increase question asking as uncertainty grows.

Intervening on causal systems can illuminate their underlying structures. Past work has shown that, relative to adults, young children often make intervention decisions that appear to confirm a single hypothesis rather than those that optimally discriminate alternative hypotheses. Here, we investigated how the ability to make informative causal interventions changes across development. Ninety participants between the ages of 7 and 25 completed 40 different puzzles in which they had to intervene on various causal systems to determine their underlying structures. Each puzzle comprised a three- or four-node computer chip with hidden wires. On each trial, participants viewed two possible arrangements of the chip?s hidden wires and had to select a single node to activate. After observing the outcome of their intervention, participants selected a wire configuration and rated their confidence in their selection. We characterized participant choices with a Bayesian measurement model that indexed the extent to which participants selected nodes that would best disambiguate the two possible causal structures vs. those that had high causal centrality in one of the two causal hypotheses but did not necessarily discriminate between them. Our model estimates revealed that the use of a discriminatory strategy increased through early adolescence. Further, developmental improvements in intervention strategy were related to changes in the ability to accurately judge the strength of evidence that interventions revealed, as indexed by participants? confidence in their selections. Our results suggest that improvements in causal information-seeking extend into adolescence and may be driven by metacognitive sensitivity to the efficacy of previous interventions in discriminating competing ideas.

What is the best way of discovering the underlying structure of a causal system composed of multiple variables? One prominent idea is that learners should manipulate each candidate variable in isolation to avoid confounds (known as the ``Control of Variables'' or CV strategy). Here, we demonstrate that CV is not always the most efficient method for learning. Using an optimal learner model which aims to minimize the average number of tests, we show that when a causal system is sparse (when the outcome of interest has few or even just one actual cause among the candidate variables) it is more efficient to test multiple variables at once. Across a series of behavioral experiments, we then show that people are sensitive to causal sparsity. When interacting with a non-sparse causal system system (high proportion of causes among variables), they use a CV strategy, changing one variable at a time. When interacting with a sparse system (low proportion of causes among variables) they are more likely to test multiple variables at once. However, we also consistently find that some people use a CV strategy even when a system is sparse. This runs counter to prior work, which showed that the CV principle can be challenging to learn.

Time is a central aspect of exploratory choice. People must balance the immediate rewards of exploiting known alternatives against the long-term rewards of exploring uncertain alternatives. However, little research has investigated how this temporal aspect affects the exploratory decisions people make. In other intertemporal choices, people display a bias towards immediate rewards. We hypothesize that in exploratory choice, this present bias will cause them to under- explore. Across three experiments, including a preregistered design, we find no evidence that present bias influences exploratory choice, but also conclude that our stimuli may not effectively induce present bias. We discuss how present bias might better be investigated, and possible reasons that present bias may not affect exploratory choice.

Acting effectively in the world requires learning and control-ling dynamic systems, that is, systems involving feedback relations among continuous variables that vary in real time. Weintroduce a novel class of dynamic control environments usingthe continuous Ornstein-Uhlenbeck process united with causal Markov graphs that allow us to systematically test people?sability to learn and control various dynamic systems as they change in real time. We find that people?s control is robust to changing goals, and exhibits heterogeneity of performance in different environments that matches closely with complexity defined by our optimal model. These results suggest peopleare capable learners of dynamic systems, able to leverage a rich representation of their environment to accomplish their goals.

Learning usually improves the accuracy of beliefs through the accumulation of experience. But are there limits to learning that prevent us from accurately understanding our world? In this paper we investigate the concept of a "learning trap" - the formation of a stable false belief even with extensive experience. Our review highlights how these traps develop though the interaction of learning and decision making in unknown environments. We further document a particularly pernicious learning trap driven by selective attention, a mechanism often assumed to faciliate learning in complex environments. Using computer simulation we demonstrate the key attributes of the agent and environment that lead to this new type of learning trap. Then, in a series of experiments we present evidence that people robustly fall into this trap, even in the presence of various interventions predicted to meliorate it. These results highlight a fundamental limit to learning and adaptive behavior that impacts individuals, organizations, animals, and machines.

The tension between exploration and exploitation is one of the primary challenges in decision making under uncertainty. Optimal models of choice prescribe that individuals resolve this tension by evaluating how information gained from their choices will improve future choices. However, research in behavioral economics and psychology has yielded conflicting evidence about whether people consider the future during exploratory choice, particularly in complex, uncertain environments. Adding to the empirical evidence on this issue, we examine exploratory decision making in a novel approach-avoid paradigm. In the first set of experiments we find that people parametrically increase their exploration when the expected number of future encounters with a prospect is larger. In the second we demonstrate that when the number of future encounters is unknown, as is often the case in everyday life, people are sensitive to the relative frequency of future encounters with a prospect. Our experiments show that people adaptively utilize information about the future when deciding to explore, a tendency that may shape decisions across several real-world domains.

What is the best way of figuring out the structure of a causal system composed of multiple variables? One prominent idea is that learners should manipulate each candidate variable in isolation to avoid confounds (known as the ?Control of Vari- ables? strategy). Here, we demonstrate that this strategy is not always the most efficient method for learning. Using an opti- mal learner model which aims to minimize the number of tests, we show that when a causal system is sparse, that is, when the outcome of interest has few or even just one actual cause among the candidate variables, it is more efficient to test mul- tiple variables at once. In a series of behavioral experiments, we then show that people are sensitive to causal sparsity when planning causal experiments.

Although a common assumption in models of perceptual dis- crimination, most models of categorization do not explicitly account for uncertainty in stimulus measurement. Such un- certainty may arise from inherent perceptual noise or noise in stimulus measurement (e.g., a medical test that gives variable results). In this paper we explore how people decide to gather information from various stimulus properties when each sam- ple or measurement is noisy. Participant?s goal is to correctly classify the given item. Across two experiments we find sup- port for the idea that people take category structure into ac- count when selecting information for a classification decision. In addition, we find some evidence that people are also sensi- tive to their own perceptual uncertainty.

Collecting (or "sampling") information that one expects to be useful is a powerful way to facilitate learning. However, little is known about how people decide which information is worth learning about, particularly during the course of learning. We describe several alternative models of how people might decide to collect a piece of information inspired by "active learning" research in machine learning. We additionally provide a theoretical analysis demonstrating the situations under which these models are empirically distinguishable, and report a novel empirical study that exploits these insights. Our model-based analysis of participant?s information gathering decisions reveal that people prefer to select items which resolve the uncertainty between two possibilities at a time rather than items that have high uncertainty across all relevant possibilities simultaneously. The results appear to challenge standard normative models of the value of information as well as accounts which argue that people are hamstrung by confirmation bias.

People usually collect information to serve specific goals and often end up with samples that are unrepresentative of the un- derlying population. This can introduce biases on later judgments that generalize from these samples. Here we show that goals influence not only what information we collect, but also when we decide to terminate search. Using an optimal stop- ping analysis, we demonstrate that even when learners have no control over the content of a sample (i.e., natural sampling), the simple decision of when to stop sampling can yield sample distributions that are non-representative and could potentially bias future decision making. We test the prediction of these theoretical analyses with two behavioral experiments.

The decision to gather information should take into account both the value of information and its accrual costs in time, energy and money. Here we explore how people balance the monetary costs and benefits of gathering additional information in a perceptual-motor estimation task. Participants were rewarded for touching a hidden circular target on a touch-screen display. The target?s center coincided with the mean of a circular Gaussian distribution from which participants could sample repeatedly. Each ?cue? ? sampled one at a time ? was plotted as a dot on the display. Participants had to repeatedly decide, after sampling each cue, whether to stop sampling and attempt to touch the hidden target or continue sampling. Each additional cue increased the participants? probability of successfully touching the hidden target but reduced their potential reward. Two experimental conditions differed in the initial reward associated with touching the hidden target and the fixed cost per cue. For each condition we computed the optimal number of cues that participants should sample, before taking action, to maximize expected gain. Contrary to recent claims that people gather less information than they objectively should before taking action, we found that participants over-sampled in one experimental condition, and did not significantly under- or over-sample in the other. Additionally, while the ideal observer model ignores the current sample dispersion, we found that participants used it to decide whether to stop sampling and take action or continue sampling, a possible consequence of imperfect learning of the underlying population dispersion across trials.

People often make repeated decisions from experience. In such scenarios, persistent biases of choice can develop, most notably the "hot stove effect" (Denrell & March, 2001) in which a prospect that is mistakenly believed to be negative is avoided and thus belief-correcting information is never obtained. In the existing literature, the hot stove effect is generally thought of as developing through interaction with a single, stochastic prospect. Here, we show how a similar bias can develop due to people's tendency to selectively attend to a subset of features during categorization. We first explore the bias through model simulation, then report on an experiment in which we find evidence of a decisional bias linked to selective attention. Finally, we use these computational models to design novel interventions to "de-bias" decision-makers, some of which may have practical application.

People often make decisions with the goal of gaining information which can help reduce their uncertainty. However, recent work has suggested that people sometimes do not select the most diagnostic information queries available to them. A critical aspect of information search decisions is evaluating how obtaining a piece of information will alter a learner?s beliefs (e.g., a piece of information that is redundant with what is already known is useless). This suggests a close relationship between information seeking decisions on one hand, and belief updating on the other. This paper explores the deeper relationship between these two constructs in a causal intervention learning task. We find that patterns in belief updating biases are predictive of decision making patterns in tasks where people must make interventions learn about the structure of a causal system.

How do people choose interventions to learn about causal systems? Here, we considered two possibilities. First, we test an information sampling model, information gain, which values interventions that can discriminate between a learner?s hypotheses (i.e. possible causal structures). We compare this discriminatory model to a positive testing strategy that instead aims to confirm individual hypotheses. Experiment 1 shows that individual behavior is described best by a mixture of these two alternatives. In Experiment 2 we find that people are able to adaptively alter their behavior and adopt the discriminatory model more often after experiencing that the confirmatory strategy leads to a subjective performance decrement. In Experiment 3, time pressure leads to the opposite effect of inducing a change towards the simpler positive testing strategy. These findings suggest that there is no single strategy that describes how intervention decisions are made. Instead, people select strategies in an adaptive fashion that trades off their expected performance and cognitive effort.

see published handbook

Empirical models of strategic games are central to much analysis in marketing and economics. However, two challenges in applying these models to real-wold data are that such models often admit multiple equilibria and that they require strong informational assumptions. The first implies that the model does not make unique predictions about the data, and the second implies that the results may be drive by strong a-priori assumptions about the informational setup. This article summarizes recent work that seeks to address both issues and suggests some avenues for future research.

Adaptive decision making often entails learning to approach things that lead to positive outcomes while avoiding things that are negative. The decision to avoid something removes the risk of a negative experience but also forgoes the opportunity to ob- tain information, specifically that a seemingly negative option is actually positive. This paper explores how people learn to approach or avoid objects with uncertain payoffs. We provide a computational-level analysis of optimal decision making in this problem which quantifies how the probability of encoun- tering an object in the future should impact the decision to ap- proach or avoid it. A large experiment conducted online shows that most people intuitively take into account both their uncer- tainty and the value of information when deciding to approach seemingly bad things.

The potential information gained from asking a question and one?s uncertainty about the answer to that question are not always the same. For example, given a coin that one believes to be fair, the uncertainty a person has about the outcome of flipping that coin is high, but either outcome is unlikely to make them believe that the coin is biased (i.e., the ?information gain? of that observation is low). In the present paper we show that people use a simple form of predictive uncertainty to guide their information sampling decisions, a strategy which is often equivalent to maximizing information gain, but is less efficient in environments where potential queries vary in their reliability. We conclude that a potentially powerful driver of human information gathering may be the inability to predict what will happen as a result of an action or query.

Self-directed learning is often associated with better long-term memory retention; however, the mechanisms that underlie this advantage remain poorly understood. This series of experiments was designed to ?deconstruct? the notion of self-directed learning, in order to better identify the factors most responsible for these improvements to memory. In particular, we isolated the memory advantage that comes from controlling the content of study episodes from the advantage that comes from controlling the timing of those episodes. Across four experiments, self-directed learning significantly enhanced recognition memory, relative to passive observation. However, the advantage for self-directed learning was found to be present even under extremely minimal conditions of volitional control (simply pressing a button when a participant was ready to advance to the next item). Our results suggest that improvements to memory following self-directed encoding may be related to the ability to coordinate stimulus presentation with the learner?s current preparatory or attentional state, and they highlight the need to consider the range of cognitive control processes involved in and influenced by self-directed study.

How do people choose interventions to learn about a causal system? Here, we tested two possibilities: an optimal infor- mation sampling strategy which aims to discriminate between multiple hypotheses, and a second strategy that aims to confirm individual hypotheses. We show in Experiment 1 that individual behavior is best fit using a mixture of these two options. In a second experiment, we find that people are able to adaptively alter the strategies they use in response to their expected payoff in a particular task environment.

An important type of decision making concerns how people choose to gather information which reduces their uncertainty about the world. For example, when learning about a novel piece of technology, like a smartphone, people often actively intervene on various aspects in order to better understand the function of the system. Interventions allow us to tell apart causal structures that are indistinguishable through observation, but only if the right variables are intervened on. Normative models of decision making developed in the machine learning literature specify a process of comparing hypotheses to identify those interventions that will allow a learner to distinguish between them. An experiment that asked subjects to decide between two causal hypotheses found that while they often chose useful interventions, they frequently perform interventions whose expected effects were typical of one causal structure but that did not always allow the two structures to be distinguished. We interpret this tendency as a type of positive-test-strategy with a preference for outcomes that are representative of a single causal structure.

Self-directed information sampling - the ability to collect information that one expects to be useful - has been shown to improve the efficiency of concept acquisition for both human and machine learners. However, little is known about how people decide which information is worth learning more about. In this study, we examine self-directed learning in a relatively complex rule learning task that gave participants the ability to "design and test" stimuli they wanted to learn about. On a subset of trials we recorded participants' uncertainty about how to classify the item they had just designed. Analyses of these uncertainty judgments show that people prefer gathering information about items that help refine one rule at a time (i.e., those that fall close to a pairwise category "margin") rather than items that have the highest overall uncertainty across all relevant hypotheses or rules. Our results give new insight into how people gather information to test currently entertained hypotheses in complex problem solving tasks.

A critical aspect of human cognition is the ability to actively query the environment for information. One important (but often overlooked) factor in the decision to gather information is the cost associated with accessing different sources of information. Using a simple sequential information search task, we explore the degree to which human learners are sensitive to variations in the amount of utility related to different potential observations. Across two experiments we find greater support for the idea that people gather information to reduce their uncer- tainty about the current state of the environment (a "disinterested", or cost-insenstive, sampling strategy). Implications for theories of rational information collection are discussed.

How do people manipulate their environment when balancing trade-offs between probability of success and payoff? Individuals in a city park played a simple lottery using a small set of marbles placed in an urn. Participants had the ability to actively improve their chances of winning but only by reducing the amount of money that they could possibly win. Hence, participants controlled the lottery?s intuitive trade-off between probability of success and potential payout. Across four different lottery structures, participants, on average, behaved systematically safer than the optimal strategy that maximizes expected gain. We explore two different accounts of this sub-optimal choice behavior: probability distortion, and intrinsic utility of winning.

This study examines how people deal with inherently stochastic cues when estimating a latent environmental property. Seven cues to a hidden location were presented one at a time in rapid succession. The seven cues were sampled from seven different Gaussian distributions that shared a common mean but differed in precision (the reciprocal of variance). The experimental task was to estimate the common mean of the Gaussians from which the cues were drawn. Observers ran in two conditions on separate days. In the "decreasing precision" condition the seven cues were ordered from most precise to least precise. In the "increasing precision" condition this ordering was reversed. For each condition, we estimated the weight that each cue in the sequence had on observers' estimates and compared human performance to that of an ideal observer who maximizes expected gain. We found that observers integrated information from more than one cue, and that they adaptively gave more weight to more precise cues and less weight to less precise cues. However, they did not assign weights that would maximize their expected gain, even over the course of several hundred trials with corrective feedback. The cost to observers of their sub-optimal performance was on average 16% of their maximum possible winnings.

A widely advocated idea in education is that people learn better when the flow of experience is under their control (i.e., learning is self-directed). However, the reasons why volitional control might result in superior acquisition, and the limits to such advantages, remain poorly understood. We review this issue from both a cognitive and computational perspective. On the cognitive side, self-directed learning allows individuals to focus effort on useful information they do not yet possess, can expose information that is inaccessible via passive observation, and may enhance the encoding and retention of materials. On the computational side, "active learning" algorithms that optimize learning by selecting their own learning experiences is an emerging research topic in machine learning. This review argues that recent advances in these related fields may offer a fresh theoretical perspective on how people gather information to support their own learning.

We investigated how subjects sample information in order to improve performance in a visuomotor estimation task. Subjects were rewarded for touching a hidden circular target based on visual cues to the target?s location. The cues were 'dots' drawn from a Gaussian distribution centered on the middle of the target. Subjects could sample as many cues as they wished, but the potential reward for hitting the target decreased by a fixed amount for each additional cue requested. The subjects' objective was to balance the benefits of increased information against the costs incurred in acquiring it. We compared human performance to ideal and found that subjects sampled more cues than dictated by the optimal stopping rule that tries to maximize expected gain. We contrast our results with recent reports in the literature that subjects typically under-sample.

Laboratory studies of human category learning tend to emphasize passive learning situations by limiting participants' control over what information they experience on every trial. In this paper, we explore the impact that active data selection has on category learning. In our experiment, participants attempted to learn standard rule-based (RB) and information-integration (II) categories under either entirely passive (observational) conditions, or by actively selecting and querying the labels associated with particular stimuli. Our primary aim was to characterize the information sampling strategy that participants adopted in the task and to examine how the passive/active learning distinction interacted with the structure of the categories. We found that participants acquired categories faster when they were able to select and query category members on their own. Furthermore, this advantage depended on learners actually making the decisions about which stimuli to query themselves rather than simply the statistics of the experienced exemplars. Model based analyses explain this effect in terms of the number of active hypotheses under consideration which is assumed to be higher in the active learning condition due to the greater engagement of the learner in the task

Perceptual aliasing arises in situations where multiple, distinct states of the world give rise to the same percept. In this study, we examine how the degree of perceptual aliasing in a task impacts the ability of human agents to learn reward-maximizing decision strategies. Previous work has shown that the presence of perceptual cues that help signal distinct states of the environment can improve the ability of learners to adopt an optimal decision strategy in sequential decision making tasks (Gureckis & Love, 2009). In our experiments, we parametrically manipulated the degree of perceptual aliasing afforded by certain perceptual cues in a similar task. Our empirical results and simulations show how the ability of the learner improves as relevant states in the world uniquely map to differentiated percepts. The results provide further support for the model of sequential decision making proposed by Gureckis & Love (2009) and highlight the important role that state representations may have on behavior in dynamic decision making and learning tasks.

This work explores the influence of motivation on choice behavior in a dynamic decision-making environment, where the payoffs from each choice depend on one?s recent choice history. Previous research reveals that participants in a regulatory fit exhibit increased levels of exploratory choice and flexible use of multiple strategies over the course of an experiment. The present study placed promotion and prevention-focused participants in a dynamic environment for which optimal performance is facilitated by systematic exploration of the decision space. These participants either gained or lost points with each choice. Our experiment revealed that participants in a regulatory fit were more likely to engage in systematic exploration of the task environment than participants in a regulatory mismatch and performed more optimally as a result. Implications for contemporary models of human reinforcement-learning are discussed.

In engineering systems, noise is a curse, obscuring important signals and increasing the uncertainty associated with measurement. However, the negative effects of noise are not universal. In this paper, we examine how people learn sequential control strategies given different sources and amounts of feedback variability. In particular, we consider people's behavior in a task where short- and long-term rewards are placed in conflict (i.e., the best option in the short-term is worst in the long-term). Consistent with a model based on reinforcement learning principles (Gureckis & Love, 2009), we find that learners differentially weight information predictive of the current task state. In particular, when cues that signal state are noisy, we find that participants' ability to identify an optimal strategy is strongly impaired relative to equivalent amounts of variability that obscure the rewards/valuations of those states. In other situations, we find that noise and variability in reward signals may paradoxically improve performance by encouraging exploration. Our results demonstrate how experimentally-manipulated task variability can be used to test predictions about the mechanisms that learners engage in dynamic decision making tasks.

Successful investors seeking returns, animals foraging for food, and pilots controlling aircraft all must take into account how their current decisions will impact their future standing. One challenge facing decision makers is that options that appear attractive in the short-term may not turn out best in the long run. In this paper, we explore human learning in a dynamic control task which places short- and long-term rewards in conflict. Our goal in these studies was to evaluate how people's mental representation of a task affects their ability to discover an optimal decision strategy. We find that perceptual cues that readily align with the underlying state of the task environment help people overcome the impulsive appeal of short term rewards. Our experimental manipulations, predictions, and analyses are motivated by current work in reinforcement learning which details how learners discount future rewards, the importance of "state" representations, and the role that exploration and exploitation play in effective learning.

Research on dynamic decision-making tasks, in which the payoffs associated with each choice vary with participants recent choice history, finds that humans have difficulty making long-term optimal choices in the presence of attractive immediate rewards. However, a number of recent studies have shown that simple cues providing information about the underlying state of the task environment may facilitate optimal responding. This study examines the mechanism by which this state knowledge influences choice behavior. We examine the possibility that participants use state information in conjunction with changing payoffs to extrapolate payoffs in future states. We find support for this hypothesis in a study where generalizations based on this state information work to the benefit or detriment of task performance, depending on the task's payoff structure.

Effective learning often involves actively querying the environment for information that disambiguates potential hypotheses. However, the space of observations available in any situation can vary greatly in potential "informativeness." In this report, we study participants' ability to gauge the information value of potential observations in a cognitive search task based on the children's game Battleship. Participants selected observations to disambiguate between a large number of potential game configurations subject to information-collection costs and penalties for making errors in a test phase. An "ideal-learner" model is developed to quantify the utility of possible observations in terms of the expected gain in points from knowing the outcome of that observation. The model was used as a tool for measuring search efficiency, and for classifying various types of information collection decisions. We find that participants are generally effective at maximizing gain relative to their current state of knowledge and the constraints of the task. In addition, search behavior shifts between an slower, but more efficient "exploitive" mode of local search and a faster, less efficient pattern of "exploration."

This work explores the influence of motivation on choice behavior in a dynamic decision-making environment, where the payoff from each choice depend on one's recent choice history. Previous research reveals increased levels of exploratory choice among participants in a regulatory fit. The present study placed promotion and prevention-focused participants in a dynamic environment for which optimal performance requires that participants sustain a single choice strategy in the face of temporary payoff decreases. These participants either gained or lost points with each choice. Our behavioral results and a model-based analysis, using an average-reward reinforcement learning framework, revealed differential levels of reactivity to local changes in payoffs - specifically, participants in a regulatory fit were less reactive to local perturbations in payoffs than participants in a regulatory mismatch and performed more optimally as a result.

We examine the interdependence between individual and group behavior surrounding a somewhat arbitrary, real world decision: selecting a name for one's child. Using a historical database of the names given to children over the last century in the United States, we find that naming choices are influuenced by both the frequency of a name in the general population, and by its "momentum" in the recent past in the sense that names which are growing in popularity are preferentially chosen. This bias toward rising names is a recent phenomena: in the early part of the 20th century, increasing popularity of a name from one time period to the next was correlated with a decrease in future popularity. However, more recently this trend has reversed. We evaluate a number of formal models that detail how individual decision making strategies, played out in a large population of interacting agents, can explain these empirical observations. We argue that cognitive capacities for change detection, the encoding of frequency in memory, and biases towards novel or incongruous stimuli may interact with the behavior of other decision makers to determine the distribution and dynamics of cultural tokens such as names.

Recent work has demonstrated robust learning traps during learning from experience ? decision-making biases that persist due to the choice-contingent nature of outcome feedback. In two experiments, we investigate the effect of outcome valence on learning trap development. Participants chose to approach or avoid category exemplars associated with rewards or losses, and, to maximize reward, must learn a categorization rule based on two stimulus dimensions. We replicate previous findings showing that when outcome feedback was contingent upon approaching exemplars, people frequently fell into the trap of using an incomplete categorization rule based on only a single dimension, which was suboptimal for long-term reward. Notably, learning trap development was attenuated in an environment with frequent loss outcomes, even when participants received explicit information about the base rates of gains and losses. The implications of these findings for theoretical models and future research are discussed.

From navigation in unfamiliar environments to career plan- ning, people typically first sample information before com- mitting to a plan. However, most studies find that people adopt myopic strategies when sampling information. Here we challenge those findings by investigating whether contingency planning is a driver of information sampling. To this aim, we developed a novel navigation task that is a shortest path find- ing problem under uncertainty of bridge closures. Participants (n = 109) were allowed to sample information on bridge sta- tuses prior to committing to a path. We developed a computa- tional model in which the agent samples information based on the cost of switching to a contingency plan. We find that this model fits human behavior well and is qualitatively similar to the approximated optimal solution. Together, this suggests that humans use contingency planning as a driver of information sampling.

Teaching is an intuitive social activity that requires reason- ing about and influencing the mind of others. A good teacher forms a belief about the knowledge of their student, asks clar- ifying questions, and gives instructions or explanations to try to induce a target concept in the student?s mind. We propose Partially Observable Markov Decision Processes (POMDPs) as a model of intuitive human teaching. According to this ac- count, teachers make pedagogical decisions with uncertainty about the knowledge state of their student. In two behavioral experiments, human participants were tasked with balancing assessments (asking questions) and instructions to help teach a student to build a tower of colored blocks. Human behavior in the task was compared to the performance of a computerized teaching algorithm optimized to solve the equivalent POMDP. Our results show that humans favor asking questions and estab- lishing common ground during teaching even at an economic cost and increase question asking as uncertainty grows.

When encountering unfamiliar physical objects, children and adults often perform structured interrogatory actions such as grasping and prodding, so revealing latent physical properties such as masses and textures. However, the processes driving and supporting these curious behaviors are still largely mys- terious. In this paper, we develop and train an agent able to actively uncover latent physical properties such as the mass and force of objects in a simulated physical ?micro-world?. Concretely, we used a simulation-based-inference framework to quantify the physical information produced by observation and interaction with the evolving dynamic environment. We used model-free reinforcement learning algorithm to train an agent to implement general strategies for revealing latent phys- ical properties. We compare the behaviors of this agent to the human behaviors observed in a similar task.

Acting effectively in the world requires learning and control-ling dynamic systems, that is, systems involving feedback relations among continuous variables that vary in real time. Weintroduce a novel class of dynamic control environments usingthe continuous Ornstein-Uhlenbeck process united with causal Markov graphs that allow us to systematically test people?sability to learn and control various dynamic systems as they change in real time. We find that people?s control is robust to changing goals, and exhibits heterogeneity of performance in different environments that matches closely with complexity defined by our optimal model. These results suggest peopleare capable learners of dynamic systems, able to leverage a rich representation of their environment to accomplish their goals.

Learning usually improves the accuracy of beliefs through the accumulation of experience. But are there limits to learning that prevent us from accurately understanding our world? In this paper we investigate the concept of a "learning trap" - the formation of a stable false belief even with extensive experience. Our review highlights how these traps develop though the interaction of learning and decision making in unknown environments. We further document a particularly pernicious learning trap driven by selective attention, a mechanism often assumed to faciliate learning in complex environments. Using computer simulation we demonstrate the key attributes of the agent and environment that lead to this new type of learning trap. Then, in a series of experiments we present evidence that people robustly fall into this trap, even in the presence of various interventions predicted to meliorate it. These results highlight a fundamental limit to learning and adaptive behavior that impacts individuals, organizations, animals, and machines.

People often make repeated decisions from experience. In such scenarios, persistent biases of choice can develop, most notably the "hot stove effect" (Denrell & March, 2001) in which a prospect that is mistakenly believed to be negative is avoided and thus belief-correcting information is never obtained. In the existing literature, the hot stove effect is generally thought of as developing through interaction with a single, stochastic prospect. Here, we show how a similar bias can develop due to people's tendency to selectively attend to a subset of features during categorization. We first explore the bias through model simulation, then report on an experiment in which we find evidence of a decisional bias linked to selective attention. Finally, we use these computational models to design novel interventions to "de-bias" decision-makers, some of which may have practical application.

see published handbook

Adaptive decision making often entails learning to approach things that lead to positive outcomes while avoiding things that are negative. The decision to avoid something removes the risk of a negative experience but also forgoes the opportunity to ob- tain information, specifically that a seemingly negative option is actually positive. This paper explores how people learn to approach or avoid objects with uncertain payoffs. We provide a computational-level analysis of optimal decision making in this problem which quantifies how the probability of encoun- tering an object in the future should impact the decision to ap- proach or avoid it. A large experiment conducted online shows that most people intuitively take into account both their uncer- tainty and the value of information when deciding to approach seemingly bad things.

This work explores the influence of motivation on choice behavior in a dynamic decision-making environment, where the payoffs from each choice depend on one?s recent choice history. Previous research reveals that participants in a regulatory fit exhibit increased levels of exploratory choice and flexible use of multiple strategies over the course of an experiment. The present study placed promotion and prevention-focused participants in a dynamic environment for which optimal performance is facilitated by systematic exploration of the decision space. These participants either gained or lost points with each choice. Our experiment revealed that participants in a regulatory fit were more likely to engage in systematic exploration of the task environment than participants in a regulatory mismatch and performed more optimally as a result. Implications for contemporary models of human reinforcement-learning are discussed.

Perceptual aliasing arises in situations where multiple, distinct states of the world give rise to the same percept. In this study, we examine how the degree of perceptual aliasing in a task impacts the ability of human agents to learn reward-maximizing decision strategies. Previous work has shown that the presence of perceptual cues that help signal distinct states of the environment can improve the ability of learners to adopt an optimal decision strategy in sequential decision making tasks (Gureckis & Love, 2009). In our experiments, we parametrically manipulated the degree of perceptual aliasing afforded by certain perceptual cues in a similar task. Our empirical results and simulations show how the ability of the learner improves as relevant states in the world uniquely map to differentiated percepts. The results provide further support for the model of sequential decision making proposed by Gureckis & Love (2009) and highlight the important role that state representations may have on behavior in dynamic decision making and learning tasks.

Successful investors seeking returns, animals foraging for food, and pilots controlling aircraft all must take into account how their current decisions will impact their future standing. One challenge facing decision makers is that options that appear attractive in the short-term may not turn out best in the long run. In this paper, we explore human learning in a dynamic control task which places short- and long-term rewards in conflict. Our goal in these studies was to evaluate how people's mental representation of a task affects their ability to discover an optimal decision strategy. We find that perceptual cues that readily align with the underlying state of the task environment help people overcome the impulsive appeal of short term rewards. Our experimental manipulations, predictions, and analyses are motivated by current work in reinforcement learning which details how learners discount future rewards, the importance of "state" representations, and the role that exploration and exploitation play in effective learning.

This work explores the influence of motivation on choice behavior in a dynamic decision-making environment, where the payoff from each choice depend on one's recent choice history. Previous research reveals increased levels of exploratory choice among participants in a regulatory fit. The present study placed promotion and prevention-focused participants in a dynamic environment for which optimal performance requires that participants sustain a single choice strategy in the face of temporary payoff decreases. These participants either gained or lost points with each choice. Our behavioral results and a model-based analysis, using an average-reward reinforcement learning framework, revealed differential levels of reactivity to local changes in payoffs - specifically, participants in a regulatory fit were less reactive to local perturbations in payoffs than participants in a regulatory mismatch and performed more optimally as a result.

Research on dynamic decision-making tasks, in which the payoffs associated with each choice vary with participants recent choice history, finds that humans have difficulty making long-term optimal choices in the presence of attractive immediate rewards. However, a number of recent studies have shown that simple cues providing information about the underlying state of the task environment may facilitate optimal responding. This study examines the mechanism by which this state knowledge influences choice behavior. We examine the possibility that participants use state information in conjunction with changing payoffs to extrapolate payoffs in future states. We find support for this hypothesis in a study where generalizations based on this state information work to the benefit or detriment of task performance, depending on the task's payoff structure.

In engineering systems, noise is a curse, obscuring important signals and increasing the uncertainty associated with measurement. However, the negative effects of noise are not universal. In this paper, we examine how people learn sequential control strategies given different sources and amounts of feedback variability. In particular, we consider people's behavior in a task where short- and long-term rewards are placed in conflict (i.e., the best option in the short-term is worst in the long-term). Consistent with a model based on reinforcement learning principles (Gureckis & Love, 2009), we find that learners differentially weight information predictive of the current task state. In particular, when cues that signal state are noisy, we find that participants' ability to identify an optimal strategy is strongly impaired relative to equivalent amounts of variability that obscure the rewards/valuations of those states. In other situations, we find that noise and variability in reward signals may paradoxically improve performance by encouraging exploration. Our results demonstrate how experimentally-manipulated task variability can be used to test predictions about the mechanisms that learners engage in dynamic decision making tasks.

We live in a world consisting of concrete experiences, yet we appear to form abstractions that transcend the details of our experiences. In this contribution, we argue that the abstract nature of our thought is overstated and that our representations are inherently bound to the examples we experience during learning. We present three lines of related research to support this general point. The ?rst line of research suggests that there are no separate learning systems for acquiring mental rules and storing exceptions to these rules. Instead, both items types share a common representational substrate that is grounded in experienced training examples. The second line of research suggests that representations of abstract concepts, such as same and different that can range over an unbounded set of stimulus properties, are rooted in experienced examples coupled with analogical processes. Finally, we consider how people perform in dynamic decision tasks in which short? and long?term rewards are in opposition. Rather than invoking explicit reasoning processes and planning, people's performance is best explained by reinforcement learning procedures that update estimates of action values in a reactive, trial-by-trial fashion. All three lines of research implicate mechanisms of thought that are capable of broad generalization, yet inherently local in terms of the procedures used for updating mental representations and planning future actions. We end by considering the benefits of designing systems that operate according to these principles.

In recent years, the cognitive neuroscience literature has come under criticism for containing many low-powered studies, limiting the ability to make reliable statistical inferences. Typically, the suggestion for increas- ing power is to collect more data with neural signals. However, many studies in cognitive neuroscience use parameters estimated from behavioral data in order to make inferences about neural signals (such as fMRI BOLD signal). In this paper, we explore how cognitive neuroscientists can learn more about their neuroimaging signal by collecting data on behavior alone. We demonstrate through simulation that knowing more about the marginal distribution of behavioral parameters can improve inferences about the mapping between cognitive processes and neural data. In realistic settings of the correlation between cognitive and neural parameters, additional behavioral data can lead to the same improvement in the precision of inferences more cheaply and easily than collecting additional data from subjects in a neuroimaging study. This means that when conducting an neuroimaging study, researchers now have two knobs to turn in a design analysis: the number of subjects collected in the scanner and the number of behavioral subjects collected outside the scanner (in the lab or online).

One powerful way children can learn word meanings is via cross-situational learning, the ability to discern consistent word-referent mappings from a series of ambiguous scenes and utterances. Various computational accounts of word learning have been proposed, with mechanisms ranging from storing and testing a single hypothesized referent for each word, to tracking multiple graded associations and selectively strength- ening some of them. Nearly all word learning models as- sume storage of some feasible word-referent mappings from each situation, resulting in a degree of learning proportional to the number of co-occurrences. While these accumulative models would generally predict that incorrect co-occurrences would slow learning, recent empirical work suggests these ac- counts are incomplete: paradoxically, giving learners incorrect mappings early in training was found to boost performance (Fitneva & Christiansen, 2015). We test this finding?s general- ity in a new experiment with more items, consider system- and item-level explanations, and find that a model with error-driven learning best accounts for this benefit of initially-inaccurate pairings.

This study examines how people deal with inherently stochastic cues when estimating a latent environmental property. Seven cues to a hidden location were presented one at a time in rapid succession. The seven cues were sampled from seven different Gaussian distributions that shared a common mean but differed in precision (the reciprocal of variance). The experimental task was to estimate the common mean of the Gaussians from which the cues were drawn. Observers ran in two conditions on separate days. In the "decreasing precision" condition the seven cues were ordered from most precise to least precise. In the "increasing precision" condition this ordering was reversed. For each condition, we estimated the weight that each cue in the sequence had on observers' estimates and compared human performance to that of an ideal observer who maximizes expected gain. We found that observers integrated information from more than one cue, and that they adaptively gave more weight to more precise cues and less weight to less precise cues. However, they did not assign weights that would maximize their expected gain, even over the course of several hundred trials with corrective feedback. The cost to observers of their sub-optimal performance was on average 16% of their maximum possible winnings.

Discovering the latent structure underlying sequential data is a central part of human learning in domains ranging from language to perception. We develop a rational framework based on Bayesian induction of hidden markov models (HMM) for understanding the principles guiding human sequence learning. The model uses nonparametric Bayesian methods to infer a distribution over possible hidden markov model (HMM) structures that are consistent with a particular training sequence. We successfully apply the model to a number of well known findings in artificial grammar learning (AGL) tasks (e.g., Gomez, 2002). In contrast to accounts based on associative learning principles (e.g., Simple Recurrent Networks), we argue that human sequence learning is better explained as a structured inference process that reflects the uncertainty over candidate structures.

We evaluate two broad classes of cognitive mechanisms that might support the learning of sequential patterns. According to the first, learning is based on the gradual accumulation of direct associations between events based on simple conditioning principles. The other view describes learning as the process of inducing the transformational structure that defines the material. Each of these learning mechanisms predict differences in the rate of acquisition for differently organized sequences. Across a set of empirical studies, we compare the predictions of each class of model with the behavior of human sub jects. We find that learning mechanisms based on transformations of an internal state, such as recurrent network architectures (e.g., Elman, 1990), have difficulty accounting for the pattern of human results relative to a simpler (but more limited) learning mechanism based on learning direct associations. Our results suggest new constraints on the cognitive mechanisms supporting sequential learning behavior.

Perceptual aliasing arises in situations where multiple, distinct states of the world give rise to the same percept. In this study, we examine how the degree of perceptual aliasing in a task impacts the ability of human agents to learn reward-maximizing decision strategies. Previous work has shown that the presence of perceptual cues that help signal distinct states of the environment can improve the ability of learners to adopt an optimal decision strategy in sequential decision making tasks (Gureckis & Love, 2009). In our experiments, we parametrically manipulated the degree of perceptual aliasing afforded by certain perceptual cues in a similar task. Our empirical results and simulations show how the ability of the learner improves as relevant states in the world uniquely map to differentiated percepts. The results provide further support for the model of sequential decision making proposed by Gureckis & Love (2009) and highlight the important role that state representations may have on behavior in dynamic decision making and learning tasks.

This work explores the influence of motivation on choice behavior in a dynamic decision-making environment, where the payoffs from each choice depend on one?s recent choice history. Previous research reveals that participants in a regulatory fit exhibit increased levels of exploratory choice and flexible use of multiple strategies over the course of an experiment. The present study placed promotion and prevention-focused participants in a dynamic environment for which optimal performance is facilitated by systematic exploration of the decision space. These participants either gained or lost points with each choice. Our experiment revealed that participants in a regulatory fit were more likely to engage in systematic exploration of the task environment than participants in a regulatory mismatch and performed more optimally as a result. Implications for contemporary models of human reinforcement-learning are discussed.

This work explores the influence of motivation on choice behavior in a dynamic decision-making environment, where the payoff from each choice depend on one's recent choice history. Previous research reveals increased levels of exploratory choice among participants in a regulatory fit. The present study placed promotion and prevention-focused participants in a dynamic environment for which optimal performance requires that participants sustain a single choice strategy in the face of temporary payoff decreases. These participants either gained or lost points with each choice. Our behavioral results and a model-based analysis, using an average-reward reinforcement learning framework, revealed differential levels of reactivity to local changes in payoffs - specifically, participants in a regulatory fit were less reactive to local perturbations in payoffs than participants in a regulatory mismatch and performed more optimally as a result.

Research on dynamic decision-making tasks, in which the payoffs associated with each choice vary with participants recent choice history, finds that humans have difficulty making long-term optimal choices in the presence of attractive immediate rewards. However, a number of recent studies have shown that simple cues providing information about the underlying state of the task environment may facilitate optimal responding. This study examines the mechanism by which this state knowledge influences choice behavior. We examine the possibility that participants use state information in conjunction with changing payoffs to extrapolate payoffs in future states. We find support for this hypothesis in a study where generalizations based on this state information work to the benefit or detriment of task performance, depending on the task's payoff structure.

In engineering systems, noise is a curse, obscuring important signals and increasing the uncertainty associated with measurement. However, the negative effects of noise are not universal. In this paper, we examine how people learn sequential control strategies given different sources and amounts of feedback variability. In particular, we consider people's behavior in a task where short- and long-term rewards are placed in conflict (i.e., the best option in the short-term is worst in the long-term). Consistent with a model based on reinforcement learning principles (Gureckis & Love, 2009), we find that learners differentially weight information predictive of the current task state. In particular, when cues that signal state are noisy, we find that participants' ability to identify an optimal strategy is strongly impaired relative to equivalent amounts of variability that obscure the rewards/valuations of those states. In other situations, we find that noise and variability in reward signals may paradoxically improve performance by encouraging exploration. Our results demonstrate how experimentally-manipulated task variability can be used to test predictions about the mechanisms that learners engage in dynamic decision making tasks.

Successful investors seeking returns, animals foraging for food, and pilots controlling aircraft all must take into account how their current decisions will impact their future standing. One challenge facing decision makers is that options that appear attractive in the short-term may not turn out best in the long run. In this paper, we explore human learning in a dynamic control task which places short- and long-term rewards in conflict. Our goal in these studies was to evaluate how people's mental representation of a task affects their ability to discover an optimal decision strategy. We find that perceptual cues that readily align with the underlying state of the task environment help people overcome the impulsive appeal of short term rewards. Our experimental manipulations, predictions, and analyses are motivated by current work in reinforcement learning which details how learners discount future rewards, the importance of "state" representations, and the role that exploration and exploitation play in effective learning.

In recent years, statistical learning (SL) has emerged as a compelling alternative to nativist theories of language acquisition (i.e., Chomsky, 1980). However, in many ways the framework of statistical learning echoes aspects of classic behaviorism by stressing the role of associative learning processes and the environment in shaping behavior. How far backwards has the needle swung? In this paper, we show how a subset of behaviors studied under the rubric of SL are in fact entirely consistent with a simple form of conditioned priming inspired by models from the behaviorist tradition (i.e. a variant of Rescola-Wagner which learns associative relationships through time).

Human behavior is intrinsically linked to the temporal and sequential characteristics of the environment. In this report, a model of human sequence learning is developed called the linear associative shift register (LASR). LASR uses a simple error-driven associative learning rule to incrementally acquire information about the structure of event sequences. The model is applied in a variety of situations including implicit learning via the serial reaction time (SRT) task and statistical word learning. The results of these simulations highlight commonalities between different tasks and learning modalities which suggests similar underlying learning mechanisms. Two novel experiments are reported which test unique predictions of the model and which differentiate between competing theories concerning the mechanisms underlying implicit sequence learning.

In this report, a model of human sequence learning is developed called the linear associative shift register (LASR). LASR uses a simple error-driven associative learning rule to incrementally acquire information about the structure of event sequences. In contrast to recent modeling approaches, LASR describes learning as a simple and limited process. We argue that this simplicity is a virtue in that the complexity of the model is better matched to the demonstrated complexity of human processing. The model is applied in a variety of situations including implicit learning via the serial reaction time (SRT) task and statistical word learning. The results of these simulations highlight commonalities between different tasks and learning modalities which suggest similar underlying learning mechanisms. LASR provides a similar account of the type of processing which underlies performance in both kinds of tasks, suggesting that they may rely on similar underlying mechanisms.

Surprise signals a discrepancy between predicted and observed outcomes. It is theorized to segment the flow of experience into discrete perceived events, drive affective experiences, and create particularly resilient memories. However, the ability to precisely measure naturalistic surprise has remained elusive. We used advanced basketball analytics to derive a quantitative measure of surprise and characterized its behavioral, physiological, and neural effects on human subjects observing basketball games. We found that surprise served to segment ongoing experiences, as reflected in subjectively perceived event boundaries and shifts in neocortical neural patterns underlying belief states. Interestingly, these effects differed by whether surprising moments contradicted or bolstered current predominant beliefs. Surprise also positively correlated with pupil dilation, processing in subcortical regions associated with dopamine, game enjoyment, and, along with these physiological and neural measures, long-term memory. These investigations support key predictions from event segmentation theory and extend theoretical conceptualizations of surprise to real-world contexts.

One goal of cognitive science is to build theories of mental function that predict individual behavior. In this project we focus on predicting, for individual participants, which specific items in a list will be remembered at some point in the future. If you want to know if an individual will remember something, one commonsense approach is to give them a quiz or test such that a correct answer likely indicates later memory for an item. In this project we attempt to predict later memory without ex- plicit assessments by jointly modeling both neural and behavioral data in a computational cognitive model which captures the dynamics of memory acquisition and decay. In this paper, we lay out a novel hierarchical Bayesian approach for combining neural and behavioral data and present results showing how fMRI signals recorded during the study phase of a memory task can improve our ability to predict (in held-out data) which items will be remembered or forgotten 72 hours later.

Knowledge tracing is a popular and successful approach to modeling student learning. In this paper we investigate whether the addition of neuroimaging observations to a knowledge tracing model enables accurate prediction of memory performance in held-out data. We propose a Hidden Markov Model of memory acquisition related to Bayesian Knowledge Tracing and show how continuous functional magnetic resonance imaging (fMRI) signals can be incorporated as observations related to latent knowledge states. We then show, using data collected from a simple second-language learning experiment, that fMRI data acquired during a learning session can be used to improve predictions about student memory at test. The fitted models can also potentially give new insight into the neural mechanisms that contribute to learning and memory.

The blood oxygen level dependent (BOLD) signal measured in fMRI experiments is generally regarded as sluggish and poorly suited for probing neural function at the rapid timescales involved in sentence comprehension. However, recent studies have shown the value of acquiring data with very short repetition times (TRs), not merely in terms of improvements in CNR through averaging, but also in terms of additional fine-grained temporal information. Using multiband-accelerated fMRI, we achieved whole-brain scans at 3mm resolution with a TR of just 500 ms at both 3T and 7T field strengths. This enabled sampling at close to the reading rate of one word per TR. By taking advantage of word timing information, we found that word decoding accuracy across two separate sets of scan sessions improved significantly with better overall performance at 7T than at 3T. The effect of TR was also investigated; we found that substantial word timing information can be extracted using fast TRs, with diminishing benefits beyond TRs of 1000 ms.

Recent functional magnetic resonance imaging (fMRI) studies have found that distinct neural systems may mediate perceptual category learning under incidental and intentional learning conditions. The present study was designed to replicate and extend previous investigations of these effects using an event-related design. In addition, the design is aimed at decoupling the influence of stimulus-encoding processes from the contribution of implicit and explicit learning on the recruitment of alternative neural systems. Consistent with previous reports, following incidental learning in a dot-pattern classification task, participants show decreased neural activity in occipital visual cortex (extrastriate region V3, BA 19) in response to novel exemplars of a studied category compared to members of a foil category, but do not show this decreased neural activity following explicit learning. Crucially, however, our results show that this pattern can be modulated by aspects of the stimulus-encoding instructions provided at the time of study. In particular, when participants in an implicit learning condition were encouraged to evaluate the overall shape and configuration of the stimuli during study, we failed to find the pattern of brain activity that has been taken to be a signature of implicit learning, suggesting that activity in this area does not uniquely reflect implicit memory for perceptual categories.

Mental localization efforts tend to stress the where more than the what. We argue that the proper targets for localization are well-specified cognitive models. We make this case by relating an existing cognitive model of category learning to a learning circuit involving the hippocampus, perirhinal, and prefrontal cortices. Results from groups varying in function along this circuit (e.g., infants, amnesics, and older adults) are successfully simulated by reducing the model's ability to form new clusters in response to surprising events, such as an error in supervised learning or an unfamiliar stimulus in unsupervised learning. Clusters in the model are akin to conjunctive codes that are rooted in an episodic experience (the surprising event) yet can develop to resemble abstract codes as they are updated by subsequent experiences. Thus, the model holds that the line separating episodic and semantic information can become blurred. Dissociations (categorization vs. recognition) are explained in terms of cluster recruitment demands.

Mental localization efforts tend to stress the where more than the what. We argue that the proper targets for localization are well-specified cognitive models. We make this case by relating an existing cognitive model of category learning to a learning circuit involving the hippocampus, perirhinal, and prefrontal cortex. Results from groups varying in function along this circuit (e.g., infants, amnesics, older adults) are successfully simulated by reducing the model's ability to form new clusters in response to surprising events, such as an error in supervised learning or an unfamiliar stimulus in unsupervised learning. Reported task dissociations (e.g., categorization vs. recognition) are explained in terms of cluster recruitment demands.

Computational models of infant categorization often fail to elaborate the transitional mechanisms that allow infants to achieve adult performance. In this paper, we apply a successful connectionist model of adult category learning to developmental data. The Supervised and Unsupervised Stratified Adaptive Increment Network (SUSTAIN) model is able to account for the emergence of infants' sensitivity to correlated attributes (e.g., has wings and can fly). SUSTAIN offers two complimentary explanations of the developmental trend. One explanation centers on memory storage limitations, whereas the other focuses on limitations in perceptual systems. Both explanations parallel published findings concerning the cognitive and sensory limitations of infants. SUSTAIN's simulations suggest that conceptual development follows a continuous and smooth trajectory, despite qualitative changes in behavior, and that the mechanisms that underlie infant and adult categorization may not differ significantly.

In recent years, the cognitive neuroscience literature has come under criticism for containing many low-powered studies, limiting the ability to make reliable statistical inferences. Typically, the suggestion for increas- ing power is to collect more data with neural signals. However, many studies in cognitive neuroscience use parameters estimated from behavioral data in order to make inferences about neural signals (such as fMRI BOLD signal). In this paper, we explore how cognitive neuroscientists can learn more about their neuroimaging signal by collecting data on behavior alone. We demonstrate through simulation that knowing more about the marginal distribution of behavioral parameters can improve inferences about the mapping between cognitive processes and neural data. In realistic settings of the correlation between cognitive and neural parameters, additional behavioral data can lead to the same improvement in the precision of inferences more cheaply and easily than collecting additional data from subjects in a neuroimaging study. This means that when conducting an neuroimaging study, researchers now have two knobs to turn in a design analysis: the number of subjects collected in the scanner and the number of behavioral subjects collected outside the scanner (in the lab or online).

We examine the interdependence between individual and group behavior surrounding a somewhat arbitrary, real world decision: selecting a name for one's child. Using a historical database of the names given to children over the last century in the United States, we find that naming choices are influuenced by both the frequency of a name in the general population, and by its "momentum" in the recent past in the sense that names which are growing in popularity are preferentially chosen. This bias toward rising names is a recent phenomena: in the early part of the 20th century, increasing popularity of a name from one time period to the next was correlated with a decrease in future popularity. However, more recently this trend has reversed. We evaluate a number of formal models that detail how individual decision making strategies, played out in a large population of interacting agents, can explain these empirical observations. We argue that cognitive capacities for change detection, the encoding of frequency in memory, and biases towards novel or incongruous stimuli may interact with the behavior of other decision makers to determine the distribution and dynamics of cultural tokens such as names.

The resurgence of interest in collective behavior is in large part due to tools recently made available for conducting laboratory experiments on groups, statistical methods for analyzing large data sets reflecting social interactions, the rapid growth of a diverse variety of on-line self-organized collectives, and computational modeling methods for understanding both universal and scenario-specific social patterns. We consider case studies of collective behavior along four attributes: the primary motivation of individuals within the group, kinds of interactions among individuals, typical dynamics that result from these interactions, and characteristic outcomes at the group level. With this framework, we compare the collective patterns of non-interacting decision makers, bee swarms, groups forming paths in physical and abstract spaces, sports teams, cooperation and competition for resource usage, and the spread and extension of innovations in an on-line community. Some critical issues surrounding collective behavior are then reviewed, including the questions of "Does group behavior always reduce to individual behavior?," "Is `group cognition` possible?", and "What is the value of formal modeling for understanding group behavior"

Our laboratory has been studying the emergence of collective search behavior from a complex systems perspective. We have developed an Internet-based experimental platform that allows groups of people to interact with each other in real-time on networked computers. The experiments implement virtual environments where participants can see the moment-to-moment actions of their peers and immediately respond to their environment. Agent-based computational models are used as accounts of the experimental results. We describe two paradigms for collective search: one in physical space and the other in an abstract problem space. The physical search situation concerns competitive foraging for resources by individuals inhabiting an environment consisting largely of other individuals foraging for the same resources. The abstract search concerns the dissemination of innovations in social networks. Across both scenarios, the group-level behavior that emerges reveals influences of exploration and exploitation, bandwagon effects, population waves, and compromises between individuals using their own information and information obtained from their peers.

Just as neurons interconnect in networks that create structured thoughts beyond the ken of any individual neuron, so people spontaneously organize themselves into groups to create emergent organizations that no individual may intend, comprehend, or even perceive. Recent technological advances have provided us with unprecedented opportunities for conducting controlled laboratory experiments on human collective behavior. We describe two experimental paradigms in which we attempt to build predictive bridges between the beliefs, goals, and cognitive capacities of individuals and patterns of behavior at the group level, showing how the members of a group dynamically allocate themselves to resources and how innovations diffuse through a social network. Agent-based computational models have provided useful explanatory and predictive accounts. Together, the models and experiments point to tradeoffs between exploration and exploitation -- that is, compromises between individuals using their own innovations and using innovations obtained from their peers and the emergence of group-level organizations such as population waves, bandwagon effects, and spontaneous specialization.

Is cognition an exclusive property of the individual or can groups have a mind of their own? We explore this question from the perspective of complex adaptive systems. One of the principle insights from this line of work is that rules that govern behavior at one level of analysis (the individual) can cause qualitatively different behavior at higher levels (the group). We review a number of behavioral studies from our lab that demonstrate how groups of people interacting in real-time can self-organize into adaptive, problem-solving group structures. A number of principles are derived concerning the critical features of such "distributed" information processing systems. We suggest that while cognitive science has traditionally focused on the individual, cognitive processes may manifest at many levels including the emergent group-level behavior that results from the interaction of multiple agents and their environment.