Abstract：
:Direct reciprocity is a chief mechanism of mutual cooperation in social dilemma. Agents cooperate if future interactions with the same opponents are highly likely. Direct reciprocity has been explored mostly by evolutionary game theory based on natural selection. Our daily experience tells, however, that real social agents including humans learn to cooperate based on experience. In this paper, we analyze a reinforcement learning model called temporal difference learning and study its performance in the iterated Prisoner's Dilemma game. Temporal difference learning is unique among a variety of learning models in that it inherently aims at increasing future payoffs, not immediate ones. It also has a neural basis. We analytically and numerically show that learners with only two internal states properly learn to cooperate with retaliatory players and to defect against unconditional cooperators and defectors. Four-state learners are more capable of achieving a high payoff against various opponents. Moreover, we numerically show that four-state learners can learn to establish mutual cooperation for sufficiently small learning rates.
journal_name
Bull Math Bioljournal_title
Bulletin of mathematical biologyauthors
Masuda N,Ohtsuki Hdoi
10.1007/s11538-009-9424-8subject
Has Abstractpub_date
2009-11-01 00:00:00pages
1818-50issue
8eissn
0092-8240issn
1522-9602journal_volume
71pub_type
杂志文章abstract：:We develop an age-structured ODE model to investigate the role of intermittent preventive treatment (IPT) in averting malaria-induced mortality in children, and its related cost in promoting the spread of antimalarial drug resistance. IPT, a malaria control strategy in which a full curative dose of an antimalarial med...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-018-0524-1
更新日期：2019-01-01 00:00:00
abstract：:In this paper, we use multiple scale homogenisation to derive a set of averaged macroscale equations that describe the movement of nutrients in partially saturated soil that contains growing potato tubers. The soil is modelled as a poroelastic material, which is deformed by the growth of the tubers, where the growth o...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-019-00656-3
更新日期：2019-10-01 00:00:00
abstract：:In this paper we introduce and study a model for electrical activity of cardiac membrane which incorporates only an inward and an outward current. This model is useful for three reasons: (1) Its simplicity, comparable to the FitzHugh-Nagumo model, makes it useful in numerical simulations, especially in two or three sp...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1016/S0092-8240(03)00041-7
更新日期：2003-09-01 00:00:00
abstract：:Most of present mathematical models for biological activity consider just the molecular structure. In the present article we pretend extending the use of Markov chain models to define novel molecular descriptors, which consider in addition other parameters like target site or biological effect. Specifically, this math...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-005-9013-4
更新日期：2006-10-01 00:00:00
abstract：:A stochastic analog to a deterministic model describing subpopulation emergence in heterogeneous tumors is developed. The resulting system is described by the Fokker-Planck or forward Kolmogorov equation. A finite element approach for the numerical solution to this equation is described. Four biological and clinical s...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/BF02459658
更新日期：1989-01-01 00:00:00
abstract：:A rigorous Bayesian analysis is presented that unifies protein sequence-structure alignment and recognition. Given a sequence, explicit formulae are derived to select (1) its globally most probable core structure from a structure library; (2) its globally most probable alignment to a given core structure; (3) its most...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1006/S0092-8240(98)90002-7
更新日期：1998-11-01 00:00:00
abstract：:Alcohol consumption and abuse is widespread in sub-Saharan Africa where most HIV infections occur and has been associated with risky sexual behaviors. It may therefore be one of the most common, potentially modifiable HIV risk factors in this region. A deterministic system of ordinary differential equations incorporat...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-012-9747-8
更新日期：2012-09-01 00:00:00
abstract：:Degradable scaffolds represent a promising solution for tissue engineering of damaged or degenerated articular cartilage which due to its avascular nature, is characterized by a low self-repair capacity. To estimate the articular cartilage regeneration process employing degradable scaffolds, we propose a mathematical ...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-013-9897-3
更新日期：2013-12-01 00:00:00
abstract：:The Shapley value, a solution concept from cooperative game theory, has recently been considered for both unrooted and rooted phylogenetic trees. Here, we focus on the Shapley value of unrooted trees and first revisit the so-called split counts of a phylogenetic tree and the Shapley transformation matrix that allows f...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-018-0392-8
更新日期：2019-02-01 00:00:00
abstract：:Synchronization and desynchronization is of great interest in the study of circadian rhythms, metabolic oscillations and time-dependent cell aggregate behaviors. Several recent studies examine synchronization and other dynamics in models of repressilators coupled by a quorum sensing mechanism that uses a diffusive sig...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-016-0187-8
更新日期：2016-06-01 00:00:00
abstract：:We study a recently discovered class of models for plateau bursting, inspired by models for endocrine pituitary cells. In contrast to classical models for fold-homoclinic (square-wave) bursting, the spikes of the active phase are not supported by limit cycles of the frozen fast subsystem, but are transient oscillation...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-007-9241-x
更新日期：2008-01-01 00:00:00
abstract：:Many dynamic programming algorithms consist of a 'forward pass' computation to optimize a cost function, followed by a 'choice recovery' computation to construct a configuration that optimizes the cost function. 'Checkpointing' is a method to perform choice recovery using limited storage. During a forward pass, checkp...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1016/j.bulm.2004.10.002
更新日期：2005-07-01 00:00:00
abstract：:In a companion paper (Lof et al., in Bull. Math. Biol., 2008), we describe a spatio-temporal model for insect behavior. This model includes chemical information for finding resources and conspecifics. As a model species, we used Drosophila melanogaster, because its behavior is documented comparatively well. We divide ...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-008-9329-y
更新日期：2008-10-01 00:00:00
abstract：:Nosocomial outbreaks of bacteria are well documented. Based on these incidents, and the heavy usage of antibiotics in hospitals, it has been assumed that antibiotic resistance evolves in hospital environments. To test this assumption, we studied resistance phenotypes of bacteria collected from patient isolates at a co...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-016-0232-7
更新日期：2017-01-01 00:00:00
abstract：:To help manage the fluctuations inherent in fish populations scientists have argued for both an ecosystem approach to management and the greater use of marine reserves. Support for reserves includes empirical evidence that they can raise the spawning biomass and mean size of exploited populations, increase the abundan...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1016/j.bulm.2004.11.006
更新日期：2005-09-01 00:00:00
abstract：:We study the emergence of collective spatio-temporal objects in biological systems by representing individually the elementary interactions between their microscopic components. We use the immune system as a prototype for such interactions. The results of this detailed explicit analysis are compared with the tradition...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1016/S0092-8240(03)00007-7
更新日期：2003-05-01 00:00:00
abstract：:To analyze signals measured from human blood flow in the time-frequency domain, we used the wavelet transform which gives good time resolution for high-frequency components and good frequency resolution for low-frequency components. Five characteristic frequency peaks, corresponding to five almost periodic rhythmic ac...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1006/bulm.1998.0047
更新日期：1998-09-01 00:00:00
abstract：:Developing a quantitative science of sustainability requires bridging mathematical concepts from fields contributing to sustainability science. The concept of substitutability is central to sustainability but is defined differently by different fields. Specifically, economics tends to define substitutability as a marg...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-014-9963-5
更新日期：2015-02-01 00:00:00
abstract：:The mechanical behavior of the actin cytoskeleton has previously been investigated using both experimental and computational techniques. However, these investigations have not elucidated the role the cytoskeleton plays in the compression resistance of cells. The present study combines experimental compression techniqu...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-013-9812-y
更新日期：2013-08-01 00:00:00
abstract：:Biological invasion is an important area of research in mathematical biology and more so if it concerns species which are vectors for diseases threatening the public health of large populations. That is certainly the case for Aedes aegypti and the dengue epidemics in South America. Without the prospect of an effective...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1016/j.bulm.2004.08.005
更新日期：2005-05-01 00:00:00
abstract：:A mathematical control model for the transmission dynamics of tuberculosis (TB) in South Korea is developed on the basis of the reported active-TB and relapse-TB incidence data. In this work, optimal control theory is used to propose optimal TB prevention and control strategy and rearrange the government TB budget for...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-014-9962-6
更新日期：2014-07-01 00:00:00
abstract：:DNA and protein sequence comparisons are performed by a number of computational algorithms. Most of these algorithms search for the alignment of two sequences that optimizes some alignment score. It is an important problem to assess the statistical significance of a given score. In this paper we use newly developed me...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/BF02459930
更新日期：1992-09-01 00:00:00
abstract：:We exhibit a stochastic discrete time model that produces the Eigen model (Naturwissenschaften 58:465-523, 1971) in the deterministic and continuous time limits. The model is based on the competition among individuals differing in terms of fecundity but with the same viability. We explicitly write down the Markov matr...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-010-9525-4
更新日期：2011-01-01 00:00:00
abstract：:In this paper we formulate a multi-patch multi-species model in which the per-capita emigration rate of one species depends on the density of some other species. We then focus on Turing instability to examine if and when this cross-emigration response has crucial effects. We find that the type of interaction matters g...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1006/bulm.2002.0328
更新日期：2003-01-01 00:00:00
abstract：:The mammalian kidney consumes a large amount of energy to support the reabsorptive work it needs to excrete metabolic wastes and to maintain homeostasis. Part of that energy is supplied via the metabolism of glucose. To gain insights into the transport and metabolic processes in the kidney, we have developed a detaile...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-016-0188-7
更新日期：2016-06-01 00:00:00
abstract：:Coalescent models of evolution account for incomplete lineage sorting by specifying a species tree parameter which determines a distribution on gene trees, and consequently, a site pattern probability distribution. It has been shown that the unrooted topology of the species tree parameter of the multispecies coalescen...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-018-0456-9
更新日期：2019-02-01 00:00:00
abstract：:In this paper, a simple semi-discrete (ticks' feeding is assumed to occur only during the summers of each year) model for tick population dynamics is presented. Conditions for existence, uniqueness, and stability of a positive equilibrium are found; the system is then studied numerically using parameter estimates cali...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1016/j.bulm.2004.03.007
更新日期：2004-11-01 00:00:00
abstract：:Continuing the discussion of how children can modify and regularize linguistic inputs from adults, we present a new interpretation of existing algorithms to model and investigate the process of a learner learning from an inconsistent source. On the basis of this approach is a (possibly nonlinear) function (the update ...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-017-0250-0
更新日期：2017-03-01 00:00:00
abstract：:Bacillus subtilis cells may opt to forgo normal cell division and instead form spores if subjected to certain environmental stimuli, for example nutrient deficiency or extreme temperature. The resulting spores are extremely resilient and can survive for extensive periods of time, importantly under particularly harsh c...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/s11538-010-9530-7
更新日期：2011-01-01 00:00:00
abstract：:Aggregation of membrane molecules is a crucial phenomenon in developing organisms, a classic example being the aggregation of post-synaptic receptors during synaptogenesis. Our understanding of the molecular events involved is improving, but most models of the aggregation or concentration process do not address bindin...
journal_title：Bulletin of mathematical biology
pub_type： 杂志文章
doi：10.1007/BF02461845
更新日期：1995-09-01 00:00:00
