I will discuss the learning dynamics of a small set of players who interact repeatedly in a given game, and who learn from past experience adapting to their opponents' actions. Such dynamics lead to to modified replicator equations in a deterministic limit, corresponding to an infinite number of observations made between adaption events. If only a finite number of moves of the opposing players are sampled between adaption steps, the learning dynamics becomes stochastic, similar to noise corrections observed in evolutionary systems in finite populations. I will here discuss the effects of memory-loss and of noise on the learning of agents for some simple two-player games, and show that the combination of both may affect the attractors considerably. Memory-loss may promote co-operation in social dilemmas, and the agents fail to retrieve the Nash equilibrium. In cyclic games, memory-loss drives the system towards stability in deterministic learning, but fixed points may be removed if sampling is stochastic, resulting in coherent random cycling.