RT @ThierryAaron: "We conclude that a collapse of the Atlantic Meridional Overturning Circulation cannot be considered a low-probability ev….

RT @boucherhayes: Yesterday 44 of the world’s leading climate scientists wrote an open letter about collapse of the Atlantic Ocean’s circul….

RT @wjeffjohnston: When does modular structure emerge in neural networks?.What are the consequences of this structure for learning and beha….

RT @chingfang17: New work with @Jack_W_Lindsey, Larry Abbott, Dmitriy Aronov, and @selmaanchettih! We propose a model of episodic memory wh….

RT @wjeffjohnston: How does the brain bind action to value?.How does it navigate a tradeoff between this binding and generalization to nove….

Many thanks to all my co-authors, in particular Jeff, with whom I've worked closely for a lot of my PhD. If you don't know, he's a star postdoc in the lab, and I hope he gets an awesome faculty job this year!.

We interpret the “integration” of new information with the first delay representation as a “manipulation” of working memory – like rewriting on a scratch pad. In that sense, we show that swap errors emerge when correctly-remembered information is manipulated in memory.

Synthesis of these results: first piece of information (color or cue) is encoded and maintained correctly; second piece (cue or color) is perhaps encoded correctly, but then incorrectly integrated with the first. The result is that the first thing now appears “swapped” in memory.

The prospective trials make things more interesting. Briefly: the representation also looks correct in the first delay period, but during the second delay the representation looks like a misinterpreted cue rather than a color-space misbinding – i.e. the opposite of retro.

Our interpretation: upon seeing the stimuli, the upper color is slotted into the “upper” subspace and vice versa; after the cue is shown, the correct color-space associations are now wrong. So, an error was introduced between the two time periods.

In the second delay period, we distinguish between two types of swaps: one in which color and space are misbound, and one where the cue is simply misinterpreted. These predict different representations. Punchline: it looks like misbinding in monkey E, kind of both in monkey W.

In the retro task: during the first delay period, the representations look mostly like the non-erroneous prediction (although less so in monkey W).

Our general approach to modeling neural activity is to fit conjunctive color-space representations, e.g. f(red, up) and f(blue, down) which allows us to generate predictions for different erroneous representations – e.g. f(red, up) + f(blue, down) vs f(red, down) + f(blue, up)

With this many swap trials, we have enough statistical power to examine the neural representations in comparison to correct-response trials. We will break our analysis into the two task types (retro vs pro) and two time periods (before the cue vs before the response).

We observed a fairly high rate of swap errors in both versions of the task, which you can see by centering responses on the uncued color. (Even the prospective task, which we found kind of surprising.) Using a mixture model, we estimate swap errors occur in around 15% of trials.

@timbuschman and @mattpanichello trained monkeys to report one of two colors based on a trained cue – e.g. circle for upper, square for lower. What’s cool is there are two versions of the task: the color can be shown before the cue (“retrospective”) or vice versa (“prospective”).

In any case, one idea in the literature is that the different features of the stimuli are mis-associated from the start; another is that you misinterpret the cue telling you which item to report. We show that neither of these is the whole story.

As an aside, I think it’s interesting to know why they happen because knowing how a system fails can tell you about how it works. Also, when vanilla RNNs are trained on working memory tasks, they don’t make nearly as many swap errors as primates, so might be worth knowing why.

When you think of memory failures, you might imagine just forgetting something, maybe because of a capacity limitation. But sometimes instead of forgetting, you remember something wrong. These are called swap errors, and it isn’t clear why they happen.

We live with the limitations of our memory, but don’t really know where they come from. Our new paper ( studies "swap errors", which we argue arise during memory manipulation – see thread for more! @timbuschman, @MattPanichello, @wjeffjohnston