An arresting summary in This Week in Psychological Science, of “Learning, Memory, and Synesthesia” by Nathan Witthoft and Jonathan Winawer (in Psychological Science for January 10, 2013, (24)1):
Individuals with color-grapheme synesthesia experience color when viewing written letters or numerals. Although some studies examining whether there is a learning component to synesthesia have returned negative results, these studies have examined very small numbers of individuals. Witthoft and Winawer revisit this question with the benefit of a larger sample. Eleven individuals with color-grapheme synesthesia completed a color-letter matching task in which they indicated the shade of the color they associated with each letter of the alphabet. The researchers found that participants’ color-letter associations closely matched those found in Fisher Price magnetic letters sets — which all but one of the participants had owned as a child. The authors suggest that these findings demonstrate a need to include learning and memory components into explanations of synesthesia.
The Fisher Price alphabet:

The full article (available only to subscribers) is very clear that these results don’t mean that some number of synesthetes have simply learned the letter-color associations from refrigerator magnets. For one thing, a huge number of children have been exposed to the Fisher Price alphabet, but the number of synesthetes is small. Witthoft and Winawer suggest that a small number of children are inclined to synesthesia, and that for them, exposure to colored letters and numbers can provide models for their associations.



