“Respected expert and director of the institute…” Fail!

This first appeared on January 7, 2010, as a feature at ScientificBlogging.com.

“Respected expert and director of the institute…” These are the words you hear as you are being introduced at a black-tie speaking engagement. You are an inventor, scientist, or artist, and this flattering introduction is music to your ears; had you seen these words written in the paper you would have saved a copy to show Mom. Finally, you are at the place every creative mind wishes to reach. The words wash back over you. “Respected”: The members of your community appreciate you. “Expert”: Your more than twenty years of dedication to the field have not gone unnoticed. “Director”: You have powerful tools and competent personnel to support your efforts. And “Institute”: Your work has attracted the funding of government, benefactors or investors.

You are liked, smart, powerful and rich! You’ve really made it!

Or have you? As an artist, scientist or inventor, success is defined in terms of your ideas – how many did you have that panned out, and how many were big? Being liked, smart, powerful and rich may be nice, but one can have these things and not have had the ideas that count toward the successful creative life. In fact, these seemingly nice things – being respected in one’s community, being an expert, having powerful tools, and having financial support – are a scourge on one’s creative potential. In order to harvest your full creative potential one must be…indifferent.

Indifferent to one’s community, indifferent to one’s previous talents or successful endeavors, indifferent to the tools one might have thus far accrued, and indifferent to sources of funding. Masters of ceremonies at black-tie events are unlikely to introduce a speaker as “the not particularly well-respected jack-of-all-trades and luddite penny-pincher,” but that is the signature of the creative individual extraordinaire. The actual introduction by the master of ceremonies sounds much nicer than this, but it is the signature of a creativity that was long-ago crushed; it is a eulogy for the dynamic idea-generating person you never came to be.

But why would indifference be helpful to creativity? Indifference helps an individual’s creativity because it helps the brain act more like a community of brains, and it is communities of brains where we find the greatest success stories for idea generation. Scientific, artistic and engineering communities are fantastically creative because there are many individuals working in parallel, each competitively striving for the next great idea.

Although most individuals in a community may not be successful at finding the next big idea, there will inevitably be some individuals who will be successful, even if only by accident. Individual scientists, artists and engineers tend to be utterly unlike these dynamic communities. Individuals tend to work serially, not in parallel; and individuals tend to concentrate their digging in one spot, rather than many. These tendencies for individuals are fine for the health of a creative community, but if one wants to be a creative individual, then one must ensure that one’s strategy for digging optimizes one’s own chance at hitting gold.

That sounds simple enough: in order for an individual to act like a community of idea-seekers, one must just carry out multiple directions of idea-generation in parallel. Dig many holes, not just one. However, it is exceedingly difficult for people to actually do this. The difficulty is not intellectual – we are, in principle, able to act like a (small) community of idea-generating individuals. The difficulty, instead, is psychological. We may be the smartest animals on Earth, but we are still animals, great apes in particular.

As such, we come with a suite of psychological attributes that, although especially helpful for surviving and reproducing among other humans in our ancient evolutionary environment, handicap us as idea hunters. Our handicaps center around the fact that we cannot help but desire to be the “respected expert and director of the institute,” a desire that inevitably kills the internal community needed inside a creative individual, and, instead, places our mind firmly within an external creativity-smothering community. The cure is to become indifferent, detached, aloof. … from communities, money, tools and even oneself.

(See also this ScientificBlogging piece on the benefits of being aloof: http://www.scientificblogging.com/mark_changizi/value_being_aloof_or_how… .)

Aloofily yours,

Mark Changizi

Mark Changizi is a professor of cognitive science at Rensselaer Polytechnic Institute, and the author of The Vision Revolution (Benbella Books).

Read Full Post »

The Topography of Language

Originally a piece in ScientificBlogging, September 17, 2009…

Reading pervades every aspect of our daily lives, so much so that one would be hardpressed to find a room in a modern house without words written somewhere inside. Many of us now read more sentences in a day than we listen to. Not only are we highly competent readers, but our brains even appear to have regions devoted to recognizing words. A Martian just beginning to study us humans might be excused for concluding that we had evolved to read.

But, of course, we haven’t. Reading and writing is a recent human invention, going back only several thousand years, and much more recently for many parts of the world. We are reading using the eyes and brains of our illiterate ancestors. Why are we so good at such an unnatural act?

Here I describe recent evidence that, although we have not evolved to be good at reading, writing appears to have culturally evolved to be good for the eye. More specifically, recent research supports the exciting hypothesis that human visual signs look like nature, because that is what we have evolved over millions of years to be good at seeing. This ecological hypothesis for letter shape not only helps explain why we are such good readers, but answers the question, Why are letters and other visual signs shaped the way they are?

The Variety of Visual Signs

Given the tremendous variety of visual signs over human history, it may at first glance seem that there could be no simple answer to the question, Why are visual signs shaped as they are? After all, we have been making visual signs for at least 40,000 years, starting with tool decorations and cave paintings. The evolution of ornamentation, art, painting, and other non-linguistic visual signs (i.e., signs not part of language) has gone on unabated, diversifying into millions of non-linguistic symbols used over the ages, and occupying nearly all aspects of our lives, including pottery, body art, religion, politics, folklore, medicine, music, architecture, trademarks and traffic.

Writing (i.e., visual signs distinguished by use as a means of visually recording the content of spoken language) has also undergone an evolutionary explosion in variety. The earliest writing appeared several thousand years ago, and occurred independently in Sumer, Egypt and China (and much more recently in the Americas). These earliest linguistic visual signs were pictograms, evolving later to logograms (where a character denotes an object, idea or action), and a single logographic writing system (such as Chinese or Linear B) can have many thousands of distinct visual signs. It wasn’t until about 2000 years ago in Egypt that phonemic writing was invented and used, where each character stands for a constituent of speech rather than having a meaning as in logographic writing. Many hundreds of writing systems have evolved and diversified from this ancestor (e.g., Latin, Arabic, Avestan, Mongolian, Phags-pa), varying widely in geometrical shape and style, and in the aspects of speech the characters represent (e.g., alphabets represent consonants and vowels, abugidas represent just consonants, and syllabaries represent syllables).

Amongst both non-linguistic and linguistic signs, some visual signs are representations of the world—e.g., cave paintings and pictograms, respectively—and it is, of course, not surprising that these visual signs look like nature. It would be surprising, however, to find that non-pictorial visual signs look, despite first appearances, like nature. Although writing began with pictograms, there have been so many mutations to writing over the millenia that if writing still looks like nature, it must be because this property has been selectively maintained. For non-linguistic visual signs, there is not necessarily any pictorial origin as there is for writing, because amongst the earliest non-linguistic visual signs were non-pictorial decorative signs. The question we then ask is, Why are non-pictorial visual signs shaped the way they are?

Previous efforts at answering this question have primarily concentrated on the differences. In particular, some of the shape differences among different (non-pictorial) visual signs are due to the kind of writing implement used, whether impressions in clay tablets with a blunt reed, rounded writing on leaves, or the physical details of a modified feather-tip point. Little attention has been devoted to uncovering the similarities, however, and as we will see here, there are deeper visual regularities that hold across human visual signs, independent of the writing mechanism (regularities that are also found in nature). It is as if someone had noticed that throat size causes male and female voices to sound differently, without noticing that male and female speech possesses a critical deeper regularity, namely that they utter the same set of phonemes, morphemes, words and sentences as one another (within a single language speaking community). We will find that, despite superficial differences in their shapes, visual signs appear to possess similar underlying “visual phonemes.”

The Shapes of Visual Signs

Uncovering deeper visual regularities that might govern visual signs is crucial in any attempt to explain why visual signs are shaped as they are, and, in turn, to explain why we are so good at reading. After all, we cannot explain why visual signs are shaped as they are if we do not first determine how visual signs over history are, in fact, shaped! A difficulty in trying to find such regularities is that it is not straightforward to describe how even a single letter is shaped, for a single letter can undergo considerable distortion (e.g., from person to person, or from font to font) without losing its identity. How can one hope to scientifically address the kinds of shapes found among visual signs, when it is awkward to even rigorously say what the shape of a single letter is?

To solve this problem I decided to use a topological notion of shape, where the details of the geometry do not matter, and what matters is only the manner in which strokes intersect, or join, with other strokes. A straight line, a C and an S have the same topological shape because each is topologically just a single stroke. L, T and X are the three distinct kinds of topological shape having two strokes. For example, a V has the same topological shape as an L because each consists of two strokes meeting at their endpoints. This notion of shape will be helpful later in measuring the shapes of nature, because while geometrical shape can change quickly as a function of a person’s viewpoint, topological shape is more viewpoint invariant, providing a more robust characterization of the shapes in nature. This topological notion of shape is not merely useful, but possesses psychological justification as well: experts in psychology (e.g., Irving Biederman’s work on intermediate-level representations) and computer vision believe that our visual systems may represent shape in a topological manner.

In earlier research I had shown (along with my collaborator Dr. Shinsuke Shimojo) that letters have on average three strokes, and this average does not vary as a function of the number of letters in the writing system. Because of this, I considered topological shapes with three or fewer segments, in particular the 36 topological shapes that can be drawn with three straight lines (even though each topological shape covers curvy geometrical shapes as well). In addition to a single-stroke, and L, T and X, there are five three-stroke configurations having a single junction (i.e., a single point of intersection of the strokes) exemplified by Y, K and Y. There are 11 configuration types having three segments and two junctions, exemplified by characters such as ] (or, equivalently, Z), 1, F, I, p, and ≠. Finally, there are 16 topological shapes with three segments and three junctions, such as D and A.

With these 36 kinds of topological shape primitive in hand, we (Dr. Shinsuke Shimojo, and two Caltech undergraduate students, Qiong Zhang and Hao Ye) set about rigorously measuring how common these shapes occur among visual signs. We began by measuring from three distinct classes of non-pictorial visual sign: phonemic writing systems (non-logographic), Chinese characters (logographic), and non-linguistic symbols. The set of phonemic letters were taken from about one hundred phonemic writing systems over history, and the topological shape of the entire letter was measured (if it was one of the 36 types in our repertoire). For Chinese characters and non-linguistic symbols, the signs typically have more than three strokes, and we measured all the topological shapes that occur as part of the whole sign.

What we discovered is that the shapes across these three very different kinds of visual sign are similar. For example, Ls and Ts are in each case common, but Xs rare. And across the 32 different kinds of topological shape with three segments, these three classes of visual sign highly correlate with one another. For example, Ys tend to be common relative to Õs, Zs and Fs more common than1s, and Hs more common than ps. That is, despite the seemingly unrestrained variability in shape among these visual signs, they in fact possess a similar topological shape “signature.” Now we are in a position to more meaningfully ask why visual signs are shaped as they are. Namely, why do visual signs have this signature?

They don’t get this signature by chance. For example, if one were to randomly place strokes onto the writing surface, the most common two-segment topological shape would be X. Ts would be rarer because they require a coincidental alignment of one endpoint along the edge of another. Ls, in turn, are even rarer because they require the double-coincidence of two endpoints touching. Among the topological shapes with two or three junctions, the shapes with more Xs will be more common in a randomly generated sign, and the shapes with more Ls the rarest; e.g., ≠ is the most common topological shape with two junctions, and Z the least common. This is not at all the case for the visual sign signature, where Ls and Ts are more common than X, and where, for example, ≠ is actually much rarer than Z. Another mechanism for the random generation of visual signs would be the act of scribbling, which is similar to the random-stroke case just mentioned, except that for scribbles Ls are now common, not rare. That is, for scribbles Ls and Xs are much more common than Ts, leading to a distribution of topological shapes unlike that of human visual signs.

[Figure 4 to be put near here.]

Designed for Reading or Writing?

Thus far, we have seen that human non-pictorial visual signs appear to possess a characteristic signature, and we have seen that this signature is not a result of chance. Before attempting to explain this signature, a natural first question is, Does this signature appear to be good for the eye, or good for the hand (or any other writing mechanism)?

There are at least two reasons for expecting that visual sign shapes are designed (by cultural selection) for ease of reading, not ease of writing. First, visual signs are written once, but can be read many times. Second, writing speed is typically limited not by the motor system, but by the time taken for the writer to compose the sentence; that is, writing is not like talking, where we can talk effortlessly without feeling as if we are composing our thoughts.

Shorthand is an example kind of visual sign that violates both of these reasons—it is typically not read more than once, and it is written without the writer having to compose the sentences (instead, the boss is orally dictating). Accordingly, shorthand is designed for the hand at the expense of the eye. We measured the topological shapes across six different shorthand writing systems, and found that their topological shapes are radically different from that found in visual signs more generally.

In contrast, consider trademark logos, which are designed to be seen at the expense of being written (they are, in fact, typically not written at all). We discovered that trademark logos possess the same shape signature found in visual signs.

That is, when we look at signs we know are designed for the eye at the expense of the hand, the signature matches the general signature we saw earlier, but when we look at signs we know are designed for the hand at the expense of the eye, the signature is altogether different.

As further support for this, we found that the visual sign shape signature correlates well with the number of angles in the shape (a measure of visual complexity), but does not correlate at all with the number of hand motions required to write the topological shape (a measure of motor complexity).

Together this makes a strong argument that the topological shapes of visual signs have been selected for reading, not writing.

Natural to the Eye

The topological shapes of non-pictorial visual signs are, then, for the eye, not the hand. But we are still left with the question, Why does the eye like these shapes? Here is where the evolutionary, or ecological, hypothesis enters into the story. Because over millions of years of evolution our visual systems have been selected to be good at processing the conglomerations of contours occurring in nature, I reasoned that if visual signs have culturally evolved to be easy to see, then we should expect visual signs to have natural topological shapes.

Where are these topological shapes in nature? What were conglomerations of strokes for visual signs are now conglomerations of contours for natural scenes. Contours are the edges of objects (as seen by the eye), not, of course, strokes in the world. For example, an L occurs in the world when exactly two edges of an object meet at their endpoints, like an elbow. A T occurs in the world when the edge of an object goes behind another object in the foreground. A Y occurs, for example, at the inside corner of a rectangular room. We measured how common these and the other topological shapes occur in natural scenes, and were stunned to find that nature possesses the shape signature we saw earlier for visual signs. That is, visual signs are shaped like nature, confirming our ecological hypothesis for the shapes of visual signs.

If visual signs look like nature, one might first suppose that the shape signature of nature depends significantly on which natural environment one considers. However, to our surprise, we found that the shape signature is highly robust, differing hardly at all whether we measured images of ancestral environments (e.g., tribal villages, savannas) or urban environments (buildings, walkways). Because of the robust topological notion of shape we used in our analysis, any environment with opaque objects strewn about will tend to have the same shape signature. This underlies why the diverse kinds of visual sign have a similar signature despite the diverse environments from which they spring, and one may speculate that aliens might for this reason possess visual signs that look reminiscent of our own.

We have been considering visual signs generally, but let us now specifically consider letters in phonemic writing system, for there is an additional question one might have about letters. We saw a moment ago that letters look like natural object junctions. Our ecological hypothesis expects letters to look natural, but why natural junctions? Why not have letters shaped like natural single contours? Or, alternatively, why not have letters shaped like whole objects? Instead of one stroke or a dozen strokes, letters in fact tend to have about three strokes (independent of the size of the writing system), and thus are at an intermediate level between edges and objects.

The answer may lie in the following pair of facts: (i) we wish to read words, not letters; and (ii) we have evolved to see objects, not object-junctions. In this light, we expect culture to select words to look like objects, so that words may be processed by the same area in visual cortex responsible for recognizing objects. Logographic characters (e.g., Chinese) and non-linguistic symbols do tend to be more object-like, possessing many more than three strokes. For phonemic writing, however, there are severe limits to how closely words can match natural objects, for the manner in which letters combine is determined by speech. However, by having letters shaped like natural object-junctions—rather than natural contours or natural whole objects—written words become combinations of natural junctions, and thus more similar to objects and more easily processed by our visual system.

Evolution by natural selection is too slow to design our brains for reading, and so cultural selection has come to the rescue, designing (without any designer) visual signs for our brains. Because our visual systems have evolved to be good at perceiving natural objects, cultural evolution has created non-linguistic symbols, logographic symbols, and written words in phonemic writing that tend to be built out of object-junction-like constituents, and are thus object-like. In particular, this explains why letters tend to have around three strokes and have the topological shapes they do. We expect that these insights will be useful in designing optimal alphabets or visual displays.

Because culture is capable of designing for the eye, the visual signs of our culture are a fingerprint of what our visual systems like. Akin to the linguistic study of the auditory productions humans make, the “visual linguistic” study of the visual productions people make is a currently under-utilized tool for vision research. There is every reason to believe that the study of visual linguistics will aid traditional lab experiments on vision and brain design as much as linguistics has supplemented lab experiments on cognition.

Mark Changizi is a professor of cognitive science at Rensselaer Polytechnic Institute. This research – and other work of his on the evolution of color, illusions and stereo vision – are the topic of his new book, The Vision Revolution (Benbella Books).

[A relevant ScienceDaily piece.]

Short Bibliography:

Biederman I & Cooper EE (1991) Priming contour-deleted images: evidence for intermediate representations in visual object recognition. Cognitive Psychology 23: 393–419.

Changizi MA & Shimojo S (2005) Character complexity and redundancy in writing systems over human history. Proceedings of the Royal Society of London B 272: 267-275.

Changizi MA (2006) The optimal human ventral stream from estimates of the complexity of visual objects. Biological Cybernetics 94: 415-426.

Changizi MA, Zhang Q, Ye H & Shimojo S (2006) The structures of letters and symbols throughout human history are selected to match those found in objects in natural scenes. The American Naturalist 167: E117-E139.

Changizi MA (2009) The Vision Revolution (Benbella Books, Dallas).

Daniels PT & Bright B (1996) The World’s Writing Systems. New York: Oxford University Press.

Read Full Post »

Scientific American piece: Why does music make us feel?

By Mark Changizi   

As a young man I enjoyed listening to a particular series of French instructional programs. I didn’t understand a word, but was nevertheless enthralled. Was it because the sounds of human speech are thrilling? Not really. Speech sounds alone, stripped of their meaning, don’t inspire. We don’t wake up to alarm clocks blaring German speech. We don’t drive to work listening to native spoken Eskimo, and then switch it to the Bushmen Click station during the commercials. Speech sounds don’t give us the chills, and they don’t make us cry – not even French.

But music does emanate from our alarm clocks in the morning, and fill our cars, and give us chills, and make us cry. According to a recent paper by Nidhya Logeswaran and Joydeep Bhattacharya from the University of London, music even affects how we see visual images. In the experiment, 30 subjects were presented with a series of happy or sad musical excerpts. After listening to the snippets, the subjects were shown a photograph of a face. Some people were shown a happy face – the person was smiling – while others were exposed to a sad or neutral facial expression. The participants were then asked to rate the emotional content of the face on a 7-point scale, where 1 mean extremely sad and 7 extremely happy. 

The researchers found that music powerfully influenced the emotional ratings of the faces. Happy music made happy faces seem even happier while sad music exaggerated the melancholy of a frown.  A similar effect was also observed with neutral faces. The simple moral is that the emotions of music are “cross-modal,” and can easily spread from sensory system to another. Now I never sit down to my wife’s meals without first putting on a jolly Sousa march.

Although it probably seems obvious that music can evoke emotions, it is to this day not clear why. Why doesn’t music feel like listening to speech sounds, or animal calls, or garbage disposals? Why is music nice to listen to? Why does music get blessed with a multi-billion dollar industry, whereas there is no market for “easy listening” speech sounds?

In an effort to answer, let’s first ask why I was listening to French instructional programs in the first place. The truth is, I wasn’t just listening. I was watching them on public television. What kept my attention was not the meaningless-to-me speech sounds (I was a slow learner), but the young French actress. Her hair, her smile, her mannerisms, her pout… I digress. The show was a pleasure to watch because of the humans it showed, especially the exhibited expressions and behaviors.

The lion share of emotionally evocative stimuli in the lives of our ancestors would have been from the faces and bodies of other people, and if one finds human artifacts that are highly evocative, it is a good hunch that it looks or sounds human in some way.

…continue reading at Scientific American…

Mark Changizi is Professor of Cognitive Science at RPI, the author of The Vision Revolution (Benbella, 2009) and The Brain from 25,000 Feet (Kluwer, 2003).

Read Full Post »

Wall Street Journal review of THE VISION REVOLUTION

This is the Wall Street Journal review of The Vision Revolution that appeared earlier this year.

Why the Eyes Have It
We can read words, gauge distance and see color. How did that happen?

By Christopher F. Chabris

Why are we ­humans so good at seeing in color? Why do we have eyes on the front of our heads rather than on the sides, like horses? And how is it that we find it so easy to read when written language didn’t even exist until a few thousand years ago—a virtual millisecond in evolutionary time?

Details

The Vision Revolution

By Mark Changizi

Ben Bella, 215 pages, $24.95

Read an excerpt of “The Vision Revolution”

Most of us, understandably, have never given much thought to questions like these. What is surprising is that most cognitive scientists ­haven’t either. People who study the brain generally ask how it works the way it does, not why. But Mark Changizi, a professor at Rensselaer ­Polytechnic Institute and the author of “The Vision ­Revolution,” is indeed a man who asks why, and lucky for us: His ideas about the brain and mind are fascinating, and his explanations for our habits of seeing are, for the most part, persuasive.

Mr. Changizi takes care not to call himself a practitioner of evolutionary psychology. This is the one discipline of the mind sciences that focuses on why questions, but it often answers them by telling just-so stories that cannot be ­disproved. (Why do men have better spatial ability than women? Because a long time ago, in Africa, men needed spatial skills to track prey and to kill at a distance—a plausible theory but one that is difficult to test with experiments.) Instead Mr. Changizi calls ­himself a “theoretical neuroscientist,” seeking explanations for the design of the mind that are based on mathematical and physical analysis. He has his own stories, it is true, but they are grounded solidly in neuroscience, and they are backed up by data about a surprising range of human activities, from ­the colors found in historical ­costumes to the ­correspondence between the shapes found in written letters and the shapes found in ­nature.

Let’s start with the question of color. It is such a natural part of our visual experience that we don’t stop to wonder why we can see it at all. ­Without color television there would have been no “Miami Vice,” of course, but were we really missing out on so much when we had only black and white? The consensus explanation for our superior ability to perceive color is that primates evolved it to see fruit—you can’t order dinner if you can’t read the menu.

Mr. Changizi thinks otherwise. He proposes that color vision is useful for distinguishing the changes in other ­people’s skin color—changes that are caused by shifts in the volume and oxygenation levels of the blood. Such shifts, like blushing, often signal emotional states. The ability to see them is adaptive because it helps an observer to “read” states of mind and states of health in others, information that is in turn useful for predicting their behavior.

Our brains evolved in a time when people lived their entire lives without ever seeing someone with a skin color different from their own. Thus the skin color we grow up seeing, Mr. Changizi says, is “neutral” to us: It serves as a kind of baseline from which we notice even minor deviations in tint or hue. Almost every language has distinct words for some 11 basic colors, but none of them aptly describe the look of skin, which seems colorless (except in our recent multicultural societies, where skin color is newly prominent). As one might expect, primates without color vision tend to have furry faces and hands and thus less need to perceive skin color; ­primates with color vision are more “naked” in this respect, humans most of all.

James Steinberg

Conventional wisdom may be similarly misleading when it comes to binocular vision. It is said that we have two forward-facing eyes, which send our brains two separate images of almost everything in our field of view so that the brain can compare those images to estimate the distance of objects—a generally useful thing to know. But people who are blind in one eye, Mr. Changizi notes, can perform tasks like driving a car by using other cues that help them to judge distance. He offers a different explanation: that two eyes give us a sort of X-ray vision, allowing us to see “through” nearby objects to what is beyond.

You can experience this ability yourself by closing one eye and holding your forefinger near your face: It will appear in your field of vision, of course, and it will block what lies beyond or behind it. If you open both eyes, though, you will suddenly perceive your finger as transparent—that is, you will see it and see, ­unblocked, the full scene in front of you. Mr. Changizi observes that an animal in a leafy environment, with such an ability, gains an advantage: It can lurk in tall grass and still see what is “outside” its hiding place. He correlates the distance between the eyes and the density of vegetation in the habitats of animals and finds that animals with closer-set eyes do tend to live in ­forests rather than on plains.

As for reading, Mr. Changizi stops to observe how remarkable this ability is and how useful, giving us access to the minds of dead people (i.e., deceased writers) and permitting us to take in words much faster than we can by merely listening to them. He claims that we learn to read so easily because the symbols in our written alphabets have evolved, over many generations, to resemble the building blocks of natural scenes—­exactly what previous millennia of evolution adapted the brain to perceive quickly. A “T,” for example, appears in nature when one object overlaps ­another, like a stone lying on top of a stick. With statistical analysis, Mr. Changizi finds that the contour patterns most common in nature are also most common in letter shapes.

Mr. Changizi has more to say about our visual experience—about optical illusions, for instance, which he sees as artifacts of a trick the brain uses to cope with the one-tenth of a second it takes to process the light that hits our eyes and to determine what is actually in front of us. He calls for a new academic discipline of “visual linguistics,” and he tells us why there are no ­species with just one eye.

What does all this add up to? Provocative hypotheses but not settled truth—at least not yet. As a theoretician, Mr. Changizi leaves it to others to design experiments that might render a decisive ­verdict. Someone else will have to study how accurately people can perceive mental states from shifting skin tones, and someone else will have to ­determine whether, in most cases, looking at another ­person’s skin adds any useful information to what is easily known from facial expression, tone of voice and body ­language.

Still, the novel ideas that Mr. Changizi outlines in “The Vision Revolution”—together with the evidence he does present—may have a big effect on our understanding of the human brain. Their implication is that the environments we evolved in shaped the design of our visual system according to a set of deep principles. Our challenge now is to see them clearly.

—Mr. Chabris is a psychology professor at Union College in Schenectady, N.Y.

.

[Some related pieces in ScienceDaily: visual computer, "x-ray" vision, color empath, letter shaped like nature, illusions of future]

.

Mark Changizi is Professor of Cognitive Science at RPI, and the author of The Vision Revolution (Benbella, 2009) and The Brain from 25000 Feet (Kluwer, 2003).

Read Full Post »

Why we see illusions, as it ran in the New York Times

This is the article that appeared in 2008 in the New York Times about my perceiving-the-present theory of illusions (the topic of Chapter 3 of The Vision Revolution)…

Mind

Anticipating the future to ‘see’ the present

By Benedict Carey

Staring at a pattern meant to evoke an optical illusion is usually an act of idle curiosity, akin to palm reading or astrology. The dot disappears, or it doesn’t. The silhouette of the dancer spins clockwise or counterclockwise. The three-dimensional face materializes or not, and the explanation always seems to have something to do with the eye or creativity or even personality.

That’s the usual cue to nod and feign renewed absorption in the pattern.

In fact, scientists have investigated such illusions for hundreds of years, looking for clues to how the brain constructs a seamless whole from the bouncing kaleidoscope of light coming through the eyes. Brain researchers today call the illusions perceptual, not optical, because the entire visual system is involved, and their theories about what is occurring can sound as exotic as anyone’s.

In the current issue of the journal Cognitive Science, researchers at the California Institute of Technology and the University of Sussex argue that the brain’s adaptive ability to see into the near future creates many common illusions.

“It takes time for the brain to process visual information, so it has to anticipate the future to perceive the present,” said Mark Changizi, the lead author of the paper, who is now at Rensselaer Polytechnic Institute. “One common functional mechanism can explain many of these seemingly unrelated illusions.” His co-authors were Andrew Hsieh, Romi Nijhawan, Ryota Kanai and Shinsuke Shimojo.

One fundamental debate in visual research is whether the brain uses a bag of ad hoc tricks to build a streaming model of the world, or a general principle, like filling in disjointed images based on inference from new evidence and past experience. The answer may be both. But perceptual illusions provide a keyhole to glimpse the system.

Orbison illusion, or Ponzo illusion variant.

When shown two images in quick succession, one of a dot on the left of a screen and one with the dot on the right, the brain sees motion from left to right, even though there was none. The visual system has apparently constructed the scenario after it has been perceived, reconciling the jagged images by imputing motion.

In an experiment originated by Nijhawan, people watch an object pass a flashbulb. The timing is exact: the bulb flashes precisely as the object passes. But people perceive that the object has moved past the bulb before it flashes. Scientists argue that the brain has evolved to see a split second into the future when it perceives motion. Because it takes the brain at least a tenth of a second to model visual information, it is working with old information. By modeling the future during movement, it is “seeing” the present.

Changizi and his colleagues hold that it is a general principle the brain applies to a wide variety of illusions that trick the brain into sensing motion.

“It’s likely that there are many different neural mechanisms involved in perceptual illusions,” said Jacob Feldman, a Rutgers psychologist. “But the idea that there may be some overarching explanation that accounts for these separate mechanisms is compelling and satisfying to some scientists.”

Timothy Hubbard, a psychologist at Texas Christian University, said the principle of perceiving the present was sound, adding, “If a person’s response to an object, to catch, hit, block, whatever, is to be optimal, that response should be calibrated to where the object would be”— not a split second earlier, when the perception occurred.

This is why identical squares arranged around the center of a spoked-wheel image appear misshapen, said Changizi, who writes about it in a book due in 2009, “The Vision Revolution.” The sides of squares closer to the center appear to bulge. The sides farther out appear shorter. The radiating lines in the pattern trick the brain into perceiving motion forward, so it projects objects forward, making those nearer the center appear closer to the eye.

The same effect can be seen by leaning forward toward a precise checkerboard. The image seems to bulge forward, this time because the eyes are moving.

Changizi says such illusions can also occur in real life. When a golf ball or baseball rolls through the grass and suddenly drops into a hole, the brain sometimes perceives a trace of the ball on the other side of the hole.

“But these are things that we don’t experience very often,” he said, “because the brain is so good at covering up its mistakes.”

===

The paper itself is here. A ScienceDaily piece, and an interview in Scientific American.

Mark Changizi is Professor of Cognitive Science at RPI, and the author of The Vision Revolution (Benbella, 2009) and The Brain from 25000 Feet (Kluwer, 2003).

Read Full Post »

Excerpt of THE VISION REVOLUTION, as it ran in the Wall Street Journal

(BOOK EXCERPT, Wall Street Journal ; WSJ review here)

‘The Vision Revolution’ (Benbella Books, 2009)

By Mark Changizi

Introduction

Super-Naturally

In the movie “Unbreakable” by M. Night Shyamalan, the villain Elijah Price says, “It’s hard for many people to believe that there are extraordinary things inside themselves, as well as others.” Indeed, the story’s superhero, David Dunn, is unaware of his super strength, his inability to be injured (except by drowning), and his ability to sense evil. Dunn would have lived his life without anyone—including himself—realizing he had superpowers if Unbreakable’s villain hadn’t forced him into the discovery.

At first glance we are surprised that Dunn could be so in the dark about his abilities. How could he utilize his evil-detection power every day at work as a security guard without realizing he had it? However, aren’t most powers—super or otherwise—like that? For example, our ability to simply stand requires complex computations about which we are unaware. Complex machines like David Dunn and ourselves only function because we have a tremendous number of “powers” working in concert, but we can only be conscious of a few of these powers at a time. Natural selection has seen to it that precious consciousness is devoted where it’s most needed—and least harmful—leaving everything else running unnoticed just under the surface.

The involuntary functions of our bodies rarely announce their specific purposes. Livers never told anyone they’re for detoxification, and they don’t come with user’s manuals. Neurosurgeons have yet to find any piece of brain with a label reading, “Crucial for future-seeing. Do not remove without medical or clerical consultation.” The functions of our body are carried out by unlabeled meat, and no gadget—no matter how fancy—can allow us to simply read off those functions in a lab.

Powers are even harder to pin down, however, because they typically work superbly only when we’re using them where and when we’re supposed to. Our abilities evolved over millions of years to help us survive and reproduce in nature, and so you can’t understand them without understanding the environment they evolved for, any more than you can understand a stapler without knowing what paper is.

Superpowers, then, can’t be introspected. They can’t be seen with a microscope. And they can’t be grasped simply by knowing the ins and outs of the meat. Instead, the natural environment is half the story. Lucky for us there are ways of finding our powers. Science lets us generate a hypothesis concerning the purpose of some biological structure—what its power is—and then test that hypothesis and its predictions. Those predictions might concern how the power would vary with habitat, what other characteristics an animal with that power would be expected to have, or even what that biological structure would look like were it really designed with that power in mind. That’s how we scientists identify structures’ powers.

And that’s what this scientist is doing in this book: identifying powers. Specifically, superpowers. Even more specifically, superpowers of vision—four of them, one from each of the main subdisciplines of vision: color, binocularity, motion, and object recognition. Or in superhero terms: telepathy, X-ray vision, future-seeing, and spirit-reading. Now, you might be thinking, “How could we possibly have such powers? Mustn’t this author be crazy to suggest such a thing?” Let me immediately allay your fears: there’s nothing spooky going on in this book. I’m claiming we have these four superpowers, yes, but also that they are carried out by our real bodies and brains, with no mysterious mechanisms, no magic, and no funny business. Trust me—I’m a square, stick-in-the-mud, pencil-necked scientist who gets annoyed when one of the cable science channels puts a show on about “hauntings,” “mystics,” or other nonsense.

But then why am I writing about superpowers? “No magic, no superpowers,” some might say. Well, perhaps. But I’m more inclined to say, “No magic, but still superpowers.” I call each of these four powers “superpowers” because each of them has been attributed to superhuman characters, and each of them has been presumed to be well beyond the limits of us regular folk.

That we have superpowers of vision—and yet no one has realized it—is one of the reasons I think you’ll enjoy this book. Superpowers are fun, after all. There’s no denying it. But superpowers are just a part of this book’s story. Each of the four superpowers is the tip of an iceberg, and lying below the surface is a fundamental question concerning our nature. This book is really about answering “why”: Why do we see in color? Why do our eyes face forward? Why do we see illusions? Why are letters shaped the way they are?

What on Earth is the connection between these four deep scientific questions and the four superpowers? I’d hate to give away all the answers now—that’s what the rest of the book is for—but here are some teasers. We use color vision to see skin, so we can sense the emotions and states of our friends and enemies (telepathy). Our eyes face forward so that we can see through objects, whether ourown noses or clutter in the world around us (X-ray vision). We see illusions because our brain is attempting to see the future in order to properly perceive the present (future-seeing). And, lastly,letters have culturally evolved over centuries into shapes that look like things in nature because nature is what we have evolved to be good at seeing. These letters then allow us to effortlessly read the thoughts of the living . . . and the dead (spirit-reading).

Although the stories behind these superpowers concern vision, they are more generally about the brain and its evolution. Half of your brain is specialized for performing the computations needed for visual perception, and so you can’t study the brain without spending about half your energies on vision; you won’t miss out on nearly as much by skipping over audition and olfaction. And not only is our brain “half visual,” but our visual system is by far the most well-understood part of our brains. For a century, vision researchers in an area called visual psychophysics have been charting the relationship between the stimuli in front of the eye and the resultant perception elicited “behind” them, in the brain. For decades neuroanatomists such as John Allman, Jon Kaas, and David Van Essen have been mapping the visual areas of the primate brain, and countless other researchers have been characterizing the functional specializations and mechanisms within these areas.

Furthermore, understanding the “why” of the brain requires understanding our brain’s evolution and the natural ecological conditions that prevailed during evolution, and these, too, are much better understood for vision than for our other senses and cognitive and behavioral attributes. Although about half the brain may be used for vision, much more than half of the best understood parts of the brain involve vision, making vision part and parcel of any worthwhile attempt to understand the brain.

And who am I, in addition to being a square, stick-in-the-mud, pencil-necked cable viewer? I’m a theoretical neuroscientist, meaning I use my training in physics and mathematics to put forth and test novel theories within neuroscience. But more specifically, I am interested in addressing the function and design of the brain, body, behaviors, and perceptions. What I find exciting about biology and neuroscience is why things are the way they are, not how they actually work. If you describe to me the brain mechanisms underlying our perception of color, I’ll still be left with what I take to be the most important issue: Why did we evolve mechanisms that implement that kind of perception in the first place? That question gets at the ultimate reasons for why we are as we are, rather than the proximate mechanical reasons (which make my eyes glaze over). In attempting to answer such “why” questions I have also had to study evolution, for only by understanding it and the ecological conditions wherein the trait (e.g., color vision) evolved can one come to an ultimate answer. So I suppose that makes me an evolutionary theoretical neuroscientist. That’s why this book is not only about four novel ideas in vision science, but puts an emphasis on the “evolution” in “revolution.”

Excerpted with permission of the publisher, BenBella Books, Inc. All rights reserved.

Mark Changizi is Professor of Cognitive Science at RPI, and the author of The Vision Revolution (Benbella, 2009) and The Brain from 25000 Feet (Kluwer, 2003).

[Some related pieces in ScienceDaily: visual computer, "x-ray" vision, color empath, letter shaped like nature, illusions of future]

Read Full Post »

Next book: HARNESSED

Working now on my third book, called HARNESSED: How Language and Music Mimicked Nature and Transformed Ape to Man. Here is the short overview…

If one of our non-speaking ancestors were found frozen in a glacier and revived, we imagine that he would find our world jarringly alien. The concrete, the cars, the clothes, the constant jabbering – it’s enough to make a hominid jump into the nearest freezer and hope to be reawoken after the apocalypse. But would modernity really seem so frightening to our guest? Although cities and savannas would appear to have little in common, might there be deep similarities? Could civilization have retained vestiges of nature, easing our ancestor’s transition?

Although we were born into civilization rather than thawed into it, from an evolutionary point of view we’re an uncivilized beast dropped into cultured society. We prefer nature as much as the next hominid, in the sense that our brains work best when their computationally sophisticated mechanisms can be applied as evolutionarily intended. One might, then, expect that civilization will have been shaped over time to possess signature features of nature, thereby squeezing every drop of evolution’s genius for use in the modern world.

Does civilization mimic nature? In his new book, HARNESSED, Mark Changizi argues that the most fundamental pillars of humankind are thoroughly infused with signs of the ancestral world. Those pillars are language and music. Cultural evolution over time has led to language and music designed as a simulacra of nature, so that they can be nearly effortlessly utilized by our ancient brains. Languages have evolved so that words look like natural objects when written and sound like natural events when spoken. And music has come to have the signature auditory patterns of people moving in one’s midst.

But if the key to our human specialness rests upon powers likely found in our non-linguistic hominid ancestors, then it suggests we are our non-linguistic hominid ancestors. Our thawed ancestors may do just fine here because our language would harness their brain as well. Rather than jumping into a freezer, our long-lost relative may choose instead to enter engineering school and invent the next generation of refrigerator. The origins of language and music may be attributable not to brains having evolved language or music instincts, but, rather, to language and music having culturally evolved brain instincts. Language and music shaped themselves over many thousands of years to be tailored for our brains, and because our brains were cut for nature, language and music mimicked nature. …transforming ape to man.

Mark Changizi is Professor of Cognitive Science at RPI, and the author of The Vision Revolution (Benbella, 2009) and The Brain from 25000 Feet (Kluwer, 2003).

[See related pieces on music in ScienceDaily and Scientific American.]

Read Full Post »

We don’t know jack

To get my blog started, here is a short piece I had written for ScientificBlogging.com about my new book, THE VISION REVOLUTION (Benbella Books, June 2009):

Your color vision is not for seeing red sunsets or green grass; rather, it evolved as a kind of empath sense, optimized to detect the changes in blood physiology in the skin of the faces (and rumps) of others, thereby sensing their emotions. Your forward-facing eyes are not for seeing in depth, but, rather, for significantly enhancing how much you can see in the cluttered forest habitats of your ancestors. Perceptual illusions are not errors your visual system makes in trying to make sense of three-dimensional scenes, but, instead, are due to your brain attempting to foresee the near future, so that by the time the brain generates a perception – which takes a tenth of a second – your perception is of the present. And you have the ability to read not because you’re an especially smart ape (no offense), but because writing has culturally evolved to look like nature, just what your ape visual system is good at processing. My new book – THE VISION REVOLUTION (Benbella Books, June 2009) – is about these four stories; about why we see as we do; about our evolutionary origins; about how our visual capabilities mesh with the world around us. It is about the visual powers you never knew you had.

In the book I also implicitly make a broader point about scientific progress in understanding the brain. Outsiders to the cognitive and brain sciences can sometimes get the impression that we brain scientists have nearly unraveled the riddles of the brain. While it is true that we are making great strides, the real question is, How far away from the finish line are we? Alas, I believe we are nowhere near the finish line; I put my money on several hundred years of brain-slogging left to go. Keep in mind that your brain is more complicated than the rest of the universe combined (minus all the other brains). Truth is, relative to what needs to be known, we don’t know jack.

The reason we have so much work left is that we’re not built like the Arnold Schwarzenegger robot in the Terminator movie. Inside the Terminator’s body – or at least back in a lab where he was designed and manufactured (by other robots) – there are design specifications indicating what all his parts are for. If the Terminator were to become curious about what one of his brain parts is for, he could just gander at the “user’s manual” wherein all his capabilities are enumerated. And many of the Terminator’s perceptions have transparent functions, because his perceptions are often explicitly labeled with what they’re for (e.g., “body-heat sensing camera activated”). Our brains aren’t nearly as scientifically friendly as the Terminator’s. Try as you might, you’ll find no user’s manual in our heads listing our capabilities. You’ll just find gray meat of questionable palatability. And when we perceive, we do so without the benefit of internal written labels explaining to us what the perception is for. Evolution didn’t select us to have user-friendly parts; we weren’t designed to wear our functions – our powers – on our sleeves. What is missing in our understanding of the brain is this enumeration of our functions. Put simply, we don’t even know what we humans can do! And if we don’t know our powers, then we don’t even know what we need to explain. You can’t figure out how the brain carries out X if you don’t yet know we can do X!

The four stories in my book are, as I mentioned earlier, about four powers we didn’t know we have: color is an empath power like that of the annoying Deanna Troi character in Star Trek; forward-facing eyes gives us a kind of “x-ray vision” power to see much better in cluttered habitats; illusions are the signature of our future-seeing power which allows us to perceive the present; and reading itself is a power, only made possible via a clever strategy culture used to make writing easily absorbable by our illiterate visual system. These four heretofore undiscovered powers fundamentally change our view about what our brains can do, and consequently lead to fundamental shifts in the questions we must ask about the underpinnings in the brain. But if these fundamental human powers have only recently been uncovered, one can only imagine the teems of powers that are waiting to be discovered! The brain sciences are filled with brilliant people, but most are not looking to answer such “why” questions. I hope that THE VISION REVOLUTION will excite more people to set their eyes on discovering our enigmatic powers. Only then will we understand what needs to be explained in the brain, a necessary step toward an eventual “brain revolution.”

Mark Changizi is Professor of Cognitive Science at RPI, and the author of The Vision Revolution (Benbella, 2009) and The Brain from 25000 Feet (Kluwer, 2003).

.

[Some related pieces in ScienceDaily: "x-ray" vision, color empath, letter shaped like nature, illusions of future]

Read Full Post »