A Plea for More (Visible) Sex and Violence on TV (Or, Why Your Color Television Is Not Really in Color)

This first appeared on December 7, 2009, as a feature at ScientificBlogging.com.

Dear TV and Movie Producer Person,

I realize that you receive letters all the time complaining about the gratuitous sex and violence on television and in movies. This is not one of those letters. In a sense, I want more sex and violence. Let me explain.

It is worth reminding ourselves why we watch TV and movies. First and foremost, we watch to be entertained. And, secondly, we watch because we get to watch. That is, we watch TV and movies because the visual modality of the experience brings an evocativeness of its own, one that we seem to like. Sure, we like the dialog and the plot twists, but we could have dialog and plot twists via reading or listening to books. We watch TV and movies because, in addition, we get that visual evocativeness.

And one of the best ways to be visually evocative is to show us the characters in the story expressing their emotions. There are a variety of ways to see the emotions in the characters, and gestures and facial expressions are two of the most obvious. But a third avenue for visually communicating the emotions of the characters is the color of the skin, whether on the face or other bare spots. These color signals are especially primal and evocative, and although one is not always consciously aware of sensing these color signals, our color vision is highly optimized for sensing the blood modulations in the skin underlying the color changes.

I want to see these color signals on TV and in movies. I want to see the anger spread over the bad guy’s face when his bomb is found and disarmed. I want to see the subtle blush on the female lead when she’s engaged in frothy banter. I want to see the ashen face of the zombie. And I want to see the veins in the neck of the victim just before the vampire chomps.

But here is the problem, TV and movie producer person: You don’t show us these visually evocative color signals on the skin of your characters. I want movies that don’t merely show skin, but show the emotion on the shown skin. Without this, your shows and movies are emotionally flat, and could be severely enriched.

But instead of aiming to raise the emotionality of our viewing experience to the emotional-3D, you bring us more pixels. Is the greater resolution of HD-TV giving us a truly more evocative experience? The visual detail is indeed impressive, and the educational and sports uses of TV probably benefit from this. But for what I take to be the mainstay of TV and movies – the stories, with actors – I doubt HD-TV makes any difference. In fact, I tend to wear contact lenses that are several years behind the actual needs of my ever-declining eyes. I can hardly read the volume-readout on the screen of my television, much less see the two-million-pixel detail in my expensive HD-TV. But, believe me, I get plenty of evocativeness from the bodies of the humans around me in my life. (Mostly, alas, just my kids’ emotions.) Color signals on bodies, you see, are not in HD.

What we really want is skin-TV, not HD-TV.

TV and movies do give us skin-TV, but only in cartoons. For example, Disney cartoons have long used color modulations on the faces of their characters to bring the emotion to life. Nevermind that many of these characters are furry-faced mammals, or are sponges that don’t even have blood. Cartoonists know how to be evocative, and liberally color signal. Yet you TV and movie people won’t give us this level of evocativeness when real people are on the screen.

There’s a reason you won’t give us skin-TV. It’s that you can’t. Television and movie projection can show these color signals, which is why Spongebob can be seen to blush. The problem is that the cameras used in TV and movies cannot pick up these skin color signals. Cameras have color filters that are very different from the sensitivities of the cones in our eyes. Whereas our cone sensitivities are optimized for sensing the underlying physiological modulations of blood in the skin (they are, in essence, oximeters), cameras are designed to be general-purpose three-filter spectrometers, and can’t detect these skin color changes.

The color dimension that we primates have, but other mammals do not, is the red-green one, and this color dimension is optimized for sensing modulations of oxygenation of the hemoglobin in the skin. Red-green modulations are about the emotions associated with these modulations in oxygenation. The color cameras are able to record modulations in red and green, of course, but not in the one place where it appears to matter most for the evolution of this color sense: on skin. Color television and movies are, in a sense, then, not in color at all. The red-green modulations are found in all the non-skin places, but not on the skin where our eyes evolved to see it. That’s what would have to be fixed in order to raise the level of evocativeness of TV and movies to the emotional-3D.

But designing cameras to let us see skin on screen as our eyes see it in person turns out to be a difficult engineering problem, and I have no insights on how to solve it. My aim with this open letter is to provide new motivation for trying harder to solve it. Lacking the appropriate technology does not merely result in perceptions of the screen that differ from what it would look like in person. Rather, it results in a difference where it matters most: on the skin, and the resulting signaled emotion. If camera technology is not the route to solving this problem, then the other route is to use advances in CGI to add color signals to the bare skin of the characters. …to bring more sex and violence to television and movies, Disney style.

Mark Changizi is a professor of cognitive science at Rensselaer Polytechnic Institute, and the author of The Vision Revolution (Benbella Books).

On LateNightLive, ABC Radio National

Today I was on LateNightLive (of ABC Radio National) with host Phillip Adam.  In addition to divulging my enjoyment at running over pigeons, I got a chance to talk about sex, blood, the blind, writing, rabbit-heads and other topics from The Vision Revolution.

Late Night Live

Late Night Live

Check out the segment here. (Download the mp3.)

Mark Changizi is a professor of cognitive science at Rensselaer Polytechnic Institute, and the author of The Vision Revolution (Benbella Books).

Darwin in the Hot Tub, or The Jacuzzification Of Evolution

This first appeared on November 23, 2009, as a feature at ScientificBlogging.com, coincidentally aligning with the 150th anniversary of The Origin of Species.

“Come on into the hot tub,” I told my three year old boy. But he wouldn’t budge. No way was he joining his older sister in there. “It’s warm, and it feels nice!” I urged, “There’s nothing to be afraid of.” But it was only when I turned off the jets that I could eventually coax him in.

“Why would my boy be so afraid of a hot tub?” I wondered. But as I reflected upon my panty-waist boy, I decided that perhaps I wasn’t being fair to him. In fact, in hindsight, I think he was behaving rationally. Hot tubs are frightening. They violently churn and bubble, as if they are actually boiling. I have spent so much time in hot tubs over the years that I now hardly notice the foam, the burning temperature, the Pseudomonas bacteria and the skin-ripping, high-pressure jets.

We get used to things, and not just to jacuzzis. My jacuzzification also happens for intellectual matters (a topic of an earlier piece, The Value of Being Aloof: Or, How Not to Get Absorbed in Someone Else’s Abdomen). One generation’s jacuzzi is another generation’s maelstrom.

In particular, we get used to evolution. We scientists, especially. We’re so accustomed to evolution that when we find skeptics of evolution, we think of them as poor, blind, close-minded saps who can’t see the most obvious truths.

Darwin's jacuzzi

But how obvious is evolution, really? And how close-minded are those who don’t yet accept evolution?

Let’s start with the obviousness of evolution. First and foremost…evolution ain’t obvious! Evolution is perhaps the craziest true theory ever!   “Let me get this straight: Add a teaspoon of heritable variation, a ton of eating one another, and epochs of time…get yourself a superzoo of fantastically engineered creatures. Yeah, that’s not crazy!”

The only reason most of us scientists don’t find evolution crazy is that we’re jacuzzified to a wrinkley pulp. And this level of comfort with the bizarre theory of evolution can be counterproductive when trying to explain evolution to the uninitiated. You won’t convince my three-year-old to get into the hot tub by suggesting that there is no bubbling or churning – he can see the bubbling and churning with his own eyes. (BTW, no intent to analogize evolution skeptics with three-year-olds! Just a useful analogy that popped up.) If you’re so jacuzzified that you fail to see the churning, you will be incapable of addressing the real worry: that the churning might hurt.

Similarly, if you’re so used to evolution that you fail to see how weird it is, you’ll be in a poor position to explain why it isn’t as crazy as it at first sounds. Better to say, “Yes, evolution is crazy, but there’s overwhelming evidence that it is, indeed, the mechanism underlying the emergence of life in all its glory.” (And you should also admit that, although we have mountains of evidence that evolution is the mechanism, we are very far from understanding how exactly it does it, just as we’re sure the brain underlies our thoughts but do not comprehend how the brain works. This was the topic of an earlier ScientificBlogging.com piece titled Is Evolution Fast Enough?’ How I Responded.

The fact that evolution wins the prize for “non-obviousishness” should already begin to change one’s view about the supposed close-mindedness of evolution’s skeptics. Evolution is extraordinary, and extraordinary theories take extraordinary evidence. Extraordinary evidence indeed exists, but you can’t communicate the evidence in a simple one-liner. (Much less in a one-liner addressing the other as a “close-minded sap”.)

Religious folk surely have their hang-ups (whereas I am utterly hang-up-less), but religious doctrine has come a long way over the centuries. Few still believe the Earth is at the center of the universe, for example, something that was once perhaps just as central to the religious world view as creation. But the evidence for the Earth not being at the center is overwhelming. And more important than being overwhelming, the idea that the Earth is not at the center of the universe is not nearly as crazy as evolution.

Religion can, then, be convinced of scientific discoveries it is initially opposed to. And, it is reasonable to expect that the more intrinsically implausible a theory sounds, the longer it will take for religion to become convinced. Evolution is the king of the implausible, and perhaps that’s why it is one of the last major scientific truths not having infiltrated all the corners of religion.

But evolution won’t infiltrate religion if we scientists can’t address the skeptic’s worries. And we won’t be able to address the worries if we’re so overcooked in evolution that we are incapable of seeing just how preposterous it seems.

====

There were some interesting comments at ScientificBlogging.com, which can be read here. One quote worth repeating here is a response of clarification of mine:

“For the Grand Canyon, I can see how more and more erosion, with self-organizing drainage networks, leads to deeper and deeper and wider and wider etc., etc., etc.

But imagine that I told you that, after all that erosion, the result wasn’t the Grand Canyon, but a modern football stadium, with seats, bathrooms, flat field, fake grass, box seats — the works.  That is, imagine after more and more blind activity, one gets a highly engineered complex structure that *does* amazing stuff.”

That’s what makes the hypothesis of natural selection so crazy. I’d go so far as saying that if you don’t appreciate how crazy it is, you don’t really get it.

 

Mark Changizi is a professor of cognitive science at Rensselaer Polytechnic Institute, and the author of The Vision Revolution (Benbella Books).

Why Doesn’t Size Matter…for the Brain?

This first appeared on November 16, 2009, as a feature at ScientificBlogging.com

No one draws pictures of heads with little gears or hydraulics inside any more. The modern conceptualization of the brain is firmly computational. The brain may be wet, squooshy, and easy to serve with an ice cream scooper, but it is nevertheless a computer.

However, there is a rather blaring difficulty with this view, and it is encapsulated in the following question: If our brains are computers, why doesn’t size matter? In the real world of computers, bigger tends to mean smarter. But this is not the case for animals: bigger brains are not generally smarter. Most of the brain size differences across mammals seem to make no behavioral difference at all to the animal.

Instead, the greatest driver of brain size is not how smart the animal is, but how big the animal is. Brain size doesn’t much matter – instead, it is body size that matters. That is not what one would expect of a computer in the head. Brain scientists have long known this. For example, take a look at the plot below showing how brain mass varies with body mass. You can see how tightly correlated they are. If one didn’t know that the brain was the thinking organ and consequently lobbed it into the same pile as the liver, heart and spleen (FYI, I keep my pile of organs in the crawl space), then one would not find it unusual that it increases so much with body size. Organs do that.

But the brain is supposed to be a computer of some strange kind. And yet it is acting just like a lowly organ. It gets bigger merely because the animal’s body is bigger, even though the animal may be no smarter. The plot below, from a 2007 article of mine (in Kaas JH (ed.) Evolution of Nervous Systems. Oxford, Elsevier) shows how behavioral complexity varies with brain mass. There is no correlation. Bigger and bigger brains, and seemingly doing nothing for the animal!

It has long been clear to neuroscientists that what does correlate nicely with animal intellgence is how high above the best-fit line a point is in the brain-versus-body plot we saw earlier. This is called the encephalization quotient, or EQ. It is simply a measure of how big the brain is once one has controlled for body size. EQ matches our intuitive ranking of mammalian intelligence, and in a 2003 paper (in the Journal of Theoretical Biology) I showed that it also matches quantitative measures of their intelligence (namely, the number of items in ethograms as measured by ethologists). The plot is shown below, where you can see that the number of behaviors in each of the mammalian orders rises strongly with EQ.

But although this is well known by neurobiologists, there is still no accepted answer to why brains get bigger with body size. Why should a cow have a brain 200 times larger than a roughly equally smart rat, or 10 times larger than a clearly-smarter house cat? One of my older research areas, in fact, aimed to explain why brains change in the ways they do as they grow in size from mouse to whale (http://www.changizi.com/changizi_lab.html#neocortex), and yet, embarrassingly, I have no idea why these brains are increasing with body size at all. If a dull-witted cow could just stick a tiny rat brain into its head and get all the behavioral complexity it needs, then brains would come in just one size, and I would have had no research to work on concerning the manner in which brains scale up in size.

So, here’s a plan. I would like to hear your hypotheses for why brains increase so quickly with body mass (namely as the 3/4 power). I will let you know if the idea is new, and I will see if I can give your idea a good thrashing. What’s at stake here is our very framework for conceptualizing what the brain is. Perhaps you can say why it is a computer, and that greater body size brings in certain subtle computational demands that explain why brain volume should increase as it does with body mass. Or, more exciting, perhaps you can propose an altogether novel framework for thinking about the brain, one that makes the enigmatic “size matters” issue totally obvious.

To the comments!…

This is where the fun of the piece begins, because at ScientificBlogging.com there were more than 70 comments, all quite productive (no trolls). So, go here and scroll down to the comments.  …and leave one!

Mark Changizi is a professor of cognitive science at Rensselaer Polytechnic Institute, and the author of The Vision Revolution (Benbella Books).

o one draws pictures of heads with little gears or hydraulics inside any more. The modern conceptualization of the brain is firmly computational. The brain may be wet, squooshy, and easy to serve with an ice cream scooper, but it is nevertheless a computer.
However, there is a rather blaring difficulty with this view, and it is encapsulated in the following question: If our brains are computers, why doesn’t size matter? In the real world of computers, bigger tends to mean smarter. But this is not the case for animals: bigger brains are not generally smarter. Most of the brain size differences across mammals seem to make no behavioral difference at all to the animal.

Instead, the greatest driver of brain size is not how smart the animal is, but how big the animal is. Brain size doesn’t much matter – instead, it is body size that matters. That is not what one would expect of a computer in the head. Brain scientists have long known this. For example, take a look at the plot below showing how brain mass varies with body mass. You can see how tightly correlated they are. If one didn’t know that the brain was the thinking organ and consequently lobbed it into the same pile as the liver, heart and spleen (FYI, I keep my pile of organs in the crawl space), then one would not find it unusual that it increases so much with body size. Organs do that.

But the brain is supposed to be a computer of some strange kind. And yet it is acting just like a lowly organ. It gets bigger merely because the animal’s body is bigger, even though the animal may be no smarter. The plot below, from a 2007 article of mine (in Kaas JH (ed.) Evolution of Nervous Systems. Oxford, Elsevier) shows how behavioral complexity varies with brain mass. There is no correlation. Bigger and bigger brains, and seemingly doing nothing for the animal!

It has long been clear to neuroscientists that what does correlate nicely with animal intellgence is how high above the best-fit line a point is in the brain-versus-body plot we saw earlier. This is called the encephalization quotient, or EQ. It is simply a measure of how big the brain is once one has controlled for body size. EQ matches our intuitive ranking of mammalian intelligence, and in a 2003 paper (in the Journal of Theoretical Biology) I showed that it also matches quantitative measures of their intelligence (namely, the number of items in ethograms as measured by ethologists). The plot is shown below, where you can see that the number of behaviors in each of the mammalian orders rises strongly with EQ.

But although this is well known by neurobiologists, there is still no accepted answer to why brains get bigger with body size. Why should a cow have a brain 200 times larger than a roughly equally smart rat, or 10 times larger than a clearly-smarter house cat? One of my older research areas, in fact, aimed to explain why brains change in the ways they do as they grow in size from mouse to whale (http://www.changizi.com/changizi_lab.html#neocortex), and yet, embarrassingly, I have no idea why these brains are increasing with body size at all. If a dull-witted cow could just stick a tiny rat brain into its head and get all the behavioral complexity it needs, then brains would come in just one size, and I would have had no research to work on concerning the manner in which brains scale up in size.

So, here’s a plan. I would like to hear your hypotheses for why brains increase so quickly with body mass (namely as the 3/4 power). I will let you know if the idea is new, and I will see if I can give your idea a good thrashing. What’s at stake here is our very framework for conceptualizing what the brain is. Perhaps you can say why it is a computer, and that greater body size brings in certain subtle computational demands that explain why brain volume should increase as it does with body mass. Or, more exciting, perhaps you can propose an altogether novel framework for thinking about the brain, one that makes the enigmatic “size matters” issue totally obvious.

To the comments!…

Comments

It seems to me that total brain mass vs. Body size doesn’t account for different parts of the brain.
Intellegence seems to me to be more related to the percentage of brain mass dedicated to the the Frontal cortex vs the total brain mass. Larger Animals may have need for more brain mass to process more nerve receptors in the larger amount of skin for example, or dedicated to processing Smell. But the part of the brain dedicated to higher level functions may be smaller by some measure (either total mass, or percentage of the rest of the brain mass, etc.)

Mark Changizi's picture

Hi Chuck

“to process more nerve receptors in the larger amount of skin”
Nice. That’s one common hypothesis. And not only more skin and thus more sensory receptors, but more musculature, and so on. But *that* would seem to imply that bigger mammals should have disproportionately larger somatosensory and motor areas, but they don’t.

“dedicated to processing Smell”
But why should larger animals need bigger olfactory neural tissue?

The motor processing functions of an animals brain may be proportionate, but the brain as a whole has to take the total motor processing input and output into account; when you are large and have a complex environment to deal with, you need a concordantly larger brain to deal with it.

My Interview with Iran’s Mullahs

Recently I was interviewed by Pouria Nazemi, Science Editor of the Jam-e-Jam Daily Newspaper. Jam-e-Jam is the principal Iranian newspaper and is controlled by the government. In the wake of Iran shutting down its leading business newspaper last week and three pro-reform newspapers in October I thought this would be interesting to readers, since it appeared between these two events.

If your Farsi is up to par, here is the link (and pdf version here).   The interview was done via email in English so I have corrected some minor grammar but otherwise it is as we corresponded.  This originally appeared in English at ScientificBlogging.com .

By the way, before the piece appeared I didn’t realize that Jam-e-Jam was a major newspaper in Iran, much less the state-run newspaper. To my ears, and given that I do not know Farsi, the name sounds “light”, reminiscent of BoingBoing. Do not take this piece as an endorsement of the dictatorship!

Pouria Nazemi : Cognitive science is a new science that we hear more about every day.   Can you briefly describe what it is?

Mark Changizi : Cognitive Science likes to define itself at the intersection of many disciplines, including psychology, neuroscience, linguistics, philosophy and computer science. But, in reality, you’d be hard pressed to pin us down. …other than to say that we’re all interested in understanding the principles underlying thinking, seeing, and other complex brain powers.

Pouria Nazemi : The brain is an amazing thing. We understand the world around us using it but how much do we know about brain itself?

Mark Changizi : Not much – in fact, I wrote a recent blog story titled “We don’t know jack” (Does that translate well?!). There are many avenues for being pessimistic about what we know – or don’t know – about the brain, but one that I often focus on is our powers, or functions. If some alien stumbled upon a calculator or a stapler, would you say that the alien understood these artifacts if the alien did not know that calculators are for math and staplers are for binding paper together? The aliens can take apart, catalog, and watch the workings of calculators and staplers for eternity, and if they haven’t figured out their function, we will be confident they haven’t come to understand them.

We’re in a similar situation as these aliens for our brains. We’ve had significant successes in taking apart the brain and watching its mechanistic workings, but the problem is that much of what our brain can do – most of the functions it is capable of carrying out – are simply not known by us. We’ll be in a good position to make sense of all our mushy meat only when we have a good idea about the functions the meat was selected to implement. And in order to do that, we have to study the human animal in a more ecological setting – that is, we must understand not just the brain, but the complexity of the environment for which it evolved, and how the brain (and body) fit the environment (often) like a glove. For example, my own research often focuses on showing that we have powers no one has noticed. You can be sure that if I’m finding new powers, then there must be tens of thousands more!

Pouria Nazemi: According to Scientific American : “Although many neuroscientists are trying to figure out how the brain works, Mark Changizi is bent on determining why it works that way”; so do you think we can learn why the brain works by having a better understanding of its structure?

Mark Changizi : Another way of saying the same thing is that I want to reverse-engineer the brain. That’s what evolutionary types like me aim to do: figure out the principles governing our “design”.

Pouria Nazemi : When I read about you I find that you have many interesting experiments and theory from writing systems to optical illusions and similarity of brain and highway systems. So what is the main goal of your research in these categories?

Mark Changizi : The research on writing systems asks why our brains, which do not have areas specialized for reading, can read so well. Could it be that the symbols and letters used in writing systems have culturally evolved over time to have the shapes our visual brains are innately good at processing? And, what is our visual brain good at processing? The shapes from nature, in particular from objects strewn about in a three-dimensional world. Could letters have come to look like nature, explaining why we’re such capable readers? In fact, that’s what I found: the contour conglomerations found in natural scenes tend to be the same ones found in human writing.

The brain and highway systems research comes from earlier work of mine trying to explain why brains change in the way they do from mouse to whale. My research shows that much of the anatomical changes that occur as brains increase in size (and there are a lot) can be explained by brains “trying” to maintain a fixed level of total-brain interconnectivity. It struck me more recently that cities have some similarities to the cortex: cities lie on the surface of the Earth and the cortex is a flattenable sheet; highways serve a similar role to white-matter-projecting neurons in the cortex; and highway exits a similar role to synapses. With my understanding of brain scaling in hand, I wondered the extent to which city highway systems scale similarly to the brain as a function of size. To my surprise, there were deep similarities in the scaling laws.

Another major research direction concerns color vision, where I have shown that our kind of color vision is nearly optimal for detecting oxygenation changes in blood under the skin. That is, I have been able to provide evidence that color vision is for seeing the emotions and other socio-sexual signals on the faces (and rumps) of others.

Is there anything tying my research together? Yes and no.

“Yes,” in that I tend to focus on “design principles,” i.e., on the fundamental engineering principles explaining why it would have evolved in the first place. I also bring a similar style to my research directions, aiming for broad unifying theories, ones that are rigorous, ones that can be tested, and ones where I can actually do test. (Rather than many physics journals, say, which publish biological theories without any data.)

But, “no,” in the sense that I do not try to build an incremental program of research. I have always actively tried to remain aloof from previous research problems, and from research communities, so that I am psychologically open to stumbling onto new ideas. Thus the crazy suite of research directions I am embarrassed to admit to.

Pouria Nazemi : One of the most interesting things is your research about the brain’s ability to see into the future (1/10th second) so would you please explain more about that and if it is an ability we can hope to develop?

Mark Changizi: Well, you can’t actually see into the future. The point is that your brain has to anticipate a tenth of a second into the future – and generate a perception of it – because by the time it is done with its anticipating, a tenth of a second has elapsed, and so the anticipated future is of the present. That is, in order to at all times try to perceive the world as it is at that time – to “perceive the present” – the brain has to anticipate the near future.

My contribution here was to show how this simple idea is sufficiently rich that whole swathes of illusions can be explained as cases where the brain incorrectly anticipates the future.

Pouria Nazemi : many of us enjoy optical illusions.  You are studying illusions as a way to understand how our brain works. I think illusions are the result of some error in our mind.  Would you please explain more about how these tricks tell us about our brain.

Mark Changizi: Let me explain one specific case, the Hering illusion shown below, where the two vertical lines are parallel but appear to bow out. Radial lines like those in the illusion do occur very often in real life, in particular whenever you move forward. At these times, the objects in the world flow outward in your visual field away from a center point. In fact, they even often blur on your retina, because your retina is not an infinitely fast “camera”. So, when you fixate on the illusion, your brain sees all those lines emanating from that center point, and says, “When I usually see this kind of radial blur stimulus, it is because I’m moving forward in the direction of the center point.” (I don’t actually mean your brain is saying this! I only mean it has evolved to have mechanisms that figure out where the observer is headed on the basis of blur cues like this.)

Now the brain has a good guess as to where it is headed. Recall that it takes about a tenth of a second to build a perception from the retinal stimulation. The brain wants to generate a perception of the two vertical lines not as they actually projected onto the retina, but as how they will project a tenth of a second later, after the observer has moved forward toward the center a little bit. Think about how the look of two vertical poles, or the sides of a doorway, change their shape as you move forward. When far away they appear vertical in your visual field. But as you near them, to pass between them, they flow outward in your visual field, but do so most quickly at eye level. To see this, imagine walking through a tall cathedral doorway, where when you are close, the upper parts of the door look like they are approaching one another up in the sky (like railroad tracks). That is, when you move forward through a doorway, the sides of the doorway bow outwards in your visual field, just like you perceive the vertical lines in the illusion. You perceive them that way because that is how they would project in the next moment were you moving in the direction your brain has been tricked into thinking it is going. Of course, it is being tricked in this case, so it counts as an illusion. But in real life it typically encounters such radial-line stimuli only when it actually is moving forward toward the center, as I mentioned above.

Pouria Nazemi : Would you please explain about these categories of illusions?

Mark Changizi: The same explanation I just gave concerning the Hering illusion turns out to radically generalize. Radial lines are just one of seven cues I was able to identify for where the observer may be headed in the next moment. And it is not just the visual geometries that can distort in the next moment if you are moving forward; this is just one of four qualities that can distort in the next moment (the others concern speed, brightness contrast, and distances to objects). That is, I eventually realized that the explanation above extended to a 7 by 4 table of 28 predictions, the Hering-type classical geometrical illusions falling in just one of these 28 slots. And I provided evidence that the pattern of illusions predicted by this unified account was, in fact, the case.

Pouria Nazemi : Your recent well-received Book “The Vision Revolution” (that I didn’t have chance to get but would very much like to do) also is about our vision. Would you please tell us about the main focus of this book?

Mark Changizi: The book is about four “powers” of vision. Color vision is for sensing emotion, not for seeing fruit as it has been argued. Forward-facing eyes evolved for seeing better in cluttered forest environments, not for stereo-3D vision as it is usually argued. Illusions are the brain’s (failed) attempt at seeing the future…in order to perceive the present. And letter shapes have culturally evolved to look like nature, turning ancient illiterate visual areas in the brain into capable reading machines.

Pouria Nazemi : We understand our world by our brains. There is nothing out there that we can understand without our brains. Also, we know that our brain sometimes (like in optical illusions) have misunderstandings. Is it possible that some parts of the world that we think we understand are really the result of such misunderstanding? I mean, how we can talk about reality if each brain is that object that determines what reality is?

Mark Changizi: Our brain was selected to provide perceptions that help us survive and reproduce, whether or not those perceptions actually gave us a more objective view of our real universe. The brain could, instead, give us perceptions that are “useful fictions” (which is the term people often use). However, very often the most useful perception to have is the one that actually does truthfully represent the world. As liars know, it takes a lot of work to string together lies in such a way that there are not contradictions. And often the best way to predict the world and not get eaten is to see the world as it is. So much of what we experience is veridical. But not all. For example, as I discuss in The Vision Revolution, color (colors are not out there), stereo vision (we see a single view from a perspective where we have no eye), and illusions (we see a guess) are all cases where useful fictions are at work.

Pouria Nazemi : Your new study show similarity between human brains and highway systems. How can man-made structures can be similar to our brain that evolved during millennia?

Mark Changizi: The idea is that, in each case, there is selection pressure shaping the organization. For the brain it is natural selection, which consists mostly of lots and lots of animals being eaten. For cities, although being eaten does sometimes make the news, the selection pressure is mostly due to multitudes of political and economic forces over many decades, which serve to slowly “push” a city to have a more efficiently functioning highway design.

Pouria Nazemi : Is it possible that someday we can map our brains and understand completely how it works? And, if yes, how long will it take and what our the challenges along the way?

Mark Changizi: Yes. And I’d say hundreds of thousands of years, optimistically. Sorry for the pessimism. I mentioned some of the difficulties above. Another way to put things in context is to consider Caenorhabditis elegans, a little roundworm with 302 neurons, with about a thousand connections, and where we have nearly a “God’s eye view”. It is the most well understood organism on Earth, if not the universe. Despite everything we know about its details, we are a very long way from really understanding how the neural network relates to the complex sets of behaviors it carries out. (Probably because, I would say, we don’t completely understand its behavior.) Our brain has a wee bit more neurons than 302, and is the most complicated machine in the universe. We’re in for a long haul.

Pouria Nazemi : If such a thing happens, everything would change because we could program brains to do many different things. Can you tell us more about the effects of such theoretical mapping in human life?

Mark Changizi: As for programming brains, I have actually thought about that. I wondered whether it may be possible to create images that provoke the visual system to carry out computations. Our visual system would be the hardware, the image would be the software, and the output of the hardware when run on the software would be our perception itself. Why not put our visual brain to work, making it do complicated calculations, and yet it wouldn’t feel like work to us, because all those visual computations are done unconsciously? So I created a class of stimuli called “visual circuits” that can do this, albeit not very well at this point. Here is a Wired story on the research… http://www.wired.com/wiredscience/2008/07/scientists-sugg/

Pouria Nazemi : What is the next step in your studies?

Mark Changizi: For the last year or two I have been studying the origins of language and music. Like letters, I believe that the sounds of speech, and the sounds of music, culturally evolved to sound like nature. Too much to this story to get into here, but they will be the subject of my third book which I hope to finish by the end of the year: HARNESSED: How Language and Music Mimicked Nature and Transformed Ape to Man.

Mark Changizi is a professor of cognitive science at Rensselaer Polytechnic Institute, and the author of The Vision Revolution (Benbella Books).

On The Lionel Show

I was on the Lionel Show / Air America this morning, which was a blast!  Got to talk about my recent book, and about evolution, autistic savants, intelligent design, color, forward-facing eyes, illusions, and more. I really must get off the elliptical machine next time I do a radio show. Here’s the segment with me (or mp3 on your computer).

Mark Changizi is a professor of cognitive science at Rensselaer Polytechnic Institute, and the author of The Vision Revolution (Benbella Books).

E.T. Evolved in Forests

This first appeared on October 26, 2009, as a feature at ScientificBlogging.com

 

Later this evening I’ll be giving a talk to a group of astronomers on what its like to see like an alien. The beauty of this is that I can speculate until the cows come home without fear of any counterexamples being brought to my attention. And even if an alien were to be among the audience members and were to loudly object that he sees differently than I claim, I can always just say that the jury is out until we get more data, and then advise him not to let the door slam into his proboscis on the way out.

E.T. the extra-terrestrial

E.T.'s forward-facing eyes suggests its ancestors evolved in forests

Although it may seem wild-eyed to discuss the eyes of aliens, if we understand why our vision is as it is, then we may be able to intelligently guess whether aliens will have vision like ours.

And in addition to the fun of chatting about whether little green men would see green, there are human implications. In particular, it can help us address the question, How peculiar is our human vision? Are we likely to see eye to eye with the typical alien invader? Or does our view of the world differ so profoundly that any alien visual mind would remain forever inscrutable?

Let’s walk through four cases of vision that I discuss in my book The Vision Revolution and ask if aliens are like us.

Do aliens see in color like us?

Let’s begin with color. I have argued in my research that our primate variety of color vision evolved in order to sense the skin color signals on the faces, rumps, and other naked spots of us primates. Not only are the naked primates the ones with color vision, but our color vision is at the sweet spot in design space allowing it to act like an oximeter and thereby see changes in the spectrum of blood in the skin as it oxygenates and deoxygenates. (See the journal article.)

Aliens may be interested in eating our brains, but they have no interest whatsoever in sensing the subtle spectral modulations of our blood under our skin. Aliens will not see color as we do, and will have no idea what we’re referring to when we refer to “little green men.”

Little green men may not think they look green

This can take the wind out of many people, namely those who feel that their senses give them an objective view of the world around them. But evolution doesn’t care about objective views of the world per se. Evolution cares about useful views of the world, and although veridical perceptions do tend to be useful, little-white-lie perceptions can also be useful. We primates end up with colors painted all over the world we view, but our color vision (in particular the red-green dimension) is really only meaningful when on the bodies of others. Although we feel as if the objects in our world “really” have this or that color, no alien would carve the world at the color-joints we do.

Do aliens have forward-facing eyes?

How about our forward-facing eyes we’re so proud of? I have argued and presented evidence that forward-facing eyes evolved as an adaptation to see more of one’s surroundings when one is large and living in leafy habitats. Animals outside of leafy cluttered habitats are predicted to have sideways-facing eyes no matter their body size, but forest animals are predicted to have more forward-facing eyes as they get larger. That is, in fact, what I found. (See the article.)

So, would aliens have forward-facing eyes? It depends on how likely it is that they evolved in a forest-like habitat (with leaf-like occlusions) and were themselves large (with eye-separation as large or larger than the typical occlusion width). My first reaction would be to expect that such habitats would be rare. But, then again, if plant-like life can be expected anywhere, then perhaps there will always be some that grow upward, and want to catch the local starlight. If so, a tree-like structure would be as efficient a solution as it is for plants here on Earth. The short answer, then, is that it depends. But that means that forward-facing eyes are fundamentally less peculiar than our variety of color vision. Aliens could well have forward-facing eyes, but it would not appear to be a sure thing.

Do aliens suffer from illusions?

One of the more peculiar things our brain does to us is see illusions. I have provided evidence that these illusions are not some arcane mistake, but a solution to a problem any brain must contend with if it is in a body that moves forward. When light hits our eye, we would like our perception to occur immediately. But it can’t. Perception takes time to compute, namely about a tenth of a second. Although a tenth of a second may not sound like much, if you are walking at two meters per second, then you have moved 20 cm in that time, and anything perceived to be within 20 cm of passing you would have just passed you – or bumped into you – by the time you perceive it. To deal with this, our brains have evolved to generate a perception not of the world as it was when light hit the eye, but of how the world will be a tenth of a second later. That way, the constructed perception will be of the present. Although there is no room in this piece to describe the details, I have argued that a very large swathe of illusions occur because the visual system is carrying out such mechanisms. (See the paper.)

Are aliens buying books of illusions and “ooh”ing and “ah”ing at them like we are? If they are moving forward (and have non-instantaneous brains), then they probably are buying these books. This is because the optic flow characteristics that underlie the explanation of the illusions are highly robust, holding in any environment where one moves forward. Aliens are, then, likely to suffer from illusions. The illusions we humans suffer from, then, may not be due to some arcane quirk or mistake in our visual system software, but, instead, a consequence of running the efficient software for dealing with neural delays.

Is alien writing shaped like ours?

I have provided evidence that our human, Earthly writing systems “look like nature,” in particular so that words have object-like structure. And I have shown that for writing like ours where letters stand for speech sounds, letters look like sub-objects, namely object junctions. Certain contour-combinations happen commonly in natural scenes, and certain combinations happen rarely. I have shown that the common ones in such environments are the common letters shapes found in human writing systems. Culture has selected writing to have the visual shapes our illiterate brains can see, which is why we’re such capable readers. (See the paper, a popular piece, and an excerpt from The Vision Revolution on this.)

Would alien writing look like this?

In this light, would alien writing look like nature as well? It depends on how specific one is when one says “like nature.” If, say, our human writing looks specifically like a savanna – i.e., if our writing mimicked signature visual features of the savanna – then it would appear very unlikely that aliens would have our kind of writing. But what if human writing looks like a very general notion of nature, so general it is likely to apply to most conceivable aliens? In my research I have provided evidence that the “nature” that appears relevant for understanding the shape of human writing is, indeed, highly general: namely, “3D environments with opaque objects strewn about.” Although highly general, aliens could float in a soup of cloudy transparent blobs, which is a kind of “nature” radically different than the one that human writing looks like. But it does seem plausible that most aliens will be roaming around opaque objects in 3D, and if that’s the case, then (so long as their culture has selected their writing to harness their visual object recognition system) their writing may look similar to human writing. Alien writing, if thrown into a pile of samples across our human writing, might just fit right in!

—-

So let’s take stock.

Would aliens have our color vision? No. Definitely not. Ours is due to our peculiar hemoglobin.

Would aliens have forward-facing eyes? Maybe. If they evolved in leafy habitats and were large.

Would aliens see our illusions? Probably.   If they move forward.

Would aliens have writing that looks like ours? Probably. If they live in a 3D world with opaque objects.

Mark Changizi is a professor of cognitive science at Rensselaer Polytechnic Institute, and the author of The Vision Revolution (Benbella Books).

Interview on This Week in Science

KirstenSanford

Kirsten Sanford

Kirsten Sanford (shown here) and co-host Justin Jackson (sorry Justin, you understand) of This Week in Science interviewed me last week about my research and my recent book, The Vision Revolution.

Here’s the interview, and I don’t start jibber-jabbering until about 33 minutes in.  Notice how they sucker-punch me right in the belly button. (That’s what they mean by “the kickass science podcast”.)

Mark Changizi is a professor of cognitive science at Rensselaer Polytechnic Institute, and the author of The Vision Revolution (Benbella Books).

“Is Evolution Fast Enough?” How I Responded

Originally a piece in ScientificBlogging, October 15, 2009…


I receive a lot of inquisitive emails from intelligent laymen, and today I received a nice one that asked, in so many words, “Is natural selection fast enough to explain the complex biology we find in our world?”

My knee-jerk response was to say, “Well, of course natural selection is fast enough, because here we are?” But I didn’t do that.

I also didn’t respond by taking out my Dawkins-certified religion-bludgeoning stick. I’m not partial to that pedagogical approach, and I figure it only got Dawkins uncomfortably familiar with Ms. Garrison of South Park.

Instead, I responded in what I think was a more helpful fashion, and my answer was not what the questioner expected. Here is what I wrote:

*****

On how evolution could be fast enough to get things like us, one distinction worth mentioning is that one can be confident that X is the mechanism underlying some phenomenon B, without being clear about how exactly X in fact manages to do B.

An an example, consider the case where X is the brain and where B is our thinking. We are sure that the brain is the mechanism underlying our thinking, but we are still very mystified at how the brain can really engender all this thinking and experiencing we do.

In this light, now let X be evolution and B be the speed at which it can create fancy things like us. Just as we’re sure the brain underlies thinking, we can be sure that the (ugly, sloppy, lengthy) mechanism of evolution underlies the complex biological stuff we find on Earth. And despite being overwhelmingly convinced that this is the case, we can still be thrown for a loop as to how natural selection can do it in the time frames allowed.

I don’t mean to suggest that we don’t know a lot about how the mechanism of natural selection leads to the biology. We do know a lot. And we also know a lot about how the brain leads to our mental life. But in each case there are still huge tracts of unanswered questions. (I’m on the pessimistic side, actually; I believe we’re millennia away from understanding the brain and genes.)

For evolution, knowing the genome is only the first step. It may be hundreds of years or more before they can comprehend how it works together in a unified computational fashion. And with that understanding, they may better appreciate that the range of possible offspring is much much lower — and the survivability and functionality much much higher — than one would expect if genes were only flipping at the individual level.

*****

The general point is that figuring out “that X is the mechanism for B” is a radically simpler problem than figuring out “how X works as a mechanism for B.” This is so obvious, and yet easy to overlook. For example, it doesn’t take a rocket scientist to conclude that my computer is the mechanism that allows these letters I’m now typing to appear before me. But it does take a rocket (or computer) scientist to understand how my computer manages to make this happen. Showing that evolution is the mechanism of life has an astronomically lower bar than showing how evolution exactly does it.

What was unexpected about my response was my admission that we don’t completely understand how evolution can be fast enough. The reason this is unexpected is that we scientists are often unwilling to show any traces of uncertainty about evolution to intelligent-design folks, probably out of a fear that it “may only encourage them.” But we are, in fact, uncertain about some aspects, and if we claim otherwise and get called out, our credibility will be blown.

Instead, we scientists should be glad to display uncertainty about evolution. …but only in regards to how exactly it works in detail to achieve the complexity found on Earth. We should reserve our claims of certainty to where we truly have them, namely in the claim that evolution is the mechanism underlying life on Earth. …and we should emphasize that the bar for showing this is vastly lower than for showing how.

=========

Here is something I wrote in response to some of the comments at ScientificBlogging…

To be clear, my main point is to distinguish between the difficulty in knowing how evolution produces the complexity of life, and the vastly simpler problem in coming to know that evolution is the mechanism that produces the complexity of life.

I did not begin to enumerate the mountains of evidence for the latter — and there are mountains of evidence.  The argument for the latter (i.e., “that” evolution is the mechanism) is not (merely) that one of the alternative mechanisms — God — is not viable.

Similarly, I did not mention any of the evidence for the conclusion that the brain underlies our thinking. The evidence for this, too, is utterly enormous. …and goes way beyond the argument that the alternatives — including, say, that something incorporeal underlies our thinking — are inconceivable.

For any skeptics of evolution out there, my point is that an argument of the form, “But scientists cannot explain how the mechanism of natural selection leads to the complexity we see in the world,” is a fantastically higher bar for scientists to have to fulfill if their goal is to merely show that evolution is the mechanism. By that standard, we should doubt that the brain underlies our thinking. …and we should doubt that livers underlie detoxification, for that matter. For most complex biological structures that do stuff, we scientists have a radically incomplete understanding of how the meat does it. …but that doesn’t make us doubt that the meat does indeed do it, because we have loads of evidence that it is the meat, and not something else, that is (somehow) responsible.

Mark Changizi is a professor of cognitive science at Rensselaer Polytechnic Institute, and the author of The Vision Revolution (Benbella Books).

The Belly Button: Evolution’s Hot Button

Originally a piece in ScientificBlogging, October 10, 2009…

I need most of my body parts. I figure I have my various meaty chunks for good evolutionary reasons, and far be it from me to sell any, no matter how often that creepy guy shows up at my door with a cooler of dry ice offering me money. But if I ever were going to unload one of my body parts, I’d pick the most useless one of all, one that is even more useless than the appendix (although even it has recently been suggested to not be so useless, “Your appendix and your eyes.”)

bellybutton

I would sell my belly button. To illustrate just how useless it is, can you recall when you last used it? I bet you can’t. It is well known that the more common “innie” variety accumulates lint from your clothes (mostly your underwear) over many months, which could be useful if you’re interested in knowing what the average underwear color of your date is – although if you’re close enough to get a good lint sample, my bet is that there is little mystery about the color of the underwear.

Is underwear color documentation the only use of the belly button? Could it have some better function? Has anyone done the perfect experiment where one removes the belly button? Maybe we’re dead in 15 minutes without a belly button! That does seem unlikely.

At any rate, anyone who knows anything will assure you that there is, of course, absolutely no function for the belly button. It is just the remnant scar from the umbilical cord, the cord which was useful while in the womb.

But I’m not so sure.  Mind you, unlike much of my research where I put forth and test an actual hypothesis for what some biological feature is good for (e.g., color vision, forward-facing eyes, the shapes of writing, illusions, number of limbs, brain anatomy), for the belly button I don’t have any hypothesis at all. For the navel I’m drawing mostly a blank. Nevertheless, there are several reasons which suggest that the belly button is not simply a developmental leftover.

First, do you have any idea how many strange things happen to our bodies during development?! Our wee bodies undergo massive transformations, with certain anatomical features appearing at one stage (e.g., tail-like appendages), and completely going away later. All this is crucial for building fancy machines like us. Just as buildings have scaffolding during construction, but don’t end up with scaffolding scars left on them, biology can build structures during development that are needed during development, and that completely go away later. All these developmental back-flips occur, and then get “covered over.” Why not the belly button?

Second, in most mammals the belly button does, in fact, get covered over. Have you ever rubbed the tummy of your dog or cat and felt its belly button? No, you haven’t, because you can’t feel it. And unless you’re a veterinarian or are in desperate need of a hobby, you have never shaved its tummy and visually found it either. Mammal belly buttons tend to be like scars – smoothed over and lintless. Our belly buttons, on the other hand, tend to be much more morphologically well-defined. Easy to see. Easy to feel. Why is ours so conspicuous?

Third, it is generally a bad idea to have skin with an inward divot or outward protuberance. Divots are potential places for infection, and protuberances are at extra risk of abrasion. That’s why the skin over your entire body tends to be divotless and protuberanceless. “Smooth” is good for skin and the bodies they cover, so the fact that belly buttons stick their noses up at this design criterion may suggest they have a reason for existing. That is, it seems to indicate that there is a good reason for belly buttons…a reason good enough to outweigh the disadvantage of having a divot or protuberance.

So, there is a case to be made that morphologically visible belly buttons may be selected for in humans. They could be covered over but are not in us, they in fact are covered over in other animals, and there is an apparent disadvantage to having them.

But what on Earth could belly buttons be for? I don’t know, although my suspicion is that it is linked in some way to sexual selection, and to the fact that we’re the naked upright ape. Other animals’ belly buttons would not be visible to anyone because of fur or because of its being located on the underside, or both. But in our case we’re naked and standing upright, so our navel is something others can potentially see. I have argued elsewhere (“The Hue Of Hefner: How Color Made An Empire Possible“) that perhaps we’re the naked ape because we are color signaling over our entire bodies, not just the face; could it be that the navel is somehow part of the socio-sexual signaling we engage in?

It has not passed my notice, in this light, that the belly button is definitely sexy. Not everything that’s visible on our bodies is necessarily considered sexy; e.g., eyebrows, nostrils and elbows get few fetish videos made about them…I am told. But we can be shy about our belly buttons, and perhaps that is a hint as to what they’re doing there.

Mark Changizi is a professor of cognitive science at Rensselaer Polytechnic Institute, and the author of The Vision Revolution (Benbella Books).