Fascinating Art

I have only recently been made aware of the artist Liu Bolin, a modern Chinese artist who paints himself into pictures. This is apparently done without photo editing tricks, and is quite impressive. I wonder what would some of these shots would look like when physically at the scene, and I am also quite curious what he uses to paint himself.


You can read an article about Liu Bolin, and then view this more extensive gallery of his work.

Top-down Processing in Visual Perception Part IV: Ramifications

This is the final instalment of my series on top-down processing in the visual system (links to part I introducing the topic, part II discussing faces and anthropomorphizing, and part III discussing artificial edges). While I find the topics of vision and optical illusions to be fascinating in their own right, I think the analysis of perception and cognition is also vitally important. This is by no means an original outlook, as David Hume made the statement in the introduction to his A Treatise of Human Nature:

'Tis evident, that all the sciences have a relation, greater or less, to human nature; and that however wide any of them may seem to run from it, they still return back by one passage or another. Even Mathematics, Natural Philosophy, and Natural Religion, are in some measure dependent on the science of Man; since they lie under the cognizance of men, and are judged by their powers and faculties.

While couched in somewhat archaic English, Hume's statement strikes me as remarkably astute. In many ways, our brains function as vast pattern-matchers. Understanding the underlying cognitive tricks we use to analyse perception is an important endeavour for making sense of our own observations, and avoiding mistakes in our interpretation of experimental results. Of course, the most pertinent application of perceptual understanding is in automated sensory processing applications (like machine vision which I have discussed before), but as Hume pointed out, it also matters in the way our thought processes interact with every other endeavour. We must be wary of our tendency to anthropomorphize, or to view causal connections that are not actually there. Realising our tendency to perform processing without being consciously aware of it helps reinforce the necessity of mathematical, logical, and statistical tools on which to rest one's theories.

Quick Shot of Physiology

One thing that I remember always puzzling me as a kid was why my voice sounded different to me when played back on a recording device than when I spoke, but everyone else's voices sounded pretty much the same. While I am sure there are likely other factors to this auditory dichotomy, while studying for my physiology exam I realised I had one of the reasons in front of me. I also decided it was interesting enough to share.

In order to transfer sound waves from the air to the liquid inside the cochlea (the inner ear structure which actually contains the nerve cells of the auditory system), there is a rather interesting structure called the middle ear. The primary components of the middle ear are the tympanic membrane (commonly called the ear drum), which vibrates in response to sound, and the three smallest bones in the human body. These bones are collectively called the ossicles, and individually are (in order from outside to inside) the malleus, incus, and stapes. The ossicles form a chain linking the tympanic membrane to the cochlea, thereby transferring vibrations from the air to the liquid inside. Due to the physical limitations of the middle ear, not all frequencies of sound are transferred equally. There is an upper frequency limit bounded by the mass of the ossicles (they can only vibrate so fast) and a lower frequency limit bounded by the stiffness of the system (as an interesting aside, one of the main reasons smaller creatures like cats and mice can hear higher frequencies than humans is because they have smaller and lighter ossicles than we do).

In addition to the membrane and the bones, there is also a pair of muscles: the tensor tympani which attaches to the malleus, and the stapedius which attaches to the stapes. These muscles can contract and increase the stiffness of the system, thereby reducing overall sound transmission and protecting the inner ear from possible damage due to loud noises (this only works effectively, however, on either loud noises which are expected or long-term noise due to the time latency of the muscle reflex). Increasing the stiffness of the system, however, does not reduce the level of sound transmission equally across all frequencies. Since this protective muscular reflex engages in an individual person when he talks, a person consistently has different frequency transmission properties when he speaks versus listening to an auditory playback of his own voice.

Now it is time to get back to studying.

Top-down Processing in Visual Perception Part III: Artificial Edges

This was supposed to be the concluding chapter in my series on top-down visual processing started in part one and continued in part two, but it got quite a bit longer than I expected and will thus be expanded in an upcoming fourth part. In the first installment I introduced the concept of top-down and bottom-up processing and gave a low-level example of top-down processing in the Necker Cube. In the second part I discussed faces in the context both of anthropomorphizing objects through the visualization of faces as well as a preponderance of optical illusions involving faces.

In this installment I am going to discuss another artifact of top-down visual processing which I am going to call artificial edges (I'm not actually sure if there is a better or more technical term for it, so if anyone knows of one, feel free to let me know). I find this phenomenon interesting from both a physiological aspect (in terms of providing evidence for top-down processing) as well as a machine vision aspect (in terms of duplicating our object recognition abilities). The basic idea is that our brain is fairly good at joining edges which belong to the same object but which have been in some manner obscured (either through occlusion, camouflage, or illumination problems). What is interesting, however, is that our visual processing system is so good at this that we can actually create edges and object boundaries that are not there. Two classic examples of this are shown in figures 1 and 2.

Figure 1

Figure 2

There is not actually a white square in figure 1, but the pieces removed from the black circles give the illusion of a white shape occluding them. Our mind then fills in the boundaries of the square to separate it from the white background to the point where we are able to discern boundary lines that are not actually there (of course, those lines disappear when one focuses on them since there isn't actually a change in hue). Figure 2 shows a somewhat more complicated version of the same phenomenon in which the image of a dalmatian is hidden within the scattered ink blots. The artificial edges created in these images are a consequence of our ability to group objects together and differentiate foreground from background. Part of our ability to do this rests in our expectations of what objects are most likely to appear in an image as well as our expectation of how such objects might be arranged. If figure 2 were shown to an individual who had never seen a dog (or even a dalmatian or similarly coloured dog), he would most likely have a very difficult (if not impossible) time spotting it.

That said, I am going to make a brief digression and point out one of the things that makes psychology such an annoying subject - differences in individual processing. Just as some people mentioned they had a difficult time spotting the old lady in the previous part, my girlfriend told me she does not see four circles occluded by a white square in the first image, but rather she sees four Pac-men biting a square. At the least the square is still there so she doesn't completely spoil my premise.

This ability to group contours and blobs together into expected objects is a massive advantage when it comes to image understanding, and it is one of the biggest problems in machine vision. Outside of tightly controlled circumstances, object contours rarely display consistent properties. This is hard for people to even spot, because our mind automatically accentuates valid contours and minimizes invalid contours as shading and texture.

Figure 3

To demonstrate this, I have included a picture I pulled off the internet of a chrome stapler (figure 3). As one can see, there are plenty of strong and weak edges in this image (when I speak of the strength of an edge, I mean roughly the rate at which pixel intensity changes. For a more thorough explanation, see the Wikipedia article). People have no difficulty picking out the stapler in this image and could easily outline the object if one were to ask, despite the fact that this is a monochromatic image and several of the boundary edges are much weaker than internal edges caused by shadow and geometric variations in the object's surface. For example, if you look at the two rearmost edges of the stapler, you can see that the posterior edge is virtually nonexistent while the lateral edge starts fairly strong near the bulb at the front of the stapler but fades as one moves toward the posterior. Our minds have no problem mentally accentuating that lateral edge along its entire length, however, and recognizing that it is a continuous edge despite its vast variation in edge strength. If you looks at the opposite side, however, you see a continuous dark band that extends the length of the stapler, forming two powerful edges. Neither of these strong edges actually depicts one of the object's boundaries, rather they are an artifact of the object's geometry, the view angle, and lighting. Thus, even a computer system whose sole purpose is to determine if one has a picture of a stapler or not would have a great deal of difficulty with that task without some pretty hefty processing on top of the edge detection (even then it would highly unlikely to be as reliable as a person, and we can recognize far more objects than just staplers) or some ability to constrain the view angle, lighting, and object variation. A great deal of these concepts are actually discussed in Gestalt psychology (if you follow the link to the Wikipedia article, you will see some familar images too. It looks like I could have just acquired links from there rather than searching randomly through the internet if I had looked at Wikipedia earlier).

I had planned on speaking about the ramifications of what I have discussed, but I have already been working on this post on and off for several weeks and it is starting to get cumbersome in length. I will therefore be expanding this series into a fourth post to be published in the not too distant future.

Continue reading in Part IV: Ramifications.

Top-down Processing in Visual Perception Part II: Faces

I started this series of posts a couple months ago with Part I on the definition and role of top-down processing. When I originally wrote that post, I had meant to expound upon the topic in a more timely fashion, but I clearly became distracted and forgot about it. If there are other topics which you think I have been neglecting of late, please do not hesitate to leave a comment and I will endeavour to correct such lapses.

As I was saying in the first part of this series, optical illusions are a great way to get one thinking about how one's perceptual system works. In one particular vein of optical illusions are those that 'jump' between interpretations, the most basic being that of the Necker Cube mentioned in the previous post. I think it is particularly revealing about our visual system, however, that when one surveys a large number of optical illusions of that nature, the vast majority are devoted, in at least one of their interpretations, to faces. One of the classic examples of this is shown in figure 1, in which both the back and side of a young lady's face are visible along with the direct side profile of an old woman.

Figure 1: Old lady and young woman

There is good reason for our focus on facial perception, as it is our primary method for recognizing other individuals in social interaction. The ability to differentiate between individuals is an exceedingly important aspect of social intelligence, as there would be, for example, no way without it to differentiate between cheaters and trustworthy members of a tribe. The supreme prevalence of our nuanced ability to analyse faces, however, is often discounted by people. Interestingly, there is a condition known as prosopagnosia in which sufferers lack the ability to distinguish individual faces. There is no problem with the person's sight, but rather faces look as indistinguishable from each other as any other body part (for example, if you could only see peoples' torsos, it would be quite difficult to correctly identify others. There would of course be certain indications like weight and muscle tone, but telling the difference between a pair of scrawny teen boys or flabby middle-aged business men would be awfully difficult). The fact that something can be so selectively lost is rather indicative of quite specialised neuronal processing involved in the identification and distinguishment of faces (although it may be that there are other cognitive impairments that are less obvious). Our predisposition to seeing faces in ambiguous images or in anthropomorphising objects most readily with the appearance of a face (figure 2, 3, , and 4) indicates just how greatly our brain tries to match incoming sensory data with the expecation of seeing a face.

Figure 2

Figure 3

Figure 4

Of course, there are other forms of anthropomorphism available, but the appearance of a face resounds more greatly within us and accentuates the illusion of life. The next post in this series will look at another area of top-down processing as well as some of the ramifications.

Continue reading in Part III: Artificial Edges.

Top-down Processing in Visual Perception Part I: Introduction and Some Examples

One of the subjects I have written about before is machine vision and the incredible difficulty of developing a robust visual processing system that can equal the robustness of our own visual system. It shouldn't be entirely surprising, though, that our visual system is as incredibly powerful as it is, since a huge proportion of our brain is utilized primarily for visual processing. One of the interesting debates in perception psychology and neuroscience is whether the brain performs bottom-up or top-down processing. As with most things (especially in psychology), neither one is entirely correct and your brain utilizes a combination of the two. Optical illusions and trick images are one relatively simple way to explore the way our brain processes visual information, and they are also fairly fun to look at.

Bottom-up processing basically means your brain reads in the raw visual information captured by the retina and gradually figures out what it means as one moves farther along the processing chain that is your cerebral cortex. Top-down processing means you start with an idea of what you ought to be seeing (most likely determined by recent sensory information, other sensory clues, and your past experience). Your brain clearly does some bottom-up processing, since you react to raw changes in the visual stimuli even if there was no reason to expect that change. What is fairly surprising, though, is top-down processing is also clearly involved in visual processing. Effectively introducing top-down processing into artificial visual systems, however, is quite difficult, and it would seem that the top-down algorithms instituted by our brains (and their handy parallel architecture) are what keep us currently so far ahead of computers.

One example of top-down processing that is fairly easy to demonstrate is the blind spot. In your retina you have a small area devoid of receptors where nerves and blood vessels enter and leave your eye. This is normally not a problem since the blindspot of each eye falls on a different area of your visual field, so the sensory perceptions of one eye can compensate for the other. Also, your eyes are almost constantly performing saccades (small jumps around to focus on different regions of the visual field). However, if you close one eye and keep your other eye locked on a specific target, your blind spot becomes anchored in place. You do not realise this, though, because your brian manages to fill in that area of your visual field with its best guess as to what is there. A quick way to demonstrate this is to take a piece of scrap paper and put two X's on it about eight centimeters apart. Then close one of your eyes and stare at the opposite mark with your open eye (for example, if you closed your left eye, look at the left X with your right eye). Hold the paper about half an arm's length in front of you and gradually move it closer. At a certain point, the X on the periphery of your vision should disappear. When it does, it is sitting in your blind spot, and your brain fills in that area with it's best guess (in this case, blank white paper).

Another example that occurs slightly higher up in your visual processing is the Necker Cube, shown here.

This simple drawing forms a three dimensional clear cube. It is ambiguous, though, whether it is intended to be in one of two possible orientations: are you looking slightly down onto the cube, or slightly up at it (in other words, are the bottom two corners corners on the front or back face of the cube)? For most people, there is a default orientation when they first see it. However, after staring at the cube for a few moments, they can cause it to 'flip' into the other orientation. At no point, though, can both orientations be held in one's head at once (at least, I cannot manage to do that). It would seem that your brain takes the visual information provided about the cube's edges and then tries to fit an interpretation on it. Since more than one interpretation is possible, your brain alternates between them. However, whenever one particular interpretation is selected, the others are suppressed to avoid conflicting interpretations of a visual scene.

Continue reading in Part II: Faces.

Vision is Hard

Vision is a remarkably difficult problem. One of my favourite "trick" images that demonstrates some of the challenging aspects of vision is the one displayed above. In the image, there are two squares labelled A and B. Those squares are identical in colour. It takes a bit of effort for most people to accept that, however. If you don't believe me, try loading the image into an image editing program and either get colour information from each square, or copy and paste regions of each next to each other.

Once satisfied that the two squares are actually identical in colour, it is worth stopping for a moment and thinking about why they look so remarkably different. In my experience, most peoples' initial reactions to this image are "how could our eyes screw up so badly?" However, if you think about it, it could equally be viewed a mistake to see the two squares as being identical in colour. It all comes down to the job one wishes to do. In most complex vision tasks, however, grasping the overall pattern of the scene is quite important, which makes viewing the two squares as different shades the more desirable perception to have. However, attaining the perception that a human effortlessly has (in fact, it takes a fair bit of effort to overcome the apparent difference in magnitude of grey) is actually quite difficult for a computer.

Say, for example, that you wish to take an image of a checkerboard like the one in the above image, find all the exposed squares, and label each square as either a dark or a light square. Since we are allowing the possibility of occlusion, variable illumination, and shadows, clearly there can be no simple global threshold used to label each pixel as either a dark or light pixel and then simply group like pixels with each other. If one attempted to do that, A and B would be labeled in the same manner and, therefore, one would be incorrect. So, perhaps you decide to be a little more complex and a little cleverer than that. You decide to locate the top left corner of your checkerboard (assuming that it is never occluded, which can sometimes be a hefty assumption, but we'll allow it) and then find its boundaries using either a blob tool (group all nearby pixels within a certain range of the starting pixel's value) or edge finder. Then, once all the boundaries have been located, compare neighbouring square average pixel values to determine which ones are the light and which ones the dark squares. Unfortunately, there are several problems with this approach. The first is the possibility of falsely splitting a square because of a shadow lying across only part of it. The second is that the thresholds used to find the boundaries of each square will not work at all illumination levels. Algorithms do exist for trying to dynamically find appropriate thresholds, but they are beyond the scope of this brief discussion and are not completely reliable.

Thus, you might decide to try one last method of being to clever to fix the downfalls of your previous method. You model the size of the squares with predetermined values, so once you find one you will be able to know where the others are even if the edge or blob detector fails to find a boundary or finds an extra boundary. However, this will only work if the orientation and apparent size of the board are completely fixed, which is not a reasonable assumption for any but the most constrained environments.

This is not an impossible task. It is not even a particularly difficult vision task if one were able to constrain the position and illumination of the checkerboard. Constraining at least one would still make it much simpler. However, I hope this simple discussion has made it clear just how much more complicated even a simple task like this might be, especially once the light levels and orientation are allowed to vary. When the task is scaled up to allow a wide assortment of objects, it becomes virtually intractable. I say virtually intractable because it clearly must not be, since it is a task that nearly every person on the planet manages to accomplish every day. Figuring out just how that is done, though, is a very fun problem to cogitate.