DPR (Dynamic Physical Rendering) Research at Intel: Touchable, Feelable Nano holograms using “Claytronics”
Imagine you could touch a hologram of anything you choose to create: Maybe a live football game with the little 2 inch players running across your “field”, a person you are having a conversation with over the internet, etc. Amazing stuff is being researched over at Intel. It is called “Dynamic Physical Rendering” (DPR).
“In a hospital in Houston, two surgeons appear to be performing a difficult procedure on a cardiac patient. In fact, only one of the doctors in the room is real. The other is a replica-a lifelike physical model whose shape, appearance and movements precisely mimic those of a specialist in Tokyo who is performing the actual work.
This scenario may seem like science fiction, but research required to realize it has already begun, in a collaborative research project between Carnegie Mellon University and Intel. The goal of the project, which Intel has labeled Dynamic Physical Rendering (DPR), is to create a new form of media the researchers call pario-Greek for “to bring forth” or “to make.”
What the researchers propose to make are moving, physical, three-dimensional replicas of people or objects, so lifelike that human senses would accept them as real. This would eliminate the need for cumbersome virtual reality gear and overcome the viewing angle limitations of modern 3D approaches. The replicas would mimic the shape and appearance of a person or object being imaged in real time, and as the originals moved, so would their replicas. These 3D models would be physical entities, not holograms. You could touch them and interact with them, just as if the originals were in the room with you.
When you finished using a replica for one purpose, you could transform it into another useful shape. A human replica could morph into a desk, a chair could become a keyboard, a lamp could be transformed into a ladder.
Replicas could be scaled up or down in size, to suit the needs of a particular application. A damaged nerve ending could be rendered at ten times its actual size, making it easier to repair. The Super Bowl could be replicated in miniature, with players two inches tall scrambling across a tabletop “field”.
The DPR project was begun at Carnegie Mellon University, spearheaded by Seth Goldstein, an Associate Professor in the Computer Science Department. The project is the brainchild of Goldstein and Todd Mowry, Director of Intel Research Pittsburgh, who first discussed the idea at a conference in 2002. Mowry wanted to improve on two-dimensional videoconferencing, and Goldstein was interested in nanotechnology. They decided to merge their interests. They determined that, by taking advantage of advances in nanoscale assembly, they might create human replicas from ensembles of tiny computing devices that could sense, move, and change color and shape, enabling more realistic videoconferencing. The same meeting environment, with people and objects, could appear at each location, in real form or as replicas. A movement or interaction at any location would be reproduced at all of them. Every meeting could be face-to-face.
What began as a novel idea has evolved into an ambitious collaboration involving almost 30 researchers. Jason Campbell, a senior researcher at Intel Research Pittsburgh, is the Principal Investigator for the DPR project. Goldstein is leading the project for Carnegie Mellon, and Mowry provides additional leadership. The project is being funded by Intel, Carnegie Mellon University, the National Science Foundation, and the Defense Advanced Research Projects Agency (DARPA).
Creating the Replicas
At a high level, there are two steps in dynamic physical rendering: capturing a moving, three-dimensional image and rendering it as a physical object. Much research has already been done in 3D motion capture, as illustrated in blockbuster movies like “Polar Express” and “Lord of the Rings.” Researchers at Carnegie Mellon University also are exploring 3D image capture, in the Virtualized Reality project. They have developed technology that points a set of cameras at an event and enables the viewer to virtually fly around and watch the event from a variety of positions. The DPR researchers believe a similar approach could be used to capture 3D scenes for use in creating physical, moving 3D replicas.
The major challenge for the DPR researchers is how to reproduce images as physical, moving replicas. Replicas will be created from a form of programmable matter-a kind of high-tech modeling clay which the Carnegie Mellon collaborators have dubbed claytronics. Claytronics can be formed into different shapes, and it can change color, through light-emitting diodes on its surface. Embedded photo cells will enable it to sense light, so that a human replica can “see.”
Figure 1. Creating a claytronics replica from a 3D image. Claytronics might even simulate the texture of the person or object being replicated. A replica will have computing capabilities, but these will be accessed through touch, voice, or another natural interface rather than a keyboard or mouse.
The basic unit of claytronics is what the Carnegie Mellon researchers refer to as a catom (claytronics atom). Replicas will be formed from ensembles of tiny catoms. Catoms will be as close to spherical as possible to support multiple packing densities.
An ensemble might contain millions or billions of catoms, which must coordinate and cooperate in order for the ensemble to function. So researchers must consider both the function of individual catoms and their behavior as part of an ensemble.
Each catom will have the minimum combination of computation and actuation needed to contribute to the ensemble. To support scaling, the researchers are looking for algorithms whose running times are proportional to the longest diagonal of the ensemble. If the algorithms require more running time than this, they will quickly become intractable. In addition to programming individual catoms, researchers are focusing on developing programming models which can facilitate the task of coordinating massive numbers of catoms, all of which are simultaneously executing within an ensemble.
From a hardware standpoint, researchers are striving to minimize the complexity of each catom. This will reduce its cost and improve manufacturability. And with fewer parts that could break, simple catoms will be more robust.
Capabilities of Catoms
While catoms will be simple in design, each will have four capabilities:
Computation: Researchers believe that catoms could take advantage of existing microprocessor technology. Given that some modern microprocessor cores are now under a square millimeter, they believe that a reasonable amount of computational capacity should fit on the several square millimeters of surface area potentially available in a 2mm-diameter catom.
Motion: Although they will move, catoms will have no moving parts. This will enable them to form connections much more rapidly than traditional microrobots, and it will make them easier to manufacture in high volume. Catoms will bind to one another and move via electromagnetic or electrostatic forces, depending on the catom size.
Imagine a catom that is close to spherical in shape, and whose perimeter is covered by small electromagnets. A catom will move itself around by energizing a particular magnet and cooperating with a neighboring catom to do the same, drawing the pair together. If both catoms are free, they will spin equally about their axes, but if one catom is held rigid by links to its neighbors, the other will swing around the first, rolling across the fixed catom’s surface and into a new position.
Electrostatic actuation will be required once catom sizes shrink to less than a millimeter or two. The process will be essentially the same, but rather than electromagnets, the perimeter of the catom will be covered with conductive plates. By selectively applying electric charges to the plates, each catom will be able to move relative to its neighbors.
Power: Catoms must be able to draw power without having to rely on a bulky battery or a wired connection. Under a novel resistor-network design the researchers have developed, only a few catoms must be connected in order for the entire ensemble to draw power. When connected catoms are energized, this triggers active routing algorithms which distribute power throughout the ensemble.
Communications: Communications is perhaps the biggest challenge that researchers face in designing catoms. An ensemble could contain millions or billions of catoms, and because of the way in which they pack, there could be as many as six axes of interconnection.
Another unique feature of catom networks is that catoms are homogeneous. Thus, unlike cell phones or other communications devices, the identity of an individual catom is sometimes (but not always) unimportant. An application is more likely to care about routing a message to the catoms comprising a specific physical part of an ensemble (for instance, the catoms comprising a “hand”) rather than sending the same message to specific catoms based on their serial numbers. Furthermore, catoms may be in motion periodically, as the shape of the ensemble changes.
To address these challenges, researchers are investigating new routing techniques that focus on the location and function of the catoms at a given point in time. They are also focusing on building communications highways within ensembles, to limit the complexity of the routing problem.
Research Progress
Figure 2. Prototype of a catom of programmable matter, 44 mm (roughly 2″) in diameter. The perimeter is covered with 24 electromagnets which enable catoms to attract one another and move throughout an ensemble.
The DPR researchers have made good progress in tackling the challenges of motion, power, and communications. And they have achieved an important milestone: the development of a working, two-dimensional catom prototype that is 44 millimeters (roughly two inches) in diameter (Figure 2). Now they are striving to make the hardware more reliable and easier to manufacture. Once they have refined the prototype, they will begin to experiment with shape control, communication and routing algorithms.
The hardware goal for the coming year is to develop catoms that are reasonably robust, and inexpensive enough that kits containing 100 or so catoms could be sold to other researchers who could perform their own experiments and explore new programming models and applications. The software goal is to develop infrastructure for creating simple but interesting software programs to control catom ensembles.” Rest of the Article at Intel
Updated Article at Intel (May 2010)