Tactile feedback is feedback generated through the sensation of touch. Unlike force feedback, it does not involve a force as such but makes use of the sensation of texture and of vibration. In traditional acoustic musical instruments this feedback comes to the performer from the resonating body of the instrument. The performer’s body is in contact with the instrument and so the performer feels the vibrations of the instruments body caused by the generation of sound. With a guitar for instance the performer feels vibration of the strings through the fingertips and the vibration of the guitar body through contact with the torso. Wind performers will feel vibration at the mouthpiece and using their fingers at the toneholes or keys.

One of the advantages of digital musical instruments is that they can separate the sound production mechanism from the control mechanism. This allows a digital musical instrument to produce a range of sounds and to make use of a range of control gestures which would just not be possible for acoustic instruments. For instance it is possible to develop a digital musical instrument which involves no contact between the user and a physical instrument at all (such as the Theremin, Tarabella’s Twin Towers, the Dimension Beam, or instruments developed with camera systems like Eyesweb or Rokeby’s Very Nervous System). Even when the performer is in direct contact with a physical interface, sound production generally takes place through synthesis software on a computer connected to amplifiers and loudspeakers, which are often spatially located away from the performer.

This means that the performer loses at least one channel of feedback from the instrument (in the case of completely non-contact instrument, the performer also loses the haptic feedback from the instrument). Therefore, if we wish to make use of tactile feedback in a digital musical instrument, it is necessary to somehow generate vibrations which the performer can feel.

This project deals with two different forms of vibrotactile feedback. The first form simulates the vibrations of an acoustic instrument. In this case it is necessary to generate tactile feedback which would result from the sound of the instrument, were the instrument generating that sound acoustically. The second form of tactile feedback is perhaps of more use for the non-contact instruments. In this case, the instrument is often more abstract and gesture-based and often involves playing in space. It may therefore be useful to use tactile feedback as a communication channel between the performer and the system, perhaps indicating parameter values or changes, or in the case of movement through space to indicate regional boundaries.

Specifically, this project involves research into the ways in which we can provide vibrotactile feedback in digital musical instruments, both to simulate acoustic instrument vibrations and also to add an extra channel of feedback in non-contact digital musical instruments, aswell as the use of this feedback and its effects on composition and performance.

The project is taking place in collaboration with composer Heather Hindman and cellist Chloe Dominguez and is funded by the Center for Interdisciplinary Research into Music Media and Technology (CIRMMT) and also as part of the Digital Orchestra Project. It is divided into two phases, the first involving an evaluation of devices for creating vibrotactile feedback and the development of prototype instruments which incorporate this feedback to simulate acoustic instrument vibrations and the second involves the development of devices for vibrotactile feedback in non-contact instruments as well as the construction of 2 prototype instruments which make use of this feedback.

The initial phase of the project has already taken place. This involved the evaluation of a number of devices for the creation of vibrotactile feedback and also the development of some prototype instruments incorporating this feedback to simulate acoustic instrument vibrations. This section will give an overview of the results of this phase. Further information is available in my paper (150K) from the NIME06 conference.

There are a number of possible actuators which can be used to generate this feedback and I will now discuss them. These are:

• Tactors
• Vibrating motors
• Piezo-electric elements
• Voice-coils
• Solenoids

Tactors are devices designed specifically to generate vibrotactile feedback for interaction design. While it has become more common to refer to any vibrotactile feedback device as being a tactor, more correctly they are specific devices using a pivoting plate to generate vibrations. They are generally designed to be driven from an audio signal and to reproduce all of the frequencies from 40 Hz to 1 kHz, although in most cases they show a major resonance at 250 Hz, the frequency to which we are in general most sensitive. They are generally not readily available except from the manufacturers, generate little or no audible noise and are the most expensive of the devices described here. However, they are one of the most commonly used devices in the design of tactile information systems.

Vibrating motors are generally used in pagers and mobile phones in order to alert the user to incoming information. They use a small rotating motor with an offset weight attached to the shaft. This weight upsets the balance of the motor causing it to vibrate as it spins. The frequency of vibration is generally dependant on the speed of rotation of the motor and is usually quite low (generally less than 100Hz). A high-speed motor could be modified to generate a much wider frequency range by adding an offset weight to its shaft, but these motors are generally larger and more expensive. By varying the voltage sent to one of these motors it is possible to vary the speed of rotation, but this is limited as most motors have a minimum voltage to activate them and so cannot give a full 0-100% speed range. By instead sending a Pulse Width Modulated signal to the motors they can then be used for their full speed range, but this complicates the system design. It is also important to note that the speed and amplitude of vibration are directly tied to one another and that there is no control over the waveform of the vibration.

Piezo-electric elements are crystal elements that vibrate when an AC signal is applied to them. They are often used in low cost buzzers and speakers, but many of them have a limited frequency range (with many having a lowest vibration point at about 400 Hz). Also, they generate extremely loud sounds when they vibrate. However, as they can be driven by a waveform this gives control over amplitude, frequency and waveform of their vibration.

Voice-coils are electromagnetic pieces held in the field of a regular magnet. As the electromagnet is driven with an AC signal, this causes the polarity of the magnet to continually reverse, causing it to move towards and away from the fixed magnet. By varying the frequency and amplitude of the AC signal, the changes in the electromagnetic field can be controlled, allowing for the control of the vibration of the electromagnet. This technology is used in loudspeakers in order to generate sound. This allows such a loudspeaker to be used to generate audio signals, giving control over amplitude, frequency and waveform. As such this was the method suggested for simulating acoustic vibrations in a digital musical instrument.

Solenoids also make use of electromagnets. A pin is held in the field of an electromagnet and that electromagnet is turned on and off by applying and removing a DC voltage. This causes the pin to move in and out of the solenoid and results in a vibration. By varying the speed of the switching of the DC source, the frequency of vibration of the solenoid can be changed. However, the amplitude and waveform of the signal are fixed.

As can be seen, with regard to control, some of these systems offer more than others. Also, certain systems (the tactor, piezo-electric element and voice-coil) are more easily driven as they can make use of an audio signal rather than requiring variable voltage DC or a variable PWM control signal.

Two prototype instruments were developed in order to examine the simulation of acoustic instrument vibrations in digital musical instruments. The first of these was the The Viblotar, which was a monochord-like instrument that incorporated two small audio amplifiers and loudspeakers in order to generate sound internally to the instrument. This results in an instrument which vibrates based on the actual sound being produced, which is most like the vibrations of an acoustic instrument. More information of The Viblotar can be found on its project page.

The second prototype instrument was the Vibloslide. This instrument was built form a short piece of ABS plastic tube, with a single small audio amplifier inside and a loudspeaker at the end where the opening would be in most acoustic wind instruments. It was played by blowing into it to create sounds, the pitch of which was controlled by touching a position sensitive strip on the top of the instrument. This allowed for continous pitch control (similar to a slide whistle), with the sound and resulting vibration generated at the instrument itself.

The second phase of the project is to examine the use of vibrotactile feedback in non-contact instruments. In order to do this, a number of things have to be developed. The first of these is a series of devices to allow for the generation of vibrotactile feedback for the instruments. Following from the results of the evaluation in the previous phase, it would seem that a voice-coil based system would be the most useful and so a system using miniature speakers and amplifiers is in devlopment.

Parallel to this must be the development of 2 prototype non-contact instruments with which to investigate the possiblities offered by the communication of vibrotactile feedback to the performers. This has resulted in the development of the instruments known as t-box and The FM Gloves. The linked pages provide more information on each of these instruments.

Both prototypes have been built and are currently being examined by my colleagues, Heather Hindman and Chloe Dominguez, in order to determine the strengths and weaknesses of each instrument from a composition and performance point of view. Following from these we will determine how best to augment the instruments with vibrotactile feedback devices in order to increase their musical potential.

• Mark T. Marshall and Marcelo M. Wanderley. Vibrotactile Feedback in Digital Musical Instruments. In Proceedings of the 2006 International Conference on New Interfaces for Musical Expression (NIME06), Paris, France, pp 226-229, 2006. Download (150K)

• Mark T. Marshall and Marcelo M. Wanderley, Investigating methods of producing vibrotactile feedback in digital music instruments, Poster for CIRMMT general assembly, 2006. Download (**WARNING: LARGE FILE 50M**)

• Mark T. Marshall, The Viblotar: a big box that makes noise (and vibrates too!), unpublished technical report, 2005. Download (158K)