A Detailing of Sound Synthesis Methods for Embedded Games

Daniel Lopez1

Abstract Sound synthesis originally came from the analog

world, where techniques were derived from mathematical
theory in order to provide low memory consumption and
minimum hardware overview. After an analysis of these, they
may fall under three major categories, Linear, Non-Linear, and
Sample-based synthesis. This paper will discuss how some of
these methods were used to leverage some of the downsides of
the technology of the time. More specifically, how the overall
technology shaped our now standard heuristic for hardware
circuitry for sound synthesis and how, even with our current
technology, these methods are still standards with which we
make synthesis programs.

I. INTRODUCTION
This paper provides a brief overview of the theory and the
implementing techniques used for preparing sound to be synthesized through chips for embedded games developments.
Sound synthesis in games does indeed imply the construction of sound waves dynamically in response to the actions
that occur within the game. This can be accomplished with
different techniques, some of which have been implemented
in devices such as the SID chip in the commodore 64. Bullets
being fired, and the intensity of a racecar being revved are
examples of aural elements that may controlled by the game
during game play in order to add a new layer of realism
and add to the experience of playing. The evolution of the
technology that has been used has arguable been driven in
an attempt to heighten this experience. Elements such as the
ones aforementioned, thusly, need to have certain parameters
that may be adjusted depending on the gameplay. When it
comes to categorizing techniques for synthesis, there are two
main schools of thought, linear and non-linear. In this paper,
these synthesis techniques for quick reproduction of these
sounds are discussed as well as the historical electronics
that were implemented that gave way to even more powerful
pieces of sound creation. Sounds that will also be discussed
will the discrete notes used for video game music and the
techniques used to implement those.

which will be discussed later), or some other sinusoidal

waveform produced by real world input. An example of
the real world input is a guitarist that connected his/her
instrument to an amplifier; the waveforms would really be
an analog (or in the modern world, digital) input of the
vibrations from the string. In any case, the trigger input
is some waveform which is enveloped in the time domain.
This means that the ballistic properties of this waveform
may deconstructed into its four basic components: Attack,
Sustain, Decay, and Release.
The pitch is another signal that describes the desired
frequency. The higher the frequency, the higher the perceived pitch. To produce vibrato, the act of modulating
the frequency of the pitch to go slightly up and down,
the pitch input is combined with some oscillating function,
which is represented in the time domain in the figure.
To mask a certain range of frequency, there is extra
filtering, represented in the middle box in the figure. This,
however, is represented in the frequency domain, where the
x axis represents the range of frequencies, and the waveform
represents the relational dynamic between that range of
frequencies and its amplitude. Different filtering methods
will be described along with the unique approach taken by
the different synthesis, respectively.
Notice that the pitch to filtering, and pitch to oscillating
direction passes through a plus signed combination, whereas
the combination is represented with a multiplication sign.
This is resonant1 with the distinctions between linear and
non-linear synthesis types, as well as the digital approaches
taken by sample-based synthesis.

II. M AIN H EURISTIC

Figure 1 shows the main idea that is applied when attempting sound synthesis, despite the various categories that will
be discussed later. However, the distinction between the time
domain and the frequency domain is not made clear through
the photo.
The trigger input is produced either through some other
set of oscillator functions (in the case of additive synthesis,
*This work was not supported by any organization
1 Daniel Lopez is a student of Computer Science and Engineering at the
University of Nevada, Reno. This work was done under the course work of
Dr. Dwight Egbert. daniellopez@nevada.unr.edu

Fig. 1. All synthesis sounds, through some level of abstraction, go through

this process to produce the desired waveform.

III. L INEAR S YNTHESIS

This is one of the most common and easiest forms of
combination of waveforms. The name linear implies that,
1 Pun

not intended

much like linear algebra, results will be come from the

simple addition of multiple sub-components. [?]
A. Additive Synthesis
Additive and subtractive synthesis fall in the linear family,
as they are produced by a linear addition (or subtraction,
respectively) of multiple sinusoidal waves. The sinusoidal
waves that are added to one another are produced at harmonic
frequencies of one another, meaning the frequency of one is
half the previous before it.
In the most general case, the sum of all of these harmonic
frequencies, given a base frequency, shall produce a square
wave. In a more practice sense, this shall get as close as
possible to a square wave in the real world. The most general
case of the function can be expressed as:

just combine add up together. This technique was more clever

than innovative since it used technology that they had already
mastered, subtractive synthesis, in a new way together, but it
did have a strong influence on the synthesizer market. There
were more competitors for synthesizers than ever before.
The suite of sampled waveforms for the attacks make Linear
Arithmetic synthesizers sound more realistic than others.
IV. N ON -L INEAR S YNTHESIS
As opposed to linear synthesis, non-linear synthesis is
another combination method that employs non-linear algebra; the combination is doing through multiplication or
some other means of construction between two or more
fundamental wave patterns, sinusoidal or otherwise.
A. Amplitude and Frequency Modulation

fSquareW ave (x) =

X
sin([2i 1]x)
i=1

2i 1

This case, additive synthesis, the oscillation, filtering, and

enveloping must be determined to create sounds that are more
realistic. However, the sum of all the resonant frequencies
must be made into a high enough number that may produce a
square comparable to the result of the infinite sum. Although
this does provide the programming team with more control
over these parameters that produce sound, it does increase a
lot of memory.
B. Subtractive Synthesis
The advent of subtractive synthesis took care of the
oscillation from a preexisting complex wave form, such as
a square wave, and only had parameters for the enveloping
that broke down the frequencies into much smaller ones.
This addressed the problem exposed with additive synthesis,
which was the number of elements that are needed is so
much smaller, that there is less memory with which to keep
up.
The main reason that preexisting subtractive synthesis was
cheaper to implement is that it becomes easier to include
an oscillator into your synthesizer which contains a near
perfect implementation of a square wave2 and the generated
waveforms would subtract from that.
The simplicity of subtractive synthesis techniques made
analog synthesizers very easy to produce, hence the creation
of instruments like the Moog synthesizer.
C. Linear Arithmetic Synthesis
The Roland corporation was tantamount in creating a
cheaper alternative for subtractive synthesis that wouldnt
require a Ring Modulation circuit. Their product changed
certain aspects from the attack transient from the envelope,
whilst still implementing the rest of the waveform generated
subtractively to the tail end of their sound. [1] That way,
they had a set of attacks and a set of sustains that they could
2 Any sort of non sinusoidal waveform that is easy to make in a circuit,
such as sawtooth waves or noise

Amplitude modulation and frequency modulation exist

within the realm of non-linear synthesis, where the resulting
waveform is the product of some information-carrying wave,
and a stable oscillating sinusoidal wave. [3] For example,
amplitude modulation is when the information is encoded in
some carrier signal dictating the amplitude of the resulting
wave. The frequency, then, remains constant, since it is
produced from some oscillator. This technique is primarily
for creating effects such as tremolo. Tremolo is a technique
that may be produced with traditional instruments such as the
violin. A violinist might produce this technique by rapidly
sawwing the bow against the strings, producing a short
choppy noise that is repeated a high speed.
If we can consider the carrier signal and the modulating signal as two mathematical sinusoidal functions, one
of which is the amplitude information of the other, we
can mathematically model the resulting sound wave. Lets
consider the carrier signal, the original frequency that is
going to be heard in intervals, as fc . This is defined as
fc (t) = Ac cos(c t)
where Ac is the current amplitude of the signal, and c is
its period. The modulating signal, fm can then be represented
as
fm (t) = Am cos(m t)
The multiplication of these two is the resulting waveform.
s(t) = fc (t)fm (t)
= Ac cos(c t)Am cos(m t)
Ac Am
[cos([c m ]t) + cos([c + m ]t)]
=
2
A combination of a linear technique such as subtractive
synthesis and a non-linear technique such as amplitude
modulation lead to the circuit design of ring modulation,
which is implemented into the Commodore 64s SID chip.
[4] According to the programmers guide, the triangle wave
of oscillator 1 and oscillator 3 are multiplied together. Unless
oscillator 1 is set to produce a triangle wave, ring modulation
occurs in the chip, producing an enveloped sound. [4], [6]

B. Phase Modulation
Phase modulation is another non-linear technique that
implements an encoding scheme, much like Amplitude and
Frequency modulation. However, instead of multiplying the
results, or finding the nested result of both sinusoidal functions, the messaging carrier signal dictates the phase shift
of one of the functions. For example, if the carrier signal
mentioned earlier is redefined such that the phase shift is a
function of the messaging carrier, the resulting signal will
have nesting sinusoidal waves.
fc (t) = Ac cos(c t + c )

(1)

f (t) = A cos(t + fc (t))

(2)

From the example, Eq. 1 represents the carrier wave; the

information containing wave, where c represents the phase
shift. The resulting waveform, found in Eq, 2 contains fc (t)
as its phase shift.
Phase Distortion Synthesis is a non-linear technique which
is a practical implementation that exploits the phase shift
smoothing properties that are produced by phase modulation.
Used by Casio in its suite of CZ synthesizers, it employs
phase modulation as well as other non-linear techniques,
such as the multiplication that is found in amplitude modulation. [7] From their patent, we can see the various waveforms
that are constructively used to create a complex waveform
with varying amplitudes. [7]
From the patent, we can see the progression of the
waveforms that are used while exploiting the mathematical
properties of these multiplication and phase modulation.
At stage A, we have the original triangular waveform and
at stage B, theres another triangular wave with which to
modulate it. However, the triangle at stage B need not have
the same period as the triangle in stage A to produce this
effect, as seen in their Figure 19, our figure 3. The only
requirement that the waveform from B needs to follow is
that at every master period, it resyncs. This is more evident
in 3; there is a small triangle that starts to create which is
quickly shutdown to create when the next period starts.
The sinusoidal wave at stage C, however, is made from
the resonant frequency of the wave at stage B; meaning
they share the same period. This is especially important and
is more apparent in figure 3. The wave at stage D is the
perfect inverse of the wave from A, and stage E shows the
multiplication of the wave from C and D. As you can see,
the waveform is sharply cut off at the end when the master
period is reset due to it having a close periodic correlation
with the wave from stage B. Stage E then has two bumps
that are of different amplitude. These figures, along with
several others were considered as th enovelty of their patent
behind their technology, using Phase Modulation synthesis.
[7]
Although this is more apparent in 3, this is also included in
figure 2. In figure 3, we can see at stage C, where the sharp
change in the sinusoidal exists due to the resonant repeating,
stage E does not contain that sharpness. Stage E

Fig. 2. Figure that showing the progression of multiplication, showing the

use of Phase Modulation

From the patent we can see that

V. S AMPLE BASED SYNTHESIS
There exist a few more digital forms of synthesis which
include using other modular waveforms from which to build
complicated waveforms in an incremental fashion rather than
a combinatorial fashion. However, the original waveforms
from which all other changes are derived, are digital representations of samples recorded from the real world.
A. Concatenative Synthesis
This is method consists of short samples of recorded
sound, which are incrementally placed (one right next to each
other) in order to closely resemble another set of sounds. [2]
These small sound bites are known as units.
A real world example of this is the vocal synthesis found
in many modern day smart phones. Most smart phones will
try to recreate words aural from some manner of dictation
or reading from some text. This recreation is done after a
short analysis of the source text. If the word is small enough
not to be in the phones look up table, some aural structural
analysis is applied to see what combination of modular unit
sounds it can make to reproduce it.

[5] E. Kuehnl, Granular Synthesis, in SoundScience, 1995. [Online].

Available: https://www.erickuehnl.com/granular-synthesis/. Accessed:
Dec. 8, 2016.
[6] T. Klose, MIDIbox SID V2 user manual lead engine,
in MIDIbox SID User Manual, 1998. [Online]. Available:
http://ucapps.de/midibox sid manual l.html. Accessed: Dec. 8,
2016.
[7] M. Ishibashi, ELECTRONIC MUSICAL INSTRUMENT, U.S.
Patent 4,658,691, Apr, 21, 1987

Fig. 3. When a non-periodic frequency is used at Stage B, there exists

strong cuts from Stage C which gets smoothed out at stage E.

B. Granular Synthesis
Granular synthesis is quite similar in that there does indeed
exist small sound bites. However, these sound bites are
typically between the 1-50 ms range. [3]. As a result, more
of these bites are needed for to recreate even the simplest of
sounds. Moreover, these bites are not just placed one right
after another; they are also placed on top of one another, and
maybe even multiplexed. (For multiple voice reasons). [5]
Several layers of filtering may also be applied to parts and/or
whole sections of sounds generated this way to smooth out
the end of one and the beginning of another.
R EFERENCES
[1] G. Reid, Issue navigator, in SoundOnSound, 2016. [Online].
Available:
http://www.soundonsound.com/music-business/historyroland-part-3. Accessed: Dec. 8, 2016.
[2] W. Latta, Constructing simple vehicle sounds, Music
&
Audio
Envato
Tuts+,
2009.
[Online].
Available:
https://music.tutsplus.com/tutorials/constructing-simple-vehiclesoundsaudio-2138. Accessed: Dec. 8, 2016.
[3] J. Bello, NYU Classes, in New York University. [Online]. Available:
http://www.nyu.edu/classes/bello/FMT files/11 synthesis.pdf.
Accessed: Dec. 8, 2016.
[4] H. W. Sams, Ed., Commodore 64 programmers reference guide.
Wayne, PA: Commodore Business Machines, 1982.