The Complexity of Maintaining an Array

and Computing Its Partial Sums

MICHAEL L. F R E D M A N

Umversity of Cahforma at San Diego, La Jolla, Cahforma

ABSTRACT Given an array A (k), I _< k <_N, it is desired to perform two tasks' (a) A (i) ~ A (0 + x, and
(b) compute A(I) + A(2) + .-. + A(j), where t, j, and x are arbitrary inputs These tasks are to be
performed on-hne Two computational models are defined relative to which computational complexity is
assessed' an algebraic model, and an reformation-theoretic model The informatlon-theoreuc complexity
measure counts the number of accesses and changes to a random access memory. Upper and lower bounds
are derived, and a trade-off between the complexmes of (a) versus (b) is observed
Categories and Subject Descriptors: E.! [Data] Data Structures, F 1 1 [Computation by Abstract Devices]: Models of Computation--relations among models; F.2.3 ]Analysis of Algorithms and Problem Complexity]. Trade-offs among Complexity Measures, G 2.m [Discrete Mathematics]. Miscellaneous H.I.I
[Models and Principles]' Systems and Information Theory--reformation theory
General Terms: Algorithms, Measurement, Performances, Theory
Addmonal Key Words and Phrases

on-hne complexity

1. Introduction
Let A ( j ) , 1 _< j _< N, be a n a r r a y o f fixed length N. C o n s i d e r the two p r o g r a m s ,
Update(j, x)
A ( j ) := A ( j ) + x

Retrieve(k)
O u t p u t ~ = 1 A (i)

(1)

( j a n d x are p r o v i d e d as inputs to the first p r o g r a m , 1 <_ j _ N, a n d k is p r o v i d e d as

i n p u t to the second p r o g r a m , 1 _< k _ N). T h e s e p r o g r a m s c a n be i n t e r p r e t e d in the
following setting. I m a g i n e a s t a n d a r d i z e d e x a m i n a t i o n given to large n u m b e r s o f
i n d i v i d u a l s over a n indefinite p e r i o d o f time. A s s u m e that e a c h i n d i v i d u a l will attain
a n integer score in the interval [1, N]. I f a n i n d i v i d u a l gets j points, this i n f o r m a t i o n
will be r e c o r d e d b y executing the p r o g r a m U p d a t e ( j , 1). O u r i n t e n t i o n is t h a t A ( j )
represent the n u m b e r o f i n d i v i d u a l s to d a t e h a v i n g scored j points. T o c o m p u t e the
percentile c u r r e n t l y associated with a p a r t i c u l a r score k, we w o u l d n e e d the c u m u lative s u m p r o v i d e d b y executing R e t r i e v e ( k ) .
N o w c o n s i d e r r e d e f i n i n g these p r o g r a m s as follows:
Update(j, x)

For i := j until N do
S(i) :-- S(i) + x

Retrieve(k)
Output S(k)
(2)

This work was supported m part by the National Soence Foundation under Grant MCS 76-08543
Author's address' Department of Electrical Engineering and Computer Sciences, University of California
at San Diego, La Jolla, CA 92093
Permission to copy without fee all or part of this material is granted provided that the copies are not made
or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication
and its date appear, and notice is given that copying is by permission of the Association for Computing
Machinery To copy otherwise, or to republish, requires a fee and/or specific permission
1982 ACM 0004-5411/82/0100-0250 $00.75
Journal of the Association for Computing Machinery, Vol 29, No I, January 1982, pp 250-260

Maintainmg an Array and Computing Its Partial Sums

251

In (2) we interpret S(j) to be the number of individuals having scored j or fewer

points. If A(j) and S(j) are initially all zeros and an intermixed sequence, al,
oz. . . . . or,, of the programs Update(j, x) and Retrieve(k) is executed, the same
outputs will be generated independently of whether the program definitions in (1) or
(2) are used. Thus the programs in (1) and (2) solve the same on-line computational
problem. However, in (1) the Update program takes constant time and the Retrieve
program can take time N (worst case), whereas in (2) the opposite holds.
A third solution to this problem involves using a balanced binary tree on N leaves
as a data structure. In t h e j t h leaf from left to right we store the number of individuals
who have scoredj points. In each internal node we store the sum of the values in the
leaves of the subtree determined by that node. Update(j, x) is then accomplished by
adding x to the values in the nodes consisting of the j t h leaf and its ancestors.
Retrieve(k) is performed by summing the values stored in an appropriate set of
nodes. In the worst case, approximately log2N node values are either incremented or
summed for the update and retrieve commands, respectively.
These three solutions suggest the possibility that a trade-off exists between the
complexities of performing updates and retrieves. The purpose of this paper is to
analyze the class of on-line algorithms for implementing the update and retrieve
operations. We derive both upper bounds and intrinsic lower bounds on the complexity of optimal algorithms. We indeed exhibit a general trade-off between the
costs of updates and retrieves, and we are also able to improve upon the (intermediate)
logzN-cost solution; specifically, the log2N solution is replaced by a logs.83N solution.
Strong evidence as presented to suggest that log5 83N is best possible. That is in fact
the case according to a particular algebraic model of computation. We also define an
information-theoretic model of computation wherein we derive a log N/loglogN
lower bound on the worst case complexity.
Our motivation is studying this topic, which is hereafter referred to as the array
maintenance problem (abbreviated AMP), is that it provides a toy problem which is
both tractable and surprisingly interesting. It is expected that our methods of analysis
will find other applications.
In Section 2 we defme a linear algebraic computational model relative to which
bounds are derived. In Section 3 the same is done relative to an information-theoretic
model.

2. Algebraic Complexity
In this section we assume that the array A (j), 1 _<j_< N, stores values in an arbitrarily
specified commutative group G. We consider a class of straight-line programs which
are constrained to function correctly independently of the choice of G.
The three approaches outlined in the introduction have the following features in
common. A set of variables is maintained (organized as an array in (1) and (2),
identified with tree nodes in the third approach). Retrieve(k) is performed by
evaluating a linear function of these variables, and Update(j, x) is performed by
incrementing these variables by amounts which depend linearly on x. Our algebraic
model comprises all programs having these features.
Thus our algebraic model consists of triples, (V, R, U), where V = {vl, . . . , vm) is
a set of variables, R is an N x m matrix, R = (ak3, and U is an m N matrix,
U = 680) (m can vary). Associated with a triple (V, R, U) are the programs,
Update(j, x)
for l := 1 until m do
v~ := vt + Box

Retrieve(k)
output ~l% C~klVz
(3)

252

MICHAEL

L. F R E D M A N

To accommodate arrays which store values in arbitrary commutative groups, we

assume that the matrices R and U are over the integers (positive, negative, or zero).
The following lemma provides a condition on R and U equivalent to the correctness
of the induced programs in (3).
LEMMA 1. Let ,IN be the N x N triangular matrix consisting of O's above the main
diagonal and l's on or below the main diagonal. Then the programs in (3) are correct
if and only if R U = JN (matrix product).
PROOF. Consider an execution of the sequence Retrieve(k), Update(j, x), Retrieve(k). Let zl and z2 denote the two respective outputs,
zl =

C~kzVl

zz = ~ Olkl(Vl "1- flljX).

and

lsl

l~1

It is readily apparent that our programs are correct if and only if

if j<_k,
if j > k .

{0

z2-zl=

But z2 - zl --- (Y~I aktBo) x, and the lemma follows immediately.

[]

We define the complexity of Update(j, x) to be the number of nonzero coefficients

B0, 1 _< l _< m, and the complexity of Retrieve(k) to be the number of nonzero
coefficients akz, 1 _< l _< m. It follows that in the worst case the complexity of an
update is the maximum number of nonzero entries in any column of U, and the
worst case complexity of a retrieve is the maximum number of nonzero entries in any
row of R. We define the complexity of the matrices U and R accordingly.
We are now ready to derive upper bounds by constructing a class of programs and
data structures which fall within the confines of our model.
2.1 (u, r)-SxsTEMS. A (u, r)-system is a recursively constructed data structure for
maintaining an array of length at most N = N(u, r)(N(u, r) is defined below), such
that the worst case complexity of an update is u and the worst case complexity of a
retrieve is r. We assume that the array being maintained is A (j). These systems are
easily described in terms of diagrammatic notation. Our diagrams consist of the
following items.
(a) A horizontal line partitioned into consecutively numbered equal length intervals,
for example,
I

I
1

I
2

I
3

I
...

I
N

(b) Horizontal, labeled line segments above and parallel to the line in (a), whose
endpoints appear directly above interval boundary points of the line in (a), for
example,
t

I
1

I
2

I
3

I
4

I
5

Maintaining an Array and Computing Its ParUal Sums

253

(c) Zero or more instances of ~ - ~ or ~

appearing above and parallel to the
line in (a), extending over exactly N(i, j) intervals (i and j are integers). We
define
i

N(i, j)

if j = l ,

J(i N

1, j) + U(i, j +N(i-l,j-l)+l

if

1)

i = 1,

if i > 1, j >

1.

Our diagrams are interpreted as follows: The line segments uniquely correspond
to variables. The values of these variables will be maintained so that at a given
instant, the variable associated with the line segment passing over the intervals
numbered g i + 1. . . . . j will have value A (i) + . . . + A(j). The label of a line
segment will also be the name of its associated variable. Now consider the following
dmgram:
r
s
I

t
I

v
I

w
I

.v
1

a
I

b
I

d
c

I
8

I
9

e
I

10

(4)
I

11

I
12

This diagram corresponds to a data structure for maintaining A (1) . . . . . A (12).

Update(j, x) is performed by adding x to each variable associated with a line segment
passing over mtervalj. Thus Update(7, x) is accomplished by executing
a := a + x,

z := z + x.

Retrieve(k) is performed by adding and subtracting appropriate variables, in effect

combining (adding) and cancehng (subtracting) line segments so as to cover the
intervals 1 through k. Thus Retrieve(9) is accomplished by evaluating r + z - c.
Observe that a data structure for maintaining an array for the purpose ofcomputmg
the partial sums,
J

Y,A(i),

I<_j<_N,

can be directly "translated" into a data structure which computes the pamal sums,
re+J--1

A(i),

1 <_ j_< N,

(5)

t=m

as well as "translated and reflected" into a data structure which computes the partial
sums,
m

A (i),

1 <_j <_ N.

(6)

t=m-J+l
-....-.4

The notation ~
refers to the complex of hne segments comprising a (possibly)
translated (u, r)-system such as described in (5); it would extend precisely over the
intervals m, m + 1. . . . . m + N 1, where N = N(u, r). The notation
analogously refers to the line segments of a translated and reflected (u, r)-system as
in (6).

254

MICHAEL

L. FREDMAN

Su

$3

S2
al

a2

a3

ar

Sl

I
1

(a)

(b)

FIG 1 (a) A (1, r)-system (b) A (u, l)-system

II

(u-- l,r)

...

m~

i
. .

(u,r-

1)

I
?/12

..

N = N(u,

r)

ml= N ( u - l,r)
m2= N ( u - l,r)+ N ( u - l , r - 1)+ 1
FIG. 2

A (u, r)-system for u > I, r > 1.

W e are n o w ready to define a (u, 0-system. A (1, 0 - s y s t e m is given by the d i a g r a m

in Figure la; a (u, 1)-system is given b y the d i a g r a m in Figure lb. T h e p r o g r a m s (1)
and (2) clearly correspond to Figures l a and b, respectively. F o r u > 1, r > 1,
a (u, r)-system is given in Figure 2.
T o p e r f o r m U p d a t e ( j , x), i f j < m2, then the variable v is incremented as well as
the ~ u -

1 variables whose line segments (contained in I(u (

1, r)l i f j -< ml,

contained in I(u - 1, r - 1)1 i f j _> ml + 2) pass over interval j. I f j _> mz + 1, then

the _ u variables identified with line segments (contained in [(u, r - 1)[ ) passing
over i n t e r v a l j are incremented. It is clear that in the worst case, u variables m a y be
incremented
T o p e r f o r m Retrieve(k), if k _ m~, then the appropriate linear expression revolving
I

_<r variables within I(u I-

expression involving _<r -

1, r)[ is evaluated. I f ml + 1 _< k _< m2, then the h n e a r

(

1 variables in [(u -

1, r -

l)1, equivalent to the

sum

A(k + 1) + . . . + A(mz), is evaluated, and the result is subtracted f r o m v. Retrieve(mz)

simply returns the value in v. If k > mz, then the appropriate linear expression
m /

involving _<r - 1 variables in [(u, r - 1)1 ,equivalent toA(m2 + 1) + . . . + A(k), is

evaluated, and the result is added to v It is clear that in the worst case, a h n e a r
expression involving a total o f r variables is evaluated In s u m m a r y we have
h

THEOREM 1. There exist programs Update(j, x) and Retrieve(k), 1 <_ j, k <_

N(u, r), for mamtaimng and evaluating partial sums of an array of length N(u, r) such
that the cost of executing Update(fi x) does not exceed u and the cost of executmg
Retrieve(k) does not exceed r. N(u, O is given by
u

zf r = l ,

N(u-

tf u = 1,
f u,r > 1.

N(u, r) =
l,O + N(u,r-

1) + N ( u -

1, r -

1)+ 1

Maintaining an Array and Computing lts Partial Sums

255

An asymptotic analysis shows that N(k, k) ~ c~k/x/-k, where ~ = 3 + v/2 -- 5.83...

and c -- x/~,/cr/29/4 = 0.29 . . . . Inverting, we conclude that the U p d a t e and Retrieve
p r o g r a m s can be constructed for an array o f length N having worst case complexity
l o ~ 83N + O(log log N).
T h e R and U matrices which are induced by these p r o g r a m s are over ( - 1 , 0, 1 }.
Thus the arithmetic associated with these p r o g r a m s is very simple, consisting o f
additions and subtractions.
2.2 LOWER BOUNDS ON COMPLEXITY. W e have shown that log~s3N +
O(log log N ) is an achievable u p p e r b o u n d on the worst case complexity o f U p d a t e
and Retrieve p r o g r a m s which m a i n t a i n an array o f length N. W e proceed to show
that log~s3N - O(l) is an inherent lower b o u n d on the average complexity o f these
tasks.
THEOREM 2. Let Update(j, x) and Retrieve(j) be programs described by the
algebraic model which solve the array maintenance problem for an array of length N.
Then the average cost of executing these programs is at least logxN - O(1), where
= 3 + 2x/2 = 5.83.
By L e m m a 1 our analysis o f the A M P reduced to a study o f factorizations J2v =
R U, where R and U are matrices over the integers. T h e cost o f executing Retrieve(j)
is the n u m b e r of nonzero entries in the j t h row o f the associated matrix R, and the
cost of executing U p d a t e ( j , x) is the n u m b e r of nonzero entries in t h e j t h c o l u m n o f
the associated matrix U. F o r the purpose o f deriving a lower b o u n d on the average
of these costs, we analyze factorizations J2v = R U for matrices over the integers
m o d u l o 2, as lower bounds pertaining to these kinds of factorizations are certainly
inherited as lower bounds pertaining to factorizations over the integers.
So let R = (a,j) and U = (b,j) be matrices over the integers m o d u l o 2 such that
R U = J N . L e t A , = {kla,k = 1}, 1 <_i<_N, a n d l e t B , = (klbk, = 1), 1 _< i_< N. T h e n
R U = (6,j), where 6,~ = [A, f3 B j l m o d 2 . Since R U = JN, IA, t3 BjI ~ IA~ n B,I if
i # j. Therefore A, f3 Bj # Aj f3 B, if i # j. Since IA, I is the n u m b e r o f nonzero entries
in the ith row o f R and [ B, [ is the n u m b e r o f nonzero entries in the ith c o l u m n o f U,
T h e o r e m 2 is an i m m e d i a t e consequence of the following.
THEOREM 3. Let A1
AN, Bx
BN be a family of finite sets such that
At f3 Bj # Aj f3 B, whenever i ~ j. Then ~N1 ([At I + I B, l) --> 2logaN!
. . . . .

. . . . .

PROOF. First we define some notation. Let [1, j] denote (1, 2 . . . . . j}. G i v e n a
permutation r over [1, N ] and a set S C [1, N ] , we let or(S) denote { J l j ffi ~r(0 for
some i E S}. I f S C__[1, N ] a n d j ~ S, we let r a n k s ( j ) denote the n u m b e r o f elements
in S which are _<j.
N o w let s = A1 tO . . . tO AN to B~ tO . . . to BN = {al . . . . . am}, say. G i v e n
a ~ ~ , we let Ta = {ila E A, and a ~ B,}, X~ = {ila ~ A, and a ~ B,}, and
Y~ = {ila f~ A, and a ~ B,}. T h e sets T~, X~, and Y~ are disjoint. W e let Sa =

T~ to Xa t_J Y~ = (ila ~ A,ora ~ B,}.

Observe that
N

Y, (IA, I + In, I) -- Y, (IZ~l + 21X~l + I Y~I).

t~l

(7)

a~_zag

O u r strategy is to utilize the sets T~, X~, and Y~, ranging over a ~ .~, to uniquely
encode each of the N! permutations r over [1, N]. G i v e n r and a ~ ~, we define
T~,~ = {tJl = rank~(s~)(j)
Xa,. = {i I i = rank.ts,~(j)
Ya., = {i] i = rank~cso)(j)

for some
for some
for some

j E ~r(T~)),
j E ~r(X~)},
j E ~r(Y~)).

256

MICHAEL

L. FREDMAN

Because Ta, X,, and Y, are disjoint, so are r(T& n(X,), and n( Ya). Moreover, so are
T,,,, Xa,,,, and Y,,,. Further!:iore, Ta,n U X,,,, U Ya,, = [ 1, ] S, 11.We let P,,, denote the
ordered triple (T,,,, X,,,,, Y,,,,). Thus Pa,,, defines a partition of [ 1, 1S, I] into three
blocks, one of size ] T, 1, one of size I X, I, and one of size I Y, I. Now let L, denote the
vector (Pa,,,, Paz,=,. . . , P,,,,,). (RecalI that & = {al, . . . , a,) .) The jth component of
L, can assume at most

I Ta,I+ I&l + I Ya,I

( IT&l, IXd~ I Y&l)
possible values as 7r ranges over the N! permutations of [l, N], since this is the
number of possible partitions in question. Thus L, assumes at most

e = fi I Ta,I+ l&l + l ya,l

J1 lTa,l, &I, 1ya,i
possible values as 7~ranges over the N! permutations.
L,, # L, if ni # 7~2.Therefore we get

(8)

Lemma 2 (below) implies that

N! I 8.
Lemma 3 (below) combined with (8) and (9) gives
yoo 2 N!
(1 + JZ)Cd72+w+l

(9
(10)

Since (1 + J2)2 = 3 + 2&, we complete the proof of Theorem 2 by taking the

logarithm of both sides of (10) and combining the result with (7). Cl
LEMMA

2. If v1 # 7~2,then for some a E sd, Pa.,, # Paswz.

PROOF.
Choose z so that K(y) = n;i(y) for ally < z and n:(z) # ~Gl(z). Let
i = r;*(z) and j = ril(z). Since A, tl B, Z A, n B,, there exists a E &such that, say,
a E A, n B, and a $! A, n B,. (The symmetrical case is treated the same way.) We
show that P,,,, # P,,,. Because a E A, and a E B,, both i, j E S,. Since z = ri(i) =
qz(j), z E n@,) and z E ~@,4. Moreover, rank,,(saj(z) = rank,,csJz) because for
each y < z, y E ?ri(&) if and only if y E 7rz(S,), in accordance with our choice of z.
Because a E A, tl B, and a $ AI tl B,, i and j do not appear in the same set, Ta, X,,
or Y,. Let r = rank,,(s,,(z) = rank+,,(z).
It follows that r does not appear in
corresponding blocks of Pa,n, and Pa,flz.In other words, Pa,=, # Pa,,,, as asserted. 0
LEMMA

( 1

;,+; + k 5 (1 + &y+2J+k.
9

PROOF.
(By induction of i + j + k.) If i, j, or k = 0, then (:,-,i)
coefficient, and so,
5

2r+J+k

(1 +

,/$+J+k

(1 +

4r+2J+k.

If i, j, k > 0, then

(1 +

,J$r+2J+k--l

+ (1 +

+ (1 + A)L+2J+k--1

&)r+2J+k-2

(by hypothests)

= (1 + &)~+2~+k--2(1 f 2(1 + Jz)) = (1 + @+2J+!

is a binomial

Maintaining an Array and Computing lts Partial Sums

257

As described above, Theorem 3 implies Theorem 2, completing our lower bound

derivation.
2.3 SEMI-(U, r) SYSXEMS. Next we consider the effect of imposing the restriction
that integer matrices U and R are 0, 1 matrices. This discussion could be motivated
by a desire to maintain an array which stores values in a commutative semigroup.
Accordingly, we refer to this setting as the semigroup model of computation. We will
observe that this added restriction moderately degrades the efficiency with which an
array can be maintained.
First we describe a data structure which we call a semi-(u, r) system. A semi-(u, r)
system is a recursively constructed data structure for maintaining an array of length
at most (u + r) _ 1, such that the worst case complexity of an Update is u and the
worst case complexity of a Retrieve is r. If u = 1 or r = 1, then a semi-(u, r) system
is identical to a (u, r)-system. We use the notation ~
to refer to the complex of
line segments comprising a (possibly) translated (but never reflected) semi-(u, r)
system. Then for u > 1 and r > 1, a semi-(u, r) system is given by
V

(u-l,r)

(u,r-1)
I

]
I

I
N

where m = (Uu_l~'r ) - 1 a n d N ffi (u+r

u ) - 1. Arguing as for (u, r)-systems, we easily
conclude that semi-(u, r) systems have the indicated Update and Retrieve complexities
and correspond to U and R matrices with 0, 1 entries. Let N(u, r) denote the
maximum length of an array which can be maintained by Update and Retrieve
programs with worst case complexities u and r, respectively, and which fall within
the semigroup model framework. The semi-(u, r) systems show that 37(u, r) >_
(~u+ r) - 1. In [2] it is shown that bT(u, r) _< (u + ~) _ 1. Therefore semi-(u, r) systems are optimal w~thin this setting. We conjecture that the analogous statement
concerning (u, r)-systems is likewise true. Since N(k, k) ~ 4 k / 4 ~ , whereas N(k, k)
0.29(5.83)k/,cCk, we conclude that the semigroup model restriction is significant
from the standpoint of worst case complexity.
Next we show that semi-(k, k) systems are near optimal within the semigroup
framework, from the standpoint of average complexity. Our consideration of semi(k, k) systems shows that log4N + O(log log N) is an achievable upper bound for the
worst case complexity (and hence the average complexity) of maintaining an array
of length N within the semigroup model.
THEOREM 4. Let Update(j, x) and Retrieve(j) be programs described by the
semigroup model which solve the array maintenance problem for an array of length N.
Then the average cost of executing these programs is at least log4N - O(1).
SKETCH OF PROOF. If we impose a further condition that A, fq B, = ~ in the
hypothesis of Theorem 3, then the conclusion can be strengthened to read
N

(IA, I +

IB, J)-> 2 N l o g 4 N - O(N).

(11)

This follows because the sets Xa (see proof of Theorem 3) are always empty, and
therefore the partiuons Pa,, are in actuality partitions into two blocks instead of three
blocks. In place of (10), the stronger equality,
2E,~o,(I7",1+1Y,I)> N!
(12)
results. Inequahty (11) follows from (12).

258

M I C H A E L L. F R E D M A N

Now define A, = {j]a,j = 1), 1 <_ i <_ N - 1, where R = (ast), and define B~ =
{llfl,j+a -- 1}, 1 _<j_< N - 1, where U = (fist). I f R U = ,IN, then

IA, n B~I --

{0

if i<_j,
if i > j .

In particular, the sets A1 . . . . . AN-I, nl . . . . . BN-1 satisfy the hypothesis of Theorem

3 along with the added condition A~ tq B~ = O. Hence (11) follows, which proves
Theorem 4. []
3. Information-Theoretic Complexity
In this section we study the complexity of the array maintenance problem for arrays
having values in Z2 (the integers modulo 2) but broaden the class of algorithms we
are wilhng to consider. Complexity is measured in terms of the number of memory
accesses and memory changes performed on a random access memory consisting
of addressable l-bit words. The author uses this measure in [3] to analyze a different problem. We refer to this complexity measure as the information-theoretic
complexity.
Since we are considering algorithms which maintain arrays having values in Z2,
we always assume that the value of x in Update(j, x) is 1 and use the simpler
notation, Update(j). We assume that N is fixed and that there are 2N programs: N
programs Update(j), 1 __j _< N, and N programs Retrieve(j), 1 _<j _< N. Considering
the nature of our measure, we define a program to be a binary tree whose internal
nodes are labeled with memory addresses and whose leaves each contain a (possibly
empty) list of assignment statements of the form b, := b, + 1 mod 2 (the contents of
the bit having address i is inverted) as well as an optional label reflecting the output
being reported, if any. (In our case these labels will be in (0, 1}.) When a program
is executed, control begins at the root. When control is located at the internal node
n having label i, the bit whose address is i is probed, and control passes to the right
son of n if the contents of that bit is 0; otherwise control passes to the left son of n.
When control reaches a leaf, the list of assignment statements in the leaf is executed,
:hereby inflicting side effects. The computational cost of an execution is expressed in
terms of the number of internal nodes along the path defined by the execution, which
represents the number of information gathering probes to memory, and the number
of assignment statements in the leaf at the end of the path, that is, the number of
memory changes. Information-theoretic complexity is a natural extension of decision
tree complexity [1], appropriate within an on-line setting.
The programs implied by a (u, r)-system can be implemented within this model so
that the total cost of an Update does not exceed u, and the cost of a Retrieve does
not exceed r. Thus, update and retrieve programs having worst case complexity
O(logN) exist. The main result of this section is that one cannot do better than
O(log N/log log N).
THEOREM 5. Given a collection of 2N programs for maintaining an array of length
N having values in Z2 (Nprograms Update(j), 1 <_j <_N, and Nprograms Retrieve(j),
1 <_ j <_ N), there exists a c > 0 (independent of N ) such that for each m > 1, a
sequence of m consecuttve executions of these programs extsts, which has a total
computational cost >_ cm log2N/logzlog2N.
The proof of Theorem 5 uses the following lemma.
LEMMA 4. Let Vl, v2. . . . . VN be binary vectors having Hamming weight at most k
(each vector has at most k ones). Let T be a binary search tree which dtstingutshes

Maintaining an Array and Computing Its Partial Sums

259

among these N vectors (no two of these vectors, when provided as input, define the same
path through T) and has hetght h. Then

(Observe that when k = oo, this gives the classical bound, 2 h > S . )
PROOF. We show how to construct a one-to-one mapping from the vectors v~into
vectors v,* having Hamming weight at most k and dimension h. Since the possible
number of such vectors v,* is (~) + . . . + (h), Lemma 4 is an immediate consequence.
Let nl, n2. . . . , nt, I <_ h, be the nodes on the path through T defined by input v,, in
descending order beginning from the root. Let cj be the vector component number
examined by node nk, 1 _<j _< /. We define v,* to be the binary vector of dimension
h, having a one in itsjth component if and only if v, has a one in its c~th component.
To see that v~* # v,*, when t # i', let nj be the node at which the paths defined by v,
and v,, diverge. Then v, and v,, differ in the cjth component, and so v~* and v,*, differ
in the j t h component. []
PROOF or THEOREM 5. To simplify the exposition, we assume initially that the
memory configuration Co with all bits set to zero represents the situation where the
maintained array A(i) has a zero stored in each location. We also identify each
memory configuration C with the binary vector whosejth component is given by the
contents, in configuration C, of the bit with addressj. Let Cj, 1 _<j _< N, denote the
memory configuration resulting from an execution of the program Update(j), starting
from Co. We construct, in the manner described below, a binary search tree which
distinguishes among C1, C2. . . . . CN.
Let TN be a mimmal height ([log2N]) binary tree with N leaves numbered (left-toright) from 1 to N. Also label the internal nodes of TN from 1 to N - 1 in symmetric
(left-to-right) order. Let wj, 1 _<j _< N, denote the binary vector of dimension N - 1
with zeros in the firstj - 1 components and ones in the remaining N - j components.
The tree TN can be regarded as a search tree which identifies the vectors wj as
follows. An internal node with label i tests the ith component. Control passes to the
left son if the value of the ith component is 1, otherwise control passes to the right
son. Now let T' be the following algorithm which identifies the vectors C1, C ~ , . . . ,
CN. The algorithm T' is obtained by modifying the algorithm T2vso that when control
is centered at an internal node with label i, the branching choice is determined by the
output produced by the program Retrieve(t) when it is executed on input Cj. (Observe
that the ith component of wj has the same value as the output returned by Retrieve(i)
executed from configuration Cj. Therefore the algorithm T' identifies the Cj vectors,
since TN identifies the wj vectors.) The algorithm T' can be expanded into a binary
search tree which directly examines the components of the C~ vectors, namely, by
utilizing the internal node portions of the binary trees corresponding to the Retrieve(t)
programs. Let T denote this expanded search tree. If the heights of the Retrieve(t)
trees do not exceed r, then the height h of T does not exceed r[log2N]. More strictly,
we can choose r to be the maximum length of those paths defined by the input
vectors Cj, 1 <_ j <_ N, through each of the Retrieve(i) program trees. Now let k
denote the maximum Hamming weight of the C s vectors, 1 _<j _< N. (Observe that
the Hamming weight of Cj is a lower bound on the cost of executing Update(j) from
configuration Co.) By Lemma 4 we have that
N_(h)+

... +(h)_<(k+

l ) h k < - 2 k r k ( l + l o g 2 N ) k,

260

MICHAEL L. FREDMAN

from which we conclude that either

k>
_

log2N
2 log21o~N '

r _>

log2N
2 log21og2N"

(13)

or

(14)

If (13) holds, then for somej the program Update(j) takes time _>log2N/(2 log21og2N)
when executed starting from Co. If (14) holds, then for some pair (j,/) the program
sequence Update(j) Retrieve(/) takes time >_log2N/(21og21og2N) when executed
starting from Co. The reader should be able to see that the above argument readily
generalizes and allows us to conclude the following. Given an arbitrary memory
configuration C corresponding to a storage state of the maintained array A(I), there
exists a pair (j, 0, depending on C, such that the program sequence Update(j)
Retrieve(/) takes time >_(log2N)/(2 log21og2N) to execute starting from configuration
C. Theorem 5 now follows by induction on m. []
We conjecture that the lower bound in Theorem 5 can be strengthened to read
m log~.83N for sufficiently large m. Some evidence supporting this conjecture is the
following, which we state without proof. If the N programs Update(j), 1 _<j _ N, are
implementations within the information-theoretic model of programs having the
form in (3), then from the standpoint of choosing Retrieve programs having minimal
complexity, one cannot do better than choosing Retrieve programs having the form
in (3). Moreover, the dual statement, obtained by interchanging the occurrences of
Update and Retrieve in the above statement, is likewise true.
4. A n Open Question

The computational problem addressed in this paper is generalized by the following

d-dimensional variation. An array A(il . . . . . id), 1 <-- il . . . . . ia <-- N, is to be represented for the purpose of facilitating efficient implementation of the on-line tasks
(a) A(il . . . . . ia) := A(il . . . . . ia) + x, and (b) compute~jl_<, 1 . . . ~j~_,~A(jj . . . . . ja).
Both the algebraic and information-theoretic models of computation can be extended
to accommodate this problem. Relative to the semigroup model, cOogn) a upper and
lower bounds are derived in [4]. An eqmvalent lower bound, however, has not been
derived relative to the general algebraic model, which allows for subtraction.
REFERENCES
1 Arlo, A V , HOPCROFT, J E , AND ULLMAN, J D The Design and Analysts of Computer Algomhms
Addison-Wesley, Reading, Mass., 1974
2 BURKHA~D, W A , FRED~tAN, M L , AND KLEITMAN, D J Inherent complextty trade-offs for range
query problems Theor. Comput. Sc~. (to appear)
3. FaEDMAN, M L Observations on the complexity of generating quasi-Gray codes SIAM J Comput.
7, 2 (1978), 134-146
4 FgEDMAN, M L A lower bound on the complexity of orthogonal range quenes. J A CM 28, 4 (Oct
1981), 696-705.
RECEIVED AUGUST 1978, REVISED MAY 1980, ACCEPTED JUNE 1980

Journal of the Assoclatlonfor ComputingMachinery.Vol 29. No 1.January 1982