Preliminaries

Personal introduction.
Finding me:

Jim Bernard
Meyer Hall 447 (moving to Timberline #2
Room 17 during the semester)
Email:
jbernard@mines.edu
Office Phone:
303-384-2180
Office hours:
TR 1112, 34; W 34; or by appointment
If you cant make it to one of my office hours, please let me know, and Ill be
happy to set up an alternative time to meet. Also, please feel free to email me
at any time.
Office:

Our textbook:

Robert DeHoff, Thermodynamics in

materials science, 2nd ed. (CRC Press,
Boca Raton, FL, 2006) ISBN 9780849340659.

Note email of syllabus and lecture notes.

Grade weightssee the syllabus for details:
One-hour exams
Homework
Final exam

15% each
45%
25%

Note roughly equal weights given to HW and exams in overall average.

Homework:
Incremental assignments:
May assign one or more problems in any class, with no initially
specified due date.
Hint: You can start working on each problem as soon as it is assigned.
Close a problem bundle when it has accumulated a reasonable number
of problems. The bundle then constitutes an assignment.
1

Due date for the assignment specified at closure (usually one week
after closure).
Summary of problems in each assignment will be sent by email shortly
after closure.
Hint: You dont have to wait for closure or the summary to start
work on the problems. Get started early!
Help each other, but its crucial to develop and write your own solutions
if you are to learn. You cannot learn thermo by observationyou must
do the work yourself.
Verbal explanations are a critical component of a written solution, so I
will allocate approximately 20% of the homework credit to the extent and
quality of explanations. See the syllabus for details.
Brush up on math.

Outline, scope
Overview of system, equilibrium, measurement, variables.
Entropy and the tendency toward equilibrium.
Thermodynamic states, processes, and laws.
Thermodynamic relations.
Phase transitions, phase diagrams.
Multicomponent systems and solutions.
Statistics of open and closed systems.

Chapter 1

Introduction
1.1

Thermodynamics and statistical mechanics

Our goal is to understand the macroscopic properties of materials using methods

whose applicability is extremely broad because they follow from the statistical
behavior of assemblies of many particles. Those particles may be atoms or molecules,
or sometimes more exotic things, but the important thing is that there be lots of
them so that statistical fluctuations in the macroscopic behavior are small. These
microscopic particles, the constituents of a system, act in some sense in concert
to determine its average macroscopic behavior, the behavior that can actually be
observed and quantified by measurements in the lab.
In practice, even quite small systems on the scale of our everyday lives have
numbers of particles of the order of 1020 or more and can be considered macroscopic.
This would be an impossibly large number of particles to observe individually in the
lab or to analyze theoretically using Newtons equations of classical mechanics or the
Schr
odinger equation of quantum mechanics. The reason we can make progress at all
is that under identical conditions a sufficiently large system in equilibrium exhibits
stable macroscopic behavior that does not depend on the microscopic details of the
collective state of the constituent particles. Furthermore, that equilibrium behavior
is repeatable and independent of the initial state of the system, or any other part of
its prior history. It only depends on the external conditions imposed on the system
by the outside world, the experimenter.
Thus, macroscopic properties such as temperature, pressure, volume, entropy,
etc., exhibit the same values when measured repeatedly in systems in equilibrium
that have identical composition and are subject to identical macroscopic constraints.
This occurs in spite of the fact that the microscopic constituents undergo extremely
rapid changes in position and momentum or quantum state. The reason for this
stable macroscopic behavior, as we will see, is statistical.
A not-so-macroscopic analogy illustrates the point. Consider a large crowd,
mere thousands, nowhere near the 1010 mentioned above, in a convention center.
The people are constantly milling about, fidgeting, blowing their noses, practicing
speeches, chatting with each other or with folks elsewhere via cell phone, going
to the bathroom, grabbing a bite to eat, etc. These are the microscopic details
of the collective state of the constituents, and it would be possible, though a big
job, to keep track of all of their activities and interactions and the contents of all
of their conversations on an individual level. Now, all of this activity generates a
lot of sound, and from the outside looking in, its just noise. The sound level of
that noise is a macroscopic property, one that can be measured and used as partial

CHAPTER 1. INTRODUCTION

characterization of the macroscopic state of the system of conventioneers. Other

macroscopic properties include their number N , the volume of the convention hall
V , and their density N/V .
We can glean some of the key features of the relationship between the details of
microscopic states and the properties of macroscopic states from this analogy:
The microscopic activities of individual constituents do not determine the
macroscopic properties directly; rather, they contribute to those properties
in an average, or statistical, sense. If we could develop a model for the sound
produced by a typical conventioneer, and perhaps its variation with time, we
could use statistical methods to predict the macroscopic sound level produce
by the whole convention. We could even predict how large would be typical
fluctuations or changes in the sound level. A similar approach could be used
to predict the number of people in the convention center at any given time,
and fluctuations in that number.
There are inevitably some fluctuations in most macroscopic properties with
time. A cheer or a round of applause can raise the sound level substantially,
for example, those tend to be outside the bounds of the typical variations
mentioned above. The average size of fluctuations relative to the mean value
tends to be larger for small systems than for large ones, becoming huge for a
single person/constituent. This is a qualitative rephrasing of the content of
a theorem from probability theory called the law of large numbers. One of
the key consequences of having a huge number of particles in a macroscopic
material system is the reduction in fluctuations because N is so large. This
is the fundamental reason why the macroscopic properties of a system in
equilibrium are so stable and repeatable.
The macroscopic properties may be all we care to know about a system
who really cares exactly what every conventioneer is saying at all times. Or
we may simply not be capable of handling the microscopic details on the
single-constituent level. This is typically the case with macroscopic material
systems.
There are two distinct but complementary approaches to understanding the
macroscopic properties of material systems: thermodynamics and statistical mechanics. Thermodynamics is a study of general relationships among equilibrium
macroscopic properties. It makes no reference whatsoever to the microscopic structure, dynamics, or even the existence of the constituent particles of a system. Since
the general relationships of thermodynamics are consequences of fundamental conservation laws and the statistical behavior of aggregates of large numbers of particles, it
is applicable to almost any sufficiently large system, including solids, liquids, gases,
stars, black holes, refrigerators, engines, magnets, capacitors, superconductors, semiconductors, etc. However, since it only relates macroscopic properties to each other,
it is capable of predicting actual values of macroscopic properties only when values
are already known for a large-enough set of material-specific parameters under the
conditions of interest, so that the actual functional relationships among the variables
can be determined. For example, if you know from thermodynamics that there is
a functional relationship between macroscopic properties A, B, C, and D that is
given by D = f (A, B, C), then to find a value for D, you must already know values
for A, B, and C. Thermodynamics gives you the function f , not the values of the
variables. Thus, external input is required to make use of the general relationships
of thermodynamics to predict numerical values in applications. That input can come
from experimental measurements or it can come from theoretical analysis of suitable
physical models.

CHAPTER 1. INTRODUCTION

Statistical mechanics is a collection of tools for statistical analysis of atomistic

physical models of systems to obtain predictions of their macroscopic properties.
While its predictive power is greatest for systems having very large numbers of
particles, its statistical methods can predict not only the mean values of macroscopic
properties, but also the magnitudes of typical fluctuations in those properties. This
makes it potentially useful even for smaller systems in which fluctuations may not
be negligible. The mean values provided by statistical-mechanical models can often
be used as input values in thermodynamic calculations in lieu of experimental data.
Development of a successful atomistic model also enhances ones understanding of the
behavior of a material system beyond what is achieved by experimental measurement
of properties, because it helps one to understand why things behave the way they
do, not just what they do. Perhaps most intellectually appealing, though, is the
fact that statistical mechanics provides a foundation for thermodynamics, in that
it gives clear meaning to the concept of entropy, which plays a truly starring role
in thermodynamics, yet is rather mysterious in that context. This gives us the key
to understanding why a macroscopic system gravitates toward a stable equilibrium
state that is independent of the details of its history.
While our principal topics are thermodynamics and its applications in materials
science, we will also develop and use some of the tools of statistical mechanics when it
provides useful input or understanding of functional relationships in specific systems,
as well as to elucidate the foundations of thermodynamics.

1.2

Systems, boundaries, and external constraints

We will nearly always direct our attention to some macroscopic subset of the entire
universe and refer to it as the system of interest, with everything surrounding
it being the environment. The thermodynamic treatment of the system of interest
will naturally be much more detailed than that of the environment. In making the
distinction, we must establish a real or imagined boundary between the two and specify
some constraints, sometimes called boundary conditions, on the system that are
generally enforced at the boundary. The environment, including the physical boundary
if any, serves to implement the external constraints, and these will frequently, but
not necessarily, correspond to actual experimental conditions. For example, chemical
reactions may be performed with the system in a fixed-temperature environment
and/or a fixed-pressure environment, or they may be performed with fixed energy
and fixed volume. We define specific terminology for a few of the most commonly
useful constraints:
Isolated system: One that is unable to exchange energy, volume, or particles with
its environment. This corresponds to having the system in a rigid, closed box that
is well insulated to prevent heat flow. The total energy is then fixed.
Closed system: One that can exchange thermal energy (heat) with its environment,
but that is otherwise isolated. This permits the energy to vary, but not the volume,
and particles cannot flow in or out.
Open system: One that can exchange both energy and particles with its environment.
It will often be useful to identify more than one system of interest, generally
referred to as subsystems of a combined supersystem. The constraints applied at
the boundaries between the subsystems may then be specified independently of
each other and of the boundary conditions between the combined system and its
environment.

CHAPTER 1. INTRODUCTION

1.3

Equilibrium

We will confine our attention to macroscopic systems that are in states of equilibrium. For an isolated system, the equilibrium state has two key characteristics:
its macroscopic properties are (1) independent of time and (2) independent of the
history, the initial conditions, of the system. For example, if we initialize a sealed,
thermally insulated container of gas with the gas confined to half the container by
a partition, after removing the partition we must wait long enough for the density
to become uniform and mass currents to damp out before we can make meaningful
measurements of the temperature and pressure. Furthermore, no matter how we
initialize the system, so long as its total energy, total volume, and the amount and
composition of the gas are the same, the measured temperature and pressure in
equilibrium will be the same.
Note that there are plenty of examples of materials that appear to be temporally
stable, yet whose properties have a significant dependence on preparation history.
Ordinary glass is perhaps the most familiar exampleits constituents are stable
in the crystalline state, but the microscopic structure of glass is amorphous and
its macroscopic properties depend, for example, on the cooling rate when it was
formed. Though glass is not in an equilibrium state, the mobility of its atoms is so
tiny at room temperature that the time required to reach the true equilibrium state
is effectively infinite.
Its useful, even at this early stage, to contemplate the reason why isolated
macroscopic systems tend toward internal equilibrium. Consider the isolated container
of gas again, with the partition initially confining the contents to half the full volume.
If we open a small hole in the partition, we expect the gas to flow from the initially
filled side to the initially empty side until the density becomes uniform throughout.
But why does that occur? It is, quite simply, driven by probability, for the molecules
undergo rapid, random motion, and soon after the hole is opened, the fact that there
are larger numbers of molecules on the filled side than on the empty side implies
that there is a higher probability for molecules to pass through the hole from the
filled side than from the empty side, producing a net flow that tends to equalize the
concentrations of molecules on the two sides. The initial imbalance between the flows
in the two directions would lead to overshoot, and thus to oscillation, but friction
inevitably damps out the oscillations, leading eventually to a macroscopically static
uniform distribution of molecules. There are, of course, ongoing random fluctuations
in the local molecular densities, but these are quite small and rapidly corrected
in a large system. We are led to suspect then that the equilibrium state is the
most-probable macroscopic state.

1.4

Measurement

Measurements of macroscopic properties are made on a spatial scale that is quite

coarse compared to the sizes of the molecular constituents. A fluid-pressure gauge,
for example, determines the average force per unit area over an area that is very large
compared to molecular sizes, even though it is generally much smaller than the total
surface area of the containing vessel. In a similar way, macroscopic measurements
yield averages over times that are long compared to the time scale of the motion of the
particles of the system. A pressure measurement, for example, spans many collisions
of fluid molecules with the area sampled by the pressure gauge, generating a time
average, as well as a spatial average. As a consequence, measurements yield mean
values taken over a large number of microscopic configurations of the system. Those
mean values are the macroscopic variables of thermodynamics, and the strategy of

CHAPTER 1. INTRODUCTION

statistical mechanics is to evaluate those averages using models of the microscopic

configurations and their probability distributions.

1.5

Extensive and intensive variables

Some macroscopic properties, such as the volume V and the number of particles
N are obviously proportional to the size, or extent, of the systemindeed, they
define it. These are examples of extensive variables. Less obvious, perhaps, are
the entropy S, the internal energy E, and the various thermodynamic potentials,
such as the Helmholtz free energy F , the enthalpy H, and the Gibbs free energy G.
In contrast, intensive variables do not scale with the size of the system, becoming
asymptotically constant instead. These include things like temperature T , pressure
P , and chemical potential .
Since macroscopic size is an important requisite for the success of ordinary
thermodynamics and statistical mechanics, it is sometimes important to define it
carefully through the concept of the thermodynamic limit. This is the limit as
the extensive variables characterizing the size of a system are scaled up consistently.
For example, if a closed isolated system has N0 particles in volume V0 , we scale both
of them via a linear factor s, so that scaled versions of the system have
N = s3 N0

and V = s3 V0 .

(1.1)

The thermodynamic limit is then taken by letting s . The distinction between

extensive and intensive variables is then that in the thermodynamic limit extensive
variables scale like s3 , while intensive variables become independent of s. The
relationships of the equilibrium thermodynamics of macroscopic systems are strictly
accurate in the thermodynamic limit, but may require corrections for small systems.
As well, fluctuations about the mean values of thermodynamic variables become
vanishingly small in the thermodynamic limit, but may not be for smaller systems.
Intensive variables commonly arise in several ways, from ratios of pairs of extensive
variables or from partial derivatives of extensive variables with respect to other
extensive variables or of intensive variables with respect to other intensive variables.
For example, the entropy per particle s = S/N and energy per unit volume = E/V
are intensive, as are the partial derivatives






E
E
P

=T,
= P , and
= P ,
(1.2)
S V,N
V S,N
T V,N
T
where P is the constant-pressure volume-thermal-expansion coefficient and T is
the isothermal compressibility. In contrast, the heat capacity at constant pressure


E
CP =
(1.3)
T P,N
is an extensive quantity, since E is extensive and T is intensive.
While we naturally think of intensive variables as having uniform values throughout a system, under certain circumstances they can acquire position dependence. This
happens, for example, in a gas in an external gravitational field, which confers height
dependence on both the density and pressure; in a two-phase mixture, such as ice and
water at 0 C, in which the particle density may be different in one phase than in the
other; and in a solid having a spatially inhomogeneous magnetic susceptibility, which
leads to spatial dependence in the magnetic-moment density in an external magnetic
field. It also features in nonequilibrium extensions of thermodynamics, where one

CHAPTER 1. INTRODUCTION

must deal with transport under nonequilibrium conditions. A good example is a

metal bar whose ends are maintained at different temperatures. The temperature
difference drives flows of heat and electrons through the bar, leading to steady-state
variations in the temperature and the electron chemical potential along the bar.
In such cases, the local values of position-dependent intensive variables must be
defined within macroscopic subregions of the full system, so that each subregion can
be treated as a macroscopic system in its own right.

1.6

Strategy

Our first task is to establish the statistical foundations of thermodynamics, so that

we can develop an intuitive understanding of the nature of equilibrium and the
central role that entropy plays as a quantitative tool that one can use to understand
macroscopic properties. Then we will build the basics of thermodynamics on that
foundation, learning how various thermodynamic quantities are defined, how they
are related to each other, and how to manipulate them efficiently. In all of this early
work, we will consider only simple systems, by which I mean three-dimensional
systems containing only a single type of particle (atom or molecule) in a single phase,
and able to interact with other simple systems or the environment only through
mechanical linkages and heat flow. This reduces the number of variables we must
juggle and makes it easier to see and understand how and why the formalism works
without getting bogged down in the details specific to more-complicated systems.
However, dont be lulled into thinking that every system you will need to deal with
is like the simple ones we think about while developing the formalism. Most interesting
systems are much richer, having multiple components (types of constituent particles),
multiple phases coexisting, nonmechanical interactions, such as electromagnetic
interactions and chemical reactions, less than three dimensions (surfaces or lines), etc.
We will address some of these, particularly the case of multiple components, later in
the semester. Generalizations to lower dimensions or to other types of interactions
follow the same general patterns established for simple systems. All one need do is
learn the jargon relevant to some unfamiliar phenomenon and learn which variables
pertinent to that phenomenon play roles analogous to those used for simple systems,
then simply reuse the old familiar formulas with new variable names.

Chapter 2

Entropy and thermal

equilibrium
2.1

Introduction

As we saw in considering of flow of a gas through a small hole in a partition

separating two parts of a container in which the gas density differs between the
sides, the process of equilibration is driven by probability. In that example, the net
flow exists because the density difference leads to a greater probability for molecules
to pass through the hole in one direction than the other. This gives us a hint that
the equilibrium macroscopic state, in which there is no statistical preference for
flow in either direction because the density is uniform, is somehow a more probable
state than states with nonuniform density. It turns out that, quite generally in
isolated systems, equilibrium macroscopic states are maximal-probability states
when compared to alternative macroscopic states. A particularly convenient measure
for comparing the relative likelihoods of different macroscopic states is the entropy,
which is a property of probability distributions that can be directly related to the
thermodynamic quantity of the same name. This, together with the understanding
of equilibrium as a statistical effect, provides the foundation for the second law of
thermodynamics, and thereby for all of thermodynamics.
In order to develop the necessary machinery for determining the probabilities
of occurrence of macroscopic states, we begin by thinking a bit about the general
notion of probability and developing some techniques for calculating probabilities.
We will then apply these to a couple of models of phenomena that occur in the solid
state, using them to illustrate the statistical prediction of equilibrium properties.

2.2

Probability

Probability is simply the quantification of the intuitive notion of likelihood. One

can assign numerical probabilities to propositions, such as, It will rain today. or,
The Rockies will win the World Series. Our interest lies principally in propositions
declaring the occurrence of events related to the states of large systems, like, The
systems come to equilibrium in a macroscopic state in which their energies are E1
and E2 = 2E1 . We will compare, for example, the probabilities of events in which
the total energy of two systems is shared in different ways, in order to determine
which partitioning of the energy is the one actually observed in equilibrium.
The value of the probability of some proposition x is a real number P (x) such

CHAPTER 2. ENTROPY AND THERMAL EQUILIBRIUM

10

that
0 P (x) 1 ,

(2.1)

with zero indicating certain falsehood, and one indicating certain truth. Thus, one can
view reasoning with probabilities as a generalization of the usual logic of propositions
that can only be either true or false. Probabilities satisfy two properties that will
prove to be especially useful in manipulations:
Sum rule for mutually exclusive events: If two events are mutually exclusive,
so that at most one of them can occur, then the probability of either occurring is
the sum of the probabilities of each. An example is the roll of a dieexactly one
number results, each being mutually exclusive of the others. The probability of
rolling a two or a five is the sum P (2) + P (5).
Product rule for independent events: If two events are independent, that is,
the probability of either is unaffected by whether the other occurs or not, the
probability of both occurring together, their joint probability, is the product of the
probabilities of each. An example is the flipping of two coins consecutivelyneither
result influences the other. The probability of obtaining heads on both is the
product P (head on first) P (head on second).
Both of these properties are trivially extended to multiple events.
When dealing with probabilities of the results of experiments, such as the occurrence of a particular measured value x of some property, it is often convenient
to interpret probability in terms of frequency of occurrence. Imagine performing a
long sequence of measurements of some physical quantity, always under the same
conditions. For each possible result x, determine the ratio
Number of occurrences of x
,
Total number of measurements

(2.2)

which is the average relative frequency of occurrence of x. If that ratio converges to

a well-defined value in the limit of an infinite number of measurements, that relative
frequency of occurrence is taken to be the probability of occurrence P (x). This is
especially useful if one can generate or find enough data to estimate the relative
frequencies of events. Even when that is not possible, it still can be helpful to keep
in mind that probability can be related to relative frequency.

2.2.1

Probability distributions

In any particular context, such as the performance of a particular type of measurement

on a particular system, one can generally identify the complete set of events that
could possibly occur.1 For example, one could enumerate all the possible measured
values for the total energy of some finite system, like a crystal of silicon. It is
especially convenient, given the rule for probabilities of mutually exclusive events, to
deal with sets of events that are mutually exclusive. That is the case for the energy
measurement just mentioned, since the energy cannot have two different values at
the same time. The set of probabilities for a complete set of mutually exclusive
events is called a probability distribution. Since all possible events are included,
one of them must occur; so the probability of any of them occurring is exactly one,
a statement of certainty. And since they are mutually exclusive, the probability of
1 In

what follows, we also assume the set of possible values is finite. While it may seem that the
allowed energies, for example, constitute an infinite set, for a confined system, they are quantized.
As well, extremely large values can be ruled out (e.g., greater than the total energy of the universe),
so any realistic enumeration of the set is finite.

CHAPTER 2. ENTROPY AND THERMAL EQUILIBRIUM

any of them occurring is the sum of their individual probabilities. Thus,

X
P (x) = 1 .

11

(2.3)

This is the normalization condition for any probability distribution.

It is knowledge of the probability distribution for macroscopic states of a system,
or an unnormalized counterpart, that allows us to assess the relative likelihoods
of the outcomes of macroscopic measurements. This will be a crucial tool in our
attempt to understand equilibrium as an inherently statistical phenomenon.
HW problem 1.1: Three blind dice. Throw three fair 6-sided dice. What is the
probability that at least one will show 6? Do this calculation in two ways:
(a) Make use of the probability that a given die will show 6.
(b) Make use of the probability that all three dice will not show 6.
Your answers should agree.
HW problem 1.2: Combinatorial tricks and treats. The determination of the
probability of an event can often be reduced to a counting problem. For example, in
rolling a six-sided die, we have six possible, mutually exclusive, elementary outcomes.
If the die is fair, we assign equal probabilities of 1/6 to each of these possibilities,
which might be characterized by simple propositions like The die shows 4. We
might want to know, however, the probabilities of more complicated propositions,
such as The roll yields a number n greater than 3. The probabilities of such
compound events can be obtained from those of the mutually exclusive elementary
events by using the sum rule for mutually exclusive events:
P (n > 3) = P (4) + P (5) + P (6) =

1 1 1
3
1
+ + = = .
6 6 6
6
2

Equivalently, we could simply count the number of elementary events satisfying the
proposition (3) and divide by the total number of elementary events (6) to get the
same result.
This sort of direct enumeration is easy to do for very small systems like a few dice
or coins, but for large systems that is often impractical, if not outright impossible.
However, combinatorial methods can often provide the counts we need, regardless
of the number of constituents or the number of elementary possibilities. We will
review several handy combinatorial results here. As usual, the important part of
your solution is the logic you use to answer the question, so be sure to explain your
reasoning clearly and completely in each case.
(a) Find the number of distinct sequences (order does matter) of n integers in the
range 1, . . . , N . Repetition of values is permitted, and N has no particular relation
to n, but both are to be considered to have fixed values. Pictorially, we can imagine
n boxes, distinguishable by labels bi assigned to them, into which we place numbers
selected from a pool of arbitrarily many copies of the first N integers:
Pool

Sequence
?

b1

b2

b3

bn

1 2 3 N
1 2 3 N
..
.

(b) Find the number of distinct sequences of n integers in the range 1, . . . , N , but
dont allow repetition within a sequenceall elements of each sequence are distinct

CHAPTER 2. ENTROPY AND THERMAL EQUILIBRIUM

12

(thus, we must assume N n). In this case, we can imagine selecting the numbers
from a pool consisting of just a single copy of each of the first N integers:
Sequence
Pool
?

b1

b2

b3

bn

1 2 3 N

What happens when n = N ?

(c) Find the number of distinct sets (order doesnt matter) of n distinct integers
from the first N integers. (Again assume N n.)
(d) How many ways can N uniquely numbered balls be divided into 3 (unordered)
groups containing n1 , n2 , and n3 = N (n1 + n2 ) balls in each? The numbers n1 ,
n2 , and n3 are given, fixed numbers, and the groups are distinguishable, say as left,
center, and right.
(e) How many ways can N indistinguishable objects be divided into M distinguishable
sets, each containing an arbitrary number of objects, ranging from 0 to N ?
(f ) Find the value of the sum
S=

N
X

N!
.
n!
(N
n)!
n=0

[Hint: Find an equivalent combinatorial problem thats easy to answer.]

HW set 1 closed, Tue. 8/25/2015, due Tue. 9/1/2015.

2.3

Probability distributions for macroscopic properties

The two specialized rules of probability we cited above allow us to break down
the probabilities of macroscopically characterized states in terms of those of morefundamental states at the microscopic, or atomic level, ultimately reducing their
determination to a counting problem, that of counting the microscopic states that
have a given set of macroscopic properties. To do that, we must first develop a
scheme for identifying and characterizing all of the microscopic states of a system.
In general, microscopic many-particle states can be quite complicated, not just
because there are of order 1023 particles, but also because they tend to interact with
each other in ways that preclude exact analytic treatment in either classical or quantum mechanics, even when there are as few as three particles. We will sidestep that
issue, homing in on the statistical aspects of the problem by making the simplifying
assumption that the particles are noninteracting. For a gas of atoms or molecules,
this would define the ideal gas, but we will not restrict ourselves to any specific type
of system yet. While models based on this simplifying assumption can give only
approximate predictions of the macroscopic behavior of real systems, our primary
goal at this point is to elucidate the statistical foundations of thermodynamics,
which is most efficiently accomplished without the extra burden of dealing with
interparticle interactions. In addition, these simplified models can serve as starting
points for improvements that include interactions in an approximate way.

CHAPTER 2. ENTROPY AND THERMAL EQUILIBRIUM

13

We suppose then that systems are composed of some constituents, perhaps

particles, atoms, molecules, excitations of normal modes2 of vibration of atoms in
a crystal, or whatever. The notion of constituents carries with it the implication
that each is in some sense a separate entity that exists independently of other
constituents, like atoms in a molecule or molecules in a gas. Each constituent will
generally be supposed to exist in any one of a number of possible constituent
states, independently of the states of the other constituentsthis is the assumption
of noninteracting constituents mentioned above. The constituent states will often
be called single-particle states, even in general discussions and when considering
specific systems in which the constituents arent necessarily particles. Examples of
constituent states are the different spin states of electrons attached to atoms in a
crystal and different rotational quantum states of diatomic molecules in a gas. In a
classical treatment of mechanical systems such as gases and liquids, the states of
individual particles are generally characterized by their positions and momenta.

2.3.1

Building microstates from constituent states

In a simple model system in which the constituents do not interact with each other,
the states of the constituents are independenteach one of them can occupy any
one of its possible states without regard for the states of other constituents. This
means the state of the system as a whole, viewed on a microscopic scale, the singleconstituent level, changes if even one of the constituents changes its state. That is,
the microscopic state, or microstate, of the complete system depends on the states
of all of its constituents, so that each microstate is characterized, or defined, by the
complete set of single-particle states of its constituents.
To distinguish the different states of a constituent and the different constituents
themselves (if they are distinguishable, which we assume here for simplicity), we use
a constituent-state identifier consisting of a state label, together with a constituent
label as a subscript:
sc = stateconstituent .
(2.4)
Here s might stand for some quantum number or set of quantum numbers characterizing the quantum state of a constituent, such as the quantum numbers n, l, and m
for a hydrogen atom, and c just indicates to which constituent the identifier applies.
Then a microstate can be characterized by the complete set of its constituent-state
identifiers:
microstate = {i1 , j2 , k3 , . . .} .
(2.5)
We could also express this more concisely as an ordered sequence of constituent-state
labels, just the s parts of (2.4):
microstate = (i, j, k, . . . ) ,

(2.6)

where the nth entry gives the state of the nth constituent. In this representation
the constituent specification is implicit in the order of the state identifiers, whereas
in the set representation (2.5), the order of the entries is insignificant.
Example 1. As a simple example, consider a collection of numbered coins: the
state label s for any particular coin is either H (heads) or T (tails), and the
constituent label c is the number on the coin. Thus,
sc = H31

(2.7)

2 A normal mode is a wavelike state of vibration of a crystal, in which all atoms vibrate with the
same frequency. In a perfectly harmonic crystal, these behave like independent quantum harmonic
oscillators, whose quantized excitations, called phonons, act like noninteracting particles.

CHAPTER 2. ENTROPY AND THERMAL EQUILIBRIUM

14

indicates that coin 31 is heads up. If there were three coins altogether, there would
be eight microstates
{H1 , H2 , H3 } , {H1 , H2 , T3 } , {H1 , T2 , H3 } ,
{H1 , T2 , T3 } ,

{T1 , H2 , T3 } ,

{T1 , T2 , H3 } ,

{T1 , H2 , H3 } ,
and {T1 , T2 , T3 } ,

(2.8)

in each of which the constituent-state identifiers could be listed in any order. In the
more concise sequential notation (2.6), these microstates would be written as
(H, H, H) , (H, H, T) , (H, T, H) ,
(H, T, T) ,

(T, H, T) ,

(T, T, H) ,

(T, H, H) ,
and (T, T, T) ,

(2.9)

in each of which the states of coins 1, 2, and 3 are listed in order.

The complete set of all possible microstates is called the microstate space, that
of the three-coin system being given in (2.8) or (2.9). There is an analogous concept
in probability theory called the sample space.
For more realistic quantum-mechanical systems, in which the particles interact
with each other, the microstates of a system are more complicated, but they can
always be represented as linear combinations of independent-particle microstates
like the ones described above.

2.3.2

Macrostates

The macroscopic state, or macrostate of a many-constituent thermodynamic system

as a whole is characterized by certain macroscopic variables, such as entropy S, energy
E, volume V , and the number of particles N . Generally, many different microstates
will have the same macroscopic properties, so those microstates are lumped together
and said to belong to the same macrostate. The number of microstates belonging to
a macrostate is called its multiplicity, which we denote by . The complete list of
multiplicities of all the macrostates gives the statistical distribution of the microstates
among the macrostates. That distribution proves to be our most important tool in
determining the probability distribution of the macrostates, our key goal.
Its important when working with these three classes of states to keep in mind
that both microstates and macrostates are collective, many-constituent states of an
entire system, whereas the constituent states (single-particle states) are states of
individual constituents (particles). The role of the constituent states is simply to
serve as the building blocks of the microstates in a noninteracting system, much like
in the three-coin system in (2.8) and (2.9).

2.3.3

A small example

To illustrate these ideas, lets consider a tiny two-constituent system consisting of a

tetrahedral die (four sides) and a cubic die (six sides). The constituent states of the
tetrahedral die are characterized by the number facing down and denoted i4 , with
i {1, 2, 3, 4}, and those of the cubic die are characterized by the number facing up
and denoted j6 , with j {1, . . . , 6}.
Random microstates of this many-particle system can be generated by rolling
both dice once, the outcome of which we also refer to as an elementary event. If
desired, time evolution of the system could be represented by repeated rolls, one for
each time step. Each microstate is characterized by a particular set of constituent
states of both dice:
microstate = {i4 , j6 } or (i, j) .
(2.10)

CHAPTER 2. ENTROPY AND THERMAL EQUILIBRIUM

15

16

26

36

46

56

66

14

(1, 1)

(1, 2)

(1, 3)

(1, 3)

(1, 5)

(1, 6)

24

(2, 1)

(2, 2)

(2, 3)

(2, 4)

(2, 5)

(2, 6)

34

(3, 1)

(3, 2)

(3, 3)

(3, 4)

(3, 5)

(3, 6)

44

(4, 1)

(4, 2)

(4, 3)

(4, 4)

(4, 5)

(4, 6)

Table 2.1: The 24 elements of the microstate space (sample space) of a system
consisting of one tetrahedral and one cubic die. Each cell of the table contains the
pair representation (i, j) (2.10) of the microstate whose constituent states are i4 in
the row header and j6 in the column header. These microstates constitute a complete
set of mutually exclusive elementary events for this system: exactly one of them
must be the outcome of any roll of the dice.
There are 4 6 = 24 of these states in the full microstate space, which is shown in
Table 2.1.
We can also identify compound events, as subsets of the microstate space that
satisfy a less-selective criterion or proposition than specification of both constituent
states. For example, the compound event defined by the assertion i > j corresponds
to a six-element subset at the lower left of the microstate-space table, as shown in
Table 2.2.
16

26

36

46

56

66

14

(1, 1)

(1, 2)

(1, 3)

(1, 3)

(1, 5)

(1, 6)

24

(2, 1)

(2, 2)

(2, 3)

(2, 4)

(2, 5)

(2, 6)

34

(3, 1)

(3, 2)

(3, 3)

(3, 4)

(3, 5)

(3, 6)

44

(4, 1)

(4, 2)

(4, 3)

(4, 4)

(4, 5)

(4, 6)

Table 2.2: The microstates corresponding to the compound event defined by the
requirement i > j constitute a six-element subset in the lower-left corner of the
microstate-space table.
A macroscopic property is a variable whose value can be determined for every
microstate of the system. Each possible value of the variable defines a compound
event or macrostate, the subset of microstates that share that same value. Thus,
the possible values of the variable partition the microstate space into subsets, each
of which is a single macrostate of the system. For the two-die system any function
f (i, j) constitutes such a property, and as an example, we use the sum f (i, j) = i + j
[Table 2.3], which is an extensive macroscopic variable.
While there are 24 distinct microstates of this system, there are only 9 different
macrostates, 9 different values of the macroscopic property i + j, and most of the
macrostates have multiplicities greater than one. For example, = 2 for the
macrostate having the sum 3, since there are two microstates, {24 , 16 } and {14 , 26 }
with that sum. The distribution of the macrostate multiplicities is shown in Table
2.4.
Now, the macrostates of the two-die system constitute a complete set of mutually
exclusive compound eventsexactly one value of the sum i + j occurs on every roll
and we would like to obtain their probability distribution. Since each macrostate
comprises one or more microstates, and those microstates are themselves mutually
exclusive of each other, the probability of any macrostate i + j is the sum of the

CHAPTER 2. ENTROPY AND THERMAL EQUILIBRIUM

16

26

36 46 56 66

14

24

34

44

10

16

Table 2.3: Values of the macroscopic property i+j for the microstates of the two-die
system. All microstates having the same sum belong to the macrostate characterized
by that value. The sums for the two microstates belonging to the macrostate with
sum 3 are highlighted.
Sum:
:

2
1

3
2

4
3

5
4

6
4

7
4

8
3

9 10
2 1

Table 2.4: Multiplicities of macrostates characterized by the sum of the numbers

on one tetrahedral die and one cubic die.
probabilities of its microstates. The rolls of the individual dice are independent events,
so the probability of a particular microstate is the product of the probabilities of the
corresponding values on the individual dice. That is, each microstate probability is the
product of its constituent-state probabilities. Thus, for the macrostate characterized
by the sum 3 in Tables 2.3 and 2.4,
P (sum = 3) = P ({14 , 26 }) + P ({24 , 16 })
= P (14 )P (26 ) + P (24 )P (16 ) ,

(2.11)

where Ive taken the liberty of denoting all of the probabilities by the same symbol P ,
even though they belong to four completely different probability distributionsthe
arguments should suffice to avoid ambiguity. Thus, weve reduced the calculation of
the macrostate probabilities to the calculation of the microstate probabilities, which
can be determined if we know the constituent-state probabilities.
For precise work on real dice, one might determine the constituent-state probabilities via the frequency interpretation by making a large number of measurements.
We will simply assume the dice to be fair, so that the constituent states are assigned
equal probabilities. These are 1/4 and 1/6 for the four-sided and six-sided dice,
respectively. Since each die has a uniform (all probabilities equal) constituent-state
probability distribution, the microstate (joint) probability distribution for the pair
is also uniform:
P ({i4 , j6 }) = P (i4 )P (j6 ) =

1
24

for all i and j .

(2.12)

We will universally assume that to be true of the microstate probabilities of statistically large isolated systems, though we do attempt to motivate the assumption
later.
Since the macrostate probabilities are just the sums of their uniform microstate
probabilities, all we need to do is count the microstates belonging to each macrostate.
We have succeeded in reducing the determination of the macrostate probability
distribution to a counting problem. Specifically, the probability of each macrostate is
simply its multiplicity, the number of microstates belonging to it, divided by the
total number of microstates. That is, the probability distribution of the macrostates
is just the normalized analog of the distribution of multiplicities.

CHAPTER 2. ENTROPY AND THERMAL EQUILIBRIUM

17

For example, the probability of occurrence of the macrostate of the dice having
the sum of 8 is
(8)
3
1
=
= .
(2.13)
24
24
8
Proceding in the same way for the other macrostates, we find the complete probability
distribution, shown in Table 2.5. You should verify that the probability distribution
Sum:
:
P:

2
1
1/24

3
2
1/12

4
3
1/8

5
4
1/6

6
4
1/6

7
4
1/6

8
3
1/8

9
2
1/12

10
1
1/24

Table 2.5: Multiplicities and probabilities P of macrostates characterized by the

sum of the numbers on one tetrahedral die and one cubic die.
P
satisfies the normalization requirement x P (x) = 1. Notice that if one is only
interested in relative probabilities, the normalization step is unnecessary, for the
multiplicity distribution suffices on its own to determine those.
There are two especially important features to notice about this macrostate
probability distribution: it isnt uniform, and it has a peak at the center. The most
probable macrostates are four times as likely to occur as the least probable. For
macroscopically large systems the contrast is vastly greater and the relative width
of the central peak is far narrower.
The state variable (sum) for this system in equilibrium is 6, which is both
the most probable value and the mean value of the sum that would be obtained
over a large number of rolls. Of course, the distinction between the microscopic and
macroscopic scales is ridiculously minor for a two-constituent system, but the lessons
learned here are readily applied to truly macroscopic systems.
Keep in mind that we introduce the concepts of constituent states, microstates,
and macrostates as useful aids to help us find the probabilities or relative probabilities
of observing certain macroscopic properties. Constituent states and microstates do
possess an element of reality as the physical single-particle and many-particle states
of a many-particle system, or at least approximations to those states. But macrostates
merely serve to aggregate microstates that share a common value of some macroscopic
property, so that we can calculate probability distributions over the different possible
values of that property. Thus, we will define macrostates somewhat flexibly, in terms
of whatever macroscopic property is of immediate interest.