Lava Build Tools Training

Lava Build Tools Training
Build Tools Team

January 2008
Xilinx Confidential
Course Logistics
• Feel free to ask questions during the class
– We will answer what we can and get back to you on
the ones we can not
• Additional information has been added to some

of the slides in the slide notes section
– This is for future reference - you do not need to see
these during the class
• Folks have reported that some of the numbered

bullet items may be off when using
OpenOffice.org to view this presentation
Lava Build Tools Training 2 Xilinx Confidential

Course Outline
• What’s New?
• Electric Accelerator: The Build Cluster
• Using the Build Cluster
• The Lava Build Environment
• The Cluster Manager
• Makefile Template Rules and Guidelines
• New Build System Documentation
• New DSD Build System Web Site

What’s New?
• ElectricAccelerator integrated into the
Build System
– Instead of gmake we now use emake
(ElectricMake)
• Developers have shared build

resources at their disposal
• Resource sharing can span many

groups including Relman
Other Changes in Lava
• flex / bison replaces lex / yacc
• 32-Bit Windows Compiler Upgrade

– Developers will need a license for the
new compiler (msvsn2005 )!!

Course Outline
• What’s New?

Electric Accelerator
Desktop Desktop Desktop Desktop

Build Build Build Build
Machine Machine Machine Machine
Build Cluster

Software Build Cluster

Virtual File
File Cluster
System
Build Machine Manager
System File Cache
emake
Sandboxes History
Cluster File
Agent
Machines Annotation
File

Virtual File
File Cluster
System
System File Cache
emake
Sandboxes History
Cluster File
Agent
Machines Annotation
File
• The Build Machine

– Accesses the user sandbox
• Could be from a file server or from local disk storage
– Builds using emake
• emake can break down the commands of a make rule and farm
that work to the build cluster
Virtual File
File Cluster
System
System File Cache
emake
Sandboxes History
Cluster File
Agent
Machines Annotation
File
• Cluster Agents
– Multiple agents can run on a single cluster agent machine
– Accepts and runs build jobs from emake
– Builds and uses the Virtual File System & File Cache

Virtual File
File Cluster
System
System File Cache
emake
Sandboxes History
Cluster File
Agent
Machines Annotation
File
• Cluster Manager
– Coordinates emake requests and agent assignments
• Based on priorities and available resources
– Manages agent licenses
– Provides a web based interface for managing the build cluster

Virtual File
File Cluster
System
System File Cache
emake
Sandboxes History
Cluster File
Agent
Machines Annotation
File
• Virtual File System

– A build environment replicated from the user sandbox
– Provides file caching between agents working on the same build
– All build content is copied back to the user sandbox when the build is completed
– Allows Electric Accelerator to see all file dependencies associated with a build job

History Files
• History Files:
– Store dependency data determined by
– Are used by Electric Accelerator to
determine the ordering of build jobs and
allows it to parallelize the build
efficiently across the cluster
– Are created if they do not exist and
updated if they do
– Are only used when using the build
cluster
Annotation Files
• Annotation Files:
– Log vast amounts of information about the build
collected by emake as XML
• makefile structure
• commands and command output
• list of file accesses by each job
• dependencies between jobs
• detailed timing measurements
• emake invocation arguments and environment
• performance metrics
– Are overwritten when the build job is re-run
– Are created for each build submitted to the
build cluster
Using Annotation Files
• The annotation file is XML and can be viewed directly
• Some useful information includes:
– Build command line
– Environment variable values at build time
– All files accessed during the build (read, write, create, unlink, etc.)
– Build metrics
• Electric Cloud ElectricInsight™ product can be used to

analyze annotation files
– Its value in developer builds has not been determined
– It is available if developers want to try it
– Linux: /tools/baton/ecloud/insight-2.0.1/einsight
– Windows: \\xcodevnac\xfndry\ecloud_software\einsight-2.0.1\windows\einsight.exe

xbld and Electric Accelerator
Data Files
• xbld creates unique history/annotation files
based on the platform and the target name
• xbld will store these Electric Accelerator data
files in an ‘emake_data’ directory where
emake is invoked
• File names:
– History Files:
• emake_<platform>_<target>.data
• e.g., emake_lin_libs.data
– Annotation Files
• emake_<platform>_<target>.xml
• e.g., emake_lin_libs.xml

Course Outline
• What’s New?

Build System Tools
RTF
Quality
Reports
Build
Status
RTF checkrtfs Viewer
Areas
Sandbox
allbuilds
Make
xbld Template Source
XCS Code
BuildIt Repository
Platform
Make Makefile
Build Tools Source Code Control Tools

BuildIt in Kryptonite
BuildIt
relman SSI
Build Job
Queue
BuildIt BuildIt BuildIt BuildIt

Client Client Client Client
gmake gmake gmake gmake

BuildIt in Lava
BuildIt
relman SSI
Build Job
Queue
BuildIt BuildIt BuildIt BuildIt

Client Client Client Client
emake emake emake emake
Electric
Accelerator
Cluster
Manager
Cluster
Agent Agent Agent Agent
(lin) (nt) (nt64) (lin64)

Developer / SSI Builds
xbld –nocluster… xbld –nocluster –Platform …

1
allbuilds –noremote -nocluster… 4
allbuilds –nocluster …
Desktop Desktop xbld –Platform …

Build Build 3allbuilds …
Machine Machine
Developer
emake emake
/ SSI
Shared Shared
xbld …
2
allbuilds -noremote …
emake
Machines
Developer
Machines
[ lin/lin64/nt/nt64 ] [ lin/lin64/nt/nt64 ]
Electric emake emake

Accelerator
Cluster
Manager
Cluster

Building on the Desktop
xbld –nocluster…
1
allbuilds –noremote -nocluster…
Desktop
Build
Machine
Developer
emake
/ SSI

Building on the Desktop Using
the Build Cluster
Desktop
Build
Machine
Developer
emake
/ SSI
xbld …
2
allbuilds -noremote …
Electric
Accelerator
Cluster
Manager
Cluster

Remote Platform Build using
the Build Cluster
Desktop xbld –Platform …

Build 3allbuilds …
Machine
Developer
emake
/ SSI
Shared
emake
Machines
[ lin/lin64/nt/nt64 ]
Electric emake
Accelerator
Cluster
Manager
Cluster

Remote Platform Build without
using the Build Cluster
xbld –nocluster –Platform …
4
allbuilds –nocluster …
Desktop
Build
Machine
Developer
emake
/ SSI
Shared
Developer
Machines
[ lin/lin64/nt/nt64 ]
emake

Lava Build Hardware Configuration
xbld –nocluster –Platform …
allbuilds –nocluster … Shared
Developer
Machines
allbuilds –noremote -nocluster…

xbld –nocluster… Desktop allbuilds …
Desktop
BuildIt Build Build xbld –P …
Machine Machine
relman SSI emake Developer emake
Build Job / SSI
Queue allbuilds -noremote …
xbld …
Shared Shared Shared Shared

BuildIt BuildIt BuildIt BuildIt emake emake emake emake
Client Client Client Client Machine Machine Machine Machine
emake emake emake emake [ lin ] [ nt ] [ lin64 ] [ nt64 ]
Electric
Accelerator
Cluster
Manager
Cluster

Dependency Processing
Make Template
!d *.o : *.c
!r @$(ECHO) ----- Compiling $(basename $@).c -----
!r @${RM} $@
!r ${COMPILER} $(basename $@).c $(CFLAGS) …
History
emake File
xbld
-nocluster -cluster
makefile.<platform> makefile.<platform>
foo.o : foo.c \ foo.o : foo.c
${XMM_XILENV1}/layer1/ss1/export/header1.h \ @$(ECHO) ----- Compiling $*.c -----
${XMM_XILENV1}/layer2/ss2/export/header2.h \ @${RM} $@
${XMM_XILENV1}/Layer2/ss3/export/header3.h \ ${COMPILER} $(basename $@). c $(CFLAGS) …
o
o
o
@$(ECHO) ----- Compiling $*.c -----
@${RM} $@
${COMPILER} $(basename $@). c $(CFLAGS) …

Managing Dependency Data
with and without the Cluster
• xbld will automatically use the appropriate
dependency processing based on build
cluster usage: -cluster versus -nocluster
• Users must take care switching between
building a sandbox with and without the
build cluster
– Building a sandbox without the cluster results
in no history file data
– Running a build using the build cluster in such
a sandbox may result in files not recompiling
when they should

Course Outline
• What’s New?

Lava Build Environment Rules
• If you are running in San Jose or Colorado, you must
configure your xcs_environment file to use ‘/build/sjxfndry’
or ‘/build/bcxfndry’ respectively, not ‘/build/xfndry’
– This only applies if you are using the build cluster
– For sites other then XCO and XSJ, follow the local rules
• If you run on lin64 you must ensure the LANG and LC_ALL
(LC_*) environment variables are unset in your shell
– xbld will attempt to do this for you
– This issue will be addressed in the EC4.2 release available in the
coming weeks
• If you are running on windows, all 3rd party tools (compiler,

perl, etc.) must be installed in the standard locations in order
to use the cluster
• You need a msvsn2005 license if you are building on 32-bit

windows in Lava!!
Installing emake on Windows
• emake must be installed locally on
Windows development machines
• Instructions for installing emake on

windows machines can be found here (
http://webster:88/si/cgi-bin/ElectricMake.cgi)
• Like most of our 3rd party software, there

will be requests to upgrade to new
versions periodically
Course Outline
• What’s New?

Why Access the Cluster Manager?
• A build seems to be hung
– Check the cluster manager to see the status of a
build job
• The build cluster seems slow
– Check the number of builds currently running on
the build cluster
– Check to see the status of the build cluster agents
• Find information on a build job
– Command line information
– Shell environment information
– Build job metrics
• Find Electric Cloud documentation

Accessing the Cluster Manager
• The Cluster Manager stores information
about:
– Builds that are waiting to run
– Builds that are running
– Builds that have completed
• A Cluster Manager can be accessed by

entering the cluster manager machine
name in the URL field of a web browser as
follows:
– Colorado: http://xcocm
– San Jose: http://xsjecmk700
Cluster Manager Login
• You do not need to log in

– If you get the login screen, leave both
fields blank and click the login button
Cluster Manager Home Page

Cluster Manager Build Page

Finding a Specific Build

Build Detail: Detail View

Build Detail: Configuration View

Filtering by Build Attributes

Cluster Manager: Agent Tab

Some Things We’ve Noticed
• The cluster manager remembers the last filter you ran
– Always check the filter line
• The Colorado Cluster Manager can be slow to respond when

the number of builds gets large
– There may be a delay in the query completing (indicators say
done) and the display of the actual results
– Every Monday morning we delete builds older than a week (XCO
only) to keep the build count down
– The XSJ cluster will be cleaned on an as needed basis

Course Outline
• What’s New?
• Makefile Template Rules and
Guidelines
Make Template Rules & Guidelines
1. Do not use "SHELL=" in make templates
– There has been a long term effort to remove
these from make templates (since Granite?)
– This works in gmake but NOT in emake
2. Do not use $$@ on the right hand side of a

target line
– This does not work in emake

3. Do not use parentheses to create subshells
– The following command does not work in emake:
(MAKEFLAGS=””; nmake -f Makefile
Target)
– Removing the parentheses resolves the problem
– The problems we have seen (and fixed) have been in
the third party software primarily with invoking nmake

4. Do not use $* outside of pattern rules
– $* GNU Make meaning:
• “The stem of the target filename. A stem is typically a
filename without its suffix. Its use outside of pattern
rules is discouraged.”
– In emake it is more than discouraged; It does not
work
– Replace $* with $(basename $@)
– Example:
!d *.o : ../smoke/*.c
!r -${RM} $@
!r ${COMPILER} ../smoke/$*.c ${CFLAGS} ${INCLUDES}
The last line should be:

!r ${COMPILER} ../smoke/$(basename $@).c ${CFLAGS} ${INCLUDES}

5. Do not use $?
– $? GNU Make meaning:
• “The names of all prerequisites which are newer than
the target, separated by spaces.”
– Because emake knows more about file
dependencies than might be recorded in the
makefile, it's possible for emake to execute a
make recipe even if none of the listed files is
newer than the target
• In this case, the $? macro is empty
– Replace $? with $<
– The automatic variable `$<' contains just the
first prerequisite
6. Directory and file permissions must be at least
711 when using the build cluster
– This can be done using:
chmod go+x /sandbox/path
– It is likely that the same file mode will be required
on directories all the way down to the place where
xbld is run
– This problem currently causes build cluster agents

to die
• This has been reported to Electric Cloud and a fix will be in
the next release
• 711 permissions will still be required but it will not cause
agents to die
7. Define dependencies between the exported
copy of the library and the rellibs target
– Subsystems that have “!ReleaseFiles rellibs …”

should also have:
rellibs: $(EXPORTDIR)/$(SHLIB)
– This will ensure that the exported files are kept

up to date

8. Try to turn logical targets into physical
targets
– Serialization requires serial dependencies!
– Use physical targets (files) to explicitly state your
intentions:
• Bad: The following recipe works, but tells make to copy
every file, every time make is run:
includes:
@${CP} ../i/*.h ${EXPORT_DIR}
• A Better way:
!m HEADERS = ../i/*.h:${EXPORT_DIR}/*.h
includes: ${HEADERS}
${EXPORT_DIR}/%.h : ../i/%.h
@${ECHO} Releasing exported header file $@
${RM} -f $@
${CP} -f $< $@
${CHMOD} 644 $@
9. Do not rely on the ‘sequential nature’ of the old build system
when listing commands in a make rule
– Example: When using XTESTSUMMARY define dependencies so
that smoke tests will be run before XTESTSUMMARY
– This is a common high level make template entry:
libsmoke:
!Subbld smoke/s
@${XTESTSUMMARY} -r=smoke
– When running on the cluster emake will try to run the smoke
tests and XTESTSUMMARY at the same time resulting in an
empty smoketest.sum file
– The following provides the desired behavior:
libsmoke: smoketest.sum
smoketest.sum: run_smokes
@${XTESTSUMMARY} -r=smoke
run_smokes:
!subbld smoke/s
– If you wanted to parallelize multiple directories, you could do so

with:
run_smokes:
!subbld smoke/s foo/s
10.Glob operations on directories
where the directory content can
change is dangerous
– Glob operations involve wild card
characters that require a directory
read to resolve them
– Electric Cloud File System keeps track
of all files accessed during a build
– It does not keep track of operations
performed on a directory

10.Glob operations on directories where the
directory content can change is dangerous
(continued)
– Assume the following structure in
MySubsys/MyPkg/make.t
includes:
!subbld MySubPkg1
!subbld MySubPkg2
!subbld export
– Also assume MySubPkg1 and MySubPkg2 copy

some files into the MyPkg/export directory
– In export/make.t you have something like:
!m EXPORTED_HEADERS = *.h : *.h
includes:
$(CP) $(EXPORTED_HEADERS) $(EXPORTDIR)

10.Glob operations on directories where the
directory content can change is
dangerous (continued)
– A better way:
includes: export_includes
export_includes: pkg_includes
!subbld export
pkg_includes:
!subbld MySubPkg1
!subbld MySubPkg2
– In export/make.t:
!m EXPORTED_HEADERS = *.h : *.h
includes:
$(CP) $(EXPORTED_HEADERS) $(EXPORTDIR)

Course Outline
• What’s New?

Build Tools Documentation
• Lava Build Tools Quick Start Guide
• New DSD Build System Document
• New SDS Development Environment Setup Guide
• New BuildIt User Guide Document**
• Updated xbld Users Guide
• Updated Make Template Users Guide
• Updated Make Template Reference Manual
• Updated allbuilds User Guide
• checkrtfs User Guide
• All documentation can be found on the Build Tools

home page:
– http://xinc/sites/bart/default.aspx
** BuildIt Publish Pending

Course Outline
• What’s New?

DSD Build System Web Site
• One stop shopping for all Build System needs:
– BATon
– XCS & XCS PRT
– Build Tools & Build Tools PRT
– Build System Strategic Planning Team
– Production Build (relman) home page
• Using Sharepoint for meeting management
• Using CRS Issue Request for submitting requests
http://xinc/sites/dsdbldsys/default.aspx

Questions?

Appendix
Support Slides
Xilinx Confidential
Hardware Terminology
Shared
BuildIt Server Developer
Machine Machines
Desktop Desktop
BuildIt Build Build
Machine Machine
relman SSI emake Developer emake
Build Job / SSI
Queue
Shared Shared Shared Shared

BuildIt BuildIt BuildIt BuildIt emake emake emake emake
Client Client Client Client Machine Machine Machine Machine
emake emake emake emake [ lin ] [ nt ] [ lin64 ] [ nt64 ]
Electric Shared emake

BuildIt Client
Machine
Accelerator
Cluster Machine
Manager
Cluster
Cluster Agent
Agent Agent Agent Agent Machine

Cluster Availability
• The San Jose and Colorado build clusters can
be used by developers at other sites
– Having a sandbox local to the build cluster is
preferable
• Tests building a San Jose sandbox - using the

Colorado build cluster - showed better
performance than building the same sandbox
on a desktop machine in San Jose
– Your mileage may vary depending on what you
build

Lava Build Tools Training

Caricato da

Informazioni sul documento

Descrizione originale:

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Lava Build Tools Training

Caricato da

Copyright:

Formati disponibili

Lava Build Tools Training

Build Tools Team

• Additional information has been added to some

• Folks have reported that some of the numbered

Lava Build Tools Training 2 Xilinx Confidential

Lava Build Tools Training 3 Xilinx Confidential

• Developers have shared build

• Resource sharing can span many

• flex / bison replaces lex / yacc

• 32-Bit Windows Compiler Upgrade

Lava Build Tools Training 5 Xilinx Confidential

Lava Build Tools Training 6 Xilinx Confidential

Desktop Desktop Desktop Desktop

Lava Build Tools Training 7 Xilinx Confidential

Software Build Cluster

Lava Build Tools Training 8 Xilinx Confidential

• The Build Machine

Lava Build Tools Training 10 Xilinx Confidential

Lava Build Tools Training 11 Xilinx Confidential

• Virtual File System

Lava Build Tools Training 12 Xilinx Confidential

• Electric Cloud ElectricInsight™ product can be used to

Lava Build Tools Training 15 Xilinx Confidential

Lava Build Tools Training 16 Xilinx Confidential

Lava Build Tools Training 17 Xilinx Confidential

Build Tools Source Code Control Tools

BuildIt BuildIt BuildIt BuildIt

Lava Build Tools Training 19 Xilinx Confidential

BuildIt BuildIt BuildIt BuildIt

Lava Build Tools Training 20 Xilinx Confidential

xbld –nocluster… xbld –nocluster –Platform …

Desktop Desktop xbld –Platform …

Electric emake emake

Lava Build Tools Training 21 Xilinx Confidential

Lava Build Tools Training 22 Xilinx Confidential

Lava Build Tools Training 23 Xilinx Confidential

Desktop xbld –Platform …

Lava Build Tools Training 24 Xilinx Confidential

Lava Build Tools Training 25 Xilinx Confidential

allbuilds –noremote -nocluster…

Shared Shared Shared Shared

Lava Build Tools Training 26 Xilinx Confidential

Lava Build Tools Training 27 Xilinx Confidential

Lava Build Tools Training 28 Xilinx Confidential

Lava Build Tools Training 29 Xilinx Confidential

• If you are running on windows, all 3rd party tools (compiler,

• You need a msvsn2005 license if you are building on 32-bit

• Instructions for installing emake on

• Like most of our 3rd party software, there

Lava Build Tools Training 32 Xilinx Confidential

Lava Build Tools Training 33 Xilinx Confidential

• A Cluster Manager can be accessed by

• You do not need to log in

Lava Build Tools Training 36 Xilinx Confidential

Lava Build Tools Training 37 Xilinx Confidential

Lava Build Tools Training 38 Xilinx Confidential

Lava Build Tools Training 39 Xilinx Confidential

Lava Build Tools Training 40 Xilinx Confidential

Lava Build Tools Training 41 Xilinx Confidential

Lava Build Tools Training 42 Xilinx Confidential

• The Colorado Cluster Manager can be slow to respond when

Lava Build Tools Training 43 Xilinx Confidential

2. Do not use $$@ on the right hand side of a