Built on:Jun04-2010-11:26

Built on:Aug13-2009-11:09
Build info:$Id: Release_notes.txt 1458 2010-06-02 19:10:53Z john_griffith2 $
Changes 6/02/10 13:01
-----------------------------------------------------
1. NBU.pm
Was only accepting xferlengths up to 512K
bumped this up to 1M
Changes 5/08/10 18:41
-----------------------------------------------------
1. dm_backer
modified makefile to allow 64 bit filesizes
Changes 5/08/10 14:41
-----------------------------------------------------
1. xcho.pl
Bug fixes and implemented dmraw format in xcho
Uses dm_writer
EXPERIMENTAL
Changes 5/08/10 11:41
-----------------------------------------------------
1. dm_backer
Created new Output File class
Rather than using ofstream, back to using c file handles
Implemented fdopen method to control sys open options
including O_DIRECT and O_SYNC
New -o argument when using -file
valid options: -o direct OR -o sync
2. dm_writer
Added O_DIRECT and O_SYNC options when using an output file
3. xcho.pl
Added config options for O_DIRECT and O_SYNC
when using NAS device
config file takes fsOpts=direct OR fsOpts=sync
4. NBU.pm
Added the necessary items for the dm_backer call
when using O_DIRECT or O_SYNC
Changes 4/08/10 10:11
-----------------------------------------------------
1. dmreader
Header will now interpret compression/patterns
as "Pattern :" instead of "Compression :"
2. dmx.pl/Drv.pm
Adjusted the read_handler and generate_md5 methods
to work correctly when a user specifies the 10 bit jitter pattern
Changes 4/01/10 08:18
-----------------------------------------------------
1. dmx.pl
Added "compression=100" option to write 10 bit patterns
2. dm_writer
Added "compression=100" option to write 10 bit patterns
for Dan A.
Pattern is repeating: 0010110111-1101001000
Changes 3/22/10 09:33
-----------------------------------------------------
1. xcho.pl
Added a synchlock option to xcho.pl
When synchlock=1 the test will pause
before doing a backup set and create a file in the
cwd named "synch.lock", test will not continue until
this lock file is removed.
Changes 2/22/10 20:33
-----------------------------------------------------
1. dmx.pl
Added limitRdLength option to dmx config
This will cause read xfr lengths to be
limited to the blksz that was used for the
the write of the file as obtained from the
dm-header data

This is in response to an issue with the 6G

SAS cards in LTO-5 testing not handling greater
than 64K data-in transfers
2. Drv.pm
Minor changes to Drive object to support the newly
added rd xfr-sz variable setting
3. dm_reader
Added 'limitrdxfrs' option to support functionality
of dmx enhancements.
This option ONLY affects records writen by dmx, dm_wrv
or other tests that use dm_writer. It will parse out the
dm header from the record and use the listed blksz from the
write as the read transfer length.
NOTE!! If you specify block size = 0 for your writes
DO NOT ATTEMPT TO USE THIS OPTION
Changes 2/05/10 10:33
-----------------------------------------------------
1. NBU.pm
Fixed up Results.out to include GB backed up
even in the case of EWEOM.
Changes 1/28/10 15:15
-----------------------------------------------------
1. xcho.pl
Added a bug fix for unload retry algorithm in Chgr.pm
If a changer goes off line on a linux system the retry counter
would never increment and would sit in an infinite loop of unload retries.
Changes 1/19/10 14:50
-----------------------------------------------------
1. xcho.pl
Added alternate daId assignment method for those
 that encounter overlapped daId's with xcho-dp
To use add the following line to your config:
'pid4daid=1'
2. dputil
Implemented alternate catalog block format
Rather than writing a single cat at the end
of each session, we now write one for every
2 Gig of data backed up (default DP behavior)
This will help with corner case failures of
writing millions of files and running out of
tape for cat blk
Changes 1/19/10 14:50
-----------------------------------------------------
1. xcho.pl
Changed 'use DBI()' to an eval to cover systems
that may not have the standard DBI pkg installed
This means the test will run still, it just will
not attempt to connect to the database.
Changes 1/14/10 14:36
-----------------------------------------------------
1. xcho.pl
Implemented rest of DB plugin functions
2. Drv.pm, NBU.pm, Dp.pm
All received minor changes and new fields
added that will be used for DB support
Changes 1/14/10 14:36
-----------------------------------------------------
1. xcho.pl
Added DB plugin support, just a simple ID stamp to test for now.
Changes 1/12/10 14:36
-----------------------------------------------------
1. xcho.pl
Added "policyName" option to xcho configuration
Changes 12/14/09 18:00
-----------------------------------------------------
1. xcho.pl
Added "write to file" functionality for NBU type backups.
Simply replace:
drivePath=/dev/sg3
with
drivePath=fs-/tmp/TestBackups/
2. bm_backer
Updated the write to file options in dm_backer.cpp
Changes 12/10/09 16:30
-----------------------------------------------------
1. dm_fwriter.pl
 Added new test to dm package
Changes 12/07/09 15:30
-----------------------------------------------------
1. xcho.pl
Added two new options for test runs:
Invalid boType (for testing Magellan Lite on Enterprise VLS)
Replaces the valid type (UX-FX, Ora, Exch etc with a user specified integer
value)
Config file option:
invalidType=n (where n is an integer value such as 99)
Modify File Names
Ability to cause test to append a suffix to the file names that are modifie
d.
The suffix is equivalent to the number of times the file has been modified
in the current test run.
Config file option:
modFileNames=1
2. dputil
Implements the above changes in the dp tool via the -u and -a options
3. dm_backer
Implements the above changes in the nbu tool via the -u and -a options
Changes 11/13/09 10:18
-----------------------------------------------------
1. dm_mtx
Adjusted static timeouts for all of dm_mtx functions
and fixed bug for variable timeout settings.
Bulk of changes in mtxl.c file
Inquiry = 60 seconds
Drive Related Moves = 10 minutes
Status = 10 minutes
Position Element = 10 minutes
Move = 10 minutes
Exchange = 10 minutes
Erase = 5 hours
Start/Stop = 5 minutes
Other commands use the default HBA setting of 1 - 5 minutes depen
ding on platform
Cleaned up the users "timeout" option
Cleaned up usage statement for Physical Element reporting in status, load and
unload
Changes 10/26/09 13:50
-----------------------------------------------------
1. dm_backer.cpp
Fixed double free of tmpArr value in array expansion routine
2. dm_writer.c
Allocate from the heap instead of the stack for CYGWIN
memory allocation limitations.
Changes 10/22/09 16:40
-----------------------------------------------------
1. xcho.pl
Added an additional line to strip characters from
HOSTNAME variable to make things work for SSH on
Windows 2008 systems.
2. dm_backer
Cleaned up memory allocation to try and address
stack allocation limitation in cygwin.
Changes 10/20/09 10:40
-----------------------------------------------------
1. NBU.pm
Bug where some windows 2003 machines inserted
a ctrl-M during a string append. 1 out of 3
Windows 2003 machines exhibited this problem?
Modified the append operation to handle this,
but don't have a real good feeling as to "why"
it's behaving this way on certain hosts.
Changes 10/06/09 16:40
-----------------------------------------------------
1. xcho-nbu (dm_backer)
Bug was reported this afternoon when running xcho-nbu
dm_backer utility was crashing during certain backups.
Issue is related to the removal of filesize restriction
(filesize >= xfrSize).
Added a pad block to the file backup when there isn't
enough space left for the next records header.
Changes 10/05/09 16:00
-----------------------------------------------------
1. xcho.pl
Writing build information and config file to Results.out file
2. dm_backer
Last two builds had debug mode of data refresh algorithms turned on.
While this was "ok" for most VLS cases, D2D picked up the issue
right away. Original method is turned back on now!!
Also, reduced the file size limit to allow files < xfrSz
Increased max file count to 5 million

Changes 9/24/09 12:07

-----------------------------------------------------
1. dputil.c (xcho-dp)
Removed the min file size limitation, now is 1K
Pulled out objDesc, using a loop in dpfuncs instead
Also enhanced so file count can be in the millions
(tested with 3.5 millions files)
 Two things here:
1. Changed from Object backups to Directory backup
(no longer list an Atrib blk for every file at
start of backup, just for Dir then do Data blks
for each file in the directory)
2. Fixed up so file list can span multiple blocks

2. dpfuncs.c
Rewrite of process_session
Made this look like a Dir backup object
Rather than file object backup
Also enabled file list to span MAblocks
Rewrite of backup_obj
Did this in order to allow file sizes < MAblkSz
Took out EWEOM print statements in close_dp_session
Stupid and redundant, especially now when we have
multiple writes in the close routine
Changes 8/17/09 12:07
-----------------------------------------------------
1. Implemented dm_backer NBU Exchange format
Changes 8/13/09 08:00
-----------------------------------------------------
1. Implemented dm_backer into HUPX and Cygwin builds.
Changes 7/27/09 08:30
-----------------------------------------------------
1. xcho.pl
Appended PID to hostname of BSET ID for DP jobs
Changes 7/21/09 10:30
-----------------------------------------------------
1. dm_backer added to the bundle
2. xcho modified to use dm_backer for NBU backup types
Added NBU Oracle capability
Changes 7/13/09 10:30
-----------------------------------------------------
1. dmx.pl
Fixed a bug in level5 routine regarding attempt to
take a modulus of ZERO for red/verify check.
Changes 7/09/09 08:30
-----------------------------------------------------
1. xcho.pl
Issue with dputil and mofied data.
When writing the first modified set it wouldn't
be modified by the given parameters. It would
actually be virtually 100% unique.
The root of this issue lies in dputil, but
temporary fix was to set modify values in the
create policy routine so the very first write
is performed as a modify (initiates the buffers
correctly this way).
Changes 6/19/09 09:00
-----------------------------------------------------
1. dmx.pl
Made doDumpFiles default to 0 (off)
2. dputil (affects xcho-dp and dpcho)
dp.dat.1 header file corrupt, causing jobs w/ blksz > 64K
to fail.
Changes 6/15/09 13:30
-----------------------------------------------------
1. dmx.pl, Drv.pm
Modified dmx and Drv.pm to produce binary dump files
of data read during compare operations.
This will allow a byte for byte compare of read/generated
and also check for read error by re-reading the file
2. Added compare.py script to bundle
Changes 6/09/09 15:45
-----------------------------------------------------
1. Nbu.pm, Dp.pm
Finished up implementation of read functions
Cleaned up / standardized Results.out format for BACKUP
2. xcho.pl
Set default for test / drive type to Dp
Changes 6/08/09 11:45
-----------------------------------------------------
1. xcho.pl
RC candidate for read operations on DP formatted data
Changes to config variables:
setWait
unloadDelay
readTapes
doReads
Clean up / standarization of Results files for all DM tests/Modules
Changes 6/04/09 06:45
-----------------------------------------------------
1. xcho.pl
Bug in backup id generation scheme
timing was allowing simultaneous jobs
from a single client to have the same
backupId string. This causes the scraper
to throw the job out and thus "missing jobs"
Changes 5/27/09 10:45
-----------------------------------------------------
1. nbcho.pl
Reset nextStart time in SeedLock routine so
jobs don't run back to back after seeding
Changes 3/16/09 17:24
-----------------------------------------------------
1. xcho.pl
Turned off the incremental modifications
default is now the method used in the past
of modifying only one segment each iteration
2. dputil.c
Finally have the data generation routines in dputil
working properly for both VLS and D2D

Changes 3/13/09 10:24

-----------------------------------------------------
1. xcho.pl
Carl B. found something very important that I forgot!!
I wasn't writing out the PID FILE!!!! DOHHHHH
Changes 3/09/09 15:45
-----------------------------------------------------
1. dputil
Modified set modification to begin at start of file
rather than end of file
Changes 3/06/09 06:45
-----------------------------------------------------
1. dputil
Added transfer size option (MAblkSz up to 256K)
Added -H option to specify non-default hdr file
2. xcho
New tool, replaces nbcho and dpcho
Now using a generic test for all of the
standard load/unload, policy backup and data
modification. Implemented subclass of Drv
object to handle the commands for the backup itself.
Run /opt/DM4/xcho.pl -t nbu -c my.cfg
replaces
/opt/DM4/nbcho.pl -c my.cfg
Run /opt/DM4/xcho.pl -t dp -c my.cfg
replaces
/opt/DM4/dpcho.pl -c my.cfg
SIGNIFICANT CHANGES:
a. Modifications are not iterative
e.g. if you say modify .1 and each file is 1 Meg
First pass will write the set
Second pass will modify the last 100K of each file in the set
(bytes 900K-1000K)
Third pass will modify the last 100K as in the second pass
as well as another 100K segment
(bytes 800K-900K)
Fourth pass will modify the 200K as in the Second and Third pass
as well an another 100K segment
(bytes 700K-800K)
When we've iterated over the entire file we start over
with "fresh" modification data
3. .pm files
Now using a Dm.pm general function class. This gives us
all of our excpetion handlers, log printers etc rather than implmenting
new ones in every test script.
 Implemented sub-classes for Netbackup and Data Protector specific
functions (Nbu.pm, Dp.pm). Tests call a single backup function
and we override based on the object type to get the desired command.
Changes 2/13/09 08:45
-----------------------------------------------------
1. dpcho
Modified dpcho to use DAid's based on PID
When running multiple instances of dpcho
with frequency time set below 30 minutes
it's common to have overlapped daID's for
simultaneous span operations across different
streams.
Using a base number of the perl scripts PID
helps prevent the probabiity of this happening.
Add functions in dpcho.pl and Drv.pm to pass
formed DAid and 4 character barcode to dputil
2. nbcho
Added significant new compare functionality for Terry.
Can run with compares=2 in config file to run compares
on a previous run of nbcho.
This functionality is NOT integrated in the sequencer
so must be run locally on the client. Rerun the nbcho
test in the same working directory as your original run
when you backed up the data. Test will use your Results.out
file to determine what slot to load and what checksum to compare
against.
NOTE: If you ran your test without the cksums option being set
you're out of luck for now.
If you want to run in the "recommended" mode, set cksums=1
and compares=1, this will cause the test to go through and
verify backups after each backup iteration.

Changes 1/30/09 15:45

-----------------------------------------------------
1. Modified dputil to better handle spans
Modified blkseqno counter to handle case
where no full record is on a cart.
E.G. 100 Gig carts and you try to do a 150 Gig backup
Previously this would cause an error / crash of the
the dedupe processes.
There is still a limitation where your backup MUST
fit on 2 tapes. Spanning multiple times for a single
backup session still does not work.
2. Modified dpcho and Drv.pm to utilize span counter
Steps toward fixing above limitation
Changes 1/29/09 07:45
-----------------------------------------------------
1. Added ability to specify drive index in dpcho config file
 (use for cases where VLS reports non-std serial number
not detected by mapping tool)
2. Added ability to specify an offset to dpch and nbcho.
This feature allows you to restart a test and specify
that the test starts on the last cart wrote and spaces to
the last file wrote (appends).
For example, you ran a test for a couple of days and
stopped it for some reason to make some changes. You
can then specify that the test start on that last cartridge
and file that was written rather than overwriting everything
that has been written previously.
Changes 1/26/09 06:45
-----------------------------------------------------
1. Converted dputil to use standard GNU getoptlong
2. Modified Drv.pm dpwrite utility to use getoptlong
Changes 1/18/09 22:45
-----------------------------------------------------
1. dpcho.pl
Added dpcho test to the bundle
2. New builds of all binary tools (/opt/DM4/bin)
created new build process to simplify build
and included additions for work that's been
done on the Bristol side:
* Large file support for dm_d2t
* NAS support for dm_writer, reader, dm_d2t
Changes 9/18/08 18:45
-----------------------------------------------------
1. dm_setutil
Added ability to truncates and appends to files
that have already been created and are in
the set.dat list
modified/improved the rand function to be accurate
for small sized directory trees
2. file_tool
Added the functionality for appends/truncates
Fixed a bug for addressing missing values to
mandator arguments that require them
Changes 9/18/08 07:15
-----------------------------------------------------
1. Chgr.pm
Modified unload_drive subroutine
to try and better handle error conditions
2. dm_wrv.pl and dmx.pl
Modified drive sn parsing routine to accomodate
Enterprise VLS using colons as characters in their
serial number.
Changes 9/02/08 08:00
-----------------------------------------------------
1. nbx
 Added option to provide a transfer length argument
2. nbcho
Added option to provide a transfer length argument
**Procedure for both tests is to add the line:
xfsz=256 (or other value) to your config file.
Valid values are:32, 64, 128 and 256 anything else
will be ignored.
3. nbutil
Added blocksize/xfersz parameter (-B 65536)
Changes 8/05/08 18:00
-----------------------------------------------------
1. nbx
Changed read current block command to support
DD box (changed block only bit to blocks and FM
Enhanced random data generator one last time
2. Added hidden options to nbx, nbcho and dmcho
for changers that fail the rels sn decode
Changes 7/31/08 17:00
-----------------------------------------------------
Bundle Revision 1.4C
1. dmx and dm_wrv
Added a test unit ready after the load command
this should clear up the medium changed check
condition that tends to cause issues on the
first rewind command
2. nbutil
Fixed a bug in time out values that appears to have
been setting our write timeout to 60 seconds in some
cases
Changes 6/27/08 10:00
-----------------------------------------------------
Bundle Revision 1.4B
1. nbx and nbcho
Added scsi load and unload commands to the load/unload
changer command routines. Main change for this occurs in
Drive module with the addition of load_unload method.

Changes 6/26/08 11:00

-----------------------------------------------------
Bundle Revision 1.4A
1. nbutil
Fixed bug in nbutil on compressing_fill function.
Seed values passed to function were invalid pointers
which were causing completely unique data to be written
to the VLS every time. This results in disappearing jobs
from Backup report (9200 errors on VLS)
Changes 6/03/08 16:43
-----------------------------------------------------
1. nbx.pl
 Added a sessions.out log that will be used to
perform restores/verifies of backup sessions.
2. dm_setutil.pl
Fixed a bug in parsing routine when using G suffix
3. dmx.pl
Created a perfcheck option
Test will keep track of average performance, if perfcheck=.2
It will stop the test if the write performance falls below more
than 20% of the average write performance for the test run.

Changes 5/28/08 10:43

-----------------------------------------------------
1. Drv.pm
Added some extra checks on reads to look and see
if we're in the EWEOM section. Reads don't have
to report EWEOM so we didn't necessarily catch this
before.
2. nbx.pl
Fixed bug in method used for incrementing policy.
3. nbutil
Added compression and modifcation options
Changes 5/22/08 16:00
-----------------------------------------------------
1. nbx.pl
Added netbackup simulation test to bundle
2. Mods/Drv.pm
Added support functions for nbx to module
3. bin/nbutil
Added the netbackup simulation utility to bundle
Version 0.1
Changes 5/16/08 13:00
-----------------------------------------------------
1. Mods/Drv.pm
Removed Time:HiRes dependency
2. Added nbbup utility to Linux distros
3. Updated X86 Linux distros
Changes 5/09/08 13:20
-----------------------------------------------------
1. dm_setutil
Added ability to add compression arguments of 1.6:1
for specific bristol test cases.
2. file_tool
Added period parameters to create 1.6:1 data (16 bytes)
3. dmx
Added check in dmx to see if lch variable is the same
as localhost name and if so, change lch to localhost
4. dmx
 Modified drive wait for ready after load to be up to
60 seconds to handle ESL libraries
Changes 4/21/08 17:00
-----------------------------------------------------
1. Drv.pm
Fixed verify handling of EOM conditions.
Was a bug that partial records would cause the test
to fail. Now we check media status to see if the
read was short becuase of EOM.
This change affects dm_wrv and dmx
2. dm_wrv and dmx
Bug where seed value was not always consistent
Result was that you're write data would not be identical
on seperate runs even after setting the seed variable.
Changes 4/10/08 14:00
-----------------------------------------------------
1. dm_writer
Removed scsi.log option and replaced with a generic
"log everything" -v option. Cleaned up the garbage
output that was showing up in the logs and now
overwrite the logs on every subsequent exectution.
This should keep the size down and much more reasonable
(less than 512K) depending on the size of your record.
Output is now in the resulting dmw.log file.
2. Drv.pm
Drv.pm now reads the debug parameter from your config
file to determine whether or not to write dmw.log
along with other debug information.
You can run with this on as a general rule if you like
or if you run without it and you get write errors that
you can't quite figure out with the standard logging
levels you can turn debug=1 and reproduce to get more
info.
This change effects dmx and dm_wrv tests
Changes 4/4/08 11:00
-----------------------------------------------------
1. dm_scsiutil
Updated to version 1.4
New function to set or clear PMR bit on tape drive
2. dm_wrv
Updated to version 1.C
Hidden pmr flag accepted in config file to clear
pmr bit before every unload (Work around for
Bristol VLS bug)
3. dmx
Updated to version 1.1
Hidden pmr flag accepted in config file to clear
pmr bit before every unload (Work around for
Bristol VLS bug)
4. Drv.pm
 Updated to version 1.1
Added method for pmr function
Changes 4/2/08 17:00
-----------------------------------------------------
1. dm_setutil
When creating very large number of files we were
overwriting files we already created.
Result was that set size was not accurate when
we were done. This also caused some issues
with reiterating back through the directory
structure unnecessarily.
Add --verbose option, prints out number of bytes
of requested bytes created.
2. file_tool
Changed write option to calculate MD5 as it writes
the file. Was writing the file then if you asked for
it went back and read it.
In the case of dm_setutil doing creating this created
huge overhead/time sink. Creation of 1 Gig was around
18 seconds. Reading and MD5 was another 25 seconds
(total of 38 seconds). Doing write and MD5 put us at
23 seconds total. MUCH better!!