PHARh!

ACEUTICA
ACTA
HELVETIAE
ELSEVIER Pharmaceutics Acta Helvetiae 72 (1998) 343-348

Validation of computer systems: Practical testing of a standard LIMS

Daniel Friedli a, Wolfgang Kappeler b, Susanne Zimmermann b,*
aSpirig AG, Froschacker, CH-4622 Egerkingen, Switzerland
b Cilag AG, Hochstrasse 201, CH-8205 Schajthausen, Switzerland

Received 20 May 1997; revised 28 August 1997; accepted 15 September 1997

Abstract

In recent years the introduction of computer systems for data handling in the pharmaceutical industry has increased. A standard LIMS
(laboratory information management system) is software commercially available from different suppliers not only to facilitate data
handling in laboratories but also to cover GMP-requirements. Computer systems introduced in GMP-areas of pharmaceutical companies
have to be validated. For a standard LIMS, the general validation of the program is performed by the supplier. Nevertheless, the user is
always required to cover all phases of a validation. The objective of this paper is to discuss suitable test procedures for the most critical
functions of a standard LIMS needed during the verification step of the validation process. 0 1998 Elsevier Science B.V.

Keywords; LIMS; Computer systems: Validation; Verification

1. Introduction (II) Installation Qualification: The IQ ensures that the

computer system has been installed as it has been specified
Validation is applied to “provide documented evidence
and sufficient documentation is available to prove this.
which provides a high degree of assurance that all parts
(III) Operational Qualification: The OQ is a docu-
will consistently work correctly when brought into use”
mented verification that the system works as intended
(GAMP, 1996). Consequently any computer system that is
throughout representative or anticipated ranges at the user
to be used in GMP relevant areas must undergo a full
company.
validation procedure. A complete validation consists of
(IV) Performance Qualification: The PQ is a docu-
various activities that should be described in written vali-
mented verification that the system works as intended
dation SOP’s.
throughout all operating ranges at the user company.
A full validation concept is composed of the following
(V) A review of all activities is documented in the
steps (PMA, CSVC, 1986; APV-Fachgruppe Information-
validation report. The conclusion should evaluate the re-
stechnologie, 1996; GAMP, 1996; Teuchert, 1996):
sults obtained and document the final decision whether or
(I) Design Qualification: The DQ covers the planning
not the validation was successfully completed. Based on a
step. A validation plan has to be prepared in written form.
positive conclusion the computer system can be released
Requirements have to be specified and agreed. The vendor
for use.
audit should be performed. SOP’s on validation should be
A laboratory information management system (LIMS)
checked for completeness and correctness.
is used for data handling in laboratories, especially for the
introduction and evaluation of quality relevant data and
* Corresponding author. Fax: + 41-52-6309901; e-mail: their approval and release. In addition printouts and archiv-
szimmerm@cilch.jnj.com.

0031-6865/98/$19.00 0 1998 Elsevier Science B.V. All rights reserved.

PII SO03 1-6865(97)00032-O
344 D. Friedli et al. / Pharmaceutics
ing functions are part of a LIMS. A standard LIMS is
developed by the supplier and most validation work on the
program itself is done and has to be provided by the
supplier (Huber, 1996a,b). Nevertheless, a full scale vali-
dation has to also be performed by the customer.
The scope of this paper is to focus on the verification of
critical functions of a LIMS and provide guidance on how
to perform this verification. This publication will not cover
the full scale validation. The verification part of a standard
LIMS can be confined to the comparison between the
requirements that have been specified initially during DQ
by the user and the results finally obtained during testing
of the program. The objective was to develop a commonly
applicable testplan that covers critical modules and func-
tions of a standard LIMS. Two different standard labora-
tory information management systems served as practical
examples during the elaboration of this publication.
2. Methodology
2.1. Risk analysis
The validation test plan is based on a risk analysis. In
order to guarantee as much security as possible during
routine utilisation of the program critical functions have to
be defined and certain numbers of test runs have to be
done (FDA, 1987). All critical functions and modules of
the program should be covered in the test plan. The
relation of effort to result must nevertheless stay in an
acceptable range, which means, must assure the most
security with the least possible expense. What has been
agreed upon during the vendor audit and specified during
continuous contacts with the supplier might give additional
advice to define critical functions and modules of the
program.
2.2. Validation testplan
Based on a risk analysis the following structure has
been defined and the functions mentioned below have been
identified as being essential for testing a standard LIMS.
LOG-IN
ORDER GENERATION
ORDER PROCESSING
ORDER CORRECTION
ORDER DECISION
PRINT-OUTS
TOTAL RUN
FINAL EVALUATION/CONCLUSION
Acta Helcetiae 72 (19981 343-348
Scheme I. Validation test scheme.
The proposal is not absolute, but depends on the pro-
gram and its respective application. Particular tests can
vary, as well as the structure.
2.3. Validation testscheme
The validation scheme can be divided into three parts:
During the first part the data to be keyed-in as well as the
expected result have to be defined in writing. The second
part covers the practical realisation of the test: Introduction
of data and documentation of the actual result in writing.
In the third part the actually obtained results are compared
with the expected results and a final assessment of the
particular test is done.
After carrying out the testing of all functions, a global
assessment and rating of all results is performed (Beizer
and Van Nostrand, 1995). Each single result is part of this
assessment and contributes to the final decision whether or
not the program is working as expected and according to
what has been required initially (GAMP, 1996). For each
function to be tested the validation test scheme in Scheme
1 has been applied.
3. Results and discussion
3.1. Log-in
The log-in function is very important for a computer
program since the user-password combination does not
only regulate the access to the program, but also controls
the specific access levels for different users. The following
test combinations are possible and should be checked for
one to two existing users.
 D. Friedli et al. / Pharmaceutics Acta Heluetiae 72 (1998) 343-348 345

3.1.1. Correct user-password combination istic. Two different ways are possible depending on the
The correct log-in is tested for each existing user group structure of the program: A program that independently
(e.g. technician, group leader, supervisor). creates a unique number for each following order can be
checked by generating new orders. These have to differ
3.1.2. Wrong user-password combination from the first one in at least one characteristic. With
The test should include existing users and existing various orders already generated it should not be possible
passwords, but in wrong combinations as well as not to create another additional order with an already existing
existing users and passwords in order to prove the access number. If manual numbering for order generation is pos-
refusal. sible, the test can easily be performed. Simply try if the
same identification characteristic can be used a second
3.1.3. Various log-ins of the same user at dtfberent loca- time.
tions at the same time
It has to be checked whether or not the same user can 3.2.2. Change of predejined fields
log-in at different locations. Depending on the program During order generation specific fields are automati-
characteristics the correct result can be positive or nega- cally filled in by the program. If it is not allowed to change
tive. given information, this has to be proved by trying to
change all these predefined fields.
3.1.4. Access to different program masks for different
3.2.3. Access to not yet released data
users
Information that is needed for creating an order has to
If the access to specific masks is defined with the user
be released before it can be used. Check if access to still
groups (access levels) the access can already be checked
unreleased data is denied.
with the log-in of different users out of different user
groups. If the access is defined especially for each user
3.2.4. Incomplete orders
individually, the test will be much more intensive since the
It has to be tested if an order can be generated with
test should be done for almost each individual user. Table
required information/data missing. Depending on the pro-
1 shows the possible combinations to be tested.
gram this is or is not possible. Check if the required
information/data can be added to an already existing
3.2. Order generation
order.

After the log-in and before starting the introduction of 3.2.5. Order traceability
results into the LIMS an order has to be generated. The It is important to be able to track “who did what and
following functions and modules related to order genera- when’ ’ . The name of the user, date and time should be
tion should be tested. registered together with the order created.

3.2.1. Order generation with a unique identification char- 3.2.6. Order status
acteristic At each time the order should be visible to the users
The program has to guarantee that each order created is with its respective status in order to check the progress of
registered with at least one unique identification character- the order.

Table 1
Possible tests for different user-password combinations
Access level Correct Wrong user Wrong user Correct Wrong password Wrong password
(e.g.) user name name (existing) name (not existing) password (existing) (not existing)
Technician X X

X X

X X

X X

X X

X X

X X

X X

X X
346 D. Friedli et al./Pharmaceutica
3.3. Order processing - Introduction of results
Working with orders depends very much on the type of
the program. Commercial Standard software reveals very
different possibilities for result entry. Therefore, it has to
be very well defined which of the following functions and
modules are the most critical ones.
3.3.1. Numerical results
Decimal-, exponential results
What happens if a decimal result is keyed-in instead of
an exponential one and vice versa? Does the program
complain or refuse? How is rounding off done?
Number of digits
A certain number of predefined digits in front and
behind the decimal point should be maintained during
introduction of results and calculations.
Decimal point
What happens if a comma is keyed-in instead of a
decimal point?
Units
Predefined units should be proposed by the program
during introduction of results whereas other units and other
dimensions should not be accepted. Conversions from one
unit into another should be checked as well as all units per
dimension, as these calculations are normally predefined
by the system.
Round off
Check if the results are rounded off according to the
predefined limits. The original result keyed-in should also
be kept available in the system, in order to still get correct
results after changing the number of decimal places.
Results within /without specifications
Does the program act correctly if correct and incorrect
results are introduced? Perform the following tests:
_ result far below specification limit,
_ result far above specification limit,
- result just below specification limit,
_ result just above specification limit,
_ result within specification limit.
Mean value (calculations)
Final results often rely on calculations of mean or other
values. Calculations have to be manually recalculated for
verification.
Text
What happens if alpha numerical results are introduced
instead of numerical ones?
No / incomplete results
Does the program refuse if results are totally missing or
if the result introduced is not complete?
Acta Helcetine 72 (1998) 343-348
3.3.2. Alpha-numerical results
Fixtext / variable text
If a fixtext is required it should not be possible to
introduce a variable text without the program complaining.
Capital and small letters, special symbols
For alpha-numerical results it is very important to know
the reaction of the program. Are capital and small letters
compatible? What kind of special symbols are accepted?
Text results within/without specifications
Does the program act correctly if correct and incorrect
text results are introduced? Results near the limit as men-
tioned for the numerical results will demonstrate if the
program works well.
No / incomplete text
Does the program refuse if text is totally missing or if
the text introduced is not complete?
Numerical result
What happens if a numerical result is keyed-in in a text
field?
3.4. Order processing - Correction of results
It is critical to test whether correction of results is done
strictly following GMP regulations. The tests mentioned
below have been chosen in order to also cover these
regulations.
3.4. I. Motivation/justification
Any correction can only be performed by authorised
users and has to be documented. Check if only authorised
users with the respective access level are allowed to
perform corrections and whether the program consequently
needs a written justification for each correction performed.
3.4.2. Correction registered with user, date and time
In order to assure traceability, each correction has not
only to be justified, but also to be recorded with the user’s
name, the actual date and time. Check for correct registra-
tion of corrections.
3.4.3. Corrected results
After correction of results only the new corrected re-
sults should be available in the program. The old values
should no longer be available for any calculation. Check if
the new corrected results appear and also if they are used
for further calculations.
3.4.4. Old results
Old results should be registered by the program, but
they should not appear and should no longer be used for
calculations.
 D. Friedli et al. /Pharmaceutics
3.5. Order decision
A LIMS normally provides 2 possible ways of how to
decide on an order. An order can be automatically finalised
by the program itself. An order can also be finalised
manually if control of a supervisor is still necessary or a
secondary decision is required. Manual decisions are only
to be made by competent authorised personnel as the order
decision is also a critical function within a LIMS. For the
manual order decision a special access level has to be
defined. Further on it has to be checked whether only the
user group that has been given authorisation has access to
the decision function. Generally users without a decision
permit cannot even open the decision mask or the decision
button remains inactive. The following functions should be
tested:
35.1. Release with correct results
An order will be released if all results comply with
specifications and no errors and deviations have been
recorded.
3.5.2. Release with non-compliant results
An order with one or more non-compliant results is
generally not releasable by the system. Check if it is still
possible to release this order manually. This certainly
depends upon the structure of the program and the decision
levels defined.
3.5.3. Secondary decision with non-compliant results
Orders that cannot be released because of out of specifi-
cation results have sometimes to be nevertheless continued
under certain circumstances. For this reason the program
should provide a possibility for a secondary decision which
can only be performed by authorised users and normally
also implies a written justification.
3.5.4. Rejection
An order with non-compliant results that cannot
released, not even by a secondary decision, will be
jected. It must not be possible to generate certificates
this order/product. For a rejection a motivation text
normally required as well.
3.5.5. Order decision registered with user, date and time
Each order decision, release as well as rejection, should
be recorded in the system with user, date and time.
Status of the order (Release/Reject)
After order decision the status of the order should be
visible in the system with its respective status.
Acta Heluetiae 72 (1998) 343-348 347
3.6. Print-outs
Today most networks are based on a client-server
structure which connects various PC’s (clients) and differ-
ent printers. It has to be checked whether the ‘communica-
tion’ between the different locations works as designed.
3.6.1. Verification of data introduced - Data printed
With the printouts it has to be verified whether the data
keyed-in is identical with those printed. In addition any
comments introduced can also be checked for their pres-
ence on the print-out.
3.6.2, Print-out at different printers
In order to guarantee that all printers work equally, the
following tests are recommended.
Print reports and certificates at different printers in
order to compare them for completeness and correctness.
Print reports and certificates at different PC’s (clients)
in order to compare them for completeness and correct-
ness.
Print particular reports with special characters and spe-
cific symbols in order to check for correctness.
3.6.3. Print-out of archived data
Compare original print-outs with prints-outs made out
of the archive and check whether the data is identical.
3.7. Total run
It is strongly recommended to perform some entire runs
(black box) in order to control the system from the begin-
ning to the end (Wolf, 1996). It is important to do total
runs on different product types/order types in order to
cover most of the system functionality that will be used
during routine operation. It may be advantageous to per-
form the total runs with actual data such as real products
and specifications. This already assures that specific func-
be tions that are needed have already been tested.
re-
of 3.8. Evaluation of all tests - Conclusion
is
Each single test evaluation contributes to the final as-
sessment of the entire test. The final decision can be
positive or negative. A final positive assessment is achieved
if all required tests and functions meet the expected results.
The validation report can be finalised and the program can
be released for use. If the final evaluation results in a
negative assessment the program has to be corrected and
revalidation tests are necessary.
348 D. Friedli et al. /Pharmaceutics Acta Heluetine 72 (1998) 343-348

4. Final comment kind of change or alteration has to be evaluated for the

necessity of a revalidation.
Validation of computer systems can be carried out on a
separate database with especially defined data that will be
tested or on the same database as the routine program will Acknowledgements
work afterwards. Generally both ways are applicable, but it
Our special thanks go to Dr. A. Beck-Sickinger, Dr. G.
should be decided which of the two approaches will be
Imanidis and Dr. S. Marrer for the organisation of this
used before starting validation. The following aspects
seminar and their competent guidance throughout the pro-
should be taken into consideration for the final decision:
ject.

4. I. Separate database
References
Using a separate database all data that is used during
APV-Fachgruppe Informationstechnologie, 1996. APV - Computerunter-
validation has to be introduced again for routine and
stiitzte Systeme. Pharm. Ind. 58 (71, 590-598.
consequently needs more time and effort. Nevertheless, Beizer, B., Van Nostrand R., 1995. Software Testing Techniques. New
with a separate database the validation work is very well York.
documented and reproducible. The most important issue is FDA, 1987. Guidelines on General Principles of Process Validation. 5600
that it has to be guaranteed that the program works identi- Fisher Lane, Rochville, Maryland 20857.
GAMP Good Automated Manufacturing Practice, 1996. Pharmaceutical
cally during validation and routine.
Industry Supplier Guide for Validation of Automated Systems in
Pharmaceutical Manufacture, ISPE.
4.2. Identical database Huber, L., 1996. Validierung Computergesteuerter Analysensysteme: Ein
Leitfaden fur Praktiker. Springer Verlag, Berlin, Heidelberg, New
Validating on the same database is not as time and work York.
Huber, L., 1996b. Validierung Computerunterstiitzter Systeme. Pharm.
consuming since the data is only introduced once. During
Ind. 58 (lo), 935-940.
validation the same data is tested as it is used afterwards PMA, CSVC, 1986. Validation Concepts for computer System, Valida-
for routine, which means that introduced data has already tion in the Manufacture of Drug Products. Pharm. Technol. lO(51,
been proven during validation and does not have to be 24-34
checked again for routine. The validation is only docu- Teuchert, V., 1996. Prozessleitsysteme in der Pharmaproduktion. Pharm.
Ind. 58 (lo), 930-934.
mented by the hard copies, but is not reproducible since
Wolf, H., 1996. Die Anwendung der GLP-Grundsltze auf
the same database is used. computergestiitzte Systeme (OECD 19951, Deutsche ubersetzung.
If the final conclusion after the validation process was Konsenspapier GLP u. DV/306. OECD-Originaldokument:
positive and the program has been released for use, any OCDE/GD, 1995, p. 115.