310

MEDICAL IMAGE PROCESSING

P.N.T. Wells and E.Berry

Bristol General Hospital and University of Bristol, UK

THE OBJECTIVE OF MEDICAL IMAGING The objective of medical imaging, as explained by Worthington (l), is to obtain information relevant to the detection, diagnosis, treatment and monitoring of disease. Detection relates to screening, as in X-ray mammography for breast cancer. Diagnosis is concerned with the identification of the origins of symptoms; often, the process is that of differential diagnosis, as in distinguishing between various possible causes of jaundice, and it is always linked to prognosis in that, to be worth doing at all, it should help to determine optimal patient management. In treatment, medical imaging has an increasingly important role for location and guidance, as in extracorporeal shock wave lithotripsy for the disintegration of kidney stones: indeed; interventional radiology is likely to lead to the development of new imaging applications during the next decade. Finally, monitoring the progress of natural processes and disease, and the efficacy of treatment, is an essential element in patient management; for example, in obstetrics the optimal date for delivery can be decided on the basis of ultrasonic measurements of fetal growth. Occasionally, the interpretation of a medical image is obvious; for exdmple, it presents no challenge to identify a major fracture in an X-ray picture of a broken limb. Generally, however, it is only radiologists - trained medical doctors - who have the skill necessary to extract and analyse the visual information in the image. Moreover, even radiologists can make mistakes and, as pointed out by Rosenquist (Z), it is false negative results that are likely to be more dangerous than false positives because they usually lead to failure to provide appropriate treatment. The ideal imaging system would provide an exact pictorial representation of what would be seen if the structures within the patient were to be exposed to direct vision, but with complete safety in relation to radiation and other physical exposures and without the need for injections or other invasive procedures. The traditional approach to medical imaging is based on the solution of the inverse problem of deducing the characteristics of the tissues within the body from the received field resulting from probing radiation. Ambiguities and lack of complete data prevent the image from being an exact pictorial representation of the object. Sarvazyan and Wells ( 3 ) have pointed out that, in principle, an exact representation could be produced if the philosophy of imaging were to be changed to that of solving the direct problem. The received field that would be produced by tissues with defined physical properties could be calculated by a process involving iterative convergence of the observed and calculated fields, interactively

involving an observer having knowledge of the clinical history of the patient. The traditional approach to imaging performance improvement, however, is based on devising improvements in image acquisition and image processing. IMAGE ACQUISITION SYSTEMS The most important image acquisition Systems have been reviewed by Webb ( 4 ) and Wells ( 5 ) . By far the largest fraction of the work in medical imaging is still traditional radiography using X-rays. The patient is positioned a little distance from an X-ray tube with a small focal spot; on the other side of the patient, and usually as close as possible, there is a cassette containing an image intensifying screen and a photographic film. The exposure consists of a brief pulse of X-rays. When the film is developed, the shadow image formed by the different attenuations of the various structures in the X-ray beam can be seen. Often, more information is obtained by the prior administration of contrast agents having relatively high or low attenuations. As an alternative to photographic film, an electronic image intensifier can be used as the receiving transducer, to provide a television signal for dynamic real-time studies. Moreover, the TV signal can be digitised so that image processing by computer is possible. In X-ray computed tomography (CT scanning) a series of X-ray transmission profiles through a cross-sectional plane of the patient is first collected, usually by means of a mechanical scanner. A computer is then used to back-project the filtered transmission profiles to reconstruct a two-dimensional image of the attenuation of the tissues in the scan plane. Contrast agents may be used to enhance tissue differentiation. The image is generally photographed from a TV monitor and, because of its digital nature, it may easily be processed by computer. The commonest instrument for radionuclide imaging is the gamma camera. A collimator collects gamma-rays from the radionuclide previously administered to the patient and allows only thuse rays which form a spatial image of the distribution of the radionuclide within the patient to reach a large diameter sodium iodide scintillation detector. The light which the detector then emits is collected by an array of photomultiplier tubes whose outputs are electronically processed to form a twodimensional image on a CRT display. The image is suitable for digital processing. There dre two imaging methods of established clinical value which do not depend on the USE of ionising radiation. The first of these employs ultrasound. At low megahertz frequencies, the wavelength of

31 1

ultrasound is in the millimetre range. Ultrasound is reflected and scattered within the body where there are discontinuities in acoustic impedance (the product of density and speed). Techniques analogous to those used in radar are used to provide highresolution real-time pulse-echo images of cross-sections through soft tissues and for the study of motion and flow by the Doppler effect. The image data can be obtained in digital format. The remaining method, now coming into routine use, is magnetic resonance imaging (MRI). An MRI scanner consists of a strong magnet with an opening large enough to accommodate the patient, the field of which tends to align nuclei, such as protons, that have magnetic moments. The precessional frequency of the proton depends on the magnitude of the magnetic field: consequently, the application of field gradients and pulses of rotating field allows three-dimensional information about the distribution and relaxation times of protons within the body to be obtained. A digital computer is used to form the image. Other methods of imaging, of specialised utility or still in the research stage, include thermography, light transmission and electrical impedance tomography. IMAGE PROCESSING METHODS Often, images of diagnostic quality are obtained without further processing of the output from the image acquisition system, although it has to be admitted that some systems do incorporate elementary and necessary processing circuits. Thus, traditional radiographs, provided that they are correctly exposed, are viewed directly on a light box. Ultrasonic scanners, on the other hand, need to hdve properly chosen gray scale transfer characteristics. In addition to intrinsic image processing arrangements of this kind, however, much useful information and numerical data can be obtained in many different situations by the application of additional image processing techniques which can be broadly considered under the following three separate classifications. Image enhancement and compression Image enhancement is any process that improves the visual perception of the feature or features of interest or relevance. The ability to adjust the contrast and brightness ranges of the display can often be used for image enhancement. Enhancement can also sometimes be achieved by suppressing undesirable or distracting image features, such as noise or speckle. Although some improvement may be obtained with a low pass filter, this is necessarily at the expense of image sharpness. TO some extent, this effect can be reduced by the use of an adaptive filter derived from, for example, the local variance and the local mean of the image texture: the usefulness of this has been demonstrated by Bamber and Daft (6) with ultrasonic scans which are known to have a high speckle content. Maximum entropy processing has been shown to be successful in extracting useful information from conventional images (such as photographs) blurred by defocusing or

motion. Except for noise reduction, its application to medical image enhancement has been disappointing, probably because medical images generally are solutions to the inverse problem, not merely poor pictorial representations of the object. The problem of medical image storage is discussed later. A high resolution medical digital image contains at least 1 MB if it is recorded over the full dynamic range of every pixel. Typically, however, only part of the frame contains image information and the data (such as alphanumerics) in the remainder are quite sparse; even the image itself may be capable of data compression if, for example, it is part of a sequence of images in which only a fraction contains frame-to-frame motion. For a classical review of picture bandwidth compression, see h a n g and Tretiak (7). Feature analysis Essentially, any two dimensional image is a distribution of brightness, contrast and texture. The contrast of the image depends on the dynamic range of the display. Selection of the gray level threshold and window width perinits contrast adjustment within the noise limit oi the system. This is commonly used in CT scanning. Moreover, images such as CT scans can be segmented according to ranges of density such as those corresponding to bone, soft tissue and gas: for example, this allows accurate compensation to be applied in radiotherapy treatment planning. In an extension of this kind of processing, edge detection algorithms can be used to identify spatially continuous transitions in contrast, such as those that occur along blood vessels and around heart chambers in the presence of contrast agents or in radionuclide scans: see, for example, Cahill and Knowles ( 8 ) . Another important technique, digital subtraction angiography, depends on the subtraction of sequential images of the same structures which differ only as the result of the administration of a contrast agent. For example, X-ray images of an artery before and after the injection of dilute contrast agent are stored digitally; they may be indistinguishable to the unaided eye, but, when subtracted, the blood vessels containing the contrast agent become clearly visible in the resultant image and the other structures almost disappear. This has several potential advantages: contrast agent concentration and X-ray exposure can be optimally chosen to provide the required diagnostic information with the minimal hazard to the patient. Another approdch to feature analysis is based on pattern recognition. Syntactic and statistical methods may be employed. For example, fractal modelling may lead to the classification of vascular structure related to liver disease. In ultrasound scanning, although the texture of the image of liver parenchyma is a speckle pattern not having a one-to-one correspondence with tissue structure, radiologists learn to recognise patterns related to different pathologies. In fact, human observers are good at extracting information relating to contrast and brightness but they are poor at the second order task of texture analysis: this is better dvne by a machine that can d e r i v e numerical

312

data such as gray level distributions and other statistical measures that may be helpful in differential diagnosis. For example, Schuster et a1 ( 9 ) have used local texture analysis in the objective differentiation of ultrasonic liver scans. Image manipulation Tomographic image acquisition systems can often be used to produce images of contiguous slices. As explained by Halliwell et a1 (10) such a stack of images can be considered to be a three-dimensional data set from which any two-dimensional plane or surface can be extracted and displayed. Parallel processing allows this kind of image manipulation to be performed in quasi-realtime. This approach has been used with MRI and ultrasound scans, where the dynamic range of the data set is so great that threedimensional display is impracticable due to the limitations of the human visual system. Although a similar approach could in principle be applied to CT scans, the slice thickness tends to be too large for it to be useful and, in any case, radiologists have no difficulty in examining and mentally correlating the separate images. Because of the strong contrast between bone and other materials in CT scans, however, techniques have been devised which allow surface representations of bony structures to be displayed in any orientation; see, for example, Vannier et a1 (11). Surface shading improves perception. Because the displays are so easily recognised (scans of heads, for example, look like skulls), they have seized the imagination of the public and popularised radiology through the contribution that it can make to the planning of surgical procedures. Images acquired by different techniques often demonstrate different characteristics of the body. For example, CT scans show anatomy and radionuclide images represent physiological funtion. Superimposing these two types of images allows physiology to be related to anatomical location. Generally, such images are produced with different scales and geometrical distortions, and they have to be stretched so that they superimpose accurately. The use of markers allows this process to be performed by computerised image processing. AUTOMATIC AND COMPUTER-ASSISTED DIAGNOSIS

In radiology, attempts at automatic image analysis have so far failed. The classically tedious task is that of reporting on routine chest X-rays. Attentive radiologists still outperform the most sophisticated automatic pattern recognition systems. Mammography poses similar problems, although certainly more difficult ones due to the subtlety of the radiographic appearances of breast diseases and the unreliability of the only marker of malignancy (the occurrence of calcifications) which is a candidate for automdtic identification. PICTURE ARCHIVING AND COMMUNICATIONS SYSTEMS Although seemingly mundane, the fact that around 20 per cent of X-ray films which enter a traditional kind of library cannot be found again really is a serious problem! In an attempt to solve it and, at the same time, to improve the efficiency of medical imaging departments, picture archiving and communications systems (PACS) are undergoing rapid and intensive development. Apart from traditional radiography, all the major modern imaging techniques are based on electronics and are, in principle, directly compatible with digital storage, transmission and display. Mun and Akisada (13) have pointed out that the most promising storage medium is the optical disk. Typically, an optical disk has a capacity of about 2 . 5 GB and a "juke box" of disks permits access to any one of 300 000 stored images within about ten seconds. To put this into perspective, a 600-bed hospital serving 750 outpatients per day uses 2 0 0 000 sheets of X-ray film annually, so a juke box could store all the images produced in a year. For this to be a reality, film radiography will need to be replaced by a system more compatible with digital technology. One promising approach involves the use of photostimulable phosphor plates, which have very wide dynamic range. Such plates consequently have the additional ddvantages of eliminating the need for retakes due to exposure errors and they allow the exposure to be minimised to that determined by the minimum acceptable signalto-noise ratio. Communications with the image acquisition systems and to the various viewing stations cdn be coaxial cable or optical fibre. Contemporary workstations are still unacceptable for clinical use except in the most specialised applications (because of interaction weaknesses) but it is likely that equipment improvement and cultural changes will make PACS commonplace in large hospitals within the next decade. IMAGE UISPLAY: MAN-MACHINE OPTIMISATION

Even with the aid of modern imaging equipment, diagnosis still often presents puzzles to the doctor and Shortiffe (12) has implied that the correct solutions depend as much on good luck as on science. The radiologist ought to be fully informed about clinical history of the patient and must not overlook the possibility that some littleknown disease may be involved. First of all, the radiologist needs to decide on the best examinations in the sequence and on the most appropriate settings for the machines. In this, a menu-driven expert system can ensure that the lessons of past experience are not forgotten. Then, the interpretation of the image itself depends on searching for characteristic features, and here d computer-generated check list can help to avoid errors.

It has already been pointed out that the objective of medical imaging is to obtain information relevant to the detection, diagnosis, treatment and monitoring of disease. Fundamentally, the process is that of cognition; the perception of the visual image is the mechanism by which information acquired by the imaging system is transferred from the display to the brain of the observer. As image acquisition systems are progressively improved, so the performance of the observer becomes increasingly dependent on matching the characteristics of the display to those of the eye. Hendee and Wells (14) have pointed out that further image processing development now awaits the results of research into the processes of visual perception.

313

REFERENCES

1.
2. 3. 4. 5.

Worthington, B.S., 1987, IEE PTOC. A, 134, 98-100.

- 1189-1190. 150,

Rosenquist, C.J.,

1988, Am. J. Roentg.,

Sarvazyan, A.P., and Wells, P.N.T., in press, Br. J. Radiol. Webb, S. (ed.), 1988 "The Physics of Medical Imaging", Hilger, Bristol, UK. Wells, P.N.T. (ed.), 1982, "Scientific Basis of Medical Imaging", Churchill Livingstone, Edinburgh, UK. Bamber, J.C., and Daft, C., 1986, Ultrasonics, 3, 41-44. Huang, H.K.., and Tretiak, O.J. (eds.), 1972, "Picture Bandwidth Compression", Gordon and Breach, New York, USA. Cahill, P.T., and Knowles, R.J.R., Med. Phys., l , 575-580. 2 1985,

6.

7.

8. 9.

Schuster, E., Knoflach, P., and Grabner, G., 1988, J. Clin. Ultrasound, 16, 453461. Halliwell, M., Key, H., et al., in press, Br. J. Radiol. Vannier, M.W., Marsh, J.L., et al. 1985, Electromedica. 51, 122-131. 258, Shortiffe, E.A., 1987, J. Am. Med. Ass., 61-66.

10.

11.

12. 13. 14.

Mun, S.K., and Akisada, M., 1988, Diagn. Imaginq, S. no. 1 2 , 136-149. Hendee, W.R., and Wells, P.N.T. (eds.), 1988, "Visualization Science in Engineering, Computing and Medicine", American College of Radiology, Reston, USA.