Rapport 88. Pre-studies for automatic wall-searching program. Thesis work by Annika Grandell Tove Gustavi

Transkript

1 Rapport 88 Pre-studies for automatic wall-searching program Förstudier till automatiskt väggsökningsprogram Thesis work by Annika Grandell Tove Gustavi KAROLINSKA INSTITUTET Institutionen för elektronik Enheten för medicinsk teknik KTH Institutionen för medicinsk laboratorievetenskap & teknik KI Stockholm 2001

2 ABSTRACT This work is the start of a larger project aiming to locate the inside of the heart wall in ultrasound images. It is mainly the location of the walls of the left ventricle that is of interest, since this will give an indication of the pump function of the heart. The goal is an investigation to see if three specific points are enough to make a curve that approximates the heart wall. The three points are the apex and the two points where the AV plane and the heart wall crosses. Before describing the development of the detection of these three points, a great amount of theory is reviewed both in the medical area and in the image processing area. Since ultrasound images are very noisy, much effort was made in trying to improve the images at hand. Relatively easy image processing methods were then used to select points that lay on edges. By combining this information with the fact that the apex is relatively still and that the AV plane has movement in certain parts of the heart cycle, the apex and the AV plane could be detected. To find the crossings between the AV plane and the heart wall further treatment of the original image had to be done. The resulting methods have proven to be quite stable and can be used in further developments. It is also seen that approximating a parabola to the three points and combining this curve with the AV plane will be a good starting point for the automatic detection of the entire heart wall. 2

3 SAMMANFATTNING Detta arbete är början på ett större projekt, som siktar på att detektera hela insidan av hjärtväggen i ultraljudsbilder. Det är huvudsakligen vänster kammares väggar som är intressanta eftersom det kan ge en indikation på hjärtats pumpfunktion. Målet med arbetet är att se om tre specifika punkter är tillräckliga för att anpassa en kurva som approximerar hjärtväggen. De tre punkterna är apex samt de två punkterna som uppstår i korsningen mellan AV planet och hjärtväggen. Inför utvecklingen av detektionen av dessa tre punkter behandlas en stor del teori, både inom det medicinsk tekniska området och inom bildbehandling. Eftersom ultraljudsbilder är väldigt brusiga lades mycket tid ned på att försöka förbättra de aktuella bilderna. Relativt enkla bildbehandlingsmetoder användes för att plocka ut punkter, som låg på kanter. Genom att kombinera denna information med det faktum att apex ligger relativt stilla under en hjärtcykel och att AV planet rör sig i vissa lägen i hjärtcykeln kunde apex och AV planet detekteras. För att hitta korsningen mellan AV planet och hjärtväggen behövdes ytterligare behandling av originalbilderna. De resulterande metoderna har visat sig vara ganska stabila och kan användas för vidare utveckling. Det visas också att genom att approximera en parabel till de tre punkterna och kombinera parabeln med AV planet kan man få ett bra utgångsläge för automatisk detektering av hela hjärtväggen. 3

4 ACKNOWLEDGEMENTS We would like to thank all the people that have helped us in our work; professor Håkan Elmqvist and the personnel at the division of medical engineering and persons connected to Vingmed. We would especially like to thank: Our supervisor Lars-Åke Brodin for the interest he has taken in our work. Britta Lind for the help acquiring ultrasound images. Camilla Storaa for the help during the project and for the proof-reading of the report. Anders Torp, who always found time to help us with everything related to image processing. 4

5 TABLE OF CONTENTS Abstract 2 Sammanfattning 3 Acknowledgements 4 Table of contents 5 1 Introduction 7 2 Theoretical background The anatomy and physiology of the heart Ultrasound General information on ultrasound Propagation of the ultrasound beam Interaction with matter Generation of ultrasound Resolution Artefacts and disturbances Acquiring images A-mode B-mode M-mode TGC The use of Doppler information The Doppler effect Continuous Doppler Pulsed Doppler Use of velocity information in producing images Tissue Velocity Information Echocardiography Trans-thoracic echocardiography Transesophageal echocardiography 18 3 Presentation of the problem Area of interest Pump function of the left ventricle Strain measurements Basic conditions clp-files The machine used Assumptions and conditions on the images Matlab as a development language GcMat 24 5

6 4 Image processing Methods for improving images Stretching Median filter Wiener filter Averaging Total Variation based noise removal algorithms Methods for enhancing edges Derivative filters 33 5 Implementation Improving the images by filtering Reducing the speckle noise Finding edges in an image Algorithm for finding the apex Results and reliability Comments on the source file Algorithm for finding the atrioventricular plane Results and reliability Comments on the source file Algorithm for finding the crossings between the AV plane and the heart walls Results and reliability Comments on the source files Fel! Bokmärket är inte definierat. 5.5 Fitting a parabola to three points in the heart wall 54 6 Conclusion Results and reliability Usefulness and possible improvements Further developments 59 7 References 61 8 Appendix 1: Source files 63 9 Appendix 2: Dividing the work Appendix 3: Nomenclature 99 6

7 1 INTRODUCTION One of the most frequently used methods to examine the heart is echocardiography. Echocardiography uses ultrasound to image the heart. Echocardiography has several advantages making it a very useful examination method. It is non-invasive and therefore not harmful for the patients and the result can be viewed in real-time during the examination. In an echocardiographic examination, a transducer is used to emit beams of ultrasound that are reflected against the different layers of tissue within the body. The reflected waves are detected and the time it takes for them to return to the transducer is measured. The information recorded can then be used to produce images of the scanned area. The examination can also give information of the movement of the heart tissues. The development of echocardiography in the last couple of years has above all been within the Doppler imaging techniques. Despite the development, great knowledge and experience is still needed to interpret echocardiographic images. By developing new automatic techniques, the use of echocardiographic techniques can further increase. As the interpretation of the images is done automatically it is easier for non-trained personnel to use the equipment. The objective of this work is to investigate if it is possible to automatically locate the apex and the two points corresponding to the crossings between the atrioventricular plane and the heart wall. If these points can be found, testing will be done to see if a curve, fitted to these points, can approximate the heart wall well enough for automatic border detection functions to be used. This will in turn lead to knowledge of the location of the entire heart wall in the whole heart cycle. Knowledge of the location of the heart wall in the left ventricle during a heart cycle can give much information on the pump function and the regional deformation. There exist some on-line automated border detection programs. These are mostly based on the use of integrated backscatter. The pulse repetition frequency is lower resulting in reduced speckle noise, the backscatter is integrated and analysed and the interfaces between blood and tissue are enhanced and thus easier to detect. [17, 18, 19] This work will instead be focused on the use of imaging processing methods when trying to automatically locate the three points of interest. The reading of this report requires knowledge of some basic medical engineering. However no previous knowledge of image processing is required. In Appendix 1 the files that are written particularly for this work are attached. Files that are used and are not attached are either functions provided by Matlab or files that are implemented in GcMat. 7

8 2 THEORETICAL BACKGROUND To facilitate the reading of this work this section contains a short going through of the anatomy and physiology of the heart, followed by a more detailed presentation of the use of ultrasound imaging and the theory that it is based on. 2.1 The anatomy and physiology of the heart The heart is a muscle located in the upper left part of the chest. It has the size of a normal fist and is protected by the thorax. Around the heart is a membrane called the pericardium, which in turn is attached to the epicardium. The epicardium is the outer layer of the heart wall. Between these membranes is a fluid, which makes it easy for the membranes to slide against each other. The myocardium in turn is inside the epicardium and is the cardiac muscle tissue. The innermost layer of the heart is the endocardium and is a thin layer. The heart is divided into four cavities, two atria and two ventricles. The right part of the heart handles the blood that shall be oxygenated in the lungs. This cycle is called the pulmonary circulation and consists of the pulmonary vein and artery and of course the lungs. The left part of the heart handles the oxygenated blood and provides it to the whole body, called the systemic circulation. The blood is pumped through the aorta on to the rest of the vessels of the body. The sizes of the cavities are approximately the same besides the size of the left ventricle, which is approximately twice the size of the other cavities. The left ventricle has to pump the blood to the whole body and its walls are therefore thicker than the walls of the right ventricle, which only has to pump the blood to the lungs. Figure 1 - Anatomy of the heart [1], anterior view of frontal section 8

9 The movements of the heart can be divided into a cardiac cycle consisting of diastole and systole. Diastole stands for relaxation and systole for contraction. First the atria are filled with blood. Then the atria contracts and the ventricles are filled with blood. They in turn contract and pump the blood into the pulmonary and systemic circulation. To prevent the blood from sliding back mitral valves are located between the atria and ventricles, between the left ventricle and the aorta and also between the right ventricle and the pulmonary vein. In the cardiac cycle, the heart is activated by an electrical impulse. The impulse starts in the sino-atrial node located in the right atrium. The signal is then transferred to the musculature in the heart wall. In the atrioventricular node, located between the right atrium and the right ventricle, the signal is delayed by a fifth of a second. The impulse then continues and is after a while divided into two signals which goes to respective ventricle. The signals move down to the apex where the muscle contraction then starts from the apex and moves upwards. In the contraction of the left ventricle, the atrioventricular (AV) plane moves downwards towards the apex. The apex is assumed to be relatively fixed, whereas the AV plane moves a distance of approximately a centimetre. 2.2 Ultrasound Examination with ultrasound is an important diagnostic tool. Its main advantage against other examination methods is that it does not have any harmful side effects. Further information on the subjects in this section (section 2.2) can be read in [2, 3, 4] General information on ultrasound Ultrasound waves propagate as longitudinal waves and have a high frequency above the audible frequencies. The frequency range for ultrasound is 20 khz and above. The velocity with which the ultrasound waves propagate the body is for soft tissue approximately 1540 m/s. The intensity of the wave is proportional to the square of the amplitude Propagation of the ultrasound beam As all other sound waves, the ultrasound wave needs a medium to propagate. Depending on the distance to the transducer, two regions with different characteristics are formed: the Fresnel region and the Fraunhofer region. The size of the Fresnel region, or the near field, depends on the diameter of the transducer and the frequency of the emitted wave. In this region the beam does not diverge due to interference effects. The size of the region is given by the square of the radius divided by the wavelength of the emitted wave. In the Fraunhofer region, or the far field, the beam diverges. The angle of the divergence depends on the diameter of the transducer and the wavelength of the ultrasound wave. The greater frequency or the larger transducer the smaller angle and vice versa. 9

10 2.2.3 Interaction with matter The acoustic impedance describes the properties of the matter in question. The acoustic impedance is expressed as the product of the density of the material and the velocity of sound in the material Reflection A reflection of the ultrasound beam occurs when the beam comes across an interface between two materials with different acoustic impedance. The reflected energy depends on the angle of incidence and is reduced when the beam does not strike the surface perpendicular Refraction The transmitted wave after the interface between two materials with different acoustic impedance will have a different propagation angle due to the frequency change in the transmitted medium. However if the incident wave is perpendicular to the interface no angle change will occur Attenuation The energy of the ultrasound beam is reduced with depth. The energy decreases exponentially with depth. Energy losses occur partly due to interactions with matter and the energy is therefore lost as heat. The heat is produced when the particles vibrations cause friction. Energy losses also occur when the beam is scattered due to interactions with small point-like objects Generation of ultrasound To produce ultrasound a piezoelectric crystal is used. The crystal receives an electrical impulse and converts it to a mechanical wave. The thickness of the crystal is selected to one half of the produced wavelength. This is to cause constructive interference between ultrasound waves that have been reflected inside the crystal and the first transmitted wave. To detect an ultrasound wave the crystal s function is reversed and it instead acts as a receiver converting a mechanical wave to an electrical impulse The transducer The transducer is the transmitter and receptor of ultrasound waves. The piezoelectric crystal is only one of the parts in the transducer. Behind the crystal is a backing block. Its purpose is to absorb the ultrasound waves travelling in the backward direction and also to act as a damper. The reason for wanting the damping function is that it reduces the duration of the produced ultrasound pulse. In front of the piezoelectric crystal is the matching layer, which has properties that maximizes the transmission of ultrasound waves to the body. Surrounding the parts is a cover that protects and insulates the parts. [5] Behaviour of the ultrasound beam In ultrasound scanners multiple transducers are ordered in different ways to produce beams of different shapes. 10

11 In a linear probe the transducer elements are ordered linearly and produces a rectangular scan segment. In a phased array transducer the emission of ultrasound pulses can be delayed. By increasing the time between the emissions of ultrasound pulses from the transducers, the ultrasound beam can be steered. By letting the pulse emitted be delayed linearly along the probe, the wave front of the resulting beam can be directed in a certain direction depending on the time delay. To create a larger field of view the beam is shaped as a diverging scan segment by delaying the outer transducer s emission of waves. By instead delaying the inner transducer s emission the resulting beam will converge towards a certain point, i.e. it will be focused The Q-factor The Q-factor is a measure of the duration of the ultrasound wave that is created in the piezoelectric crystal. The electrical impulse sent to the crystal causes it to vibrate and therefore the ultrasound wave transmitted is not just a peak. Depending on, above all, the backing block, the transmitted wave is damped. The more damped it is, the lower Q-factor it has. Figure 2 - High Q, i.e. light damping Figure 3 - Low Q, i.e. heavy damping Resolution The resolution is a measure of the smallest distance between two objects that can be distinguished Time resolution pulse repetition frequency The time resolution depends on how often the pulses are sent out, the pulse repetition frequency. There is however an upper limit on the pulse repetition frequency since the emitted ultrasound pulse has to be detected before the next pulse is sent out. The time resolution thus depends on how deep the ultrasound beam has to penetrate and the maximum pulse repetition frequency is derived from this demand Axial resolution The axial resolution is a measure of the distance between two objects in the beam s direction that can be separated from each other. The most important 11

12 factor is the length of the ultrasound pulse, i.e. the Q-factor. The smallest distance between two objects that can be detected is half the length of the ultrasound pulse. If the separation is larger the reflected echoes will overlap Lateral resolution The lateral resolution is a measure of the distance between two objects in the cross section of the beam that can be distinguished. If two objects are within the same ultrasound beam they will not be separable. The resolution is worse deeper inside the body than it is in superficial layers. This depends, among other things, on the divergence of the beam Artefacts and disturbances The ultrasound image contains disturbances that make the image an inaccurate replication of the actual appearance of the examined area Speckle noise The dominating disturbance in ultrasound images is the speckle noise. This noise is the reason why ultrasound images look so grainy. As described above, the ultrasound image is built-up by the reflection against tissue of the sent out pulse from the transducer. If two points in the tissue lie very near each other, their reflections can coincide and cause coherence. The points must however have a distance between each other that is smaller than the duration of the sent out pulse. For constructive coherence to occur, the (radial) distance between the reflecting points must be a multiple of the distance of half the wavelength. Then the two points will overlap and form one bright dot in the ultrasound image. If however the distance between the small objects are (2n+1)λ/4 (n is an integer) the interference will instead be destructive and the two points will be represented as a black spot. Two images taken with only a small time difference will show very similar speckle patterns. Since the movement between these images has not been large the structure of the tissue has not altered much either. This means that the speckle pattern will arise in similar manners. The higher the Q-value, the more speckle noise will appear, since the region where two reflections overlap at the receiver will be larger. Below is an illustration of the reflected signals of two points that have a constructive respective destructive interference. [11, 21] 12

13 Figure 4 - Distance between points are nl/2, overlap leads to constructive interference Figure 5 - Distance between points are (2n+1) l/4, overlap leads to destructive interference Reverberations Multiple echoes can occur when a strongly reflected signal is again reflected back into the body, i.e. multiple echoes are produced. This means that the transducer will receive false echoes that do not arise from an interface. This principally occurs near the surface where the signal has not attenuated much Shadowing and enhancement If an interface with high acoustic impedance difference (see 5.2.3) is closely followed by another interface, the following interface will be shadowed by the strong interface. This implicates that the interface will disappear or be strongly reduced in the produced image. The opposite effect with a too strong echo will occur when an interface with a weak echo is followed by another interface, which then will be enhanced Speed artefacts The velocity of the ultrasound is approximated to 1540 m/s in tissue when producing an image from the acquired data. This means that structures with higher acoustic impedance will be reproduced with a reduced thickness, since the actual velocity in this type of medium is higher than 1540 m/s. The opposite applies to structures with lower acoustic impedance that will be reproduced with an increased thickness Mirroring If the ultrasound beam strikes a strongly reflecting structure with an angle, the beam will be reflected. The reflected beam in turn can strike another material 13

14 and be reflected back to the strongly reflecting structure and ultimately reach the transducer. This will lead to a false echo, which appears in the image behind the strongly reflecting structure Increase of width As the beam moves deeper into the body, it diverges. This means that objects deeper within the body will be reproduced as bigger than they actually are. 2.3 Acquiring images There are a few different recording techniques used at ultrasound examinations. Some of the techniques present the recorded information as a two-dimensional image of the scanned area and some present the data as a function of time. Different techniques are used to visualise different features. Sometimes, the relative intensity, instead of the intensity, is used in the display and is measured in decibel. More on the subject treated in this section (section 2.3) can be read in [2, 3, 4] A-mode A-mode stands for amplitude mode and is the simplest way of using ultrasound. Ultrasound pulses are only transmitted along one beam and the reflected echoes are recorded. The intensity as a function of depth is measured. The output is thus a graph where the horisontal axis represents the time from when the pulse is sent out and the vertical axis represents the amplitude of the reflected pulses. See figure21 for a typical A-mode graph B-mode B-mode stands for brightness mode and is displayed as a two dimensional image. Either a moving transducer or an array of transducers is used. This means that a larger sector is scanned than in A-mode. The resulting echoes are recorded and an image is produced. See figure 9 for a typical B-mode image M-mode M-mode stands for motion mode and shows the time variation of one beam. Thus a stationary transducer is used. A two dimensional image is built up where the variation of the interfaces are plotted as a function of time. Figure 6 - A typical M-mode image 14

15 2.3.4 TGC TGC is short for Time Gain Compensated. This is an adjustment that is used to amplify the returned echoes. Since the ultrasound beam is attenuated in tissue, the amplitude of the returned echoes will decrease with time after emission. Therefore the returned echoes are amplified as a function of time. The ideal case is an image where all interfaces are reproduced equally strong. 2.4 The use of Doppler information The information of the movements of the blood and tissues can be calculated using Doppler information The Doppler effect When a sound wave with a certain frequency, f, hits a moving object, the reflected sound wave will have a different frequency, f, depending on the velocity of the moving object. Moreover, the change of frequency depends on, θ, which is the angle between the direction of the original sound wave and the direction of motion of the moving object. The formula for the Doppler effect is: ( f f f ') vcosθ = 2 c (eq. 1) c is the velocity of sound in air. From the Doppler shift, the velocity of the moving object can be derived. However, it is the velocity component along the beam that is being derived. If an object only moves perpendicular to the transmitted pulse, no Doppler shift will occur. Two types of Doppler methods are used: continuous and pulsed. [3, 4] Continuous Doppler Measuring velocities using continuous Doppler requires two transducers: one transmitter and one receiver. A high Q-factor is necessary. The transmitter is energised by an alternating current and sends out ultrasound continuously. The receiver then registers the frequency of the reflected waves and determines the differences in frequency between the transmitted and the received signals. The advantage of continuous Doppler is that aliasing is avoided and therefore high velocities can be measured. The main drawback however is that the various velocities cannot be determined in depth since the waves are sent out continuously. Aliasing means in this case that high velocities will be represented as low because of a too low sampling rate, f s, when deriving the velocities from the Doppler shift. The sampling rate must be so high that the signal is measured at least twice every period, f s 2f 0, where f 0 is the frequency of the measured signal. The sampling rate is nevertheless depending on the pulse repetition frequency. There is, however, an upper limit on the pulse repetition frequency as described above. This upper limit reduces the sampling rate and thus results in an upper limit on the velocities that can be measured. If a velocity exceeds 15

16 this limit, the sample rate will be too low and the aliasing phenomenon will occur. This will result in undersampling and as a consequence, higher velocities will be registered as low Pulsed Doppler The pulsed Doppler uses transducers, which acts as both transmitter and receiver. Unlike the continuous Doppler, the pulsed Doppler is depth sensitive. The operator of the transducer selects an area of interest to examine and the pulse repetition time is determined according to the selected depth. If the pulse repetition frequency is too high, velocities deep inside the body will not be measured, since the echoes from these parts will not have returned before the next pulse is transmitted. This will result in an upper limit on the pulse repetition frequency and therefore, there will also be a maximum limit on the measurable velocities. This will result in aliasing Use of velocity information in producing images In images where velocities are displayed, the colours used are often red and blue. Red to yellow represents movement toward the transducer and blue to green movements away from the transducer. [13, 15] Auto-correlation method for calculation of the velocities There are some different approaches used when determining the velocity taking the frequency shift as a starting point. However, in this text only the auto-correlation method will be described. Since it is the derivative of the phase that determines the velocity, this is the important calculation factor to speed up for the purpose of real-time illustration of velocities. The basic idea of the auto-correlation method is that the method is quick to estimate the phase factor and its derivative. The autocorrelation method is in fact an approximation of the velocities, since ultrasound images are discrete both spatially and in time. Instead of analysing the phase shift in every received signal, the auto-correlation method suggests that the phase shift can be approximated by taking two, in time, subsequent scan lines and compare these, resulting in an approximation of the phase shift. The comparison of scan lines involves calculation of the auto-correlation function in every point along the scan line. This in turn is approximated, since the amount of data is limited. The auto-correlation function with lag one is therefore discretized. The auto-correlation method is mathematically quite advanced and is thus only briefly described here. More can be read in [4, 9, 16] Tissue Velocity Information The use of Tissue Velocity Information (TVI) has become an important tool in describing the function of the heart. TVI is only one of the names used when referring to movement of tissue measured by Doppler. Other names used are Tissue Doppler Imaging (TDI) and Doppler Tissue Imaging (DTI). [8, 9, 13, 14, 15] 16

17 Extraction of the TVI Velocities in the body can be calculated by using Doppler methods and the auto-correlation method described in section However the velocity signals can arise from either movement of the tissues or the blood flows. There are two ways to separate the blood flow from the movements of tissue. First, the blood velocities are much higher than the tissue velocities and second, the amplitude of the signal coming from the reflection against blood cells is much weaker than the signals coming from tissue. m/s Velocity db Amplitude Tissue movement Blood flow Figure 7 Characteristics of Doppler signals and how to separate them from each other To extract the tissue velocity information, either a high pass filter on the amplitude information or a low pass filter on the velocity information can be used Display of the TVI Depending on its purpose, the tissue Doppler velocities can be displayed either in M-mode or in two-dimensional images. If the M-mode display is used the pulse repetition frequency is much higher, but if instead the two-dimensional view is used the velocities can be overlapped on an ordinary ultrasound image. The pulse repetition frequency is lower because the velocities have to be calculated in every segment of the heart between two subsequent displayed images. 2.5 Echocardiography Echocardiography is the term used when referring to ultrasound based methods for heart imaging. Originally it was used only for images derived from the amplitude information, but in spoken language it also refers to images derived from Doppler-measurements. Echocardiography is a frequently used method for examining the state of the heart. It is a non-invasive method and the result can be seen in real-time during the examination. A full examination often includes both M-mode and 2-D recordings, in addition to continuous, pulsed, high-pulsed and TVI [20]. The frequencies of the ultrasound used in echocardiographic examinations normally ranges between 2-5 MHz. At the examination the ultrasound probe can be placed either on the chest or it can be positioned in the oesophagus. 17

18 2.5.1 Trans-thoracic echocardiography By placing the transducer on different positions of the chest, different views of the heart can be imaged. The four standard views, so called acoustic windows, are suprasternal-, parasternal-, apical- and subcostal-view: Suprasternal Parasternal Apical Subcostal Figure 8 - Acoustic windows of the chest A rotation of the transducer results in a rotation of the scanned plane. It is up to the person performing the examination to find the appropriate angle. It is also up to the examiner to decide whether she wants to include all four chambers of the heart in the image, or whether she wants to focus on one side of the heart (resulting in a so called 2-chamber image) Transesophageal echocardiography By sending down a probe with an ultrasound transmitter in the oesophagus, it is possible to reduce some of the noise that is produced when the sound waves penetrate the chest. Furthermore, the resolution is improved because the probe is located closer to the heart. Normally, trans-thoracic echocardiography is preferred because it is easier to perform and less inconvenient for the patient. Though in certain cases the quality of the trans-thoracic image is insufficient and transesophageal echocardiography has to be used. This may be the case with, for instance, overweight patients. Transesophageal echocardiography is also used to monitor the heart during cardiac surgery. 18

19 3 PRESENTATION OF THE PROBLEM The research assignment was to develop an algorithm that can automatically find the co-ordinates of certain points in the heart wall in an ultrasound image. The points should be expressed either in screen co-ordinates (x and y referring to the position of the pixels on the screen) or in scan co-ordinates (the beam and range depth corresponding to the points, see section 3.2.1). The three points to be found were the apex and the two points in the image where the walls of the heart cross the AV plane (see section 2.1, figure 1). Figure 9 - Display of the three wanted points Initially, the task was to find the inner walls of the heart, but later it was decided that the task should be limited to finding three key points. The reason was mainly that the task of finding the entire heart wall would be too large. Having found the three points mentioned, it would, however, be easier to eventually find the rest of the heart walls. Moreover, the movement of these three points, or rather the movement of the AV plane relative apex, gives a fairly good overview of the heart function. The results obtained can therefore be used independently of a wall-finding program. 3.1 Area of interest Much can be learned if the points corresponding to the apex and the AV plane are known Pump function of the left ventricle Studying the movement of the AV plane towards the apex is interesting mainly because it has been shown that the shortening of the distance between the AV plane and the apex correlates with the left ventricle ejection fraction. The ejection fraction is a measure of the pumping function of the ventricle. It describes the quotient between the volume of the blood that is pumped into the aorta during systole and the volume of the blood that fills the ventricle just before the contraction starts. Today, the measurement of the ejection fraction 19

20 is mostly done by drawing the contours of the inner walls of the left ventricle in the end-systole respective the end-diastole by hand. This gives a highly subjective result. Two different contour drawings can result in an area difference to up to 10%. [18] In the figure below it is easy to see how the manual drawing of the contour of the left ventricle can be difficult and subjective. Figure 10 - Manual tracing of the left ventricle The measurement of the displacement of the AV plane is mostly done by using the area-length method, i.e. by hand drawing where the AV plane is in the endsystole respective the end-diastole in an M-mode image. This method is also quite subjective. By automatic location of the AV plane in the end-systole respective the enddiastole, a value of the displacement of the AV plane can be given that is objective and independent of observer. Another way of using the knowledge of the location of the apex and the AV plane is to calculate an approximation of the location of the heart wall. From this approximation the contour of the inner wall can be derived, using advanced wall-searching algorithms. This means that it would be possible to calculate the inner contour in the end-systole respective the end-diastole and to derive the ejection fraction from these results. More details on the use of finding the walls can be read in [6, 12] Strain measurements The three points (apex and the crossings between the AV plane and the heart walls) give a good estimation of the location of the heart walls. The heart walls will roughly describe a parabola connecting the three points. This estimation might be enough to enable a calculation of the strain in the heart walls, using the TVI. 20

21 Strain, ε, is a measurement of the deformation of the wall tissue and is defined as the length expansion, L 2 - L 1, of a tissue segment divided by the original length of the segment, L 1 : ε = L2 L L 1 1 (eq. 2) Strain is used to examine the moveability of the heart. For example, strain measurements are used to detect hypo- and akinesia (abnormal muscular action) after a heart infarct. The damaged tissue will still move, due to the movements of surrounding tissue, but it will no longer contract. This means that the strain for dead tissue is close to zero. v(r)=v 1 L 1 v(r+ r)= v 2 Figure 11 Illustration of the strain concept The strain is normally calculated from the TVI through the strain rate. Strain rate is a related conception defined as: r v2 v1 SR = L 1 (eq. 3) As shown in section , the strain and the strain rate are related through the following expression: t2 ε = exp( SR dt) 1 t1 (eq. 4) Measurements for calculation of strain can be done in a number of different ways. MR is one of the techniques used on humans. The advantages of measuring strain by ultrasound are that the method is both fast (real-time) and non-invasive. The major drawback is that, using the tissue Doppler technique, only velocities parallel to the ultrasound beam can be measured. Another problem is that the resolution of the TVI is low. Since the ultrasound machine has to deal with both the amplitude information and the TVI, the manufacturer of the machine has generally made the resolution worse on the TVI by 21

22 reducing the number of beams, in order to improve the resolution of the amplitude image. [7, 10] Derivation of the relation between strain and strain rate As mentioned, strain is defined as the length expansion of a tissue segment divided by the original length of the segment: ε = L2 L L 1 1 (eq. 5) By setting L 2 -L 1 = dl 1 = (v 2 v 1 )dt, and rearranging the expression above, the following result is obtained: dl1 v2 v = L L dt (eq. 6) The right hand side of the expression consists of the so-called strain rate, SR, times the time increment. Integrating both sides gives: log e L L 2 1 t 2 = SR dt t1 (eq. 7) This expression in its turn leads to the following relation between strain and strain rate: t2 ε = exp( SR dt) 1 t1 (eq. 8) 3.2 Basic conditions Although the results of this study are general and not depending on, for example, the machine used to record the data or the software used in testing the algorithms, a few things must be said about the practical conditions during the work. It is also necessary to say something about the assumptions that have been made in order to facilitate the work. The developed methods were tested on 25 sets of images clp-files The data used in the work were recorded on a Vingmed System Five machine and stored as clp-files. A clp-file contains two sorts of data: the amplitude information and the tissue velocity information. The amplitude information represents the strength of the reflection of the ultrasound wave, that is the difference in acoustic impedance at a certain depth and at a certain time. The information can be extracted from the clp-files as third order tensors of size (beams*ranges*timeframes). Beams is the number of ultrasound beams that are used in the recording. The number of ranges represents the resolution in depth and timeframes is the total number of pictures in the recording. The data 22

23 used to develop the algorithms have a resolution in depth of about 400 ranges corresponding to about 10 cm beams are used and each recording consists of time frames, which corresponds to just over two cycles of the heart. The clp-files also contain TVI-data (Tissue Velocity Information data), which is a measurement of the velocity of the heart tissue parallel to the ultrasound beams. The velocity can be measured simultaneously with the amplitude information using the Doppler effect and the result is stored in the file as complex numbers. When recording the TVI only about one fourth of the beams used to record the amplitude information are used, leading to a rather poor resolution in the beam-direction. Also, the number of ranges is less The machine used A Vingmed System Five machine was used in the recordings of the data. The machine itself is not relevant to the results of this work, however there are some things that need to be said about the filtering of the output data. The ultrasound machine has a built-in filter, which acts as a highpass filter. This affects the data so that the tissue velocities under a certain value cannot be discerned. Unfortunately this clipping of data has great consequences for the work. This will be further discussed later on. The reason for having the described highpass filter is that it reduces noise and disturbances. The value on the lowest velocity is an adjustment to how much noise that can be accepted in the ultrasound image. In the regions close to the probe, multiple reflections and strong echoes from the ribs and the outer skin-layer will disturb the TVI. Filtering the Doppler signal with a highpass filter would reduce these disturbances, since the multiple, stationary reverberations have a low velocity, i.e. a low Doppler shift. With this filter the velocity information in the areas that have a low Doppler shift, including the low velocities of certain structures in the heart, will be removed in the filtering and given a constant value Assumptions and conditions on the images Figure 12 Vingmed System Five The methods of finding the three points are not completely general. A number of assumptions are made that reduce the work substantially, but also bring along requirements on the data. The first assumption is that the images to be processed are always apical 2-chamber images of the heart. In addition to this, the methods require that both the apex and the AV plane are comprised in the image. This may seem like a matter of course, but restricting the methods to the images that fulfil the requirements makes it possible to draw some conclusions that can be used to separate the requested points from the rest of the data. For example it is reasonable to assume that apex will be located within a rather small radius from the probe. 23

24 The image quality must be reasonably good. Ultrasound images are generally quite noisy. Therefore, the methods used to find the requested points must be rather robust and tolerate a certain amount of noise. Of course, there will always be a limit on how much disturbances that can be tolerated An assumption that is not related to the quality or the extent of the image is the assumption that apex is still during the heart cycle. This is of course not exactly true, but it is a starting point and a reasonable approximation that will do for our applications. 3.3 Matlab as a development language The reason for using Matlab as a development tool is its simplicity. It is very easy to test procedures and it also has a good image processing toolbox. In the case of image processing Matlab has several built-in filters, which makes it easy to improve the pictures. The drawback of using Matlab is that it is relatively slow compared with, for instance, C++. C++ is nevertheless quite complicated compared to Matlab. One of the reasons for using Matlab is that it is easy to use its different graphical toolboxes. GcMat is an example of this GcMat GcMat is used to visualise ultrasound images. GcMat is written and developed by Vingmed. Ultrasound data can easily be treated with built-in GcMat functions. The ultrasound data is stored in clp-files when the ultrasound is registered, as described in section From these files different sorts of information (amplitude data and velocity information) can be extracted in GcMat. The extracted information can then be treated and presented as an ultrasound image would be presented. The data is, as described in section 3.1.2, of the format beams*ranges. If the data is displayed in an orthogonal coordinate system, the interpretation is difficult. If instead a built-in command that scanconverts the data is used, the data can be presented as an ordinary ultrasound image. This is illustrated below. 24

25 Figure 13 - Illustration of the data on format range*beam Figure 14 - Illustration of the scanconverted data The use of GcMat is complicated by the fact that its functions have been constructed by several different persons. As a result, some functions have similar, or the same, purpose. This is very confusing for the user, since there is a lack of documentation of the functions. Some functions with the same purpose actually return different results. 25

26 4 IMAGE PROCESSING An arbitrary image can be discretized to form a digital image. The space coordinates are discretized as well as the colour of each area element. The area elements are called pixels (short for picture element ). As a result of discretizing, an image can be seen, treated and stored as a matrix. The number of pixels that make up the image determines the resolution in a digital image. As a rule, the number of pixels along each side of the picture as well as the number of greylevels/colours in the colour scale is chosen to be integer powers of two. This is not necessary, but it is a convention that is generally accepted. Image processing on digital images is performed by letting mathematical operators work on the image matrix. In the attempt to find the three points of interest in the ultrasound images of the heart, several well-known techniques for noise reduction and image enhancement were used. This section consists of a compilation and description of the most commonly used methods, and it is divided into two parts. In the beginning of the work, much effort was focused on trying to improve the quality of the ultrasound images. The first part deals mainly with noise reduction, but it also describes a few other methods that do not change the information content of the images. These methods are only used to facilitate the viewer s interpretation of the images. The second part of this section deals with methods for enhancement of details in images. For further readings on image processing and the methods described in section 4, see [22]. 4.1 Methods for improving images Stretching Stretching is a method for improving the contrast of a picture that has a compressed colour scale. The method does not change the information in the picture, but by stretching and compressing parts of the colour scale it is possible to let the interesting details of the picture represent a wider range of the colour map. Stretching therefore improves the contrast of those parts of the picture, making it easier for the viewer to interpret the information. The implementation of the method involves mapping of the original colour map, z, onto a new colour map, z. To perform this mapping a transformation function, T, is used. z =T(z) (eq. 9) This function determines the stretching of each interval of the colour scale. The steeper the transformation function is, the larger the stretching. For a picture with a large number of dark details, a logarithmic transformation function may successfully be used. Another commonly used stretching is histogram equalisation. Below is an illustrative example of a transfer function for stretching. 26

27 Figure 15 - Transfer function for stretching Histogram equalisation Histogram equalisation is used to spread out the colours represented in a picture evenly over the colour scale. A histogram is computed, showing the number of pixels within each interval of the colour scale. Each interval of the original colour map is then mapped onto a new interval with a size that directly corresponds to the proportion of the pixels within the old interval Thresholding Sometimes it is desirable to show only the pixels with high values, taking no interest in those pixels that have lower values than a certain threshold. It is then possible to map all the pixel values above the threshold value onto one single value. This mapping renders a picture that shows all the pixel values higher than the threshold mapped onto a specific value, while the pixels with lower values are displayed in one single specific colour. The same discussion is applicable for the case where low values are of interest. Figure 16 - Transfer function for thresholding 27

28 Figure 18 Original image Figure 17 Thresholded image Median filter A median filter replaces the grey level value of a pixel with the median value of the surrounding pixels. A median filter is used to reduce noise and preserve edges. It is particularly good when the image consists of disturbances that have a limited extent and also have grey level values that differ from the surrounding pixels. These disturbances can be removed by using the median filter. The larger the filter the larger disturbances can be removed. However with larger filter, information on small details can be destroyed. The median filter is a non-linear filter. Figure 19 - Ultrasound image without filtering 28

29 Figure 20 - Median filter applied The remarkable difference in the images can more clearly be seen when looking at only one scan line, i.e. plotting the amplitude as a function of depth for one beam. Figure 21 - Amplitude information for one beam without filter and with median filter In the images it is clear how the noise is reduced Wiener filter The Wiener filter is a filter that when applied to an altered image restores it. The basic principle is that the filter uses knowledge of how the image has been changed. If this is not known, estimation has to be done. The image is restored based on the minimizing of the square of the error. The Wiener filter is a development of the inverse filter. The inverse filter is also used to restore an image, which has been modified. The process of modifying an image can be modelled as, { f ( x, )} g ( x, y) = H y (eq. 10) 29

30 where f(x,y) represents the original image and g(x,y) represents the transformed image. H represents the transformation process. The transformation process can also be expressed as a convolution. Some conditions have to be fulfilled. For instance, the transformation process, H, has to be a shift invariant linear operator. The process can then be written as: g( x, y) = H f = H ( x', y') f ( x x', y y' ) dx' dy' (eq. 11) By taking the Fourier transformation of this, the formula looks a lot easier: g ˆ( u, v) = Hˆ ( u, v) fˆ( u, v) fˆ = f ( x, y) = I When noise is present, eq. 11 then becomes: 1 gˆ( u, v) Hˆ ( u, v) gˆ Hˆ (eq. 12) g ( x, y) = H ( x, y) f ( x, y) + n( x, y) (eq. 13) The Fourier transformation of reconstructed original image the looks like: ~ f = n fˆ ˆ + Hˆ (eq. 14) Now, the function is not only singular but also when the transformation function assumes low values noise will be enhanced. To avoid these problems the Wiener filter can be used. The basic idea of this filter is to find a transfer function that when applied on the distorted image creates the best approximation of the original image. The derivation of the Wiener filter involves the concept of cross correlation. The cross correlation is a measure of the similarity between two functions and the mathematical expression for the cross correlation is: Φ fg ( x) = f *( ξ) g( x + ξ) dξ = f *( ξ x) g( x) dξ = ( f * 4 4 R R g)( x) (eq. 15) Taking the Fourier transform of this expression yields: Φˆ = fg fˆ * gˆ (eq. 16) When g=f the cross correlation is called the auto-correlation function. Another property for the correlation functions are: Φˆ = ˆ ˆ ib TΦ ii (eq. 17) 30

31 Tˆ is the wanted transfer function, which approximates the original image. Seen as a picture: i * T b The proof for eq. 17: Φ ib = i * b = ( b T) * b Φˆ ib = bˆ* Tˆ * bˆ Φ ii = i * i = ( b T) * ( b T) Φˆ ii = bˆ* Tˆ * bˆ Tˆ Q Φˆ = ˆ ˆ ib TΦ ii (eq. 18) In the case of the Wiener filter, according to eq. 13, i = H f + n. This expression combined with eq. 18 will result in an expression for the transfer function T, which shall operate on i. Φˆ Φˆ ii ib = Φˆ = Φˆ ( h f + n) b ( h f + n)( h f + n) = Hˆ * Φˆ = Hˆ 2 bb Φˆ + Φˆ bb nb + Hˆ Φˆ nb + Hˆ * Φˆ bn + Φˆ nn (eq. 19) If the noise is assumed to be uncorrelated, i.e. Φˆ ˆ bn = Φ nb = 0, eq. 19 combined with eq. 17 will result in the wanted transfer function, T. Φˆ = ˆ ˆ Hˆ * Φˆ = Tˆ( Hˆ Φˆ + Φˆ ) ib TΦ ii Tˆ = Hˆ 2 bb Hˆ * + Φˆ nn / Φˆ 2 bb bb nn (eq. 20) Below is the result of applying a Wiener filter on the same ultrasound image as figure 19: 31

32 Figure 22 - Wiener filter applied on the ultrasound image Averaging One way to reduce the noise is to take the average of a number of frames. If a large number of frames are averaged the noise will be reduced remarkably. However, segments of the image that move noticeably will be smudged over a larger area. For segments that are relatively still averaging is a good tool to reduce noise. If there is movement, some noise reduction can still be achieved by averaging images from only a few adjacent time frames. The less movement the more frames can be used in the averaging process and the noise reduction will thereby increase Total Variation based noise removal algorithms The basic idea of these kinds of image restoration methods is to minimize the noise in the image by minimizing a mathematical function under a number of constraints. The problem will thereby be reduced to finding a suitable mathematical function to minimize. If such a function can be found, the resulting optimisation problem can be solved with known methods. A noisy image, u 0 (x,y), can be thought of as the original image, u(x,y), added with additional noise, z(x,y). u = u + z 0 (eq. 21) In this case, the conditions that have to be fulfilled are: Ω u 0 dx dy = u dx dy Ω (eq. 22) 32

33 Ω 1 2 ( u u ) 2 0 dxdy = σ 2 (eq. 23) The first constraint, eq. 22, simply means that the mean value of the noise, z(x,y), computed over the entire area of the image (Ω) is zero. That is, the noise is equally spread out over the colour scale. The second constraint, eq. 23, means that the standard deviation of the noise is σ (σ > 0). The real problem is to find the function to minimize. This problem is currently an area of research, and will not be discussed in this report. For further reading on the subject, see reference [23]. It has been shown that the function u(x,y) that minimizes the magnitude of the gradient, u, represents an image in which the edges are preserved at the same time as the noise is levelled out. The function to minimize is: Ω u 2 x 2 + u dx dy y (eq. 24) To minimize this function, standard methods are used. To start with, the Euler- Lagrange equations are derived. They can then be solved numerically after discretization of the time and space co-ordinates [23]. 4.2 Methods for enhancing edges As mentioned, averaging is an efficient way to reduce the noise. However many of the details in the original image are lost in the processing. To reduce this effect of the averaging filters, highpass filtering can be used. Highpass filters are designed to enhance the edges in the image Derivative filters Averaging over pixels can be looked at as integration over the pixel values. The opposite of integrating is to differentiate. According to this line of reasoning, differentiating an image would enhance the details instead of blurring them. Derivative filters of first, second and third orders are often used in image processing to sharpen pictures. A simple kind of derivative filter for the first derivative is the Prewitt filter. There are many filters that are more advanced, both for the first order and for higher orders of derivatives, but the principle idea is the same. The derivative in one pixel is approximated from the values of the surrounding pixels. The convolution of the image and the filter is an image of the derivatives in either the x- or the y-direction. The derivative-image can then be imposed on the original image. The appearance of the Prewitt filter and the result of the application of the filter can be seen below. 33

34 Figure 23 Prewitt filters in y- respective x-direction Figure 24 - Original image Figure 25 Prewitt filter applied in the x-direction Figure 26 Prewitt filter applied in the y-direction Edges have high values of the derivatives and can easily be found by calculating the first derivative and thresholding the result. The middle of the edge, or rather the highest derivative of the edge, can be found by using the knowledge that the second derivative is there close to zero. (Keep in mind that the derivatives are only approximated, and that the second derivative therefore will not be exactly zero.) Taking the derivative over many pixels special treatment of noisy images Taking the derivative of a noisy picture enhances the noise. To avoid this the derivative can be computed over a larger interval. Taking the mean value of n pixels (n is an integer) and subtracting from that the mean value of the n following pixels will give an estimation of the derivative between the two areas. The main drawback of this method is that the larger n is, the more 34

35 details are lost in the filtration. Therefore it is important to consider the choice of n carefully for every picture. Used correctly, however, the method has great benefits. It is a stable method that does not require good quality of the pictures, and therefore it makes it possible to perform edge detection based on derivative methods on pictures that are too noisy for normal derivative filters to be used. 35

36 5 IMPLEMENTATION 5.1 Improving the images by filtering The images derived from the clp-files are images derived directly from the ultrasound machine. Although some filtering has already been done within the machine, the quality of the images is poor. Therefore some additional filtering has to be done before the actual task of finding apex and the points in the AV plane can be started. One step towards finding the three points of interest is to detect the edges as good as possible. Finding the exact position of the edges is very difficult and it is currently an area of research, but it should be possible to find at least parts of the walls that are perpendicular to the direction of propagation (which are easier to detect). Edge detection can be performed using derivative filters, but first as much noise as possible must be removed. Ultrasound images are generally very noisy. Much of the disturbances are due to so called speckle noise, which gives the images a grainy appearance. The speckle noise makes it more or less impossible to use ordinary derivative filters to detect edges since derivative filters tend to reinforce noise Reducing the speckle noise Several methods that can be used to reduce the noise in an image are presented in section 4.1. How well a method works depends on the image, or rather on the sort of noise in the image that the method is applied to. For example, median filtering is particularly useful on images that have disturbances that cause separate pixel values to deviate much from the values of the surrounding pixels. Consequently, median filtering is a method that applies well to ultrasound images. The result of using median filtering on an ultrasound image is shown in section 4.1, figure 19 and 20, where the results of Wiener filtering can also be seen (figure 22). The clp-files contain sets of images, each consisting of images recorded during only a few seconds. This means that the time interval between two images is small. Averaging over a couple of time frames should then be possible to perform without losing too much detail due to the movement of the heart. Averaging over a number of following images would reduce the uncorrelated noise, but it would not remove the speckle noise since it is timedependent and does not change much on the time between two following images. The problem with time-dependent noise can be avoided if the averaging is done over images from different heart cycles, but from the same moment in the cycle. Using electrocardiograms, images that correspond to the same position in the heart cycle can be extracted. After averaging over only three images, corresponding to the same position in the heart cycle, the improvement is very good. Of course, the averaging of images from different 36

37 heart cycles requires that the transducer is held relatively still during the examination. A drawback with this method is that the quality of the ECG output is often poor and it is therefore difficult to extract the corresponding frames in different heart cycles from the ECG information. One of the easiest points in the heart cycle to recognise from the ECG-curve is the R-peak of the electrocardiogram. This peak represents the beginning of the systolic phase of the ventricles. The images below show the difference between an ordinary image and an image, which is an average between three images from the same instant in the heart cycle. Figure 27 - Ordinary image with a large amount of speckle noise 37

38 Figure 28 - Averaged image with reduced speckle noise A more advanced noise removing method is the total variation (TV) method described in section The implemented code has been tried on both ultrasound images and a test-image (picturing black rectangles on a white background) to which noise was added. The result was not to full satisfaction. The method is implemented as an iterating method with some input parameters such as, for instance, the value of the discretization. The method made the images better for a limited number of iterations, but it turned out to be more or less impossible to choose the values of the variables in a way that made the method converge. In addition to the fact that the method is unstable, it also has the drawback that it is slow and requires a lot of computation. There are simpler methods that give better results and therefore the TV method is not used in the final algorithms for finding the requested points Finding edges in an image In finding the inner and outer heart wall in an ultrasound image, basic derivative filters, like for instance the Prewitt filter, are not that useful. This is because of the noise present in the image. In spite of the noise-reducing filtering, there is still a substantial amount of noise left in the image in the form of random intensity variations between adjacent pixels. In an image with high resolution (many pixels), local variance in the intensity can efficiently be reduced without too much loss of sharpness by averaging the intensity over an area of n*m pixels (choose n, m small compared to the total amount of pixels in the image). In ultrasound images the spatial resolution is low perpendicular to the beams, since it is limited by the number of transducers in the ultrasound transmitter. Normally the number of probes used to record ultrasound images of the heart lies between 35 and 40. Averaging over pixels in that direction could destroy important information. 38

39 The method of calculating the derivative over a number of pixels (see section ) reduces the impact of noise that cannot be completely removed. The method is very stable and has proved to be highly useful in the processing of ultrasound images. Where nothing else is implied, this is the method used to calculate derivatives in the developed algorithms. Derivative filters can be constructed to calculate the derivative in an arbitrary direction. In edge detection it is often not the direction of the derivative in a point that is interesting, but the absolute value of the gradient. Therefore it is often desirable to use a rotation invariant derivative filter. This is difficult to design for ultrasound images, partly because the spatial resolution (i. e. the discretization) is different in different directions, partly because the coordinate system of the incoming data is not orthogonal. This is the reason why rotation invariant derivative filters are not used in this work. Instead the results from derivative filters in beam- and range-direction (alternatively x- and y- direction) are added. Another problem is the detecting and depicting of edges parallel to the ultrasound beams. These edges do not reflect much of the ultrasound in the direction of the transducer and as a result, strong variations in acoustic impedance does not automatically result in high intensity in the amplitude image. 5.2 Algorithm for finding the apex As mentioned above, some requirements have to be made on the images in which the apex is supposed to be located. First of all the images have to be apical. Also, the apex has to be present in the images. After establishing that the apex is relatively still during the heart cycle, timeaveraging methods can be used. The information that can be used in finding apex is thus: Apex is relatively still during the heart cycle The mean velocity is low for apex Apex should lie on an edge in the vertical direction The first step is to apply a derivative over multiple pixels in a vertical direction. The theory for this is described in section The strongest positive values of this derivative are extracted. This is then done for all the available time frames and averaged. Every pixel then has a value between 0 and 256. If a pixel has a value of 256, it means that the pixel lies on a strong edge, which does not move, in every time frame. The averaging process leads to a reduction of noise. This is also described above in section The next step is to extract pixels that have a high derivative value and that, at the same time, have a mean velocity close to zero. A low value of the mean velocity means that the area is quite still during the heart cycle. When pixels that correspond to an edge and that have a low velocity have been extracted, it turns out that the major part of these pixels lies in the apex area. 39

40 Some pixels that do not coincide with the apex also occur, probably due to disturbances and artefacts. To reduce the candidate points to only one point, the median value of the indexes are taken in both the range direction and the beam direction. Taking the median value instead of the mean value reduces the effect of the pixels coming from the noise. Below it is illustrated how the pixels where the apex is supposed to be located are reduced from including all the pixels to only one pixel. Since the apex is supposed to lie in the same position during the whole heart cycle, the images below are in one time frame, but the same kinds of illustrations can be displayed for any time frame. Figure 29 - Step 1: Taking the derivative of the image leads to reduction of points where the apex is supposed to lie 40

41 Figure 30 - Step 2: Summation of derivative images over time Figure 31 - Step 3: Extracting the pixels with low velocities After these steps, the images from step 2 and step 3 are summarised and the resulting picture is thresholded. The remaining pixels are overlaid on the normal image and results in the following image: 41

42 Figure 32 - Step 4: Thresholding the summated image Figure 33 - Step 5: Taking the median value of the candidate points of the image in step Results and reliability The apex is located in the correct area in more or less all ultrasound images. However, in one of 25 images the location does not coincide with the right area. In this image the stationary reverberations are quite large and affects the outcome. On the whole, it can be said that stationary reverberations will disturb the algorithm, since the algorithm is, among other things, based on the relative stationarity of the apex. 42

43 5.2.2 Comments on the source file The algorithm for finding the apex is implemented in the file findapex2.m. There are several parameters that can be altered in the algorithm for finding the apex. Of course, if the values are altered the outcome will also change. The parameters and their current values are: derpictures: This variable is a three dimensional tensor, containing matrices, where each matrix in turn is a result of the application of applying a derivative filter on the original image with function der.m. In this step the threshold parameter can be determined. Since the heart wall in the apex area is not always distinct, a low value on this threshold parameter has been chosen so that the edge will not be filtered out. The value on the threshold parameter has also been chosen to be positive since the slope is known to be positive, i.e. the apex is located in an area where the pixel values are increasing with depth. If a higher value on this parameter is chosen there is a possibility that the area where the apex is located will be filtered away and thus an incorrect location, or possibly no location, will be found. If a lower value on the parameter is chosen the accuracy will be lower. The current value on the threshold parameter is 0,03. factor: This variable is a number that determines the thresholding of the TVI. This is the number sent to the function, tvitest.m (see Appendix 1). The TVI is quantified, described in section 3.2.2, and the lowest velocities are set to a constant value. factor=3 means that tissues moving with velocities up to three times this value are extracted. Consequently, this value allows the apex to have a small movement, but yet be in a relatively fixed position during the entire heart cycle, and not be filtered away. If the value on factor is chosen to be lower, there is a possibility that areas with small movements are filtered away. On the other hand, if the value on factor is chosen higher the accuracy will be less since a larger area where the apex can be located will result. The functions are now quite slow and time consuming. The main reason for this is that it takes time to take the derivative of all the images in the heart cycles. One way to speed things up is to reduce the number of frames used and only take the derivative of, for instance, every tenth image or perhaps even fewer. This will however reduce the accuracy in the determination of the location of the apex. Still, the accuracy will probably be good enough since the assumption that the apex is relatively still is in turn an approximation. Another way to speed up the calculations could be to first locate the area where the apex is located and then take the derivative of only this area instead of the whole image. 5.3 Algorithm for finding the atrioventricular plane When locating the AV plane in every time frame, it is important to start out from a frame where the AV plane is trustworthy located. This is done in multiple steps: To make use of the tissue velocity information, determine a place in the heart cycle where the AV plane has a high velocity. This place could 43

44 for instance be represented by a time frame corresponding to one fourth of a heart cycle, starting at the R-peak, i.e. a little way into the systolic phase of the ventricles. In this position, the AV plane has a high velocity and the valve between the ventricle and atrium is closed. Extract images that correspond to the chosen point in the heart cycle with help of ECG information. Take the derivative in the vertical direction of the extracted images. (Described in section ) Extract only the strongest derivatives and form thresholded images. An overlap of the derivative images results in pixels that are candidates to the location of the atrioventricular plane. Exclude the areas corresponding to the blood flow using the velocity information as described in section Extract tissue with high velocities (including the AV plane) by thresholding the tissue velocity information. This results in an image with selected points where the AV plane is supposed to lie. Combine the information from the derivative and the velocity information. This will reduce the allowed pixels further, resulting in a few candidate points. Taking the median value of these points along each beam results in a maximum of one pixel along each beam. The median value is used to reduce the effect of disturbances. Since a curve is needed, the given points have to be extra- and interpolated, demanding that the curve does not make any sudden shifts, so that every beam has a pixel value where the AV plane is located. When the AV plane has been found in one image, the information of the location of this curve is used for calculation on where the AV plane is located in adjacent frames. By predicting the location of the AV plane and combining this information with the derivative of the image it will be possible to reduce the allowed pixels where the AV plane could be. A curve is then achieved in the same way as the curve in the first reference frame. The prediction on the location of the AV plane in the next frame is done by using the ECG information to determine in which direction the AV plane moves. Below is an illustration of the sequence of images leading to a curve representing the AV plane. 44

45 Figure 34 - Step 1: Extracting the derivatives Figure 35 - Step 2: Combine derivatives and TVI 45

46 Figure 36 - Step 3: Taking the median values of the candidate points Figure 37 Step 4: Extrapolation, interpolation and adjusting the curve representing the AV plane Results and reliability The determination of the location of the AV plane in a frame where the velocity is high is stable. The AV plane is found in all the ultrasound images the function has been tested on. However, on a small amount of the images, the curve approximating the AV plane follows the inner wall upwards towards 46

47 the apex instead of crossing the heart wall in the same height as the AV plane is located. The method for tracking the AV plane has proven to be less reliable. The curve approximating the AV plane is not always exactly where the AV plane is. Yet this method works fairly well since it never gets off track completely. Still it cannot be used without any refinements. One reason for the lack of stability is probably that the location of the AV plane is difficult to find when the valve is open; since the method is based on horizontal edge finding and when the valve is open there is no distinct line to be found where the AV plane is supposed to be located Comments on the source file As for the location of the apex, there are values on several parameters that can be chosen in the source file testavplane.m. factor: Just like the algorithm for finding the apex the function der.m is used. This time the value on factor can be chosen higher since there always is a distinct horizontal edge where the AV plane is located, at least for the frame chosen where the velocity is high and the valve is closed. The value on factor, i.e. the threshold value, is chosen to 0,2. If a higher value is chosen there is a risk that there will not be enough points represented to get an accurate line representing the AV plane. For the tracking of the AV plane during the heart cycle a value on factor of 0,1 is chosen since the AV plane is not distinct in all phases of the heart cycle since the valve is open in some points. fac: This variable is a measure on how many percent of the pixels of the extracted derivative points that are selected. In this case, 30% of the derivatives with the highest values are extracted. If a lower value is chosen, the number of pixels selected will not be many enough and thereby will not result in a curve representing the AV plane. If a higher value is chosen the accuracy will be reduced and the location of the AV plane will be imprecise. s: This parameter describes how smooth the curve representing the AV plane shall be. s is a measure of how many known points that are taken into account when determining the value in a certain point. If s is large, jumps can occur, thus allowing the valve to be open. If a lower value is chosen, there is a risk that extreme values will remain. The current value of s is 3. The speed of the function locating the AV plane in one frame is quite fast, but one possible speed-up could be to determine the area where the AV plane is located before taking the derivative. The algorithm for tracking the AV plane in the entire heart cycle is relatively slow, but since there are many frames present, not much can be done to reduce the required amount of time. To improve the quality of the algorithm for finding the AV plane and tracking it, the concept of strain can be used to give the extra information needed. The strain is highest in the heart wall where the AV plane crosses. 47

48 5.4 Algorithm for finding the crossings between the AV plane and the heart walls Having found the AV plane, it is desirable to find the two points, called the AV points, that correspond to the crossings between the heart walls and the AV plane. If the walls of the heart could be found in a way similar to the way the AV plane is found, it would be an easy task to find the crossings. The problem is that the walls are parallel to the ultrasound beams, and as a consequence they do not always return a strong echo to the probe. As a result, it is very difficult to find more than parts of the walls. In detecting the walls, both the amplitude information and the Tissue Velocity Information of the heart can be used. A number of techniques for finding and enhancing the walls in an image are described below. Subtraction of the derivative in the beam direction A well-known technique for enhancing edges in an image is derivative filtering. Taking the derivative in the beam direction of the ultrasound image gives an image with high values along the inner and outer edges of the vertical heart wall and very low values (close to zero) in the middle of the wall. Subtracting the derivative from the amplitude image therefore enhances the heart wall in the image. Figure 38 - Derivative in the beam direction Subtraction of the derivative in the range direction The AV plane often returns strong reflections, which are confusing when localising the walls. If the reflections from the AV plane can be removed in the amplitude image, it will lessen the risk of finding the walls where the reflections from the AV plane are strong. The derivative taken in the vertical direction has high values at the AV plane. Subtracting the derivative from the amplitude image will decrease the intensity at the AV plane in the amplitude image. 48

49 Enhancing the strongest echoes The heart walls in an apical image should theoretically be easy to find by extracting the two strongest peaks in the amplitude at every depth of the image. Because of noise and other disturbances that arise from the tissues surrounding the heart and from the technical equipment used in the examination, this cannot be used as a reliable method for finding the walls. However, it often gives a good estimate of the location of the heart walls, especially if the result is thresholded. Thus, only the strong peaks in the amplitude image are used, and the weak ones, that are more likely to be due to noise, are removed. Figure 39 - Extraction of the two strongest peaks in the range direction Tissue Velocity Information The TVI returns a matrix containing the velocities measured in the heart tissue. The blood in the left ventricle generally has a high mean velocity compared to the velocity of the heart walls. The blood velocities are removed by highpass filtering and areas in the image corresponding to blood therefore have low velocities in the matrix containing the TVI. The AV plane, on the other hand, often has high velocities in the TVI. The TVI can thereby be used in different ways. By removing the areas with low TVI from an amplitude image, the interface between tissue and blood should be enhanced. By removing the areas with high TVI from the amplitude image it should be possible to reduce the effect of the AV plane when trying to find the walls. 49

50 Figure 40 - The Tissue Velocity Information of the heart Using the methods described above, it should be possible to find a curve that approximates the wall, but to find the AV points it is not necessary to trace the entire wall. Theoretically, the two points should be possible to find by extracting the two strongest peaks in the amplitude information just over the AV plane. This can be done in the untreated ultrasound image or in an amplitude image in which the walls have been enhanced in order to improve the stability of the method. In Appendix 1 (avpoints.m), the Matlab code for both alternatives is found. The Matlab code that renders the weighted image is found in weightimage.m. The methods used to render the weighted image are; subtraction of the derivative in the beam direction, addition of the strongest peaks and finally subtraction of the TVI in order to reduce the reflections from the AV plane. The original image and the weighted image are displayed below. 50