King scallop larva (190 mum.)
Segmented image (discussed below)
Machine Vision for Identification of Shellfish Larvae |
Work carried out by: Jonathan Campbell, John Slater, John Gillespie, Ivan Bendezu, (all Letterkenny Institute of Technology), and Fionn Murtagh (Royal Holloway, University of London). |
Summary |
| The collection of wild larvae seed as a source of raw material is a major subindustry of shellfish aquaculture. To predict when, where and in what quantities wild seed will be available, it is necessary to track the appearance and growth of planktonic larvae. One of the most difficult groups to identify, particularly at the species level are the Bivalvia. This difficulty arises from the fact that fundamentally all bivalve larvae have a similar shape and colour. Identification based on gross morphological appearance is limited by the time-consuming nature of the microscopic examination and by the limited availability of expertise in this field. Molecular and immunological methods are also being studied. We describe the application of computational pattern recognition methods to the automated identification and size analysis of scallop larvae. For identification, the shape features used are binary invariant moments; that is, the features are invariant to shift (position within the image), scale (induced either by growth or differential image magnification) and rotation. Images of a sample of scallop and non-scallop larvae covering a range of maturities have been analysed. Scatter plots of selected features indicate the feasibility of automatic identification. |
Background and Objective |
|
Diminishing fishery stocks, increased fishing pressure, and the ever present threat of pollution mean that the seas can no longer be regarded as a limitless food source. Aquaculture production is expanding rapidly as the way forward for the marine food sector to develop a sustainable industry for present and future generations. Production valued in excess of 114 million Euro occurred in 2002 and is projected to increase to 97,023 tonnes valued at over 175 million Euro by 2008. Currently approximately 60% of aquaculture volume comprises shellfish with much of this production, dependent on the collection of wild seed as a source of raw material. To predict when, where and in what quantities wild seed will be available, knowledge of the adult shellfish spawning pattern and the dispersal of planktonic bivalve larvae in the waters around our coast are required. Whilst large numbers of plankton samples are relatively easy to collect and good taxonomic keys exist for many of the specimens in such samples, one of the most difficult groups to identify, particularly at the species level are the Bivalvia. This difficulty arises from the fact that fundamentally all bivalve larvae have a similar shape, they are all brownish in colour to one degree or another, and there are no protruding parts or appendages, which can be used to aid identification. |
Related Work --- Manual Techniques |
|
Molecular methods based on antibody and oligonucleotide markers have been developed (Andre et al), (Paugham et al), (Frischer et al), and (Hare et al}. The application of an immunological method of larvae identification \emph{in situ} has recently been reported (Paugham et al, 2003}. |
Related Work --- Computational Pattern Recognition Techniques |
|
A useful survey covering the full spectrum of methods is given in (Garland and Zimmer). The paper clearly identifies the labour intensity of the problem, especially considering the large number of samples that could be generated by the necessary field surveys. The candidate methods surveyed are: (a) morphological (based on microscopy); (b) optical, e.g. fluoroscopy, but it is concluded that this is unlikely to provide species discrimination, only class discrimination; (c) molecular (immunological and DNA-based). In the context of (a) (morphological), scanning electron microscope (SEM) and optical microscope methods are mentioned. The use of simple shape features based on area and bounding rectangle dimensions is reported in (dePontual et al); however, this study seems to have been limited by the availability of methods offered by the image processing package used, and the article admits difficulties in handling the wide range of species that may be encountered. Complex texture-based image processing of polarization microscopy images is described in (Tiwari and Gallager), i.e. in contrast to (de Pontual et al) which considers only the shape outline/silhouette, this paper considers the texture (colour patterns) that show up in the (polarized) images. |
Methods |
|
A chief aim of the methods described here is to avoid the need for exotic and expensive sensing, i.e. to perform automatic image analysis of plain optical microscopy images as shown below. |
|
King scallop larva (190 mum.) |
Segmented image (discussed below) |
|
The stages of the identification system are shown below. |
Identification system block diagram |
|
We start with a raw image such as that shown above; of course, a microscope image of a plankton sample is much larger and contains a great many larvae, perhaps representing a range of species, and other matter, see for example: example 1, example 2, example 3. |
|
The first stage depends on what is to follow but a typical requirement is to segment, i.e. separate the object from the background. There are many candidate segmentation algorithms (Pal and Pal), many of them based on statistical clustering. Above we show the result of segmenting using a simple threshold technique in which the threshold is based on the image histogram (Gonzalez and Woods). This segmentation is relatively trivial. However, in other cases, the translucency of the objects is more pronounced, i.e. some parts of the object are actually less optically dense than the background, thereby defeating simple thresholding. We are currently studying methods based on Markov random field models (Solberg), (Murtagh, Barreto and Marcello) which combine spatial contiguity with colour/grey-level similarity. This matter is discussed in much greater detail in a later section on proposed future work. In the segmented image, each object is reduced to a planar shape from which shape features can be extracted. Since objects can appear anywhere, and at any orientation, we require shift- and rotation-invariant features. Invariant moments (Hu), (Nixon and Aguado), (Masters) are ideal candidates. Invariant moments (of a binarised object), ignore object texture, i.e. in contrast to (Tiwari and Gallager) discussed above. We believe that this is well founded, since manual inspection suggests that discrimination should be possible based on outline shape alone. Following feature extraction, feature vectors are passed to a classification algorithm (Therrien). |
Compendium of raw images used in the pilot study (.S in the name indicates scallop). |
Compendium of segmented images (.S in the name indicates scallop). |
|
When the first seven invariant moments (Masters) are extracted and normalised (see separate note on moments), we arrive at the following table.
Feature h1 h2 h3 h4 h5 h6 h7
Image
1 -8.6940 -7.9026 -12.2217 -4.9005 0.1141 -0.1980 0.0804
2 -7.9710 -5.4640 -11.8688 -3.7754 -0.2349 -0.7533 -0.0871
3 -8.4007 -6.8575 -11.5683 -6.0194 -0.0865 -0.3836 0.0658
4 -9.1047 -10.4998 -12.4660 -7.8341 -0.0486 -0.2176 0.0245
5 -9.3712 -13.8080 -12.8727 -9.2301 -0.0002 -0.0028 -0.0000
6 -9.0280 -9.5743 -12.4695 -8.1891 0.0332 -0.0844 0.0218
*7 -8.7786 -8.2680 -12.0012 -6.3576 -0.0683 -0.1458 -0.0551
8 -9.0596 -10.4306 -12.3343 -6.8460 -0.0597 -0.2134 0.0425
9 -9.1003 -9.2379 -12.8374 -8.7028 0.0197 0.1318 0.0068
10 -8.8725 -9.5835 -12.0148 -5.5173 -0.1274 -0.3880 0.0646
11 -8.9225 -8.3671 -12.7163 -8.1694 -0.0385 -0.2324 -0.0121
*12 -8.7167 -7.6808 -12.1323 -6.4967 0.1003 0.3857 0.0381
13 -8.4861 -6.3422 -12.8731 -9.0545 -0.0051 -0.0873 0.0034
*14 -8.8416 -8.2991 -12.1866 -6.8248 0.0825 0.3249 0.0398
15 -8.5474 -6.4951 -12.8462 -8.8891 -0.0053 0.0707 -0.0075
*16 -8.8622 -9.1319 -11.6367 -6.7841 0.1023 0.3345 0.0218
17 -8.5396 -7.8825 -11.4335 -3.9287 -0.2168 -0.5815 0.1113
*18 -8.8719 -8.6952 -11.9684 -6.5917 0.1032 0.3574 -0.0280
19 -8.2990 -6.0410 -12.2315 -5.7875 0.1053 0.3277 0.0590
20 -8.9878 -8.9661 -12.7607 -8.1713 -0.0348 -0.2139 -0.0149
Table: Invariant Moment Features (* signifies scallop)
Finally, a features h1 and h4 are displayed on a scatter plot. |
Scatter plot of h1, h4. Points 7, 12, 14, 16 and 18 are scallops. |
Publications |
Two poster publications (PowerPoint) are available: Jonathan Campbell, John Slater, John Gillespie, Ivan Bendezu, and Fionn Murtagh. Pattern recognition methods for the identification of bivalve larvae. Poster Presented at Bionet03, Galway, December 2003, and poster presented at TecNet 2004 conference. |
Conclusions |
We have described the use invariant moment shape features for the discrimination of scallop larvae in microscope images. Images of a sample of scallop and non-scallop larvae covering a range of maturities have been analysed. Based on a limited data set, the distribution of selected features suggest the feasibility of automated identification. Further work on the project will focus on: (a) collection of a more extensive data set; (b) implementation and evaluation of classification techniques; (c) development of a robust segmentation technique. |
Appendix --- Proposed Future Work | ||
Abstract | ||
|
The procurement of shellfish seed from natural settlements of wild larvae provides the foundation for mussel, oyster and scallop culture industries in Ireland. To predict when, where and in what quantities wild seed will be available, it is necessary to track the appearance and growth of planktonic larvae of these species. Identification using manual inspection is limited by the time-consuming nature of the microscopic examination and by the limited availability of expertise. Results from preliminary investigations by the proposers into automatic identification using classical computational pattern recognition methods applied to digitised microscope images have been published. Subsequent work has identified two significant topics for further study: (i) image segmentation methods since some translucent larvae are difficult to separate from their background using purely density-based segmentation methods; (ii) further development of shape recognition methods, especially the recognition of translucent objects in the presence of occlusions. The major research outcomes will be advancement of two areas of pattern recognition: (i) segmentation including spatial model based methods; (ii) shape recognition methods, including occlusion handling. In addition, the work will make a significant advancement to the understanding of the applicability of machine vision methods to image analysis problems in marine biology and related areas. | ||
Objectives | ||
|
(a) Development of spatially-oriented image segmentation methods based on: (i) active contours (snakes); (ii) Markov random field (MRF) models; (iii) other spatial models (priors); (iv) appropriate choice of edge detector; (v) level-set methods; (b) Further development of planar shape recognition methods for separated objects, for example, Fourier transform contour methods; (c) Development of shape recognition methods to handle occlusion (perhaps only minor occlusions or those where translucence of the object means that certain shape cues are retained). | ||
Detailed Description of Work | ||
|
The production of shellfish by aquaculture in suspended culture, for example mussel culture in Mulroy Bay; or by fishery enhancement involving the release into prepared seabed plots, for example bottom mussels, oysters and scallops in Lough Foyle, Lough Swilly and Mulroy Bay; requires a large, consistent and reliable supply of juvenile shellfish or spat. Researchers in the School of Science at LYIT, with funding from the Higher Education Authority, are presently comparing morphological methods for the identification of bivalve larvae in the plankton with methods based on the DNA sequence of shellfish larvae involving real-time PCR. The objective of these investigations is the development of a technique for the identification of larvae of commercial bivalve shellfish that will allow larval tracking in the plankton and thus prediction of the time and place of larval settlement. Further proposals have been submitted for funding for a complementary investigation involving the development of a multiplex technique based on fluorescent in-situ hybridization of bivalve larvae, in which scallop larvae will be made to fluoresce one colour and mussel larvae another colour to aid in their identification. The identification of bivalve larvae using morphological techniques is currently being performed on several projects funded by the Marine Institute in conjunction with University College Cork, University College Dublin, Queen's University Belfast and the University of Wales Bangor. Further expansion in the application of these manual identification methods using gross morphological appearance is limited by the time-consuming nature of the microscopic examination and by the limited availability of taxonomic expertise in marine bivalve larvae. Initial attempts at automatic identification using classical planar shape recognition methods applied to digitised microscope images (Campbell et al, 2003) and subsequent additional work, as yet unpublished, has provided clear indications of the feasibility of such an approach. The stages of the identification system are as follows: (i) segmentation; (ii) extraction of invariant shape description features; (iii) classification based on the features. The first stage depends on what is to follow but a typical requirement is to segment, i.e. separate the objects (larvae) from the background. There is a wide range of candidate segmentation algorithms, many of them based on statistical clustering. After segmentation, each object is reduced to a planar shape from which shape features can be extracted. Since objects can appear anywhere, and at any orientation, shift- and rotation-invariant features are required (Hu, 1962). An alternative solution to invariant moments are features based on the object boundary which can be obtained either by traditional gradient-based edge detection or by image morphology. Based on the object boundary/contour, Fourier descriptors (features) for example can be used to summarise the boundary shape. Finally, feature vectors are submitted to a classification algorithm. The prior research study identified three topics necessitating further work: (i) image segmentation where object grey-level/colour overlaps significantly with that of the background; (ii) enhanced recognition methods for subtly differing planar shapes; (iii) shape recognition in the presence of partial occlusion. This research project will address each of these topics as detailed below. Work on segmentation will focus on spatially-oriented methods based on: (i) active contours (snakes) ( Kass, Witkin, Terzopulos, 1988), including gradient vector flow methods (Xu and Prince, 1998) and level set methods (Chan and Vese, 2001; Malladi, Sethian and Vemuri, 1995); (ii) Markov random field (MRF) models (Geman and Geman, 1984; Besag, 1986); (iii) other spatial model based methods (Campbell et al, 1997). Work on shape recognition methods for separated objects will focus on Fourier transforms of contours, polynomial and spline based methods, eigenimage methods (Turk and Pentland, 1991; Jeevakumar et al, 1995); Work on shape recognition in the presence of occlusion (Mardia et al, 1997) will be performed. Most of the literature on shape recognition/tracking in the presence of occlusion refers to moving images (video applications); ironically, the static image case is more difficult because the object cannot be tracked from a separated state into the occluded state. It is believed that, due to the translucence of the objects (so that certain shape cues are retained), the occlusion problem in this case may be susceptible to novel solutions. Moreover, it may be possible to specify priors based on parameterisations of expected shapes. | ||
References | ||
|
Besag, 1986, On the Statistical Analysis of Dirty Pictures, J.R. Statist. Soc. B, 48, 259-302. J. Campbell, J. Slater, J. Gillespie, I. Bendezu, and F. Murtagh, 2003, Pattern recognition methods for the identification of bivalve larvae. Poster Presented at Bionet03, Galway, December 2003. J.G. Campbell, C. Frawley, D. Stanford, F. Murtagh, and A. E. Raftery. Linear Flaw Detection in Woven Textiles using Model-Based Clustering. International Journal of Imaging Systems and Technology, 10:339-346, 1999. J. Campbell, F. Murtagh, and Munevver Kokuer, 2001, DataLab-J: a signal and image processing laboratory for teaching and research. IEEE Trans. Education, 44:329--335. T.F. Chan and L.A. Vese, Active Contours without Edges, 2001, IEEE Trans. Image Processing, 10:2, 266-277. S. Geman and D. Geman, 1984, Stochastic Relaxation, Gibbs Distribution and Bayesian Restoration of Images, IEEE Trans. PAMI, 6, 721-741. K. Jeevakumar, J.G. Campbell, M. McGinnity, and W. Tschiesche. The Use of the Eigenface Method in Face Recognition. In Proceedings of IDSPCC '95, 6th Irish DSP and Control Colloquium, Queen's University Belfast, June 19-20, pages 109-116, Belfast, N. Ireland, 1995. M.Kass, A. Witkin and D. Terzopulos, 1988, Snakes: Active Contour Models, Int. J. Comput. Vis., 1, 321-331. R. Malladi, J.A. Sethian and B.C. Vemuri, 1996, Shape Modelling with Front Propagation: A Level Set Approach, IEEE Trans. PAMI, 17, 158-175. K.V. Mardia, et al, 1997, Deformable Template Redognition of Multiple Occluded Objects, IEEE Trans. PAMI, 19:9, 1035-1042. M.Turk and A. Pentland, 1991, Eigenfaces for Recognition, J. Cog. Neur. 3:1, 71-86. C. Xu and J.L. Prince, 1998, Snakes, Shapes and Gradient Vector Flow, IEEE Trans. Image Processing, 10:2, 266-277.
|
