HA NOI UNIVERSITY OF SCIENCE AND TECHNOLOGY SCHOOL OF ELECTRICAL & ELECTRONIC ENGINEERING MASTER THESIS Human chromosome classification using convolutional neutral network QUÁN THỊ YẾN yen.qt20212483M@sis.hust.edu.vn Advanced Program in Biomedical Engineering Instructor: PhD Trần Anh Vũ School: Electrical & Electronic Engineering HA NOI, 7th 2023 Instructor's signature Ha Noi, 8th 2023 SOCIALIST REPUBLIC OF VIET NAM Independence- Freedom- Happiness VERIFICATION OF THE MASTER THESIS The full name of the author: Quán Thị Yến Thesis topic: Human chromosome classification using convolutional neutral network Majority: Biomedical Engineering The student code: 20212483M The Instructor and the chairman of committee verify that the author has corrected and supplemented the thesis according to the minutes of the meeting committee with the following contents: Add the ways to classify chromosomes Correct explanatory words in the thesis Correct the names of tables and pictures according to the rules of writing a thesis Remove irrelevant images Move the models of EfficientNet, Denset and Inception networks to the Chapter 2: Methodology Describe in detail macro-average precision, macro-average recall and macroaverage F1-score and accuracy to evaluate the performance of classifier , 2023 The Author The Instructor CHAIRMAN OF THE COMMITTEE THESIS TOPIC Human chromosome classification using convolutional neutral network Instructor Sign and write full name Acknowledgements For the completion of this thesis, I would like to express my sincere gratitude to PhD Tran Anh, Lecturer at School of Electrical and Electronic Engineering, Hanoi University of Science and Technology because he guided and facilitated me during the implementation process His enthusiastic teachings have helped me a lot in the process of carrying out this research Thank you to Mr Tuan, a former research student at room 418, C9 building, Hanoi University of Science and Technology, for supporting me in my thesis work I sincerely thanks! Abstract Most cells are composed of 23 chromosomes The first 22 pairs are called autosomes The 23rd pair contains the sex chromosomes: males typically have one X and one Y chromosome in each cell, while females typically have two X chromosomes The chromosome contains all information that the body needs to grow and develop Some genetic diseases are associated with chromosomal abnormalities Diagnosis of these abnormalities has gained attention in recent years One of the most popular and useful ways to solve this problem is based on Karyotyping Karyotyping is a laboratory procedure that allows doctors to examine the set of chromosomes Karyotyping tests are performed to determine if the cell's chromosomes are normal Therefore, it plays an important role in the diagnosis of genetic disorders In fact, Karyotyping requires considerable manual efforts, domain expertise and experience, and is very time- consuming I made a thesis research with this topic to apply what I learned at school and self-study about the application of convolutional neural networks and many methods in the preprocessing step to be able to self-classify chromosome Thereby, as a basis to replace Karyotyping method, helping doctors quickly identify abnormal chromosomes At last, I have achieved the best results with the EfficientNet-b3 model in the series (Accuracy: 97.12%) of CNN models I put in to try For many reasons, in the process of implementing the thesis, I still made many mistakes and limitations Therefore, I look forward to receiving your comments and evaluations for the topic to be developed and improved Sincerely thank! STUDENT Sign and write full name CONTENTS CHAP INTRODUCTION 1.1 Chromosome 1.2 Chromosome classification 1.3 Methods to identify mutated chromosomes in real life 1.3.1 Definition 1.3.2 Performing test 1.3.3 Why chromosomal testing is required? 1.3.4 Using AI to replace Karyotyping method 1.4 Related works CHAP METHODOLOGY 2.1 Dataset Description 2.2 Data processing 12 2.2.1 Increase brightness and contrast 12 2.2.2 Data Augmentation 13 2.2.3 Crop chromosome image 17 2.2.4 Normalization 18 2.2.4.1 Resize 18 2.2.4.2 Zero-padding 18 2.2.4.3 Adaptive Resizing 19 2.3 Classification 20 2.3.1 Using CNN network for chromosome classification 20 2.3.2 CNN networks 20 2.3.2.1 DenseNet 21 2.3.2.2 EfficientNet 24 2.3.2.3 InceptionResnet 27 CHAP EXPERIMENT 30 3.1 Application in Passau dataset 30 3.1.1 Data- Processing 30 3.1.2 Chromosome Classification 34 3.1.3 Result 34 CONCLUSION 40 REFERENCES 41 SUMMARY OF THE MASTER'S THESIS 44 a) Reason of choosing the topic 44 b) Purpose, Research Object, Scope of Research 44 c) Content Summary and Author’s Contribution 45 d) Research method 46 e) Conclusion 46 LIST OF TABLES Table 3.1 The effect of pre-processing stage on performance 35 Table 3.2 Experiment results between CNN models when trained with input images of size 256 x 256 36 Table 3.3 Performance of EfficientNet B3 with resize methods using image size 256 x 256 36 Table 3.4 Performance of EffcientNet-b3 using adaptive resizing (k=1) with different input image sizes 37 Table 3.5 The effect of augmented factor in adaptive resize method 37 Table 3.6 Comparison to other approachers 38 LIST OF FIGURES Figure 1.1 Diagram of a replicated and condensed metaphase eukaryotic chromosome:(1) Chromatid, (2) Centromere, (3) Short arm, (4) Long arm Figure 1.2 A way to classify chromosome types Figure 1.3 Down syndrome is a chromosomal disorder caused by an extra chromosome 21 Figure 1.4 Klinefelter syndrome (KS) with one copy of the Y chromosome (47,XXY) Figure 1.5 Children diagnosed with Turner syndrome Figure 1.6 The process of using AI to separate and classify chromosomes Figure 1.7 Result after AI processing Figure 2.1 The proposed general model Figure 2.2 Chromosome Image Data from University of Passau 10 Figure 2.3 The principal image of chromosome from Passau University 10 Figure 2.4 The Image of chromosome after labeling 11 Figure 2.5 The Data Processing 11 Figure 2.6 Amount of data for each section test, train, valid 12 Figure 2.7 Data Enhancement Process 15 Figure 2.8 5x5 image with layer of Zero- padding 19 Figure 2.9 Example of matrix array 20 Figure 2.10 CNN Network 21 Figure 2.11 Densenet 21 Figure 2.12 A 5-layer dense block with a growth rate of k = and the standard ResNet structure 22 Figure 2.13 Block of convolution layers with results concatenated 22 Figure 2.14 DenseNet model 23 Figure 2.15 A block diagram of pre-trained DenseNet-161 23 Figure 2.16 DenseNet 201 24 Figure 2.17 The model of EfficientNet 25 Figure 2.18 The performance of some CNN networks 25 Figure 2.19 Architecture of EfficientNet-B0 with MBConv as Basic buildingblock 26 Figure 2.20 Structure of EfficientNet-B1 26 Figure 2.21 Schematic representation of EfficientNet-B3 27 Figure 2.22 Inception 27 Figure 2.23 The model of Inception V1 28 Figure 2.24 Inception V2 28 Figure 2.25 Inception V3 28 Figure 2.26 The model of InceptionResnet 29 Figure 3.1 The proposed detail Diagram 30 Figure 3.2 The chromosome image (right) after rotating 45◦ from the original image (left) 31 Figure 3.3 The chromosome image (right) after flipping from the original image (left) 31 Figure 3.4 Crop chromosome image 32 Figure 3.5 Three approaches to resize cropped image to fixed size 32 Figure 3.6 Adaptive resizing method 33 Figure 3.7 Algorithm for the adaptive resizing method 33 Figure 3.8 Model architecture 34 Figure 3.9 Equations to evaluate performance 35 Figure 3.10 Comparison of Loss curves and Accuracy curves for train and validation stage 38 CHAP INTRODUCTION 1.1 Chromosome A chromosome [1] is a long DNA molecule with part or all of the genetic material of an organism In most chromosomes the very long thin DNA fibers are coated with packaging proteins; in eukaryotic cells the most important of these proteins are the histones These proteins, aided by chaperone proteins, bind to and condense the DNA molecule to maintain its integrity These chromosomes display a complex three-dimensional structure, which plays a significant role in transcriptional regulation Figure 1.1 Diagram of a replicated and condensed metaphase eukaryotic chromosome:(1) Chromatid, (2) Centromere, (3) Short arm, (4) Long arm Chromosomes as Figure 1.1 are normally visible under a light microscope only during the metaphase of cell division (where all chromosomes are aligned in the center of the cell in their condensed form) Before this happens, each chromosome is duplicated (S phase), and both copies are joined by a centromere, resulting either in an X-shaped structure (pictured above), if the centromere is located equatorially, or a two-arm structure, if the centromere is located distally The joined copies are now called sister chromatids During metaphase the X-shaped structure is called a metaphase chromosome, which is highly condensed and thus easiest to distinguish and study In animal cells, chromosomes reach their highest compaction level in anaphase during chromosome segregation If these structures are manipulated incorrectly, through processes known as chromosomal instability and translocation, the cell may undergo mitotic catastrophe Usually, this will make the cell initiate apoptosis leading to its own death, but sometimes mutations in the cell hamper this process and thus cause progression of cancer Chromosome types There are many ways to classify chromosomes, and each way relies on different characteristics of chromosomes to distinguish Figure 3.4 Crop chromosome image ➢ Normalization To carry out normalization, it means that making the dataset has the same image size, there are three ways that will be described and applied in my thesis including Scaling up, Zero-padding, and Adaptive Resizing Each method has both advantages and disadvantages After using them to balance the image size, I realized that Adaptive Resizing is the best method which brings the most advantages and solves the disadvantage from the other methods Figure 3.5 Three approaches to resize cropped image to fixed size Adaptive Resizing Because the size of the black background will also increase, after I tried to crop to reduce the black background So, I implemented the Adaptive resizing method In this method, I still select the image with the largest size after Augmentation and Cropping Then, for each image in the output data set, the resizing method will be used but keeping the ratio of each original image I will increase the image to the largest ratio but the size is still less than or equal to the size of the image I set as 32 the standard Then with the images that are not sized as standard size images, I will use the zero-padding method to increase to the required size Figure 3.6 Adaptive resizing method To have the image as above, it is needed to apply the algorithm for the adaptive resizing as following: Figure 3.7 Algorithm for the adaptive resizing method This algorithm can both scaling up, cropping, especially can increase the number of images Augmentation by moving the position of object in some different coordinates in the image To explain this algorithm, we can see that (h,w) is the size of all images after cropping; (H,W) is the target size or the size I want to after scaling up Firstly, I used if-else command to scaling up the size of cropped image Then, to make all the image have the same size (normalization) I used fordo command to make a zero matrix and add the image I have just scaled up in this matrix Beside normalization, for-do command in this algorithm can create 33 diffenrent images by moving the object in some different position in the image Therefore, the dataset will be increased 3.1.2 Chromosome Classification This section describes in detail my model for the task of classifying chromosomes I conducted experiments with several state-of the-art CNN architectures for this task A set of state-of-the-art deep CNN models had been investigated and evaluated on the testing set including InceptionResNet-v2, EffcientNet-B0, EffcientNet-B1, EffcientNet -B3, DenseNet121, DenseNet161, DenseNet169, DenseNet201 These CNN architectures are high performing for image classification tasks In addition, I exploited the Power of the pre-trained CNN models in a large image dataset to fine-tune for my problem Figure 3.8 Model architecture After pre-processing input data was resized to a shape of B×1×H×W (B:batch size), and then fed into the model The CNN network considered the pre-trained backbone as feature extractor, after that output would be applied a global pool, then I got a tensor with a shape B×N×1×1 (N: the number of channel features) And finally, after coming through a 2D convolution layer, a SoftMax function was used to calculate the probability of each output To determine what kind of chromosome it is, I would get the index of greatest probability In the training stage, all pre-processed images with the adaptive resizing method are fed into the network with the same size of (H × W) The number of training epochs was set as 200, and the early-stopping technique was applied to avoid overfitting of training dataset This allowed to stop training once the model performance stops improving on a holdout validation dataset During the training process, I used Adam optimizer with an initial learning rate of 10 -3 To end up, I minimized the cross-entropy loss function between the ground-truth labels and the predicted labels by my model over all training samples 3.1.3 Result Classification evaluation index To evaluate the performance of classifier, we would compare some of indicators, including mean accuracy, macro-average precision, macro-average 34 recall and macro-average F1-score The formula to calculate them will be mention following: Figure 3.9 Equations to evaluate performance Where TP, TN, FP, FN are the numbers of True Positive, True Negative, False Positive, False Negative Accuracy is the ratio between the number of correctly graded points and the total score Accuracy is only suitable for problems where the size of the data layers is relatively the same True Positive (TP): the number of points of the positive class that are correctly classified as positive True Negative (TN): the number of points of the negative class that are correctly classified as negative False Positive (FP): the number of points of the negative class that were mistakenly classified as positive False Negative (FN): the number of points of the positive class that were mistakenly classified as negative Macro-average: used for multi-class classification problem Intuitively, we can reduce the multiclass classification problem to the binary classification problem by looking at each class For each class, we consider the data in that class to have a label of positive, all the remaining data have a label of negative Then the Precision, Recall, and PR curve values are applied to each layer For each class, we get a pair of precision and recall values For problems with few data classes, we can plot the PR curve for each class on the same graph However, for problems with many layers of data, this is sometimes not possible Instead, the PrecisionRecall-based metric used is macro-average Macro-average precision is the average of the precisions by class Macro-average F-Score is calculated similarly to F-score but based on macro-average precision and macro-average recall Firstly, I would like to prove that the pre-processing stage can significantly improve the performance of classification model I did the comparison with the pre-processing step on the same architecture model and dataset And as the following table shows the model’s performance increased by about 2% on all metrics Table 3.1 The effect of pre-processing stage on performance Pre-processing stage F1-score Precision Recall Accuracy Cropping 0.9655 0.9659 0.9655 0.9712 Non- cropping 0.9433 0.9437 0.9433 0.9516 35 The table below compares te quantitative performance results for some classification models with different CNN backbones on my test set It can be seen that EfficientNet-b3 is the best model with F1-score of 96,55%, Precision of 96,59%, Recall of 96,55% and accuracy of 97,12% And then, this network architecture was used to conduct the next experiment Table 3.2 Experiment results between CNN models when trained with input images of size 256 x 256 Network architecture F1-score Precision Recall Accuracy Inception-ResNet-v2 0.9404 0.9362 0.9446 0.9475 DenseNet-121 0.9653 0.9568 0.9567 0.9626 DenseNet-161 0.9511 0.9513 0.9518 0.9590 DenseNet-169 0.9519 0.9524 0.9526 0.9587 DenseNet-201 0.9532 0.9530 0.9542 0.9601 EfficientNet-b0 0.9591 0.9587 0.9598 0.9658 EfficientNet-b1 0.9634 0.9638 0.9634 0.9687 EfficientNet-b3 0.9655 0.9659 0.9655 0.9712 After that, I conducted an experiment to compare three approaches to resize images to the fixed size on the performance of the selected model (EffcientNetb3), including scaling-up, zero-padding and adaptive method with default k-factor of Besides, I also investigated the impact of image input sizes after preprocessing In this experiment, I evaluated performance on some different fixed image size (256 × 256), (175 × 135) and (256 × 200) The reason why I chose size of (175 × 135) and (256 × 200) is that I performed size statistics of all chromosomes after cropping, and got the largest size by length of 175, width of 135, and size of (256 × 200) is the same aspect ratio as (175 × 135) As a result, in Table 3.3, I observed that resizing the input images by adaptive method with k-factor of for training gave the best result Because the scaling-up method could change shape of original image, while zero-padding method had no effect for performance of classifier, as mentioned in Section 2.2.4 And then I evaluated the effects of input size after using adaptive resizing method From Table 3.4 I can see that input size of (256 × 256) achieved the best performance in this experiment Table 3.3 Performance of EfficientNet B3 with resize methods using image size 256 x 256 Resize method F1-score Precision Recall Accuracy Zero- padding 0.9445 0.9447 0.9447 0.9505 Adaptive (k=1) 0.9655 0.9659 0.9655 0.9712 Scaling-up 0.9617 0.9614 0.9621 0.9669 36 Table 3.4 Performance of EffcientNet-b3 using adaptive resizing (k=1) with different input image sizes Image size F1-score Precision Recall Accuracy 175 x 135 0.9604 0.9601 0.9615 0.9665 256 x 200 0.9619 0.9623 0.9623 0.9690 256 x 256 0.9655 0.9659 0.9655 0.9712 One of my contributions in this paper is that I proposed a new resizing method which both keeps aspect ratio and enriches data with k-factor Indeed, as a result in Table 3.5, performance of classifier increased slightly with k-factor of and compared to Although it was not a sharp rise (about 0.5% in f1-score), this could be considered as a promising approach for other tasks because of combining resize method and augmentation in an algorithm Table 3.5 The effect of augmented factor in adaptive resize method k- factor F1-score Precision Recall Accuracy 0.9655 0.9659 0.9655 0.9712 0.9662 0.9663 0.9665 0.9712 0.9695 0.9700 0.9692 0.9736 Generally, a learning curve is a plot that shows time or experience on the xaxis and learning or improvement on the y-axis Learning Curve: Line plot of learning (y-axis) over experience (x-axis) Learning curves are widely used in machine learning for algorithms that learn (optimize their internal parameters) incrementally over time, such as deep learning neural networks.The metric used to evaluate learning could be maximizing, meaning that better scores (larger numbers) indicate more learning An example would be classification accuracy It is more common to use a score that is minimizing, such as loss or error whereby better scores (smaller numbers) indicate more learning and a value of 0.0 indicates that the training dataset was learned perfectly and no mistakes were made During the training of a machine learning model, the current state of the model at each step of the training algorithm can be evaluated It can be evaluated on the training dataset to give an idea of how well the model is “learning.” It can also be evaluated on a hold-out validation dataset that is not part of the training dataset Evaluation on the validation dataset gives an idea of how well the model is “generalizing.” Train Learning Curve: Learning curve calculated from the training dataset that gives an idea of how well the model is learning 37 Validation Learning Curve: Learning curve calculated from a hold-out validation dataset that gives an idea of how well the model is generalizing It is common to create dual learning curves for a machine learning model during training on both the training and validation datasets In some cases, it is also common to create learning curves for multiple metrics, such as in the case of classification predictive modeling problems, where the model may be optimized according to cross-entropy loss and model performance is evaluated using classification accuracy In this case, two plots are created, one for the learning curves of each metric, and each plot can show two learning curves, one for each of the train and validation datasets Optimization Learning Curves: Learning curves calculated on the metric by which the parameters of the model are being optimized, e.g loss Performance Learning Curves: Learning curves calculated on the metric by which the model will be evaluated and selected, e.g accuracy Figure 3.10 Comparison of Loss curves and Accuracy curves for train and validation stage As we can see in my research in Figure 3.10, the loss curves and accuracy curves of proposed model for training and validation stage are shown in Figure below It can be seen that the accuracy scores get high value right from some first epochs and quickly converge and stabilize after about 30-40 epochs The use of pre-trained models as initialization Weights instead of random initialization is one of the reasons for achieving such good results Table 3.6 Comparison to other approaches Article Data Method Accuracy Wu et al (2018) Private data MD-GAN & CNN 63.50% Zhang et al (2018) Private data CNN (Lenet) 92.55% Qin et al (2018) Xiangya MLP 99.20% Thinh et al (2020) Passau CNN (Inception- ResNet) 92.80% Mine Passau CNN (EfficientNet- b3) 97.12% 38 In Figure 3.6, I would like to compare my result with some previous approaches which also related to the topic of my research The result I achieved is better than almost all the articles I listed here In comparison with Wu et al [25], the accuracy I achieved (97.9%) is much better than their result (63.5%) The accuracy of Zhang et al [14] is also lower than mine, which means that my method I applied in my research is better than theirs Particularly, compared to their method, the highlight step we performed better than them is the models of classification I applied and the normalization technique Besides, with the same dataset, the accuracy of my approach is higher than that of the method from Thinh et al [26] (92.8%) The reason is that I carry out more steps of pre-processing In detail, I improved the quality of dataset by cropping background of image Cropping is in fact really important because it can reduce the time we train the dataset and increase the accuracy of the classification The more black background is, the more unnecessary part we needn’t to process Normalization in my research is also the highlight feature I increased the size of all the chromosome by their rate, which can clarify chromosomal traits In addition, by experiment I choose the models of CNN that perform the highest accuracy Compared to the article of Qin et al [17], their performance (99.2%) is better than my result because their dataset is different and contains a bigger number of images than my dataset Practically, it is quite difficult to collect the big dataset like them 39 CONCLUSION In this subject, Deep learning represented by convolutional neural network has been applied to the task of chromosome detection and classification, and I have achieved the best results with the EfficientNet-b3 model in the series (Accuracy: 97.12%) of CNN models I put in to try According to my research, my proposed model has achieved better accuracy in some classifiers The resulting image enhancement and classification can provide a quantitative basis for the following physician's diagnosis The preprocessing model also helps to segment and enhance features, and then uses multiple CNN models to classify and detect single chromosomes, to achieve chromosome matching However, my study still has some limitations that need to be improved in the future First, I only used a dataset that could have a limited number of images It is possible that when trained with a larger data set, the performance of deep learning will improve markedly Second, my preprocessing step can be further improved to improve the quality of the dataset Further studies may find more data sets to test the accuracy of my classification model Besides, I can implement additional steps and technology to improve the preprocessing step to be able to optimize the data and achieve higher accuracy I hope that my study can further assist the cytologist in analyzing metaphase images to distinguish abnormal chromosomes 40 REFERENCES [1] "Wikipedia," The Free Encyclopedia, 2023 [Online] Available: https://en.wikipedia.org/wiki/Chromosome [2] "Cleveland Clinic," 2023 [Online] https://my.clevelandclinic.org/health/diseases Available: [4] "MedlinePlus," 11 2022 [Online] https://medlineplus.gov/lab-tests/karyotype-genetic-test Available: [5] Abid F, Hamami L, "A survey of neutral neutral network based automated systems for human chromosome classification," Artificial Intelligent Review, 2018 [6] IM, EI Emary, "On the Application of Artificial Neutral Networks in Analyzing and Classifying the Human Chromosome," Journal of Computer Science, 2006 [7] Dewa MSA, Grafika J, Agung S, Rafli F, Nurul H, Muhammad FR, "Comparison of image enhancement methods for chromosome karyotype image enhancement," Journal of Computer Science and Information, 2017 [8] Swati, Gupta G, Yadav M, Sharma M, Vig L, "Siamese Networks for Chromosome Classification," 2017 IEE International Conference on Computer Vision Workshops, 2017 [9] Ding W, Chang L, Gu C, Wu K, "Classification of Chromosome Karyotype Based on Faster-RCNN with the Segmatation and Enhancement Preprocessing Model," 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), 2019 [10] Hariyanti MS, Nor HS, Nor AMI, "Overlapping Chromosome Segmentation using U-Net: Convolutional Networks with Test Time Augmentation," Procedia Computer Science , 2019 [11] Somasundaram D, Vijay K, "Separation of overlapped chromosomes and pairing of similar chromosomes for karyotyping analysis," Measurement , 2014 [12] Munot M, Mukherjee J, Joshi M, "A novel approach for efficient extrication of overlapping chromosomes in automated karyotyping," Med Biol Eng Comput , 2013 [13] L B, "Toward a completely automatic neural-network-based human chromosome analysis," IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 1998 41 [14] Zhang W, Song S, Bai T, et al, "Chromosome Classifcation with Convolutional Neural Network Based Deep Learning," 11th International Congress on Image and Signal Processing BioMedical Engineering and Informatics (CISP-BMEI), 2018 [15] D S, "Reduced size multi layer perceptron neural network for human chromosome classifcation," IEEE Engineering in Medicine and Biology Society (IEEE Cat, no 3, pp 2249-2252, 2003 [16] Wei H, Gao W, Nie H, Sun J, Zhu M, "Classifcation of Giemsa staining chromosome using input-aware deep convolutional neural network with integrated uncertainty estima," Biomedical Signal Processing and Control, p 71, 2022 [17] Qin Y, Wen J, Zheng H, et al, "Varifocal-Net: A Chromosome Classifcation Approach Using Deep Convolutional Networks," IEEE Transactions on Medical Imaging, vol 38, no 1, pp 2569-2581, 2019 [18] G Ritter, G Schreib, and Le Gao, "Faculty of Computer Science and Mathematics," University of Passau, 244 11 2009 [Online] Available: https://www.fim.uni-passau.de/en/research-and-professorships/formerchairs-professorships/mathematical-stochastics/chromosome-imagedata [19] skrg1441, "geeksforgeeks," 2020 [Online] Available: https://www.geeksforgeeks.org/changing-the-contrast-and-brightnessof-an-image-using-python-opencv/ [20] D Shah, "The Essential Guide to Data Augmentation in Deep Learning," [Online] Available: V7labs.com [21] "Cloudinary," Media Guides, 2023 [Online] Available: https://cloudinary.com/guides/bulk-image-resize/python-image-resizewith-pillow-and-opencv [22] M Kumar, "Numpy Ninja," 22 12 2020 [Online] Available: https://www.numpyninja.com/post/how-padding-helps-in-cnn [23] C P Van, "VIBLO," 10 12 2020 [Online] Available: https://viblo.asia/p/deep-learning-tim-hieu-ve-mang-tich-chap-cnnmaGK73bOKj2 [25] Gunter R, Le G, "Automatic segmentation of metaphase cells based on global context and variant analysis," Pattern Recognition, vol 41, no 38-55, p 1, 2008 [26] Thinh NH, Son NHH, Huong PTV, et al, "A Web-based Tool for Semi-interactively Karyotyping the Chromosome Images for Analyzing Chromosome Abnormalities," 2020 7th NAFOSTED Conference on Information and Computer Science, pp 433-437, 2020 42 [27] M H, "Enlarging smaller images before inputting into convolutional neural network: zero-padding vs interpolation," Journal of Big Data, no 6, p 98, 2019 [28] Nguyen LD, Lin D, Lin Z, Cao J, "Deep CNNs for microscopic image classifcation by exploiting transfer learning and feature concatenation," 2018 IEEE International Symposium on Circuits and Systems (ISCAS), pp 1-5, 2018 [29] Mingxing T, Quoc VL, "Effcientnet: Rethinking model scaling for convolutional neural networks," Proceedings of the 36th International Conference on Machine Learning, Long Beach, California, PMLR 97, 2019 [30] Huang G, Liu Z, Van Der Maaten L, Weinberger KQ, "Densely Connected Convolutional Networks," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2261-2269, 2017 [31] Caruana R, Lawrence S, Giles C, "Overftting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping," Advances in Neural Information Processing Systems, p 13, 2000 [32] Diederik K, Jimmy B, "Adam: A Method for Stochastic Optimization," International Conference on Learning Representations , 2014 [33] Wu Y, Yue Y, Tan X, Wang W, Lu T, "End-To-End Chromosome Karyotyping with Data Augmentation Using GAN," 2018 25th IEEE International Conference on Image Processing (ICIP), pp 2456-2460, 2018 [34] Menaka D, Vaidyanathan S, "Chromenet: a CNN architecture with comparison of optimizers for classifcation of human chromosome images," Multidimensional Systems and Signal Processing , no 33, p 747–768, 2022 [35] V Hyma Lakshmi Tirumani, Madhu Tenneti, Ch Srikavya K, Sarat Kumar Kotamraju, "Image resolution and contrast enhancement with optimal brightness compensation using wavelet transforms and particle swarm optimization," The Institution of Engineering and Technology, vol 17, no 12, pp 2833-2840 43 SUMMARY OF THE MASTER'S THESIS Topic: Human chromosome classification using convolutional neutral network Author: Quán Thị Yến – 2021B Instructor: Dr Trần Anh Vũ Keyword: Chromosome, Classfication, Karyotyping, Neutral network Summary content: a) Reason of choosing the topic As we know, some genetic diseases are associated with chromosomal abnormalities Diagnosis of these abnormalities has gained attention in recent years One of the most popular and useful ways to solve this problem is based on Karyotyping Karyotyping is a laboratory procedure that allows doctors to examine the set of chromosomes Karyotyping tests are performed to determine if the cell's chromosomes are normal Therefore, it plays an important role in the diagnosis of genetic disorders In fact, Karyotyping requires considerable manual efforts, domain expertise and experience, and is very time- consuming Therefore, chromosome classification using convolutional neutral network can both gradually replace Karyotyping and help doctors quickly identify abnormal chromosomes b) Purpose, Research Object, Scope of Research Purpose I made a thesis research with this topic to apply what I studied at school and self-study about the application of convolutional neural networks and many methods in the preprocessing step to auto-classify chromosomes Thereby, as a basis to replace Karyotyping method, helping doctors quickly identify abnormal chromosomes Research Object Based on the identified research objectives and implementation process, the research objects of the thesis are determined to include the following groups: • A group of objects related to chromosomes, including basic theory of chromosomes, mutated chromosomes, methods of classification and detection of mutated chromosomes being applied in practice • Groups of objects related to chromosome classification using artificial intelligence methods, including pre-processing techniques such as cropping, zooming, augmentation and using machine learning algorithms to classify or Using neural network for classification brings high accuracy Scope of research Due to limited research time and data processing, students focus on solving essential scientific problems to achieve the set goals Specifically, the content of the thesis is limited to the following scope: 44 • Theory of karyotyping method: the thesis only presents an overview of the method of detecting mutated chromosomes (karyotyping method) being applied today, highlighting the advantages and disadvantages of the method This content summarizes the important foundations for the proposed use of artificial intelligence for chromosome classification designed in the thesis • Theory of chromosome processing method using artificial intelligence: the thesis systematizes the general knowledge about the basis and principle of the method, including image preprocessing techniques and systems image classification methods c) Content Summary and Author’s Contribution Content Summary My Research includes chapters: Chapter 1: Introduction In this chapter, I introduce generally chromosome: What is chromosome, chromosome classification? Why I need to classify chromosomes? Also, I find some methods to classify, especially Karyotyping is the most popular method Therefore, I researched it about definition, performing test Thanks to this, I researched AI to replace Karyotyping To method using AI, I find many related works in order to create my own method based on achievement from other research Chapter 2: Methodology Firstly, this chapter describes the dataset, how the dataset was processed to become my input dataset After that, I will describe the theory of Increase brightness and contrast, data Augmentation, cropping method and normalization Also, I mentioned the theory of some model of CNN for classification (EfficientNet, DenseNet, InceptionResnet) Chapter 3: Result and discussion Based on chapter and chapter 2, after obtaining enough necessary information, I applied this knowledge to process the dataset from Passau Finally, I present the results I achieved and give the conclusion Author’s Contribution In this subject, Deep learning represented by convolutional neural network has been applied to the task of chromosome detection and classification, and I have achieved the best results with the EfficientNet-b3 model in the series (Accuracy: 97.12%) of CNN models I put in to try According to my research, my proposed model has achieved better accuracy in some classifiers The resulting image enhancement and classification can provide a quantitative basis for the following physician's diagnosis The preprocessing model also helps to segment and enhance 45 features, and then uses multiple CNN models to classify and detect single chromosomes, to achieve chromosome matching The proposed research on chromosome classification method is a premise for further research on methods to detect mutant chromosomes The results presented in the thesis are meaningful references for research related to image processing techniques The results in the thesis can be applied in practice to classify chromosomes of any kind, capable of replacing the traditional karyotyping method Help doctors diagnose and faster medical examination time d) Research method During the research, students used many different research methods to achieve the set goals Frequently used methods are: • Pretreatment methods: students use a variety of pretreatment methods and combine them to get the best results These methods include separating stacked chromosomes, reducing the black background, resizing the image to the same size, increasing the number of images by rotation, horizontal flip, and vertical flip methods • Classification method: To be able to find the optimal classification method, giving the highest accuracy, students used many different classification measures such as machine learning algorithms, algorithms using neural networks e) Conclusion By using many models of CNN, and especially create the adaptive resizing method to normalize dataset optimally, I have achieved some success to classify chromosomes However, my study still has some limitations that need to be improved in the future I hope that my study can further assist the cytologist in analyzing metaphase images to distinguish abnormal chromosomes 46