BỘ GIÁO DỤC VÀ ĐÀO TẠO TRƯỜNG ĐẠI HỌC BÁCH KHOA HÀ NỘI HÀ THỊ THANH NGHIÊN CỨU MỘT SỐ BÀI TOÁN TRONG HỎI ĐÁP CỘNG ĐỒNG LUẬN ÁN TIẾN SĨ HỆ THỐNG THÔNG TIN HÀ NỘI−2021 BỘ GIÁO DỤC VÀ ĐÀO TẠO TRƯỜNG ĐẠI HỌC BÁCH KHOA HÀ NỘI HÀ THỊ THANH NGHIÊN CỨU MỘT SỐ BÀI TOÁN TRONG HỎI ĐÁP CỘNG ĐỒNG Ngành: Hệ thống thông tin Mã số: 9480104 LUẬN ÁN TIẾN SĨ HỆ THỐNG THÔNG TIN TẬP THỂ HƯỚNG DẪN KHOA HỌC: PGS.TS NGUYỄN THỊ KIM ANH TS NGUYỄN KIÊM HIẾU HÀ NỘI−2021 LỜI CAM ĐOAN Tơi xin cam đoan kết trình bày luận án cơng trình nghiên cứu thân nghiên cứu sinh thời gian học tập nghiên cứu Đại học Bách khoa Hà Nội hướng dẫn tập thể hướng dẫn khoa học Các số liệu, kết trình bày luận án hoàn toàn trung thực Các kết sử dụng tham khảo trích dẫn đầy đủ theo quy định Hà Nội, ngày tháng năm 2021 Nghiên cứu sinh Hà Thị Thanh TẬP THỂ HƯỚNG DẪN KHOA HỌC PGS.TS Nguyễn Thị Kim Anh TS Nguyễn Kiêm Hiếu LỜI CẢM ƠN Trong trình nghiên cứu hoàn thành luận án này, nghiên cứu sinh nhận nhiều giúp đỡ đóng góp quý báu Đầu tiên, nghiên cứu sinh xin bày tỏ lòng biết ơn sâu sắc tới tập thể hướng dẫn: PGS.TS Nguyễn Thị Kim Anh TS Nguyễn Kiêm Hiếu Các thầy ln ln tận tình hướng dẫn, bảo, giúp đỡ động viên nghiên cứu sinh suốt trình nghiên cứu Nghiên cứu sinh xin chân thành cảm ơn thầy cô Bộ môn Hệ thống thơng tin Phịng thí nghiệm Khoa học liệu, Viện Công nghệ thông tin truyền thông - Trường Đại học Bách khoa Hà Nội, nơi nghiên cứu sinh học tập tạo điều kiện cho nghiên cứu sinh tham gia nghiên cứu suốt thời gian học tập Trong thời gian này, nghiên cứu sinh nhận nhiều kiến thức kinh nghiệm từ thầy cô bạn sinh viên lab đặc biệt nhóm nghiên cứu Nghiên cứu sinh xin chân thành cảm ơn Phòng Đào tạo - Trường Đại học Bách Khoa Hà Nội tạo điều kiện để nghiên cứu sinh hồn thành thủ tục bảo vệ luận án tiến sĩ Nghiên cứu sinh cảm ơn trường ĐH CNTT&TT Thái Nguyên - nơi NCS làm việc, tạo điều kiện thời gian hỗ trợ học phí cho nghiên cứu sinh tham gia học tập nghiên cứu trường ĐH Bách Khoa Hà Nội Nghiên cứu sinh xin cảm ơn chương trình 911 Cơng ty TNHH Đầu tư Phát triển đô thị Gia Lâm thuộc Tập đoàn Vingroup hỗ trợ Quỹ Đổi sáng tạo Vingroup (VINIF) Dự án mã số VINIF.2019.DA18 tài trợ cho NCS thời gian nghiên cứu thực luận án Cuối cùng, nghiên cứu sinh xin gửi lời cảm ơn sâu sắc tới gia đình, bạn bè đồng nghiệp động viên, giúp đỡ nghiên cứu sinh, giúp nghiên cứu sinh yên tâm nghiên cứu vượt qua khó khăn để hồn thành luận án MỤC LỤC DANH MỤC CÁC TỪ VIẾT TẮT VÀ THUẬT NGỮ iv DANH MỤC HÌNH VẼ vi DANH MỤC BẢNG viii DANH MỤC KÝ HIỆU TOÁN HỌC x MỞ ĐẦU CHƯƠNG TỔNG QUAN VỀ HỆ THỐNG HỎI ĐÁP VÀ CÁC KIẾN THỨC NỀN TẢNG 1.1 Hệ thống hỏi đáp 1.1.1 Định nghĩa hệ thống hỏi đáp 1.1.2 Kiến trúc hệ thống QA 1.1.3 Các cách tiếp cận tới hệ thống QA 1.1.4 Thách thức hệ thống QA 12 1.2 Hỏi đáp cộng đồng 13 1.2.1 Kiến trúc hệ thống CQA 14 1.2.2 So sánh hệ thống QA CQA 15 1.3 Các toán hệ thống hỏi đáp cộng đồng CQA 16 1.3.1 Xác định chất lượng câu trả lời 16 1.3.2 Bài toán tìm câu hỏi tương đồng 18 1.4 Thách thức hệ thống hỏi đáp CQA 18 1.5 Tập liệu hỏi đáp 19 1.6 Kiến thức tảng học sâu 22 1.6.1 Mơ hình mạng truy hồi 23 1.6.2 Mơ hình LSTM 24 1.6.3 Mơ hình mạng tích chập 25 1.6.4 Cơ chế ý 27 1.6.5 Mô hình Transformer 29 1.6.6 Học biểu diễn từ nhúng 31 1.7 Kết luận chương 34 i CHƯƠNG BÀI TOÁN LỰA CHỌN CÂU TRẢ LỜI ĐÚNG TRONG HỆ THỐNG HỎI ĐÁP CỘNG ĐỒNG 36 2.1 Giới thiệu 36 2.2 Các cơng trình liên quan 39 2.2.1 Các nghiên cứu toán lựa chọn câu trả lời 39 2.2.2 Các nghiên cứu dựa vào chế ý NLP 44 2.2.3 Các nghiên cứu chế ý có giám sát tốn NLP 44 2.3 Mơ hình đề xuất 44 2.3.1 Mơ hình match-LSTM 45 2.3.2 Mở rộng mơ hình 46 2.3.3 Cơ chế ý có giám sát 48 2.4 Các thử nghiệm kết 49 2.4.1 Tập liệu 49 2.4.2 Chọn tham số mô hình 50 2.4.3 Kết thảo luận 51 2.4.4 Trực quan hóa trọng số ý 54 2.5 Kết luận chương 55 CHƯƠNG BÀI TỐN TĨM TẮT CÂU TRẢ LỜI TRONG HỆ THỐNG HỎI ĐÁP CỘNG ĐỒNG 57 3.1 Giới thiệu 57 3.2 Các cơng trình liên quan 59 3.3 Mơ hình tóm tắt câu trả lời cho câu hỏi non-factoid 60 3.3.1 Biểu diễn câu 61 3.3.2 Trích rút tóm tắt 64 3.4 Đánh giá mơ hình 65 3.4.1 Tập liệu 65 3.4.2 Thiết lập thử nghiệm 66 3.4.3 Kết thực nghiệm 66 3.5 Kết luận chương 70 CHƯƠNG BÀI TỐN TÌM CÂU HỎI TƯƠNG ĐỒNG TRONG HỆ THỐNG HỎI ĐÁP CỘNG ĐỒNG 71 4.1 Giới thiệu 72 ii 4.2 Các cơng trình liên quan 74 4.3 Tập liệu 76 4.4 Mơ hình BERT cho tốn tìm kiếm câu hỏi tương đồng 78 4.4.1 Mô hình BERT 78 4.4.2 BERT cho tốn tìm kiếm câu hỏi tương đồng 79 4.4.3 Các kết thực nghiệm thảo luận 80 4.5 Mơ hình SBERT 88 4.5.1 Mơ hình SBERT cho tốn tìm câu hỏi tương đồng 88 4.5.2 Các thử nghiệm kết 90 4.6 Kết luận chương 93 KẾT LUẬN 94 DANH MỤC CÁC CƠNG TRÌNH ĐÃ CƠNG BỐ 97 TÀI LIỆU THAM KHẢO 98 iii DANH MỤC CÁC TỪ VIẾT TẮT VÀ THUẬT NGỮ Viết tắt Tiếng Anh Tiếng Việt CQA Community Question Answering Hỏi đáp cộng đồng AQ Question Answering Hệ thống hỏi đáp MMR Maximal Marginal Relevance Thuật toán MMR MAP Mean Average Presision Độ đo MAP MRR Mean Reciprocal Rank Độ đo MRR LSTM Long Short - Term Memory Mơ hình LSTM RNN Recurent Neural Network Mơ hình mạng truy hồi RNN AE Auto Encoder Mơ hình AE BERT Bidirectional Encoder Represen- Mơ hình biểu diễn mã hóa hai chiều tation from Transformers từ Transformer - Mơ hình BERT SBERT Sentence BERT Mơ hình SBERT NLP Natural Language Processing Xử lý ngôn ngữ tự nhiên tf.idf Term Frequency – Inverse Docu- Trọng số tf.idf ment Frequency BM25 Best Match25 Mơ hình BM25 IR Information Retrieval Tìm kiếm thơng tin WE Word Embedding Từ nhúng PE Position Embedding Nhúng vị trí POS Part of Speech Gán nhãn từ loại NER Named Entity Recognition Nhận dạng thực thể có tên OOV Out Of Vocabulary Ngồi tập từ vựng LCS Longest Common Subsequence Dãy chung lớn ILP Interger Linear Programming Thuật toán ILP SemEval International workshop on Se- Hội thảo đánh giá ngữ nghĩa mantic Evaluation SNLI Stanford Natural Language Infer- Tập liệu suy diễn ngôn ngữ ence SVM Support Vector Machine Mơ hình máy véc tơ hỗ trợ SVM LDA Latent Dirichlet allocation Mơ hình chủ đề ẩn LDA CNN Covolutional Neural Network Mơ hình mạng tích chập CNN iv ROUGE Recall-Oriented Understudy for Độ đo ROUGE Gisting Evaluation ROC Receiver Operating Characteris- Đường cong ROC tic AUC Area Under the Curve ABCNN Attention-Based Diện tích đường cong AUC Convolutional Mơ hình mạng tích chập dựa vào Neural Network chế ý - ABCNN v DANH MỤC HÌNH VẼ 1.1 Kiến trúc chung hệ thống QA 1.2 Ví dụ hỏi đáp Yahoo!answer 14 1.3 Kiến trúc hệ thống CQA[1] 15 1.4 Mơ hình RNN 24 1.5 Mơ hình LSTM 25 1.6 Mơ hình CNN 26 1.7 Mơ hình attention tốn dịch máy Bahdanau 28 1.8 Mô hình Transformer[2] 29 1.9 Q trình tính ý 31 1.10 Mơ hình CBOW Skip-gram [3] 33 2.1 Ví dụ câu hỏi câu trả lời tập liệu SemEval 2017 38 2.2 Một ví dụ chế ý theo từ học mơ hình match-LSTM Các từ chứa nội dung của câu hỏi câu trả lời có trọng số thấp Trong đó, hình (a), ý lại tập trung vào từ dừng từ ’not’ từ ’anyone’ câu hỏi lại từ quan trọng với câu trả lời tương ứng Hình (b) ý số từ câu hỏi lại tập trung vào phần chào hỏi câu trả lời 40 2.3 Mơ hình match-LSTM [4] cho tốn lựa chọn câu trả lời 45 2.4 Mơ hình mở rộng từ mơ hình match-LSTM cho toán lựa chọn câu trả lời 47 2.5 Một ví dụ minh họa trọng số ý học mơ hình matchLSTM kết hợp với chế ý có giám sát với cặp câu hỏi câu trả lời tốt 54 2.6 Một ví dụ trọng số ý học mơ hình match-LSTM kết hợp với chế ý có giám sát với cặp câu hỏi câu trả lời khơng tốt 55 3.1 Ví dụ câu hỏi, câu trả lời đoạn tóm tắt câu trả lời theo kiểu trích rút tập liệu Yahoo!Answer 58 vi [16] Nakov P., i Villodre L.M., Magdy W., Moschitti A., Glass J.R., and Randeree B (2015) Semeval-2015 task 3: Answer selection in community question answering In SemEval@NAACL-HLT [17] Chan W., Zhou X., Wang W., and Chua T.S (2012) Community answer summarization for multi-sentence question with group l1 regularization In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume , ACL ’12, pp 582–591 Association for Computational Linguistics, Stroudsburg, PA, USA [18] Cai L., Zhou G., Liu K., and Zhao J (November 2011) Learning the latent topics for question retrieval in community QA In Proceedings of 5th International Joint Conference on Natural Language Processing, pp 273–281 Asian Federation of Natural Language Processing, Chiang Mai, Thailand [19] Wang M., Smith N.A., and Mitamura T (2007) What is the jeopardy model? a quasi-synchronous grammar for qa In EMNLP-CoNLL, pp 22– 32 [20] Nakov P., Màrquez L., Moschitti A., Magdy W., Mubarak H., Freihat A.A., Glass J., and Randeree B (jun 2016) SemEval-2016 task 3: Community question answering In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp 525–545 Association for Computational Linguistics, San Diego, California doi:10.18653/v1/S16-1083 [21] Nakov P., Hoogeveen D., Màrquez L., Moschitti A., Mubarak H., Baldwin T., and Verspoor K (August 2017) SemEval-2017 task 3: Community question answering In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp 27–48 Association for Computational Linguistics, Vancouver, Canada doi:10.18653/v1/S17-2003 [22] Patra B (2017) A survey of community question answering CoRR, abs/1705.04009 ˇ [23] Mikolov T., Karafiát M., Burget L., Cernocký J., and Khudanpur S (2010) Recurrent neural network based language model In Proceedings of the 11th Annual Conference of the International Speech Communication Association (INTERSPEECH 2010), volume 2010, pp 1045–1048 International Speech Communication Association ISBN 978-1-61782-123-3 ISSN 19909772 100 [24] Hochreiter S and Schmidhuber J (November 1997) Long short-term memory Neural Computation, 9(8):p 1735–1780 ISSN 0899-7667 doi: 10.1162/neco.1997.9.8.1735 [25] Shen Y., He X., Gao J., Deng L., and Mesnil G (2014) Learning semantic representations using convolutional neural networks for web search In Proceedings of the 23rd International Conference on World Wide Web, WWW ’14 Companion, p 373–374 Association for Computing Machinery, New York, NY, USA ISBN 9781450327459 doi:10.1145/2567948.2577348 [26] Kalchbrenner N., Grefenstette E., and Blunsom P (June 2014) A convolutional neural network for modelling sentences In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 655–665 Association for Computational Linguistics, Baltimore, Maryland doi:10.3115/v1/P14-1062 [27] Bahdanau D., Cho K., and Bengio Y (2014) Neural machine translation by jointly learning to align and translate [28] Graves A., Wayne G., and Danihelka I (2014) Neural turing machines CoRR, abs/1410.5401 [29] Luong T., Pham H., and Manning C.D (September 2015) Effective approaches to attention-based neural machine translation In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp 1412–1421 Association for Computational Linguistics, Lisbon, Portugal doi:10.18653/v1/D15-1166 [30] Bengio Y., Ducharme R., Vincent P., and Janvin C (March 2003) A neural probabilistic language model J Mach Learn Res., 3(null):p 1137–1155 ISSN 1532-4435 [31] Pennington J., Socher R., and Manning C (October 2014) GloVe: Global vectors for word representation In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 1532– 1543 Association for Computational Linguistics, Doha, Qatar doi:10 3115/v1/D14-1162 [32] Chen D and Manning C (October 2014) A fast and accurate dependency parser using neural networks In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 740– 750 Association for Computational Linguistics, Doha, Qatar doi:10.3115/ v1/D14-1082 101 [33] Yin Y., Wei F., Dong L., Xu K., Zhang M., and Zhou M (2016) Unsupervised word and dependency path embeddings for aspect term extraction In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI’16, p 2979–2985 AAAI Press ISBN 9781577357704 [34] Bordes A., Weston J., Collobert R., and Bengio Y (2011) Learning structured embeddings of knowledge bases In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, AAAI’11, p 301–306 AAAI Press [35] Bojanowski P., Grave E., Joulin A., and Mikolov T (06 2017) Enriching Word Vectors with Subword Information Transactions of the Association for Computational Linguistics, 5:pp 135–146 ISSN 2307-387X doi:10 1162/tacl_a_00051 [36] Peters M., Neumann M., Iyyer M., Gardner M., Clark C., Lee K., and Zettlemoyer L (June 2018) Deep contextualized word representations In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Papers), pp 2227–2237 Association for Computational Linguistics, New Orleans, Louisiana doi:10.18653/v1/N18-1202 [37] Bonadiman D., Uva A., and Moschitti A (April 2017) Effective shared representations with multitask learning for community question answering In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pp 726–732 Association for Computational Linguistics, Valencia, Spain [38] Joty S., Màrquez L., and Nakov P (October-November 2018) Joint multitask learning for community question answering using task-specific embeddings In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 4196–4207 Association for Computational Linguistics, Brussels, Belgium doi:10.18653/v1/D18-1452 [39] Tran N.K and Niedereée C (2018) Multihop attention networks for question answer matching In The 41st International ACM SIGIR Conference on Research Development in Information Retrieval , SIGIR ’18, p 325–334 Association for Computing Machinery, New York, NY, USA ISBN 9781450356572 doi:10.1145/3209978.3210009 [40] Wu W., Sun X., and Wang H (July 2018) Question condensing networks for answer selection in community question answering In Proceedings of 102 the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 1746–1755 Association for Computational Linguistics, Melbourne, Australia doi:10.18653/v1/P18-1162 [41] Wang X., Kapanipathi P., Musa R., Yu M., Talamadupula K., Abdelaziz I., Chang M., Fokoue A., Makni B., Mattei N., and Witbrock M (2019) Improving natural language inference using external knowledge in the science questions domain AAI-19 , The Thirty-Third AAAI Conference on Artificial Intelligence [42] Mozafari J., Fatemi A., and Nematbakhsh M.A (2021) Bas: An answer selection method using bert language model Journal of Computing and Security, 8(2):pp 1–18 ISSN 2322-4460 doi:10.22108/jcs.2021.128002 1066 [43] Wan S., Dras M., Dale R., and Paris C (01 2006) Using dependencybased features to take the “para-farce” out of paraphrase Proceedings of the Australasian Language Technology Workshop, pp 131–138 [44] Surdeanu M., Ciaramita M., and Zaragoza H (December 2008) Learning to rank answers on large online qa collections In ACL-08 , ACL-08: HLT - 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, pp 719– 727 ISBN 9781932432046 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, ACL-08: HLT ; Conference date: 15-06-2008 Through 20-06-2008 [45] Soergel D (10 1998) Wordnet an electronic lexical database [46] Sequiera R., Baruah G., Tu Z., Mohammed S., Rao J., Zhang H., and Lin J.J (2017) Exploring the effectiveness of convolutional neural networks for answer selection in end-to-end question answering CoRR, abs/1707.07804 [47] Punyakanok V., Roth D., and tau Yih W (2004) Mapping dependencies trees: An application to question answering In In Proceedings of the 8th International Symposium on Artificial Intelligence and Mathematics, Fort [48] Heilman M and Smith N.A (June 2010) Tree edit models for recognizing textual entailments, paraphrases, and answers to questions In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp 1011–1019 Association for Computational Linguistics, Los Angeles, California 103 [49] Jansen P., Surdeanu M., and Clark P (June 2014) Discourse complements lexical semantics for non-factoid answer reranking In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 977–986 Association for Computational Linguistics, Baltimore, Maryland doi:10.3115/v1/P14-1092 [50] Wang M and Manning C.D (2010) Probabilistic tree-edit models with structured latent variables for textual entailment and question answering In Proceedings of the 23rd International Conference on Computational Linguistics, COLING ’10, p 1164–1172 Association for Computational Linguistics, USA [51] Wang K., Ming Z., and Chua T.S (2009) A syntactic tree matching approach to finding similar questions in community-based qa services In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval , SIGIR ’09, p 187–194 Association for Computing Machinery, New York, NY, USA ISBN 9781605584836 doi:10.1145/1571941.1571975 [52] Severyn A and Moschitti A (2012) Structural relationships for large-scale learning of answer re-ranking In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval , SIGIR ’12, p 741–750 Association for Computing Machinery, New York, NY, USA ISBN 9781450314725 doi:10.1145/2348283.2348383 [53] Yih W.t., Chang M.W., Meek C., and Pastusiak A (August 2013) Question answering using enhanced lexical semantic models In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 1744–1753 Association for Computational Linguistics, Sofia, Bulgaria [54] Severyn A and Moschitti A (October 2013) Automatic feature engineering for answer selection and extraction In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp 458–467 Association for Computational Linguistics, Seattle, Washington, USA [55] Severyn A and Moschitti A (2015) Learning to rank short text pairs with convolutional deep neural networks In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval , SIGIR ’15, p 373–382 Association for Computing Machinery, New York, NY, USA ISBN 9781450336215 doi:10.1145/2766462 2767738 104 [56] Bromley J., Guyon I., LeCun Y., Săackinger E., and Shah R (1993) Signature verification using a "siamese" time delay neural network NIPS’93, p 737–744 Morgan Kaufmann Publishers Inc., San Francisco, CA, USA [57] Lai T.M., Bui T., and Li S (August 2018) A review on deep learning techniques applied to answer selection In Proceedings of the 27th International Conference on Computational Linguistics, pp 2132–2144 Association for Computational Linguistics, Santa Fe, New Mexico, USA [58] He H., Gimpel K., and Lin J (September 2015) Multi-perspective sentence similarity modeling with convolutional neural networks In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp 1576–1586 Association for Computational Linguistics, Lisbon, Portugal doi:10.18653/v1/D15-1181 [59] Tayyar Madabushi H., Lee M., and Barnden J (August 2018) Integrating question classification and deep learning for improved answer selection In Proceedings of the 27th International Conference on Computational Linguistics, pp 3283–3294 Association for Computational Linguistics, Santa Fe, New Mexico, USA [60] Rao J., He H., and Lin J (2016) Noise-contrastive estimation for answer selection with deep neural networks In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, CIKM ’16, p 1913–1916 Association for Computing Machinery, New York, NY, USA ISBN 9781450340731 doi:10.1145/2983323.2983872 [61] Yang L., Ai Q., Guo J., and Croft W.B (2016) Anmm: Ranking short answer texts with attention-based neural matching model In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, CIKM ’16, p 287–296 Association for Computing Machinery, New York, NY, USA ISBN 9781450340731 doi: 10.1145/2983323.2983818 [62] Wen J., Ma J., Feng Y., and Zhong M (2018) Hybrid attentive answer selection in cqa with deep users modelling In AAAI [63] Li Y., Du N., Liu C., Xie Y., Fan W., Li Q., Gao J., and Sun H (2017) Reliable medical diagnosis from crowdsourcing: Discover trustworthy answers from non-experts In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, WSDM ’17, p 253–261 Association for Computing Machinery, New York, NY, USA ISBN 9781450346757 doi:10.1145/3018661.3018688 105 [64] Yoon S., Shin J., and Jung K (June 2018) Learning to rank questionanswer pairs using hierarchical recurrent encoder with latent topic clustering In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Papers), pp 1575–1584 Association for Computational Linguistics, New Orleans, Louisiana doi:10.18653/v1/N18-1142 [65] Shen Y., Deng Y., Yang M., Li Y., Du N., Fan W., and Lei K (2018) Knowledge-aware attentive neural network for ranking question answer pairs In The 41st International ACM SIGIR Conference on Research Development in Information Retrieval , SIGIR ’18, p 901–904 Association for Computing Machinery, New York, NY, USA ISBN 9781450356572 doi: 10.1145/3209978.3210081 [66] He H and Lin J (June 2016) Pairwise word interaction modeling with deep neural networks for semantic similarity measurement In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 937– 948 Association for Computational Linguistics, San Diego, California doi: 10.18653/v1/N16-1108 [67] Wang S and Jiang J (2017) A compare-aggregate model for matching text sequences In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings [68] Wang Z., Hamza W., and Florian R (2017) Bilateral multi-perspective matching for natural language sentences In Proceedings of the TwentySixth International Joint Conference on Artificial Intelligence, IJCAI-17 , pp 4144–4150 doi:10.24963/ijcai.2017/579 [69] Bian W., Li S., Yang Z., Chen G., and Lin Z (2017) A compare-aggregate model with dynamic-clip attention for answer selection In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM ’17, p 1987–1990 Association for Computing Machinery, New York, NY, USA ISBN 9781450349185 doi:10.1145/3132847.3133089 [70] Shen G., Yang Y., and Deng Z.H (September 2017) Inter-weighted alignment network for sentence pair modeling In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp 1179– 1189 Association for Computational Linguistics, Copenhagen, Denmark doi:10.18653/v1/D17-1122 106 [71] Howard J and Ruder S (July 2018) Universal language model fine-tuning for text classification In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 328–339 Association for Computational Linguistics, Melbourne, Australia doi:10.18653/v1/P18-1031 [72] Wang Z., Mi H., and Ittycheriah A (2017) Sentence similarity learning by lexical decomposition and composition [73] Yang R., Zhang J., Gao X., Ji F., and Chen H (July 2019) Simple and effective text matching with richer alignment features In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 4699–4709 Association for Computational Linguistics, Florence, Italy doi:10.18653/v1/P19-1465 [74] Rocktăaschel T., Grefenstette E., Hermann K.M., Kociský T., and Blunsom P (2016) Reasoning about entailment with neural attention In Y Bengio and Y LeCun, editors, 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings [75] Wang S and Jiang J (June 2016) Learning natural language inference with LSTM In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 1442–1451 Association for Computational Linguistics, San Diego, California doi:10.18653/v1/N16-1170 [76] Mi H., Wang Z., and Ittycheriah A (November 2016) Supervised attentions for neural machine translation In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp 2283– 2288 Association for Computational Linguistics, Austin, Texas doi: 10.18653/v1/D16-1249 [77] Nguyen M and Nguyen T.H (August 2018) Who is killed by police: Introducing supervised attention for hierarchical LSTMs In Proceedings of the 27th International Conference on Computational Linguistics, pp 2277– 2287 Association for Computational Linguistics, Santa Fe, New Mexico, USA [78] Liu L., Utiyama M., Finch A., and Sumita E (December 2016) Neural machine translation with supervised attention In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp 3093–3102 The COLING 2016 Organizing Committee, Osaka, Japan 107 [79] Zou Y., Gui T., Zhang Q., and Huang X (August 2018) A lexicon-based supervised attention model for neural sentiment analysis In Proceedings of the 27th International Conference on Computational Linguistics, pp 868– 877 Association for Computational Linguistics, Santa Fe, New Mexico, USA [80] Liu S., Chen Y., Liu K., and Zhao J (July 2017) Exploiting argument information to improve event detection via supervised attention mechanisms In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 1789– 1798 Association for Computational Linguistics, Vancouver, Canada doi: 10.18653/v1/P17-1164 [81] Lample G., Ballesteros M., Subramanian S., Kawakami K., and Dyer C (June 2016) Neural architectures for named entity recognition In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 260–270 Association for Computational Linguistics, San Diego, California doi:10.18653/v1/N16-1030 [82] Gajendran S., D M., and Sugumaran V (2020) Character level and word level embedding with bidirectional lstm – dynamic recurrent neural network for biomedical named entity recognition from literature Journal of Biomedical Informatics, 112:p 103609 ISSN 1532-0464 doi: https://doi.org/10.1016/j.jbi.2020.103609 [83] Filice S., Da San Martino G., and Moschitti A (August 2017) KeLP at SemEval-2017 task 3: Learning pairwise patterns in community question answering In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp 326–333 Association for Computational Linguistics, Vancouver, Canada doi:10.18653/v1/S17-2053 [84] Wu G., Sheng Y., Lan M., and Wu Y (August 2017) ECNU at SemEval-2017 task 3: Using traditional and deep learning methods to address community question answering task In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp 365– 369 Association for Computational Linguistics, Vancouver, Canada doi: 10.18653/v1/S17-2060 [85] Chen Q., Zhu X., Ling Z.H., Wei S., Jiang H., and Inkpen D (2017) Enhanced lstm for natural language inference Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) doi:10.18653/v1/p17-1152 108 [86] Wang M (2006) A survey of answer extraction techniques in factoid question answering ravichandran, deepak, abharam ittycheriah, and salim roukos 2003 automatic derivation of surface text patterns for a maximum entropy based question answering system In In Proceedings of the Human Language Technology Conference and North American Chapter of the Association for Computational Linguistics (HLT-NAACL [87] Song H., Ren Z., Liang S., Li P., Ma J., and de Rijke M (2017) Summarizing answers in non-factoid community question-answering In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, WSDM ’17, pp 405–414 ACM, New York, NY, USA ISBN 978-1-4503-4675-7 doi:10.1145/3018661.3018704 [88] Yang W., Xie Y., Lin A., Li X., Tan L., Xiong K., Li M., and Lin J (2019) End-to-end open-domain question answering with bertserini In W Ammar, A Louis, and N Mostafazadeh, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Demonstrations, pp 72–77 Association for Computational Linguistics doi:10.18653/v1/n19-4013 [89] Li J., Luong T., and Jurafsky D (July 2015) A hierarchical neural autoencoder for paragraphs and documents In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp 1106–1115 Association for Computational Linguistics, Beijing, China doi:10.3115/v1/P15-1107 [90] Hinton G and Salakhutdinov R (2006) Reducing the dimensionality of data with neural networks Science, 313(5786):pp 504 – 507 [91] Carbonell J and Goldstein J (1998) The use of mmr, diversity-based reranking for reordering documents and producing summaries In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval , SIGIR ’98, pp 335–336 ACM, New York, NY, USA ISBN 1-58113-015-5 doi:10.1145/290941.291025 [92] Tomasoni M and Huang M (2010) Metadata-aware measures for answer summarization in community question answering In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL ’10, pp 760–769 Association for Computational Linguistics, Stroudsburg, PA, USA 109 [93] Le Q and Mikolov T (22–24 Jun 2014) Distributed representations of sentences and documents In E.P Xing and T Jebara, editors, Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pp 1188–1196 PMLR, Bejing, China [94] Gouws S., Bengio Y., and Corrado G (2015) Bilbowa: Fast bilingual distributed representations without word alignments In Proceedings of the 32nd International Conference on Machine Learning [95] Qiu X and Huang X (2015) Convolutional neural tensor network architecture for community-based question answering In Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI’15, pp 1305– 1311 AAAI Press ISBN 978-1-57735-738-4 [96] Severyn A and Moschitti A (2016) Modeling relational information in question-answer pairs with convolutional neural networks CoRR, abs/1604.01178 [97] Yousefiazar M (2015) Query-oriented single-document summarization using unsupervised deep learning In In Proceedings of Australasian Language Technology Association Workshop, p 210 [98] Nallapati R., Xiang B., and Zhou B (2016) Sequence-to-sequence rnns for text summarization In Workshop track - ICLR 2016 [99] Rush A.M., Chopra S., and Weston J (September 2015) A neural attention model for abstractive sentence summarization In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp 379–389 Association for Computational Linguistics, Lisbon, Portugal doi:10.18653/v1/D15-1044 [100] Chopra S., Auli M., and Rush A.M (June 2016) Abstractive sentence summarization with attentive recurrent neural networks In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 93–98 Association for Computational Linguistics, San Diego, California doi: 10.18653/v1/N16-1012 [101] Zhaochun Ren Hongya Song P.L.S.L.J.M and de Rijke M (2016) Using sparse coding for answer summarization in non-factoid community question-answering [102] Kingma D.P and Ba J (2015) Adam: A method for stochastic optimization In Y Bengio and Y LeCun, editors, 3rd International Conference 110 on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings [103] Lin C.Y (July 2004) ROUGE: A package for automatic evaluation of summaries In Text Summarization Branches Out, pp 74–81 Association for Computational Linguistics, Barcelona, Spain [104] Reimers N and Gurevych I (2019) Sentence-bert: Sentence embeddings using siamese bert-networks CoRR, abs/1908.10084 [105] Zhou G., Chen Y., Zeng D., and Zhao J (2013) Towards faster and better retrieval models for question search In Proceedings of the 22nd ACM International Conference on Information Knowledge Management, CIKM ’13, p 2139–2148 Association for Computing Machinery, New York, NY, USA ISBN 9781450322638 doi:10.1145/2505515.2505550 [106] Zhou G., He T., Zhao J., and Hu P (July 2015) Learning continuous word embedding with metadata for question retrieval in community question answering In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp 250–259 Association for Computational Linguistics, Beijing, China doi:10.3115/ v1/P15-1025 [107] Tay Y., Luu A.T., and Hui S (2017) Enabling efficient question answer retrieval via hyperbolic neural networks ArXiv , abs/1707.07847 [108] Harris Z.S (1954) Distributional structure WORD, 10(2-3):pp 146–162 doi:10.1080/00437956.1954.11659520 [109] Robertson S., Walker S., Jones S., Hancock-Beaulieu M.M., and Gatford M (January 1995) Okapi at trec-3 In Overview of the Third Text REtrieval Conference (TREC-3) [110] Blei D.M., Ng A.Y., and Jordan M.I (2002) Latent dirichlet allocation In T.G Dietterich, S Becker, and Z Ghahramani, editors, Advances in Neural Information Processing Systems 14 , pp 601–608 MIT Press [111] Ji Z., Xu F., Wang B., and He B (2012) Question-answer topic model for question retrieval in community question answering In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM ’12, p 2471–2474 Association for Computing Machinery, New York, NY, USA ISBN 9781450311564 doi:10.1145/2396761.2398669 111 [112] Tan M., Xiang B., and Zhou B Lstm-based deep learning models for nonfactoid answer selection In Workshop track - ICLR 2016 [113] Liu Y., Ott M., Goyal N., Du J., Joshi M., Chen D., Levy O., Lewis M., Zettlemoyer L., and Stoyanov V (2019) Roberta: A robustly optimized bert pretraining approach [114] Hao Y., Dong L., Wei F., and Xu K (2019) Visualizing and understanding the effectiveness of bert Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) [115] Kiros R., Zhu Y., Salakhutdinov R., Zemel R.S., Torralba A., Urtasun R., and Fidler S (2015) Skip-thought vectors In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume , NIPS’15, p 3294–3302 MIT Press, Cambridge, MA, USA [116] Conneau A., Kiela D., Schwenk H., Barrault L., and Bordes A (September 2017) Supervised learning of universal sentence representations from natural language inference data In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp 670–680 Association for Computational Linguistics, Copenhagen, Denmark doi: 10.18653/v1/D17-1070 [117] Cer D., Yang Y., Kong S.y., Hua N., Limtiaco N., St John R., Constant N., Guajardo-Cespedes M., Yuan S., Tar C., Strope B., and Kurzweil R (November 2018) Universal sentence encoder for English In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp 169–174 Association for Computational Linguistics, Brussels, Belgium doi:10.18653/v1/D18-2029 [118] Hill F., Cho K., and Korhonen A (June 2016) Learning distributed representations of sentences from unlabelled data In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 1367–1377 Association for Computational Linguistics, San Diego, California doi: 10.18653/v1/N16-1162 [119] Yang Y., Yuan S., Cer D., Kong S.y., Constant N., Pilar P., Ge H., Sung Y.H., Strope B., and Kurzweil R (July 2018) Learning semantic textual similarity from conversations In Proceedings of The Third Workshop on Representation Learning for NLP , pp 164–174 Association for Computational Linguistics, Melbourne, Australia doi:10.18653/v1/W18-3022 112 [120] Deng Y., Lam W., Xie Y., Chen D., Li Y., Yang M., and Shen Y Joint learning of answer selection and answer summary generation in community question answering In The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20) [121] Sun J., Moosavi S., Ramnath R., and Parthasarathy S (2013) Question difficulty and expertise estimation in community question answering sites In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp 85––90 [122] Sakata W., Shibata T., Tanaka R., and Kurohashi S (2019) Retrieval using query-question similarity and bert-based query-answer relevance In In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’19) [123] Pires T., Schlinger E., and Garrette D (July 2019) How multilingual is multilingual BERT? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 4996–5001 Association for Computational Linguistics, Florence, Italy doi:10.18653/v1/P19-1493 [124] Yang Z., Dai Z., Yang Y., Carbonell J., Salakhutdinov R.R., and Le Q.V (2019) Xlnet: Generalized autoregressive pretraining for language understanding In Advances in Neural Information Processing Systems, volume 32 Curran Associates, Inc [125] Yin W., Schă utze H., Xiang B., and Zhou B (2016) ABCNN: Attentionbased convolutional neural network for modeling sentence pairs Transactions of the Association for Computational Linguistics, 4:pp 259–272 doi:10.1162/tacl_a_00097 [126] Nguyen D.Q and Tuan Nguyen A (November 2020) PhoBERT: Pretrained language models for Vietnamese In Findings of the Association for Computational Linguistics: EMNLP 2020 , pp 1037–1042 Association for Computational Linguistics, Online doi:10.18653/v1/2020.findings-emnlp 92 [127] Nguyen D.Q., Nguyen D.Q., Vu T., Dras M., and Johnson M (May 2018) A fast and accurate Vietnamese word segmenter In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018) European Language Resources Association (ELRA), Miyazaki, Japan [128] Vu T., Nguyen D.Q., Nguyen D.Q., Dras M., and Johnson M (June 2018) VnCoreNLP: A Vietnamese natural language processing toolkit In Pro113 ceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pp 56–60 Association for Computational Linguistics, New Orleans, Louisiana doi: 10.18653/v1/N18-5012 [129] Wiegreffe S and Pinter Y (November 2019) Attention is not not explanation In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp 11–20 Association for Computational Linguistics, Hong Kong, China doi: 10.18653/v1/D19-1002 [130] Cui B., Li Y., Chen M., and Zhang Z (November 2019) Fine-tune BERT with sparse self-attention mechanism In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLPIJCNLP), pp 3548–3553 Association for Computational Linguistics, Hong Kong, China doi:10.18653/v1/D19-1361 [131] Schroff F., Kalenichenko D., and Philbin J (2015) Facenet: A unified embedding for face recognition and clustering 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 815–823 [132] Hadsell R., Chopra S., and LeCun Y (2006) Dimensionality reduction by learning an invariant mapping In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Volume , CVPR ’06, p 1735–1742 IEEE Computer Society, USA ISBN 0769525970 doi:10.1109/CVPR.2006.100 [133] Henderson M.L., Al-Rfou R., Strope B., Sung Y., Lukács L., Guo R., Kumar S., Miklos B., and Kurzweil R (2017) Efficient natural language response suggestion for smart reply CoRR, abs/1705.00652 114 ... dụng hệ thống hỏi đáp cộng đồng Nhận thức điều này, luận án tập trung vào việc nghiên cứu toán hệ thống hỏi đáp cộng đồng Mặc dù có nhiều nghiên cứu tốn NLP toán hệ thống hỏi đáp cộng đồng đề xuất... từ vựng câu hỏi với câu hỏi câu hỏi với câu trả lời khó phức tạp Một lý khác, nghiên cứu toán hỏi đáp cộng đồng phần lớn thực tập liệu tiếng Anh Nghiên cứu hệ thống hỏi đáp cộng đồng liệu tiếng... cách đồng thời thay thực cách Các mơ hình chứng minh phù hợp với liệu cộng đồng toán hỏi hệ thống hỏi đáp CQA Qua việc phân tích kiến trúc hệ thống hỏi đáp cộng đồng, nhiều toán đặt để đặt toán