VIET NAM NATIONAL UNIVERSITY HO CHI MINH CITY HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY FACULTY OF COMPUTER SCIENCE AND ENGINEERING GRADUATION THESIS SMART WAREHOUSE SYSTEM BASED ROBOTIC AUTOMATION AND INTERNET OF THINGS PLATFORM MAJOR: COMPUTER ENGINEERING COUNCIL: COMPUTER ENGINEERING INSTRUCTOR: Dr LE TRONG NHAN REVIEWER: Dr NGUYEN TRAN HUU NGUYEN Student 1: Le Quang Trai 1652620 Student 2: Hoang Ha Tuan Dung 1752145 Ho Chi Minh City, December 2021 - - KHOA: KH & KT Máy tính N: NGÀNH: _ MSSV: 1752145 LÊ QUANG TRÃI _ MSSV: 1652620 _ Smart Warehouse System based Robotic Automation and Internet of Things Platform Robot AGV nhà kho thơng minh p trình 30/03/2021 12/07/2021 1) Ngày tháng _ _ n án: _ TRƯỜNG ĐẠI HỌC BÁCH KHOA KHOA KH & KT MÁY TÍNH CỘNG HỊA XÃ HỘI CHỦ NGHĨA VIỆT NAM Độc lập - Tự - Hạnh phúc -Ngày 12 tháng 08 năm 2021 PHIẾU CHẤM BẢO VỆ LVTN (Dành cho người hướng dẫn) Họ tên SV: HOÀNG HÀ TUẤN DŨNG MSSV: 1752145 Ngành (chuyên ngành): Kỹ thuật máy tính Họ tên SV: LÊ QUANG TRÃI MSSV: 1652620 Ngành (chuyên ngành): Kỹ thuật máy tính Đề tài: Smart Warehouse System based Robotic Automation and Internet of Things Platform Họ tên người hướng dẫn: T.S Lê Trọng Nhân Tổng quát thuyết minh: Số trang: 77 Số chương: 08 Số bảng số liệu: Số hình vẽ: 58 Số tài liệu tham khảo: Phần mềm tính tốn: 01 Hiện vật (sản phẩm): 01 Tổng quát vẽ: - Số vẽ: Bản A1: - Số vẽ vẽ tay Bản A2: Khổ khác: Số vẽ máy tính: Những ưu điểm LVTN:  The most interest in this thesis is the simulation environment, which is developed completely by students themselves, including UI, animations and a tracking service, to capture simulation data for plotting or comparison with different approaches This simulation allows to create events in a smart warehouse, such as packet received, packet delivery, AGV Robot position for real time tracking  A modification of the Dijkstra algorithm is implemented in this project for routing many AGV Robots in a dynamic scenario This work can be inherited by many projects concerning Robot movements  An end-to-end communication is based on OPC-UA protocol, which is a new service in Internet of Things, and is deployed successfully by students This protocol is used to send comments from the main controller to the Robots By using this protocol, a new architecture of the system is proposed by students In this architecture, the Robot is a server, while the main controller, is a client  Due to the COVID 19 pandemic, students show their great efforts to integrate the whole system, with a small Mecanum Robot for a demonstration TR NG I H C BÁCH KHOA KHOA KH & KT MÁY TÍNH C NG HỊA XÃ H I CH NGH A VI T NAM c l p - T - H nh phúc -Ngày tháng n m 2021 PHI U CH M B O V LVTN (Dành cho ng ih ng d n/ph n bi n) H tên SV: LÊ QUANG TR I MSSV:1652620 Ngành (chuyên ngành): H tên SV: HOÀNG HÀ TU N D NG MSSV:1752145 Ngành (chuyên ngành): K THU T MÁY TÍNH K THU T MÁY TÍNH tài: H th ng nhà kho thơng minh d a Robot t đ ng n n t ng k t n i v n v t (Smart Warehouse System based Robotic Automation and Internet of Things Platform) H tên ng i h ng d n/ph n bi n: TS Nguy n Tr n H u Nguyên T ng quát v b n thuy t minh: S trang: 70 S ch ng:8 S b ng s li u:3 S hình v :55 S tài li u tham kh o:14 Ph n m m tính toán: Hi n v t (s n ph m) T ng quát v b n v : -S b nv : B n A1: B n A2: Kh khác: - S b n v v tay S b n v máy tính: Nh ng u m c a LVTN: In this thesis, the students have successfully • Applied pathfinding algorithm for the AGV (Dijkstra) to find the low cost path • Used OPC UA as the communication protocol between AGVs and client • Conducted a virtual environment animation tool to evaluate the control strategy • Created a physical model for demonstration of one AGV moving from an initial position to the destination desire Nh ng thi u sót c a LVTN: The simulation system should consider more on realistic characteristics such as how long a robot running with a specific battery ngh : cb ov  B sung thêm đ b o v  Không đ c b o v  câu h i SV ph i tr l i tr c H i đ ng: a Could you explain the benefits of using OPC UA for communication? b Why did you choose to implement a simulation system by your own, rather than build up from an open source? 10 ánh giá chung (b ng ch : gi i, khá, TB): Gi i i m: 9/10 Ký tên (ghi rõ h tên) Nguy n Tr n H u Nguyên Thesis COMMITMENT We commit that this project is based on our supervisor ideas and knowledge Some considers and information have not been distributed The references, numbers and measurements are quite solid and legitimate The bunch completed the proposal necessities set faculty of Computer Engineering Sincerely, Le Quang Trai Hoang Ha Tuan Dung iv Thesis ACKNOWLEDGEMENT In the beginning, we would express our deepest appreciation to our thesis supervisors, Ph.D Le Trong Nhan He has been there providing his heartfelt support and guidance at all times and has given us invaluable guidance, inspiration, and suggestions in our quests for knowledge during our university time Without his assistance and dedicated involvement in every step throughout the process, this thesis would have never been accomplished We sincerely thank the teachers, who are occupying the Faculty of Computer Science and Engineering in particular and Ho Chi Minh City University of Technology in general, he has always been imparting knowledge in the past four years Their support, encouragement, and credible ideas have been great contributors to the completion of the thesis Last but not least, It would be inappropriate if I omit to thank our friends and family The unconditional love and blessings of our late parents, the care of friends and acquaintances who never let things get dull, have all made a tremendous contribution in helping us reach this stage in our life We thank them for putting up with me in difficult moments where we felt stumped and for goading us on to reach for our passions Finally, we would like to wish you good health and success in your noble life v Thesis ABSTRACT Nowadays, IoT is becoming very popular and widely used in multiple applications Therefore, we think that it is suitable to have the capability to understand and apply this abstract to our thesis Furthermore, using autonomous robots in smart warehouses is not considered an unfamiliar phenomenon, although it is quite common in developed countries, in Viet Nam it is still kind of a new thing So we come up with an idea of combining these abstracts, create and smart warehouse containing autonomous robots using an IoT communication system to work In this project, we propose an overall design of a smart warehouse which contains the warehouse’s statistic and the AGVs system processing inside We draw out a detail map of the warehouse which represent the working environment of the AGVs, based on the map, we apply the centralize management approach (one central controller and multiple AGVS) by using the central to administer appropriate path finding algorithm and sent the result to the AGVs For testing the working accuracy of the path finding algorithm and further development, we design a tool to analyze the result comes out from the algorithm For the data transfer and communication between vehicles, we propose an appropriate communication protocol that is adaptive to the system dynamics For physical testing, we create a demonstration map include line system and RFID cards attached in desire points for location mark, then we make a simple vehicle contains of functions like line following, movement ability, self locate by auto-detech RFID attached on map For each feature, we choose the suitable hardware components and combine them into one AGV Finally, In order to create the realistic result, we make the AGV run on real physical map while connected and receive data from central controller vi CONTENTS Chapter Introduction 1.1 Introduction 1.2 Thesis introduction 1.3 Thesis overview 1.4 Conclusion Chapter Ecosystem for smart warehouse 2.1 Introduction 2.2 Environment Modelling 2.3 AGV Management type (centralize, decentralize) 2.4 Conclusion Chapter System Architecture 3.1 System architecture 3.2 Detail architecture 3.2.1 Warehouse floor - Map design 3.2.2 Communication Protocol - OPC UA connection Chapter Hardware and Software 4.1 Hardware 4.1.1 AGV Operating System 4.1.2 AGV Controller 4.1.3 AGV Sensor 4.1.4 AGV Actuator 4.1.5 Power source 4.1.6 Other 4.1.7 Line circuit diagram of the AGV 4.1.8 Detail connections of the AGV vii 1 3 5 10 10 12 12 13 14 14 14 19 22 26 28 29 31 32 Thesis 4.2 Software 4.2.1 Development Environment Chapter AGV Mobility and Strategy 5.1 Robot Control Principle 5.1.1 Basic Robot control principle 5.1.2 Mecanum Wheel Robot Control Principle 5.1.3 Implementing the IR Sensor Logic 5.1.4 Controlling Direction 5.2 Robot Navigation Rules 5.2.1 Autonomous Guide Vehicle State 5.2.2 Central Control System 5.3 Path Planning 5.3.1 Shortest Path Algorithms 5.3.2 Possible collisions and prevention method 5.4 Conclusion 33 33 36 36 36 37 38 39 42 42 44 45 46 48 49 50 50 51 51 53 53 55 56 57 59 59 59 61 61 Chapter Experiment and Validation 8.1 Validation 8.2 Evaluation 63 63 65 Chapter 6.1 OPC 6.2 OPC 6.3 OPC 6.4 OPC 6.4.1 6.4.2 6.5 OPC 6.6 OPC OPC UA based communication Unified Architecture (OPC UA) definition UA Requirement UA Architechture UA Connection and Data exchange Connection Data exchange UA Security UA Development Tool - UA expert Chapter Prototype Model 7.1 Physical model 7.1.1 Map design for demonstration 7.1.2 AGV design for demonstration 7.2 Final result viii Thesis 8.3 Conclusion ix 69 Thesis and the open document plugins The Address Space pane (lower left window) shows the UA Servers information model Depending on the Node selected in the Browser the Attribute and Reference Windows (upper right and lower right windows) show the attribute of the selected Node and its references within the meshed network of the server address space 58 CHAPTER PROTOTYPE MODEL 7.1 7.1.1 Physical model Map design for demonstration Figure 7.1: Detail map for demonstration The map which we use for our demonstration consists of a total of points (7 RFIDs) As we can see from the above figure, initially, the AGV will start at the ’1 0’ coordinates Depends on the type of cargo it received, 59 Thesis the vehicle will make its way to either destinations marked with the color blue, green, or orange accordingly Then return to the original position and wait for a new instruction to continue processing Figure 7.2: Real map view The picture shows the real map for demonstration 60 Thesis 7.1.2 AGV design for demonstration Figure 7.3: Real AGV in demonstration The demonstration AGV will be the size of 30x20cm, quite small but it has enough functions to serve our purposes 7.2 Final result Here are some pictures of AGV moving on the track: 61 Thesis (a) AGV receive package type (RFID) (b) AGV running on track Figure 7.4: AGV demonstration The two pictures show when the AGV running on a real application, the first one (a) is the steps when the user insert the RFID card, which stands for the type of package and the destination to which the AGV will be directed (based on color), then the AGV will proceed and travel through the line to the desired target (b) For more detail, here are the link direct to our demo clip: Link 62 CHAPTER EXPERIMENT AND VALIDATION 8.1 Validation Validation is an essential process to identify whether the logic and the algorithm used in the thesis are correct or not In here, I build a simple animation with a single robot car on the map then combined with my controller module to verify the execution Figure 8.1: Result of finding shortest path with single robot The figure above illustrates how the single vehicle move in the map with the shortest path algorithm In the figure, there is a yellow vehicle and a pick-up object which is an orange circle Also, the blue digits are the coordinate of the map The value above the edge is the cost of the edge from left to right, and the value below the edge is the cost of the edge from right to left The value on the left of the edge is the cost of the edge from 63 Thesis the bottom up, and the value on the right of the edge is the cost of the edge from the top - down (Note that: Cost of the edge is all the resources the car needs to travel from A to B as well as from B to A.) As we can see, with vehicle from position to position 17 to pick up object, we have the following paths: • 0, 5, 6, 11, 12, 17 =⇒ Total cost: + + 10 + + = 25 • 0, 5, 6, 11, 16, 17 =⇒ Total cost: + + 10 + + = 25 • 0, 5, 6, 7, 12, 17 =⇒ Total cost: + + 10 + + = 26 • 0, 5, 10, 11, 12, 17 =⇒ Total cost: + + + + = 27 • 0, 1, 6, 11, 12, 17 =⇒ Total cost: 10 + 10 + 10 + + = 39 • So the minimum cost path from to 17 is 25 In the example, the selected path is → → → 11 → 12 → 17 In conclusion, our control module selects the right path in order to minimize the traveling cost of the AGV In the following case, we will verify how the system works with multiple vehicle in the map In the below figure, AGV1 is located at will move to position 20; AGV2 is located at will move to position 1; AGV3 is located at will move to position 17 64 Thesis Figure 8.2: Result of finding shortest path with multiple robot The control principle is used to verify the path of each car which based on the priority and lock the path that already use during moving In the example, the priority is ordered AGV1 > AGV2 > AGV3 Let consider three vehicles request the path at the same time Based on the priority, AGV1 gets the path first Then, AGV2 gets the path from the central control In case of AGV3, there is no path which has been found because AGV1 and AGV2 block all the node that AGV3 can use Therefore, in this scenario, there are two AGVs which can move to the orange circle The AGV3 can have it own path when AGV1 reaches it destination and releases all the previous occupy nodes 8.2 Evaluation In this part of the thesis, we conduct an experiment base on our virtual animation tool The research is determined in order to evaluate the difference of time processing between two types of using AGV in warehouse: Single-AGV and multi-AGVs 65 Thesis First of all, we design a specific model for the system which helps to describe briefly on how the warehouse process The model will contain main parts: • The red dashed line area stands for the parking places where all the AGV idle an wait for instructions The vehicle will park at the purple circles inside And the maximum amount of AGVs stays at a time is twelve • rectangles with green line cover are the tasks holder area Each small circle contain one type of package represented by color AGV will move from initial places to this area to receive tasks and packages as they receive instruction from the server • The blue dashed line area placed at the far right side of the map is the destination where packages carried by AGVs will be delivered Each type of package will be bring to the according color • Beside from the line covered areas, the rest, for more specific the chessboard shaped located in the middle of the map was the line system, which contain black line to guide the AGVs Figure 8.3: Warehouse business model As pre-describe in part of the thesis, AGV inside the parking lot (red line covered) will be chosen randomly and directed to one of two packagetaking locations (blue line covered) As long as the cargo taken, the AGV 66 Thesis will be able the determine the cargo typed and request the low-cost path from the central control Then it starts to follow the path sent from control module and reach the desire destination Afther that, the AGV will once again sent to central control and receive the lowest-cost path to return to the parking area The total tasks used in this test is 50 and will be divided into two same part for conveyors which indicates that both conveyor and will consist of 25 tasks accordingly The test comprises with types of goods which differentiate by the color: red, green, orange, purple and blue The virtual tool will automatically generate tasks onto the conveyors with random types of goods The graph below illustrates the way each type represented be color being distributed inside the system In more detail, the vertical shows the amount (number) of cargo each type consists of while the horizontal stands for types of goods Figure 8.4: Packet types distribution As can be seen from the graph, each type of packet was dispensed approximately equal from the total value of 50 tasks This is considered to be an obvious result due to the fact that our intention was to create an uniform distributed experience, based on this information, we can have a general evaluation on the difference between multi-AGV control principle and only one AGV running at a time inside the system Thus, we can 67 Thesis come to a conclusion include either if one AGV or multi-AGV was the better feature and numbers of car running at once which gives out the most beneficial outcome considering time spent and tasks done aspects Figure 8.5: Serving time gantt chart of single AGV The Gantt chart shows the processing time of a single AGV in order to transfer the 50 random tasks pre-declare as mentioned above The vertical line indicates the time by unit A tasks is calculated by take the time when the system pick the vehicle until the AGV finish every steps needed to transfer the goods and then return to the initial destination From the data analyzed from the chart, we can compute that a system include a single AGV will require approximately 22000/50 = 440 unit of time for each tasks One more problem is the working pressure on this single vehicle which has to be active for the whole 22000 unit of time during the process 68 Thesis Figure 8.6: Serving time gantt chart of multiple AGVs In the second chart, similar to the previous one, the number of tasks and the task distribution is unchanged However, in this experiment, multiAGV was applied, with the number of 12 AGVs The figure indicates the amount of nearly 15000 unit of time in total Therefore, time required for each task in this case is 15000/50 = 300 unit of time Beside, the tasks begin divided to multiple vehicle significantly reduce the workload each one has to handle, therefore could extend the working efficiency and life cycle of the whole system 8.3 Conclusion In conclusion, based on the two-chart from the single-AGV and multiAGV models with the random assign task algorithm We can easily point out the first outrageous advantage considered time spent on each task (single-AGV required 440 units of time while it is 300 for multi-AGV) The next one is the workload reduction for the system, by random partition on tasks, the overload problem will not occur because each car will be assigned their own mission sequentially 69 REFERENCES [1] Nast, C., 2021 Your First Look Inside Amazon’s Robot Warehouse of Tomorrow https://www.wired.com/story/amazon-warehouse-robots/ [2] Interlakemecalux.com 2021 Warehouse layout design https://www.interlakemecalux.com/warehouse-manual/warehousedesign/warehouse-layout/ [3] Selecthub.com 2021 Smart Warehousing — About Smart Warehouse Management Systems https://www.selecthub.com/warehouse-management/smartwarehouse-systems/ [4] Pietschmann, C., 2021 Raspberry Pi Vs NVIDIA Jetson Nano Developer Kit — Build5Nines https://build5nines.com/raspberry-pi-4-vs-nvidia-jetson-nanodeveloper-kit/ [5] Store.arduino.cc 2021 Arduino Uno Rev3 — Arduino Official Store https://store.arduino.cc/usa/arduino-uno-rev3/ [6] Store.arduino.cc 2021 Arduino Mega 2560 Rev3 — Arduino Official Store https://store.arduino.cc/usa/mega-2560-r3/ [7] Seeedstudio.com 2021 IMX219-160 8MP Camera with 160° FOV Compatible with NVIDIA Jetson Nano/ Xavier NX https://www.seeedstudio.com/IMX219-160-Camera-160-FOVApplicable-for-Jetson-Nano-p-4603.html/ [8] Raspberrypi.org 2021 Camera Module https://www.raspberrypi.org/products/camera-module-v2/ 70 Thesis [9] Mecanum Wheel https://en.wikipedia.org/wiki/Mecanumw heel/ [10] Gfrerrer, A., 2008 Geometry and kinematics of the Mecanum wheel Computer Aided Geometric Design, 25(9), pp.784-791.8 https://www.sciencedirect.com/science/article/abs/pii/S0167839608000770 [11] Haider Th Salim ALRikabi1*, Abdul Hadi M.Alaidi2, Faisal Theyab Abed3 2018 Attendance System Design And Implementation Based On Radio Frequency Identification (RFID) And Arduino, Jour of Adv Research in Dynamical Control Systems, Vol 10, 04-Special Issue Attendance-System-Design-and-implementation-based-on-RadioFrequency-Identification-RFID-and-Arduino.pdf [12] Colak, I., and Yildirim, D Evolving a Line Following Robot to use in shopping centers for entertainment 2009 35th Annual Conference of IEEE Industrial Electronics, 2009 doi:10.1109/iecon.2009.5415369 [13] Huijuan Wang, Yuan Yu and Quanbo Yuan, ”Application of Dijkstra algorithm in robot path-planning”, 2011 Second International Conference on Mechanic Automation and Control Engineering, 2011 doi:10.1109/mace.2011.5987118 [14] SH Leitner, W Mahnke OPC UA – Service-oriented Architecture for Industrial Applications - ABB Corporate Research Center, 2006 http://cimug.ucaiug.org/kb/knowledge%20base/soa%20for%20industrial%20ap 71 Thesis STUDENT INFORMATION List of thesis authors: Le Quang Trai - ID: 1652620 • Phone number: 03.810.103.740 • Personal Email: 1652620@hcmut.edu.vn Hoang Ha Tuan Dung - ID: 1752145 • Phone number: 0989.788.778 • Personal Email: dung.hoang190699@hcmut.edu.vn 72 ... QUANG TRÃI _ MSSV: 1652620 _ Smart Warehouse System based Robotic Automation and Internet of Things Platform Robot AGV nhà kho thơng minh p trình 30/03/2021 12/07/2021... MSSV: 1652620 Ngành (chuyên ngành): Kỹ thuật máy tính Đề tài: Smart Warehouse System based Robotic Automation and Internet of Things Platform Họ tên người hướng dẫn: T.S Lê Trọng Nhân Tổng quát... nhà kho thơng minh d a Robot t đ ng n n t ng k t n i v n v t (Smart Warehouse System based Robotic Automation and Internet of Things Platform) H tên ng i h ng d n/ph n bi n: TS Nguy n Tr n H u