GENERAL
INTRODUCTION
Monitoring chemical, nuclear, and hazardous environments poses significant challenges for traditional robots However, employing wireless robots for surveillance and assessment enhances safety and efficiency These robots can traverse rough terrains and operate autonomously in unsafe conditions, providing operators with real-time information and updates about their surroundings, thereby ensuring greater safety and operational effectiveness.
OBJECTIVES OF THE THESIS
The integration of data collection and monitoring in robotic systems has significantly enhanced operational efficiency The next phase involves enabling robots to analyze environmental information and execute programmed tasks To address these advancements, the development team has focused on spider robots, selecting the thesis titled "Design, Implementation, and Control of Hexapod Robot Combining Image Processing on Android Platform."
The issues are being studied for the topic
Study on positive and negative kinetic methods of robots
Research image processing recognition - tracking objects on the Android platform
App Design on Android to control robots.
LIMITATION OF TOPIC
The spider robot features a six-legged design, with each leg comprising three joints powered by three RC Servo MG996R motors Each motor has a limited rotation angle ranging from 0 to 180 degrees, which restricts the robot's movement With a total of 18 servos, the robot weighs approximately 8 kg, resulting in a slower movement speed to ensure stability and control during operation.
THE CONTENT OF THESIS
The topic will be presented in parts from theoretical application to application Each issue in the article will be presented through each chapter as follows:
Chapter 3: THE APP APPLICATION DESIGN AND HARDWARE DESIGN Chapter 4: THE ALGORITHM
Chapter 5: THE EXPERIMENT RESULT/FINDING AND ANALYSIS
Figure 2 1: The leg segment model.
THEORITICAL BASIC
KINEMATIC ANALYSIS
The leg structure consists of three key segments: the coxa, femur, and tibia The coxa, located nearest to the body, will be constructed as a servo bracket assembly in the upcoming step Following the coxa, the femur serves as the second segment, while the tibia is the final segment of each leg.
The servos in the hexapod are designated based on the specific leg segments they control The coxa servo is responsible for connecting to the body, while the femur servo facilitates the lifting and lowering of each leg Additionally, the tibia servo enables the bending motion in the middle of the legs.
Table 2 1: Specification for hexapod robot
Figure 2 3: Leg model of hexapod
A hexapod is a kid of walking robot that has a high redundancy degree of freedom (DOF), coupling multi-branched, combine of series and parallel
A hexapod is a robotic structure featuring six legs, each equipped with three degrees of freedom (DOFs), resulting in a total of 18 DOFs The kinematics of a hexapod involves analyzing the joint data from all eighteen legs, determining the end positions of the legs, and understanding the mathematical relationship between the trunk's position and its orientation.
The hexapod robot features a bionic design, inspired by insect anatomy, with each leg possessing three degrees of freedom (DOFs) Its structure includes three legs on the left side of the trunk and three on the right, as depicted in figure 2.2.
In Figure 2.2, the coordinate axes are defined with x along the side of the body, y representing the direction of straight motion, and z pointing upwards The right-hand rule is applied for axis definition The alpha angle denotes the rotation between the body and the base, while the beta angle indicates the rotation between the base section and the femur Additionally, the alpha angle also represents the angle between the base section and the tibia.
Figure 2.4 illustrates the R3 leg position, defined by the rotary joint's coordinate system and the robot's foot position The Denavit-Hartenberg (D-H) parameters will be derived from the matrix conversion, as indicated in equation 2.1.
The leg structure of the robot is the same and parallel string, so making a D-H table for a single leg will apply to the related legs
Using parameters in table 2.2 for the coordinate system in figure 2.4, we calculate the conversion matrix for each link, substitute parameters in table 2.2 into equation 2.1, we obtained [2]:
Table 2 2: The D-H matrix parameter of hexapod robot
The D-H parameters are the angles of two link , link distance ,link length , and the twisting angle of link
Conversion matrix of the entire coordinate system of R3-leg:
We have c , s , c , s , c , s , c , s 1 1 2 2 3 3 23 23 are stand for :
1 1 2 2 3 3 2 3 2 3 cos( ), sin( ), cos( ), sin( ), cos( ), sin( ), cos( ), sin( )
is the endpoint coordinates of leg-R3 in contact with the moving surface, multiply two equation 2.5 and equation 2.6 :
(2 9) Figure 2 5: The coordinate system diagram
2.x l1(cos(arctan( ))) ((2.x l2(cos(arctan( ))) 2.l1.l2) (2.z l2) ) x x arctan(x l2.z l2
INVERSE KINEMATIC
After applying the Denavit-Hartenberg (D-H) method for kinematic analysis, the inverse kinematics analysis is conducted to determine the joint rotation angles required when the spatial position of the mechanical leg's foot end is known This process involves deriving inverse kinematics equations that convert the new position into the corresponding rotation angles for the servo Notably, the root, femur, and tibia are positioned within the same plane, facilitating this analysis.
We add x , y , z in equation 2.6 squared and substitute equation 2.7, equation 2.8 0 0 0 and equation 2.9 We obtain:
Figure 2 6: Rotation motion in the axis.
THE METHOD OF CONTROLLING THE AXIS
To effectively navigate real environments, we must analyze the movements of our six-legged robot, which necessitates solving the positive and negative kinetic problems This understanding enables us to program the robot's legs for coordinated motion, ensuring efficient and precise movement.
With the rotation we have 3 directions in each of the x, y and z axes called roll, pitch and yaw respectively as shown in figure 2.6
To move according to Roll direction and Pitch direction is the same principle, is shown in figure 2.7 x1 y1 x3 y3 y2 x2 x4 y4 y5 x5 x6 y6 y0 x0
' ' cos(yaw) sin(yaw) x x sin(yaw) cos(yaw) y y
' ' x x.cos(yaw) y.sin(yaw) y x.sin(yaw) y.cos(yaw)
Using rotation matrix in 0xz coordinate[4]:
Similarly with the remaining pins, when the 6 legs are updated to the new position, it will create a rotating motion
When the robot rotates around the yaw angle, all six legs simultaneously adjust their positions using a rotation matrix along the Z-axis of the central body, resulting in a coordinated rotating motion The updated positions are calculated according to equation 2.16.
To translate centroid according y0 direction with a m-distance:
To translate centroid according x0 direction with m-distance:
OPENCV LIBRARY
OpenCV, or Open Source Computer Vision Library, is a versatile software library designed for computer vision and machine learning applications It serves as a foundational infrastructure that enhances the implementation of machine perception in commercial products With its BSD license, OpenCV allows businesses to easily adopt and customize the code to meet their specific needs.
The library features over 2,500 optimized algorithms, encompassing a wide range of both classic and cutting-edge computer vision and machine learning techniques These powerful algorithms are designed for tasks such as face detection and recognition, object identification, and classification.
Figure 2 10: RGB to HSV conversion
The RGB model plays a crucial role in various applications, including tracking human actions in videos, monitoring camera movements, and identifying moving objects It enables the extraction of 3D models and the generation of 3D point clouds using stereo cameras, as well as the stitching of images to create high-resolution visuals of entire scenes Additionally, it facilitates the search for similar images in databases, the removal of red-eye effects in flash photography, and the tracking of eye movements OpenCV, a widely-used library with a community of over 47,000 users and more than 18 million downloads, is integral for companies, research institutions, and government agencies in implementing these advanced features, including scenery recognition and augmented reality overlay.
OpenCV is a versatile computer vision library that supports multiple programming languages, including C++, Python, Java, and MATLAB, and is compatible with various operating systems such as Windows, Linux, Android, and Mac OS Primarily focused on real-time vision applications, OpenCV utilizes MMX and SSE instructions for enhanced performance The library is currently enhancing its capabilities with full-featured CUDA and OpenCL interfaces With over 500 algorithms and approximately 5,000 supporting functions, OpenCV is written in C++ and features a template interface that integrates smoothly with STL containers.
In this project, We use the OpenCV library version 3.4.10 for Java programming on Android studio
HSV components serve as effective feature vectors for image retrieval and classification, representing colors through three key elements: Hue, Saturation, and Value Archana et al demonstrated the application of HSV in a multiple face detection method, achieving an impressive accuracy of 93% Additionally, the RGB to HSV conversion algorithm is essential for video processing tasks.
The RGB color space is the predominant color model utilized in computer graphics, representing colors through varying intensities of red (R), green (G), and blue (B) This color space can be effectively visualized as a cube within a three-dimensional Cartesian coordinate system.
The RGB color model, which consists of red, green, and blue components, is essential for video processing and is represented in a standard 24-bit format Each color channel utilizes 8 bits, allowing for a total of 16,777,216 possible colors, with component values ranging from 0 to 255 as unsigned integers However, in the conversion algorithm, the RGB channels must be expressed as floating-point numbers within the range of [0, 1], as illustrated in image figure 2.11.
The HSV color space consists of three key components: hue (H), saturation (S), and value (V), which together represent colors in a way that aligns with human color perception The hue component (H) specifically identifies the true colors, including red, yellow, green, cyan, blue, and magenta.
The saturation component (S) represents color purity It gives a measure of how much the true color is diluted by white Decreasing saturation (S) corresponds to increasing whiteness
The value component (V) in the HSV color model represents brightness, indicating how far a color deviates from black A decrease in value corresponds to an increase in blackness Unlike the RGB model, HSV is structured as a hexagonal cone, providing a distinct coordinate system for color representation.
The horizontal cross-section of hexaone features hexagons of various sizes, determined by the value of V In the classical layout, the color coding is as follows: 0° is represented by red, 60° by yellow, 120° by green, 180° by cyan, 240° by blue, and 300° by magenta.
The conversion from RGB to HSV utilizes the HSV hexaone algorithm, as outlined in the referenced paper The process involves selecting the maximum and minimum values from the inputted R, G, and B components The calculations for the HSV color space are detailed in equations 2.21 to 2.27.
a , if f (x, y) Threhold2 g(x, y) b, if Threshold1 f (x, y) Threhold2 c,if f (x, y) Threshold1
To detect isolated colors in an HSV image, multiple masks must be applied, partitioning pixels based on their intensity values This involves using low and high threshold masks for hue, saturation, and value Thresholding converts the image into a binary format, where pixels that meet the threshold criteria are set to white (255), while those that do not are set to black (0) The threshold algorithm is mathematically defined in equations 2.28 and 2.29.
Multiple thresholding divides pixels into multiple partitions with more Threshold values as shown in …
Figure 2 13: The image of a ball.
The threshold algorithm is designed to isolate regions in an image that correspond to the objects of interest for analysis This separation is achieved by examining the intensity differences between the pixels of the object and those of the background.
Mathematical morphology is defined from two basic algorithm They are Erosion and Dilation
An essential part of the erosion and dilation operations is structuring element used to probe the input image A structuring element is a matrix which have any arbitrary shape and size
A two-dimensional binary matrix is utilized in image processing, where the structuring elements are generally smaller than the image itself The center of the structuring element is referred to as the origin Common shapes for these structuring elements include rectangular, elliptical, and cross-shaped, with rectangular elements often employed for processing images captured by cellphone cameras.
Erosion is a fundamental operation in mathematical morphology, specifically used for binary images This operator effectively erodes the boundaries of foreground pixel regions, resulting in a reduction of their size and an increase in the size of holes within those areas.
DETECTING CIRCLE USING HOUGH CIRCLE TRANFORM
Canny edge detection is a widely used algorithm designed to optimally identify edges in images It involves five key processes: first, the image is smoothed using Gaussian convolution; next, the gradient of the image is calculated to determine edge intensity and direction The algorithm then applies non-maximum suppression to eliminate non-maximum pixel values After this step, some edge pixels may still contain noise, so the algorithm filters out pixels with weak gradient values while preserving those with high gradient values.
The final steps is edge tracking by hysteresis, this algorithm has determined what strong edges and weak edges The steps of canny edge detection algorithm in figure 2.20
Figure 2 20: Block diagram of Canny edge detection technique
Image noise can significantly impact edge detection results To mitigate this noise, applying Gaussian blur is an effective method for smoothing images This involves convolving the image with a Gaussian filter kernel, typically using sizes such as 3x3, 5x5, or 9x9 A 5x5 kernel is generally considered optimal for most applications The Gaussian filter kernel's equation for a size of (2k+1)x(2k+1) is essential for this process.
There is a 5x5 Gaussian filter with = 1.4 [10]
The gradient calculation identifies edge intensity and direction by analyzing the image's gradient To effectively detect edges, applying filters that emphasize intensity changes in both horizontal (x) and vertical (y) directions is the simplest approach.
Then, the magnitude G and the slope of the gradient are calculated as [11] :
In image processing, Gx represents the derivative in the horizontal direction, while Gy denotes the derivative in the vertical direction The edge direction angle is categorized into four main orientations: vertical, horizontal, and diagonal, with specific angle values of 0°, 45°, 90°, and 135°.
Non-maximum suppression is an edge thinning technique that processes the gradient intensity matrix to identify pixels with the highest values in the direction of the edge This algorithm evaluates the intensity of neighboring pixels along the same direction to determine if they are more or less intense than the pixel currently being analyzed.
When the gradient angle is 0 degrees, the edge direction is horizontal, with pixels A7 and A3 being processed alongside pixel A If both A7 and A3 have lower intensity than pixel A, the intensity of pixel A is set to 1 At a 90-degree angle, the edge direction becomes vertical, involving pixels A1 and A5; if these pixels are less intense than pixel A, it is classified as an edge pixel Similarly, at a 135-degree angle, the edge direction is diagonal, with pixels A2 and A6 If both of these pixels are less intense than pixel A, pixel A is also treated as an edge pixel.
The double threshold technique is used to categorize pixels into three distinct types: strong, weak, and non-relevant Strong pixels exhibit high intensity, while weak pixels have insufficient intensity for evaluation Non-relevant pixels, on the other hand, are considered irrelevant edges.
Upper threshold is used to identify the strong pixels
Lower threshold is used to identify the non-relevant pixels
All pixels have intensity between both threshold are weak pixels
Figure 2 22: The weak pixel link strong edge.
Figure 2 24: a) A-dimension function in intensity; b) Derivative of function
The algorithm enhances weak pixels, resulting in a final image dominated by strong pixels Pixels with a gradient magnitude exceeding the upper threshold are classified as strong edge pixels, while those with a magnitude between the two thresholds are identified as weak edge pixels Weak edges can be connected to strong edges, as illustrated in figure 2.22.
DETECT LOCATION OF CIRCLE
The final stage of image processing in this thesis involves detecting the center and radius of circles, utilizing principles of Canny edge detection This stage focuses on a nose-free grayscale input image, ensuring there are no random intensity variations The detection process consists of three key stages, which are illustrated in the accompanying block diagram.
2.6.1 CONVOLUTION AND NON-MAXIMUM SUPPRESSION
Finding Center and Radius of the circle
Figure 2 23: The diagram of detection circle.
Figure 2 25: The kernel for finding Horizontal gradient orientation.
The initial phase of image processing involves convolution and Non-Maximum Suppression, where a grayscale image serves as the input This stage utilizes horizontal and vertical kernels to determine the directional gradients for each pixel, rounding these values to one of eight possible directions Additionally, these kernels are employed to calculate the edge strength of the image, enhancing the detection of features.
Edges in an image signify the boundaries of objects, leading to abrupt changes in image intensity This approach facilitates the rapid identification of circles within an image while optimizing resource usage Objects typically exhibit various colors, hues, or light intensities, resulting in significant intensity variations Consequently, the geometric details represented in a line drawing are effectively captured through these intensity changes The algorithm operates under the assumption of a noise-free grayscale image, ensuring minimal random intensity fluctuations between adjacent pixels.
Figure 2.24 illustrates the process of edge detection through intensity changes In 2.24a, a one-dimensional edge is presented, and applying a convolution kernel to this signal results in the transformation seen in figure 2.24b.
Figure 2.24b shows the maximum positioned at the center of the original signal's edge By extending one-dimensional analysis to two dimensions, we utilize four 3x1 window operators for a grayscale digital image This approach allows us to determine the horizontal (Kx) and vertical (Ky) pixel directions for each pixel while considering its neighboring pixels.
X serves as the primary factor for the two kernels mentioned above In both Kx1 and Kx2, the center values of the first and last columns are set to 2, while all other values are maintained at 1 This configuration highlights the significance of the central values.
Figure 2 26: Kernel for finding vertical gradient orientation.
I _ hor(x, y) Ix2(x, y) Ix1(x, y) (2 41) values of the horizontal neighboring pixels compared to the others Similarly, below is kernels used to find vertical gradient orientation:
X serves as the primary factor for the kernels Ky1 and Ky2, where the center values of both the first and last rows are set to 2, while all other values are 1 This design highlights the significance of vertical neighboring pixels in relation to others The kernels Kx1, Kx2, Ky1, and Ky2 are employed to assess pixel direction and edge strength in both horizontal and vertical orientations.
In this section, we will explore the methods for calculating pixel direction and edge strength Pixel direction is determined by assessing both horizontal and vertical orientations Meanwhile, edge strength is calculated by analyzing horizontal and vertical orientations alongside the horizontal and vertical gradient magnitudes.
In the analysis of image processing, Equation (2.39) represents the convolution of kernel Kx1 with Image I, while Equation (2.40) denotes the convolution of kernel Kx2 with the same image The resulting values, Ix1 and Ix2, are crucial for determining the horizontal orientation of a pixel at coordinates (x, y) within the image Furthermore, Equation (2.41) illustrates the calculation of the horizontal gradient magnitude at the point I(x, y) Consequently, this process yields essential information regarding both the horizontal orientation and horizontal gradient magnitude of the image.
If Ix1(x, y) > Ix2(x, y) then the orientation I_h_ore is:
If Ix1(x, y) < Ix2(x, y) then the orientation I_h_ore is:
If Ix1(x, y) = Ix2(x, y) then the orientation is: 0
Equation 2.43 is the convolution of kernel Ky1 and Image I while equation 2.44 is the convolution of kernel Ky2 and Image I These two sets of values, Iy1 and Iy2, determine
The vertical orientation of a pixel point (x, y) in the image I is described by equation 2.44, while equation 2.45 provides the method for calculating the vertical gradient magnitude at that point Together, these two key pieces of information—vertical orientation and vertical gradient magnitude—are essential for image analysis.
If Iy1(x, y) > Iy2(x, y) then the orientation I_v_ore is:
If Iy1(x, y) < Iy2(x, y) then the orientation I_v_ore is:
If Iy1(x, y) = Iy2(x, y) then the orientation is: 0
To analyze a pixel (x, y), four essential pieces of information are required: the horizontal orientation I_h_ore, the vertical orientation I_v_ore, the horizontal gradient magnitude I_hor(x, y), and the vertical gradient magnitude I_ver(x, y) The edge strength is then determined using these horizontal and vertical gradient magnitudes, as outlined in equation 2.46.
The pixel orientation is determined by the vertical and horizontal alignment of each point in the image, based on specific pre-defined conditions outlined in Table 2.3.
Table 2 3: Conditions used to determine the pixel direction
Figure 2 27: Range of angles of pixel direction.
The aforementioned processes are applied to every pixel in the image to calculate pixel direction and edge strength These values are subsequently utilized by Non-Maximum Suppression to identify thin edges of objects.
Non-Maximum Suppression generates a binary image consisting of thin edge points, reducing edge width to just one pixel Following this, the convolved image is analyzed for pixel direction and edge strength During this scanning process, any pixel direction other than zero prompts a search for neighboring pixels with the same direction.
A linear window, measuring 3 pixels, is utilized to analyze immediate neighboring pixels based on edge strength and pixel direction In this process, the pixel with the highest edge strength within the linear window is selected, while all other pixels are set to zero An example of this Non-Maximum Suppression (NMS) technique is illustrated in Table 2.4.
Table 2 4: Image with linear window in color corresponding to the pixel direction and gradient magnitude and result of NMS
DATA TRANSMISSION
Wireless communication technology has become a cornerstone of modern communication, enabling seamless connectivity between mobile devices, robots, and other gadgets This thesis focuses on the integration of Bluetooth and Firebase technologies, utilizing smartphones to control robot movements and monitor environmental conditions The advantages of wireless technology include faster response times, real-time control, and enhanced communication between clients and hosts.
Firebase is a robust platform designed for mobile and web applications, offering essential back-end services such as data storage, user authentication, and static hosting With Firebase, developers can effortlessly create scalable apps that accommodate a growing user base, from a single user to one million.
In 2011, Envolve, the precursor to Firebase, emerged as a startup offering developers an API that facilitated the integration of online chat features into websites.
Envolve was utilized by developers not only for chat messages but also to synchronize application data in real time, such as game states, among users This innovative use of Envolve enhanced the interactivity and engagement of applications, allowing for seamless data sharing across platforms.
In April 2012, Envolve founders James Tamplin and Andrew Lee established Firebase as a separate company, focusing on Backend-as-a-Service solutions that featured real-time capabilities, following their decision to separate the chat system from the real-time architecture.
Figure 2 38: The features in Firebase.
After it was acquired by Google in 2014, Firebase rapidly evolved into the multifunctional behemoth of a mobile and web platform that it is today
Firebase helps mobile appteams succeed With firebase you can:
Firebase lets you build more powerful, secure and scalable apps, using world-class infrastructure using:
1 Cloud Firestore: is a flexible, scalable database for mobile, web, and server development from Firebase and Google Cloud Platform It is a NoSQL document database that lets you easily store, sync, and query data for your mobile and web apps
— at a global scale.It's supporting for Android, iOS and Web Platform
2 ML Kit: is a mobile SDK that brings Google’s machine learning expertise to Android and iOS apps in a powerful yet easy-to-use package Whether you’re new or experienced in machine learning, you can implement the functionality you need in just a few lines of code There’s no need to have deep knowledge of neural networks or model optimization to get started On the other hand, if you are an experienced ML developer, ML Kit provides convenient APIs that help you use your custom TensorFlow Lite models in your mobile apps It’s supporting for Android and iOS Platform
3 Cloud Functions: for Firebase lets you automatically run backend code in response to events triggered by Firebase features and HTTPS requests Your code is stored in Google’s cloud and runs in a managed environment There’s no need to manage and scale your own servers It’s supporting for Android, iOS, C++, Unity and Web Platform
4 Authentication: provides backend services, easy-to-use SDKs, and ready-made UI libraries to authenticate users to your app It supports authentication using passwords, phone numbers, popular federated identity providers like Google, Facebook and Twitter, and more It’s supporting for Android, iOS and Web Platform
5 Hosting: is production-grade web content hosting for developers With a single command, you can quickly deploy web apps and serve both static and dynamic content to a global CDN (content delivery network) You can also pair Firebase Hosting with Cloud Functions to build and host micro services on Firebase It’s supporting only Web Platform
6 Cloud Storages: is for object storage service built for Google scale The Firebase SDKs for Cloud Storage add Google security to file uploads and downloads for your Firebase apps, regardless of network quality You can use our SDKs to store images, audio, video, or other user-generated content On the server, you can use Google Cloud Storage, to access the same files It’s supporting for Android, iOS, C++, Unity and Web Platform
7 Real-time Database: is a cloud-hosted NoSQL database that lets you store and sync between your users in real-time The Real-time Database is really just one big JSON object that the developers can manage in real-time It’s supporting for Android, iOS, C++, Unity and Web Platform
Figure 2 40: Real-time Database be become A Tree of Values
The Firebase Real-time Database is a cloud-hosted NoSQL database that lets you store and sync between your users in real-time
The Real-time Database is really just one big JSON object that the developers can manage in real-time
With just a single API, the Firebase database provides your app with both the current value of the data and any updates to that data
Real-time syncing enables users to effortlessly access their data across various devices, whether on the web or mobile Additionally, a real-time database fosters collaboration among users, enhancing their overall experience.
Another amazing benefit of Real-time Database is that it ships with mobile and web SDKs, allowing you to build your apps without the need for servers
When users go offline, the Real-time Database SDKs utilize local caching on the device to manage and store changes Once the device reconnects to the internet, the local data is automatically synchronized, ensuring seamless updates.
The Real-time Database can also integrate with Firebase Authentication to provide a simple and intuitive authentication process
Figure 2 41: The Firebase application connect the other applications on smartphone
Firebase Authentication provides backend services, easy-to-use SDKs, and ready- made UI libraries to authenticate users to your app
Setting up your own authentication system can take months and require a dedicated maintenance team, but with Firebase, you can create a complete authentication solution in under 10 lines of code Firebase simplifies the process by managing everything for you, including complex tasks like account merging.
You can authenticate your app’s users through the following methods:
Using Firebase Authentication makes building secure authentication systems easier, while also improving the sign-in and onboarding experience for end users
Firebase Authentication is built by the same people who created Google Sign-in, Smart Lock, and Chrome Password Manager
Figure 2 43: The Real-time database.
INTRODUCTION TO ANDROID STUDIO
Android Studio, the official Integrated Development Environment (IDE) for Android app development, is built on the robust IntelliJ IDEA platform It not only incorporates IntelliJ's advanced code editor and developer tools but also provides additional features that significantly boost productivity for developers creating Android applications.
A flexible Gradle-based build system
A fast and feature-rich emulator
A unified environment where you can develop for all Android devices
Apply Changes to push code and resource changes to your running app without restarting your app
Code templates and GitHub integration to help you build common app features and import sample code
Extensive testing tools and frameworks
Lint tools to catch performance, usability, version compatibility, and other problems
Built-in support for Google Cloud Platform, making it easy to integrate Google Cloud Messaging and App Engine
Figure 2 48: The project files in Android view
Each project in Android Studio contains one or more modules with source code files and resource files Types of modules include:
Android Studio organizes your project files in the Android project view by default, allowing for easy access to essential source files through a module-based structure, as illustrated in figure 1.
Figure 2 49:The project files in Problems view, showing a layout file with a problem.
All the build files are visible at the top level under Gradle Scripts and each app module contains the following folders:
manifests: Contains the AndroidManifest.xml file
java: Contains the Java source code files, including JUnit test code
res: Contains all non-code resources, such as XML layouts, UI strings, and bitmap images
The Android project structure on disk differs from this flattened representation To see the actual file structure of the project, select Project from the Project dropdown (in figure
Customize your project file view to emphasize particular aspects of app development By selecting the Problems view, you can easily access links to source files that highlight coding and syntax errors, such as missing closing tags for XML elements in layout files.
The Android Studio main window is made up of several logical areas identified.
1- The toolbar lets you carry out a wide range of actions, including running your app and launching Android tools
2- The navigation bar helps you navigate through your project and open files for editing
It provides a more compact view of the structure visible in the Project window
3- The editor window is where you create and modify code Depending on the current file type, the editor can change For example, when viewing a layout file, the editor displays the Layout Editor
4- The tool window bar runs around the outside of the IDE window and contains the buttons that allow you to expand or collapse individual tool windows
5- The tool windows give you access to specific tasks like project management, search, version control, and more You can expand them and collapse them
Figure 2 51: Complete process of Kalman filter.
6- The status bar displays the status of your project and the IDE itself, as well as any warnings or messages
Maximize your screen space by organizing the main window, which includes hiding or moving toolbars and tool windows Additionally, enhance your efficiency by utilizing keyboard shortcuts to quickly access most features of the IDE.
You can easily search your source code, databases, and user interface elements in Android Studio by double-pressing the Shift key or clicking the magnifying glass icon in the upper right corner This feature is particularly helpful for quickly locating specific IDE actions that you may have forgotten how to trigger.
KALMAN FILTER
To ensure computational efficiency, a discrete-time Kalman filter is utilized, and the implementation remains straightforward, as all processing will be conducted on an Arduino microprocessor equipped with element-wise computational capabilities.
The Kalman filter is a sophisticated algorithm that processes a sequence of measurements over time, primarily utilizing data from accelerometers and gyro sensors It consists of two key components: the time update equation, which predicts future estimates based on current values and covariance, and the measurement update equation, which refines these predictions by integrating new measurements This iterative process involves continually updating the prior estimates with new data to enhance accuracy, making the Kalman filter essential for applications requiring precise tracking and estimation.
Supposing we have a simple state having only position and velocity:
The actual position and velocity of an object can be represented by a range of possible combinations, with some being more probable than others The Kalman filter operates under the assumption that both position and velocity are random variables following a Gaussian distribution, characterized by a mean value (μ) indicating the most likely state and a variance (σ²) representing uncertainty By estimating a new position based on a previous one, the filter highlights that higher velocities typically correspond to greater distances in position Additionally, the relationship between these variables is captured through a covariance matrix, which reflects how one measurement can provide insights into others.
In the prediction step, the estimate at time k, denoted as ˆx k , incorporates known external influences represented by μ and its covariance matrix P k Utilizing a matrix F, each state within the original estimate can transition to a range of potential states, with each point in xˆ k 1 being relocated within a Gaussian distribution characterized by covariance Q k This process generates a new Gaussian distribution that maintains the same mean but exhibits a different covariance The overall expression for the prediction step is achieved by expanding the covariance through the addition of Q k.
This is a linear model: states written by vectors, while variables are written in matrices
Where B is the control matrix, u is the control vector For simple application, these factors can be omitted
To enhance the accuracy of our estimates, we must refine them using measurements from sensors that likely track velocity or position Each sensor value provides indirect insights into the system's state, although the units and scales may differ from those of the tracked state The H matrix serves as the observation model for these sensors, allowing us to represent the distribution of sensor readings effectively.
Figure 2 52: Combination of two Gaussian curves.
Sensors can be unreliable, leading to a range of readings for each estimated state From these readings, we can infer the system's state, but uncertainty remains, represented by the covariance of this uncertainty, denoted as Rk Each measurement, referred to as zk, has a mean that reflects this distribution.
To determine the most probable state, we combine two Gaussian distributions: one centered around the mean of the transformed prediction and the other around the sensor reading.
Consider as 1D Gaussian bell curve with variance σ 2 and mean μ [5] is defined as:
Substitute equation (2.54) into equation (2.55) to obtain: Combination of two Gaussian curves [18]
Re-write (2.58) and (2.59) in matrix form with Σ is the covariance matrix of a Gaussian blob, and u is its mean along each axis:
K is called the Kalman Gain Now putting all together, we have two distributions: the predict measurement with ( 0 , 0 )(Hx HP H k , k T )and the observed measurement with
( , )(z k ,R k ) Plug these into equation (2.61) to find their overlap [17] :
Knock an H off the front of every term in (2.62) and (2.63) (one is hiding inside K), and an H T off the end of all terms in the equation for P ' [17]
So (2.65), (2.66), (2.67) complete equations of the measurement update step giving
' x k is the new best estimate
In essence a Kalman filter will have these steps:
Based on the measurement results to correct the estimate
Implement Kalman filter on C language:
For implementing on an IMU sensor, here the system state at time k:
Where xk is the state matrix given by:
The filter output includes the angle θ and the bias θb', which represents the gyro's drift To obtain the true rate, simply subtract the bias from the gyro measurement.
A is the prediction matrix defined as:
Next is the input control uk, in this case is angular velocity ( ) from gyroscope measurement (deg/s) at time k
B is the input-control matrix Since the u k is the angular velocity and the bias can’t be calculated directly then B is defined as:
From equation (2.68), (2.69), (2.70), (2.71), the system state is described as follows:
When tracking a two-wheel robot, it's essential to account for noise in the environment, such as wheel slippage or bumps on the ground that may hinder its movement To effectively manage this uncertainty, it's crucial to model the variability after each prediction step, ensuring more accurate tracking and performance assessment.
In the process of estimation, each point in the x k-1 is relocated within a Gaussian distribution characterized by the covariance matrix Q k Here, the bias and accelerometer estimates are treated independently, leading to Q k being equivalent to the variance of both the accelerometer and bias estimates Consequently, Q k is defined accordingly.
The covariance matrix Q k depends on the current time k, so the variance of the accelerometer and the bias is multiplied by the delta time t
Predict the covariance matrix P k of x k by using the following formula:
Where A is the prediction matrix (2.70) used above to estimate x k
Then we have the estimation of P k after expanding by adding Q k [16]
Next, move to the Correct process [17]:
Update estimate with measurement zk [16]
Figure 3 2: The main control layout.
THE APP APPLICATION DESIGN AND HARDWARE DESIGN 49
APP APPLICATION DESIGN
Figure 3 3: The members in the same team.
3 The Bluetooth devices which is available for pairing
4 The bottom Navigation bar for changing connection Mode
-Turn on this device’s Bluetooth adapter
-Turn off this device’s Bluetooth adapter
-Start discovering slave device in 120s
Figure 3 5: The Bluetooth control layout.
Figure 3 4: The navigation drawer menu for showing other screen.
-The Bluetooth Receiving Data Console’s
-The Internet Receiving Data Console’s
1 Touch coordinates: Where your finger is on screen
2 RGB color values at this coordinates
3 HSV color values at this coordinates in range H [0, 179], S [0, 255], V [0, 255]
4 HSV color values at this coordinates in range H [0, 360], S [0, 1], V [0, 1]
7 Capture screen for image processing
-The Bluetooth Console’s opened when click this button on main screen
-The Tracking mode Console’s opened when click this button on main screen
Figure 3 6: The camera phone layout.
1 Function button: Preset movements for Hexapod
4 Three basic Hexapod’s rotation angle showing by
Figure 3 7: The Internet control layout.
1 Basic navigation motion of Hexapod
2 Data read from real-time database
3 Seek-bar for controlling camera
1 The Bluetooth address that this devices is connecting with
2 Data read from real-time database
-The Internet Console’s opened when click this bottom button on main screen
-The Internet Receiving data Console’s opened
Figure 3 8: The monitoring data layout.
HARDWARE DESIGN
The Arduino Mega 2560 board, built on the ATMega2560 chip from Atmel's 8-bit AVR microprocessor, is designed for programming complex control applications It features robust ROM, RAM, and Flash memory, along with digital I/O ports, multiple terminals for PWM and analog signal output, and supports communication standards like UART, SPI, and TWI (I2C).
Table 3 1: Technical specifications for board Arduino Mega 2560
DC current per I/O pin 20mA
DC current for 3.3V pin 50mA
Figure 3 10: Module Buck DC-DC 12A.
The Arduino board can be powered through a DC jack that accepts 7-12V, or via the Vin pin Additionally, if you have a 5V external power source, you can utilize the 5V pin to power the Arduino.
• PWM pulse feet: There are 15 PWM digital pins (pins 2-13, pins 44, 45, 46) in threads using pins 44 and 45 for right and left motors
• I2C communication: 2 pins Digital I2C communication is 20 (SDA) and 21 (SCL)
• Serial communication: Supports up to 4 Serial ports 0, 1, 2, 3 (Pin order 0, 1, 19,
18, 17, 16, 15, 14) Here we use Serial 3 to communicate with Bluetooth module HC05
• Interrupts: There are 6 external interrupt pins 0, 1, 2, 3, 4, 5 (order of pins are 2, 3,
21, 20, 19, 18) Here we use interrupt 0, 1 to read 2 channels A of the two encoders of the left and right motor.
The Arduino MEGA lacks integrated 3.3V and 5V power circuits, making it challenging to provide standard voltage levels for modules or sensors Additionally, with a fixed 12V power supply for the robot, utilizing a BUCK DC-DC converter is essential to efficiently step down the voltage to 5V for the driver controller and processor.
Table 3 2: Technical specifications for module Buck DC-DC 12A
• Short circuit protection: current limitation 14A, Fuse
• Overheating protection: Yes (for temperature, automatic shutdown after disconnecting the output)
• Input reverse polarity protection: No, (if required in the input line to the diode)
• Module dimensions: 60 mm x 51 mm x 22 mm (L x W x H)
Specifications Module BUCK DC-DC 12A
Module dimensions 60 mm x 51 mm x 22 mm
The HC05 Bluetooth module provides data bandwidth of up to 720 Kbps with a range of 10 to 100 meters Unlike infrared (IrDA), which requires devices to be in close proximity, Bluetooth utilizes multidirectional radio waves, enabling communication through non-metallic barriers Operating at a frequency of 2.4GHz, Bluetooth employs continuous scalar broadcast technology for efficient data transmission.
Table 3 3: Specification for module HC05
Current 30mA when paring, after paring the normal transmission operation 8mA
Bluetooth protocol Bluetooth Specification v2.0 EDRo
Figure 3 13: MPU 6050 – Accelerometer and Gyroscope module.
The MPU6050 is a widely used Inertial Measurement Unit (IMU) sensor that combines two accelerometers and a gyroscope Commonly found in smartphones, handheld controllers, and wearable electronic devices, IMU sensors play a crucial role in determining the three-dimensional state of objects they are attached to.
The module features two auxiliary pins for optional interfacing with external IIC modules, such as a magnetometer Its configurable I2C address allows for multiple MPU6050 sensors to connect to a microcontroller via the AD0 pin Additionally, well-documented libraries make it user-friendly with popular platforms like Arduino If you're seeking a sensor to manage motion in applications like RC cars, drones, self-balancing robots, humanoids, or bipeds, this sensor could be the ideal choice for you.
Table 3 4: Pins of module MPU6050
The hardware of the module is very simple, it actually comprises of the MPU6050 as the main components as shown above Since the module works on 3.3V, a voltage regulator
Serial Clock (SCL) I2C clock pin
Serial Data (SDA) I2C data communication pin
Auxiliary Serial Data (XDA) Data pin (connected to other sensors) Auxiliary Serial Clock (XCL) Clock pulse pin (connect to other sensors)
Interrupt (INT) Interrupt pin to indicate that data is available for MCU to read
Figure 3 14: The schematic diagram. is also used The IIC lines are pulled high using a 4.7k resistor and the interrupt pin is pulled down using another 4.7k resistor
• Within the framework of the use of the default I2C address and no external interrupts and other auxiliary sensors, XDA, XCL, INT, AD0 pins will not be used
• I2C is a two-wire interface consisting of a data line (SDA) and a single wire (SCL)
In I2C, a device can be Master or Slave Each slave has a fixed address, Master has to access corresponding address to call that slave
• In communication with the Arduino, MPU6050 in Slave mode, the maximum bus speed is 400 Khz
• To program the I2C interface, we use Arduino's Wire.h library, which writes the I2C.h subroutine to read and export I2C data
The WHO_AM_I register of the MPU-60X0 has a default slave address of 0x68, which can be modified based on the AD0 pin's value If the AD0 pin is connected to a 5V source, the I2C address changes to 0x69.
Sample Rate divider: Calculates the sampling rate of the MPU
The Sample Rate for the MPU-60X0 is determined by dividing the gyroscope output rate by the SMPLRT_DIV register This Sample Rate influences the sensor register output, FIFO output, and DMP sampling The formula for calculating the Sample Rate is: Sample Rate = Gyroscope Output Rate / (1 + SMPLRT_DIV).
Set the register value to 0x07, specifying a sampling rate of 1 Khz when the DLPF is disabled (DLPF_CFG = 0 or 7) and 8kHz when the DLPF is enabled
Configuration Register: Configure synchronous sampling frame and the Digital Low Pass Filter for sensor The default value is 0x00
Gyroscope Configuration Register: Configures the angular velocity range, where the value 0x00 is selected with a range from -250 to 250 degrees / s
Accelerometer Configuration Register: Measure the angular acceleration range, here select 0x00 with range from -2g to 2g
Figure 3 15: The diagram Wiring GY521 with Arduino.
Power Management register: select the clock source for the MPU, the default clock source is the 8 MHz quartz oscillator
Data Gyro Record Registers: store the angular velocity values of the three axes
Acc Data Record Registers: store the angular acceleration values of the three axes
Servo motors typically operate within a voltage range of 4.8V to 6.5V, with most commonly used at 5V, where higher voltage results in increased torque Standard servo motors generally have a limited rotation range of 0° to 180° due to their gear design; therefore, it's essential to ensure that your project can function within this limitation or consider modifying the motor for a full 360° rotation Additionally, since the gears in servo motors are prone to wear and tear, selecting a servo motor with metal gears is advisable for applications requiring durability and extended operation.
Figure 3 17: MG996R High Torque Metal Gear Dual Ball Bearing Servo.
The torque of a motor is a crucial parameter to consider, and the MG996R Motor, with a torque rating of 2.5 kg/cm, serves as a prime example This means the motor can lift a weight of 2.5 kg when the load is suspended 1 cm away If the load is suspended at 0.5 cm, the motor can lift up to 5 kg, whereas at 2 cm, it can only handle 1.25 kg Therefore, selecting the appropriate motor torque based on the project's load requirements is essential for optimal performance.
The MG996R is a robust metal gear servo motor, capable of delivering a maximum stall torque of 11 kg/cm This servo operates within a range of 0 to 180 degrees, with its rotation determined by the duty cycle of the PWM wave sent to its signal pin.
Figure 3 18: The PWM signal cycle.
The PWM signal required for motor control operates at a frequency of 50Hz, corresponding to a PWM period of 20ms The On-Time of the signal can be adjusted between 1ms and 2ms, allowing for precise motor positioning Specifically, an On-Time of 1ms positions the motor at 0°, 1.5ms at 90°, and 2ms at 180° By varying the On-Time within this range, the motor can be effectively controlled from 0° to 180°.
Stall torque 9.4 kg/cm (4.8 V ), 11 kg/cm (6 V)
Controlling servo motors using the Arduino Servo library is straightforward, but it requires a dedicated pin for each motor, which can strain both pin availability and processing power The Adafruit 16-Channel 12-bit PWM/Servo Driver simplifies this by allowing control of up to 16 servos through I2C using just 2 pins Its built-in PWM controller operates all 16 channels simultaneously without taxing the Arduino's processing capabilities Additionally, you can connect up to 62 of these drivers, enabling the control of as many as 992 servos while still using only 2 pins.
Table 3 6: The PCA9685 pinout configuration
Wire Number Wire Colour Description
1 Brown Ground wire connected to the ground of system
2 Red Powers the motor typically +5V is used
3 Orange PWM signal is given in through this wire to drive the motor
Figure 3 20: The Wiring Arduino and the PCA9685 pinout.
To connect a servo, use the standard 3-pin female connector that fits directly into the headers of the Servo Driver Ensure that the ground wire, typically black or brown, aligns with the bottom row, while the signal wire, usually yellow or white, is positioned on the top.
GND The power and signal ground pin
VCC Logic power pin (should be 3 - 5V max)
V+ Power to the servos (5 - 6VDC)
Serial Clock (SCL) I2C clock pin
Serial Data (SDA) I2C data communication pin
OE (Output Enable) Can be used to quickly disable all outputs
Each port has 3 pins: V+, GND and the PWM output Each PWM runs completely independently but they must all have the same PWM frequency
Adafruit_PWMServoDriver pwm1 = Adafruit_PWMServoDriver(0x40);
Adafruit_PWMServoDriver pwm2 = Adafruit_PWMServoDriver(0x41);
Figure 3 21: The led matrix 8x8 module
You can chain multiple drivers, allowing control of up to 62 servos, with easy wiring thanks to headers located at both ends of the board, enabling a straightforward connection using a 6-pin parallel cable.