Design and Implementation of a Voice Controlled Multifaceted Robot

1 2 3
Revathi P Rachana R Supriya K M
Student, Student, Student,
Department of ECE Department of ECE Department of ECE
East West Institute Of Technology East West Institute Of Technology East West Institute Of Technology
Bengaluru, India Bengaluru, India Bengaluru, India
revathiprakash818@gmail.com rachanark98@gmail.com supriyaklp99@gmail.com
4
Suhas Raghavendra R
Student,
Department of ECE
East West Institute Of Technology
Bengaluru, India
suhashassan21@gmail.com

Abstract—In the modern world, the Robots play a vital for sensing gestures and controlling robots. Glove based
role in replacing manual works especially in assisting technique is a well-known means of recognizing hand
handicapped people. In the traditional method the Robots gestures. It utilizes a sensor attached to a glove that directly
were considered to perform solitary task assigned to it. measures hand movements [2].
The proposed work is designed to operate in multifaceted The proposed robot identifies and tracks a black line on
modes, such as RF Transmitter-Receiver mode (remote white surface area works as a Line following robot which is
mode), Hand Gesture mode, Voice Controller mode, Line able to detect a specified line and maintain track on it and
Follower mode using a microcontroller Arduino does the assigned job [4],[8]. This work can be modified quite
Mega2560 board. In addition to this Wireless Camera, easily to include a camera well that can stream the videos to
APR module, Temperature sensor, Obstacle detection the user over Wi-Fi using Wi-Fi module.
and Medicine dispensing feature made this Robot
versatile which can be used in various application. II. LITERATURE REVIEW
In the traditional method the Robots were considered to
Keywords— Gesture Control, Voice Control, and perform solitary task assigned to it. The proposed method is
Medicine Dispensing. designed in such a way that the robot can be operated in
multiple modes, as the unique features in isolated Robots are
I. INTRODUCTION integrated. Montashir Ahmed, Hasan U.Zaman, Md.Manjidul
Innovation and development in technology is to provide Haque, S.M. Tarek Aziz [8] designed and implemented a line
comfort and ease to human life. The Robots have the potential following robot that follows a line which specifically is black
to be deployed for disinfection, delivering medications and line on white surface that travels on a designated path, in
food, and in which the proposed Robot can assist addition to identifying obstacles in 3 ways. Aravindan B,
handicapped people [6]. The proposed work is to solve the Sakthivel S, Shrinithy K, Kathirvel P. [7] designed Multi –
issues encountered in assisting handicapped people and Way Controlled Robot can be operated under hand gesture
senior citizens. This work enables the physically disabled mode, mobile and voice-controlled modes. In hand gesture
people to return to their normal life, by multifaceted mode; robot can be controlled by tilt made by hand. The
controlling modes, as this Robot can be modeled as purpose of this work is to implement a wheel chair for
wheelchair [7]. disabled people to navigate themselves using different modes
By Voice control mode, one can operate the Robot by giving of operation. M.Jain, M.Sehgal, Y.Kolra, P.Jain,
Wireless command from the application using the functions N.Aggarwal, R.Singh, D.Kumar, T.K.Bera [2] presents a four
already programmed in the app, when the Bluetooth – wheel robot with gesture controlled arm manipulator
application is turned on and is connected with the current system. The robot detects the object from its surrounding
system via Bluetooth [1]. The vehicle will have motion in environment and gesture controller is developed to control
four directions: Forward, Backward, Right and Left. During the motion of arm that also operates pick and place operation.
Forward movement, all four motors will movement in same Phung Quang Anh, Tran Duc Chung, Tran Tuan, Mka
direction, which is the reverse case in the backward direction. Ahamed Khan [3] work aims to design and implement an
Whereas in case of Left and Right motion two motors obstacle avoidance mobile controlled robot operated by
direction will be inversely proportional to other two motors controlling android – based mobile device via Bluetooth.
anti clockwise and clockwise respectively. Instructions are User launches android controller interface on his mobile that
given to the motors through the Bluetooth app of Android is connected to Bluetooth and has to select the options to
Smartphone by the user. make robot move in any four directions by choosing
Gesture recognition technologies are much younger in the unblocked path when facing the obstacles. Shomitro Kumar
world of today. At this time there is much active research in Ghosh,Md.ToheenBhuiyan, Raihan Chowdhury,
the field and little in the way of publicly available Md.IsmailJabiullah [4] presents a line following robot that
implementations. Several approaches have been developed detects visual line on smooth surface area embedded on the

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©2021 IEEE

floor and follows it on a path of black on white or white on interconnected to a RF Transmitter and an Accelerometer
black using IR sensor and Ultrasonic sensor for obstacle with an input Power supply is as shown in Fig 1. The signals
detection. Adithya Chaudhry, ManasBatra, Prabhakar Gupta, are transmitted with respect to the angle of tilt made by the
SahilLamba, Suyash Gupta [5] proposes a system which hand through the transmitter which is then received by the
focuses on the concept of a robot that can be controlled by Receiver which is connected to the Robot and act
human voice. An android app is used as a medium for the accordingly. The block diagram shown in Fig 2 consists of a
transmission of human commands to microcontroller. The microcontroller unit Arduino Mega board which is inter-
basic movements such as moving forward, backward, turning connected to Ultrasonic Sensor 1, Ultrasonic Sensor 2, IR
to left and right can be controlled by human voice commands. sensor 1, IR sensor 2, Four-Channel switch, Temperature
Vishaal M, VishwaBharathi R, Sri Vishnuvardhan K, Usha A sensor, LCD, power supply module, Bluetooth module,
[6] work presents a prototype robotic system that esp8266 module, Motor driver 1, Motor driver 2, Motor
automatically dispenses medicines to patients in the hospital. driver 3, RF Receiver, Finger print scanner and Speaker
The robot uses the line follower method to deliver the pills to which is connected to APR Voice Recorder.
patient resided in the room. The robot uses a GSM module to The input to the microcontroller unit is given by the 12V
send message to check whether the patient has taken their Power Supply and the expected output from the block
medicines to the caretaker’s mobile. Similar work is adopted diagram is as follows:
in the proposed method where the Robot can dispense the • Instructions are given to the motors through
medicine with the authentication of the patients and assist Bluetooth app of Android Smart phones by the
them. Jongil Lim, Girma S. Tewolde , JaerockKwon , user.
Seyeong Choi [9] work presents the design of a network
robotic framework using a Smartphone based robotic
• Voice control mode can be achieved through
platform. The implementation uses a Smartphone as a
Bluetooth module which is connected to Arduino
network bridge between mobile robot and the computer to
Mega Board through which Robot movements in
distribute complex task into multiple networked
four directions: Forward, Backward, Left and
computational resources, as well as, a sensor package that
Right can be achieved.
allows the mobile robot to navigate with low power
consumption. This work is intended for a single robot that has
a network connection through a smart-phone communicating • Object detection can be achieved by Ultrasonic
with the sensing and motion data with Arduino Mega through sensors.
the Bluetooth module. Similarly, in the proposed work the
single Robot is integrated with multiple tasks. Shubh • Line follower can be accessed by IR sensors.
Srivastava, Rajanish Singh [1] work aims to build a voice-
controlled robot car based on arduino microcontroller using
Bluetooth. The control unit is coordinated with Bluetooth • Human body temperature can be detected through
gadget that transmits the control signals on receiving the Temperature sensor.
voice commands. Instructions are provided via smart android
device with Bluetooth application. JustinasMišeikis, Pietro
Caroni, Patricia Duchamp, Alina Gasser, Rastilav Marko, • The transmitted signals are received through RF
NelijaMišeikieně, Frederik Zwilling, Charles de Castelbajac, module in the Receiver as per the instructions
Lucas Eicher, Michael Früh, HansruediFrüh [10] work given by the user which tends to drive the wheels
presents the robot named Lio as the mobile robot platform through Motor driver 1 and Motor driver 2.
with a multifunctional arm explicitly designed for human-
robot interactions and personal care assistant tasks. The robot • The stored commands are recorded in the APR
has a compliant motion controller and the combination of Voice Recorder and the feedback is received
visual, audio, laser, ultrasound and mechanical sensors used through the speaker.
for safe navigation and environment understanding. Similar
work is adopted in the proposed method where the Robot • Medicine dispensing application can be achieved
assists the disabled people. Using the above literature papers by Motor driver 3 accessing finger print scanner.
as a base, a design is proposed which fulfills the drawbacks
of all the systems and was implemented.

III. SYSTEM DESIGN

Methodology gives the systematic, theoretical analysis
applied to design the block diagrams as shown in Fig 1 and
Fig 2 for the transmission and reception of the signals for the
Robot operation. The block diagram shown in Fig 1 is the
Gesture control using Arduino Uno microcontroller that Fig 1: Block Diagram of Gesture Control using
cooperates to act as the transmitter for controlling the robot
Arduino Uno Transmitter.
through Gesture controlling mode of operation.
The block diagram of Gesture Control Transmitter consists
of microcontroller Arduino Uno board which is
 To set up the I2C lines, SDA (Serial Data) Pin on MPU
6050 is connected to the Analog Pin 4 of Arduino UNO
and Serial Clock (SCL) Pin on MPU 6050 to Analog Pin
5 of Arduino UNO.
The MPU 6050 is a six DOF (degrees of freedom) or six
axis IMU (Inertia Measurement Unit) sensor, which
means that it gives six values of output: three values from
the accelerometer and three from the gyroscope. An
accelerometer works on the principle of piezoelectric
effect, the current is produced from the collision on the
Piezoelectric walls determining the direction of
inclination and its magnitude. Gyroscope works on the
principle of Coriolis acceleration that is the acceleration
due to the rotation experienced by particles moving along
the surface.
The 433MHz Wireless RF Transmitter is interfaced with
Arduino UNO by connecting its four digital pins to the
Fig 2: Block Diagram of Design and Implementation of a four switches on the RF Transmitter board (encoder
Voice Controlled Multifaceted Robot Receiver.
HT12E) for forward (FWD), reverse (REV), left (LEFT),
right (RIGHT) motions. The data is transmitted four
times in succession. It consists of different lengths of
positive going pulses for ‘1’ and ‘0’. The frequency of
IV. IMPLEMENTATION these pulses may lie between 1.5 and 7 KHz.
In the Traditional method the Robots were considered to
perform solitary task assigned to it. The proposed method is
designed in such a way that the Robot can operate in multiple
modes, as the unique features in isolated Robots are
integrated. The dependency on a caretaker for patients can be
prevented. The proposed work is designed to operate in
multifaceted modes, in which the Gesture Control mode of
operation can be achieved from the circuit built as shown in
Fig 3.

Fig 4: Circuit Diagram of Design and Implementation of a

Fig 3: Circuit Diagram of Gesture Control using Arduino Transmitter. Voice Controlled Multifaceted Robot Receiver.

The proposed work is designed to operate in The circuit diagram shown in Fig 4 consists of a
multifaceted modes, in which the Gesture Control mode microcontroller Arduino Mega board which has fifty-four
of operation can be achieved from the circuit built as digital input/output pins and sixteen analog input pins.
shown in Fig 3. These digital pins of Arduino Mega are connected by
Ultrasonic sensor, Infrared sensor, APR33A3 voice
The circuit diagram shown in Fig 3 is the Gesture
module, Receiver, LCD, four-channel switch, Fingerprint
Control using Arduino UNO microcontroller that
combines to act as the transmitter for supervising the scanner and Motor drivers. The temperature sensor,
Robot through Gesture Control mode of operation. output pin is connected to analog pin of Arduino Mega
board. The Bluetooth module, transmitter pin is
The MPU 6050 communicates with the Arduino through
the I2C protocol. The program will be running in the connected to Arduino Mega board receiver pin and
Arduino Board taking advantage of Arduino's Interrupt Bluetooth module, receiver pin is connected to Arduino
Pin by connecting to Digital Pin 2 (Interrupt Pin 0) of Mega board transmitter pin. The whole circuit is powered
by 12V power supply. The Ultrasonic sensor is used for
Arduino UNO to the INT Pin on MPU 6050.
obstacle detection and avoidance, Infrared sensor is used
for line following technique, Temperature sensor is used
for detection of human temperature, APR33A3 voice
module is used to play voice files for the robot, by the
eight pins and can store eight voice/sound. The
433MHzwireless receiver is connected to the Arduino
Mega board which can be controlled in two ways: firstly,
through Remote control mode that is through wireless
transmitter and secondly through Gesture control mode
that is through the angle of tilt made by the hand
(Accelerometer connected to the wireless transmitter and
Arduino Uno board). The fingerprint scanner is
initialized, enrolled and evaluated by four-channel switch
and which is displayed through LCD that is for Medicine
dispensing feature, it will show whether the patient has
taken the medicine or not, and this message is also
received through telegram with the help of esp8266 Wi-
Fi module. HC05 Bluetooth module is a simple wireless
communication device based on the Bluetooth protocol.

The Robot performs series operations according to the

instructions given through Bluetooth application
‘VOICEBOT’. The Bluetooth App receives the voice
commands and converts it to text and sends the wireless
signal to the Microcontroller Unit (MCU) which is
annexed with sensors and modules. The instructions
given in the form of voice commands can be listed as
follows:
Fig 5: Voice Controlled Multifaceted Robot Flowchart.

A. Audio
Table 1: Voice Commands given through Bluetooth.
B. Voice
C. Radio
D. Line Connector Commands Features
E. Object
F. Temperature A Audio Voice feedback
G. Medicine/Tablet
Basically, the Microcontroller and the Robot circuit is B Voice Controlling
powered on by giving 12V power supply. When the movements
Bluetooth app is turned on through the mobile through voice
application and is connected with the current system via C Radio Remote control
Bluetooth HC-05. If the connection is secured precisely and gesture
then the application is ready to connect with Robot or control
else if the connection is not secured precisely then the
application is not ready to connect with Robot as shown D Line Line follower
in Fig 5. Here, the commands are given through the
Bluetooth application called “Voicebot”. The commands E Object Object
given to Voicebot are as shown in Table 1. detection and
re-route path
F Temperature Temperature
detection

G Medicine/Tablet Medicine
dispensing
 V. RESULTS AND DISCUSSION F Temperatu Temperature High
re detection temperature
Low
temperature
G Medicine / Medicine Medicine
Tablet Dispensing Dispensing

A Audio Commands

When the commands Hi, Name and Doing are given through
Voicebot application with assistance Bluetooth module the
voice feedback such as Hello, RIA and Executing commands
respectively are obtained.
Fig 6: Experimental Setup with all the sensors and controller interfaced.
B Voice Commands

When the commands Forward, Backward, Left, Right and

Stop are given through Voicebot application with the
assistance of Bluetooth module according to the command
the Robot works.

C Remote control and Gesture control

Fig 7: Procedure for Bluetooth connection to give commands.

Table 2: Voice commands and their results.

Connector Command Feature Result

Hi Hello
A Name Voice RIO
Doing feedback Executing
commands
Forward Forward Fig 8: Controlling Robot through Remote control (RF Transmitter module).
B Backward Controlling Backward
Left movements Left
Right through Right
voice
C Radio Remote Forward
control Backward
Gesture Left
control Right
Head Rotation
D Line Line Line Follower
Follower
E Object Obstacle Object Fig 9: Controlling Robot through Gesture Control.
detection detected at
and front Results for the command RADIO can be fetched in two
avoidance or ways such that by Remote control mode shown in Fig 8 and
Object by Gesture Control mode shown in Fig 9.
detected at
back
And re-routes
path
D Line Follower authentication, notifying the caretaker through the Telegram
application as shown in Fig 13 the same is shown in Fig 14
through the LCD.

Fig 10: Line Follower.

Fig 10 represents that the Robots follows the black line on

the white surface area acting as the Line Follower Robot. Fig 13: SMS received by the Caretaker through Telegram Application

E Obstacle detection and avoidance

When the Object command is given through the Voicebot

application with the assistance of Bluetooth module the
Robot speaks as “object detected at front” and re-routes path
if it detects the object at front and speaks as “object detected
at back” and re-routes path if the object is detected at back.
F Temperature Detection

Fig 14: LCD display of Medicine/Tablet Dispensing.

VI. CONCLUSION
The possibility to migrate features from different individual
Robots collectively into a sole Robot with multifaceted (that
is to have many different aspects or features), by using a
microcontroller reducing the target system. The Robot is
Fig 11: Results of temperature detection. designed and implemented using simple technologies rather
than complex ones, to achieve maximum efficiency with
Temperature detection is achieved through voice feedback reduced area and cost. Thus, the Robot has been designed to
and by LCD after sensing the sensor. Robot detects human work effectively at restaurants, office and hospital
body temperature as “high temperature” or “low environment.
temperature” as per the limits given while coding as shown
in Fig 11.
VII. FUTURE ENHANCEMENT
G Medicine/ Tablet Dispensing Real time clock with GSM module can be implemented for
Medicine dispensing application such that at a particular time
coded, the Robot would automatically dispense the medicine
without giving voice commands.

Fig 12: Fingerprint Authentication of the Patient.

The command MEDICINE or TABLET that includes the sub-

features such as locating the patient residing in the room,
identifying the patient through fingerprint authentication as
shown in Fig 12. The medicines are dispensed soon after the
 REFERENCES “Design And Implementation of A Robotic Technique Based Waiter”
2017 3rdInternational Conference on Electrical Information
Communication Technology (EICT),7-9 December 2017 .
[1] Shubhash Srivastava, Rajanish Singh, Department of Computer
Science and Engineering, Raj Kumar GoelInstituteofTechnology, [14] Kaula Lumpur, Malaysia, Yong Chai Tan, Bentley Lew, KhimLeng
Ghaziabad affiliated to Dr. A.P.J Abdul Kalam Technical University, Tan,KaeVin Goh,KienLoong Lee, ZhiChao Khor, Faculty of
Lacknow, Uttar Pradesh, India, " Voice controlled Robot car using Engineering and Science, Universiti Tunku Adbul Rehman (UTAR),
Arduino ", International Research Journal of Engineering and Kaula Lumpur, Malaysia, “A New Automated Food Delivery System
Technology (IRJET), eISSN:2395-00056, p-ISSN:2395-0072. Using Autonomous Track Guided Centre-Wheel Drive Robot”
Proceedings of the 2010 IEEE Conference on Sustainable Utilization
[2] M.Jain, M.Sehgal, Y.Kalra, P.Jain, N.Aggarwal, R.Singh, D.Kumar
and Development in Engineering and Technology Universiti Tunku
and T.K.Bera, Mechanical Engineering Department, Thapar Institute
Abdul Rehman, 20&21 November 2010, Faculty of Engineering .
of Engineering and Technology, Patiala, "Object detection and gesture
control of four-wheel mobile robot", 2019, Proceedings of the Fourth [15] K.N.V Sriram, Suja Palaniswamy, Department of Computer Science
International Conference on Communication and Electronics Systems and Engineering, Amrita School of Engineering, Bengaluru, Amrita
(ICCES 2019), IEEE Conference Record #45898. Vishwa Vidyapeetham, India, “Mobile Robot Assistance for Disabled
and Senior Citizens using Hand Gestures”.
[3] Phung QuangAnh, Tran Duc Chung, Taran Tuan, M.k.a. Ahamed
Khan, Computing Fundamental Department, FPT University,155300,
Hanoi, Vietnam, Faculty of Engineering Technology and Built
Environment, UCSI University, Kuala Lumpur, Malaysia, "Design and
development of an obstacle avoidance mobile-controlled robot", 15 –
17 of October 2019, IEEE Student Conference on Research And
Development (SCOReD).

[4] 1st Shomithro Kumar Gosh, 2nd Md.ToheenBhuiyan, 3rd Raihan

Chowdhury, 4th Md.IsmailJabiullah, Department of Computer Science
and Engineering, Daffodial International University, Dhaka,
Bangladesh , “ A line follower robot with obstacle detection by
ultrasonic”, 16 December 2019, IEEE Conference Record.

[5] AdityaChaudhry, ManasaBatra, PrakharGupta, SahilLamba, Suyash

Gupta, Information Technology, Bharati Vidyapeeth's College of
Engineering, New Delhi 2019, "Arduino based voice controlled robot”,
International Conference on Computing, Communication, and
Intelligent Systems (ICCCIS).

[6] Vishaal.M, VishwaBharati. R, Sri Vishnuvardhan. K, Usha.A,

Department of Electronics and Communication Engineering, Easwari
Engineering College, Chennai, India 2019, "Automated medical
dispensing system using robotics ", IJCESR , ISSN (print) : 2393 –
8374, Volume-6,Issue-4.

[7] Aravindan B, Sakthivel S, Shrinithy K, Kathirvel P, UG Students,

Department of Intrumentation and Control Engineering, INDIA, “
Multi- way controlled robotic vehicle”, International Journal of
Engineering and Management Research, volume-7, Issue-2,
MarchApril 2017, Page Number:171-174.

[8] Hasan U.Zaman, Md.Majidul Haque Bhuiyan, Montashir Ahmed, and

S.M Tarek Aziz, Department of Electrical and Computer Engineering,
North South University, Dhaka-1229, Bangladesh, “ A novel design of
line following robot with multifarious function ability”, 2016, IEEE
International Conference on Microelecronics, Computing and
Communication (Microcom).

[9] Jongli Lim, Girma S. Tewolde, JaerockKwon, Seyeong Choi, “ Design

and implementation of a network robotic framework using a
smartphone based platform”, IEEE Journal, Volume: 7, May 17, 2019.

[10] JustinasMišeikis, Pietro Duchamp, Alina Gasser, Rastilav Marko,

NelijaMišeikieně, Frederik Zwilling, Charles de Castebajac, Lucas
Eicher, Michael Früh , Hansruedi Früh, “Lio – a personal robot
assistant for human – robot interaction and care applications”, IEEE
Journal, Volume: 5, Issue: 4, June 2020.

[11] Jeevan Reddy K, A Navneeth, A Vishwam, G Pradeep, Associate

Professor, Students, Electronics and Communication Engineering,
Sreenidhi Institute of Science and Technology “Microcontroller Based
Assistant System for Dumb People” International Journal for Research
in Applied Science and Engineering Technology
(IJRASET),ISSN:2321-9653,IC Value:45.98,SJ Impact
facto:6.887,Volume-7, Issue IV,April 2019 .

[12] Narsingoju Adithya, Sridevi Chitti, “Hand Gesture Controlled Robot”

International Journal of Recent Technology and Engineering (IJRTE),
ISSN:2277-3878,Volume-8, Issue-1S4, June 2019.

[13] Khulna,Bangladesh, Md.Kamruzzaman, Md.Tareq, Department of

Electrical and Electronic Engineering, Uttara University, Bangladesh,