Revista CIENCIA Y TECNOLOGÍA

ISSN 1810-6781 Rev. Cienc. Tecnol. 19(4): 101-111, (2023)

Esta obra está publicada bajo la licencia

CC BY 4.0

Prototype of Virtual Personal Assistant based on Artificial Intelligence applied to the IT offices of the Peruvian Public Sector

Richard  Mantilla León¹ ; Juan  Gamarra Moreno¹

Faculty of Systems Engineering and Informatics, National University of San Marcos, 350 Carlos Germán Amezaga Avenue -Cercado de Lima, Lima, Perú

*Corresponding author:  richard.mantilla@unmsm.edu.pe (R. Mantilla) DOI: 10.17268/rev.cyt.2023.04.07

Submitted: 10-26-23
Accepted: 12-27-23
Published: 12-30-23

ABSTRACT

Organizations incur economic losses due to the failure of Traditional Information Technology (IT) management, which requires the implementation of more efficient technologies such as Artificial Intelligence. The objective of this research was to implement a Prototype of Virtual Personal Assistant based on Artificial Intelligence applied in IT offices of the Public Sector, in order to assist IT leaders in Project Management. It was carried out in three stages: General review of the IT project management process, design and implementation of the prototype, and evaluation of the developed prototype. As a result, a prototype was implemented, developed with the Python programming language, on a Microsoft Windows 10 platform, which interacts with voice and natural language and fulfills three functions: Creation of an IT Project, Generate a record of queries and observations of suppliers under market study, Generate Selection Committee record. The prototype assists the IT leader in Project management, simplifying and optimizing time and resources in the activities of organizing, capturing and recording information on IT Projects in the IT Offices of Public Sector organizations.

Keywords: Virtual Personal Assistant; Natural Language Processing; IT Projects; Artificial Intelligence.

1. INTRODUCTION

Today, public and private organizations, regardless of their size, implement Information Technology solutions. The responsibility of IT Solutions Management must be carried out comprehensively, encompassing all the activities that management involves such as the formulation, approval management, planning, execution, monitoring, control, and improvement actions, of all the processes of putting into production, training and closing. It is the responsibility of the leaders of the IT offices to manage and guarantee the success of the Information Technology solutions they propose.

Information Technology projects have now become elements that provide a competitive advantage in many organizations, and they also better position companies in the face of risks (Bloch et al., 2012).

Studies carried out by Michael Bloch, Sven Blumberg, and Jürgen Laartz in collaboration with the University of Oxford report that 45% of projects exceed budget, 7% miss-delivery time, and 56% are delivered with less useful value than what was offered (Bloch et al., 2012).

The problem is that the tasks take longer than expected and this increases the completion time of the projects (Chen & Hall, 2020). In this context, emerging technologies based on artificial intelligence appear, especially Virtual Personal Assistants to assist humans, which are Information Technology solutions based on natural language processing (NLP), machine learning, and artificial intelligence (Schuetzler et al., 2018) with a high potential for application in various fields and sectors.

According to the investigations of A. Batura (Batura, 2019) the use of VPAs has been increasing, positioning Google Assistant at a popularity level for 2018 with 13.9%, Siri with 13.1%, Amazon's Alexa with 8.9%, Microsoft's Cortana with 3.5%. It is estimated that the VPA market will grow in the world to reach the value of 45.1 billion dollars. Chatbots and smart speakers with listening capacity are gaining popularity and assistants like Alexa and Google start to gain market in 2019 (Talwar & Koury, 2017).

In recent years, the trend of integrated technology products and services has been seen under the terms of smart technologies and smart cities, a wave that is being used massively and in this regard, South Africa experiences the intensive use of these technologies, which is why it is very common the use of smart watches, smartphones and intelligent virtual personal assistants such as Amazon's Alexa, however, even though the adoption of these technologies is very promising, they present challenges regarding the legal or regulatory field and individual and collective well-being, noting that new technologies pose unique challenges in themselves, challenges in their implementation and something that is evident that the legal and regulatory frameworks do not cover what is suffered in the field of their use, for example in the aspects of access to information and protection of personal data (M Botes, 2023).

VPAs with interaction based on natural language are been used in organizations for different areas, facilitating processes, management activities and provide the ability to respond to current demands for speed with benefits such as massiveness, simplicity, traceability, among others to automate different activities resulting from the use of this growing technology (Batura, 2019).

This article aims to provide value in terms of Information Technology applications through the use of VPA as support to the human and also provide conceptual and technical knowledge of the developed prototype that serves as a guide for the development of VPA solutions, that is applied to assist the IT leader in managing the information developed during the management of IT Projects in the IT offices of the Peruvian Public Sector.

About Literature Review.

 1.1 Virtual Personal Assitant

Its origin is an artificial military intelligence project called CALO, "cognitive assistant that learns and organizes." Adam Cheyer, the administrator of the CALO project, brought together people from the SRI International that was the artificial intelligence center and started the creation of Siri. A virtual assistant is a software agent that helps users, automating tasks with little interaction, naturally, using the voice to assistant processes, interprets and gives a response in the same way (Helode et al., 2017). From that moment on, its use expanded rapidly, with products such as Apple's Siri, Google Assistant and Amazon's Alexa (Dokukina & Gumanova, 2020), (Thirupathur Kailashnathan Vijay Vangara, 2020). Both Siri and Google's VPA are embedded in the IOs and Android operating systems (Android | The Platform Pushing What’s Possible, n.d.) respectively.

 1.2 Natural Language Processing

Natural Language Processing (NLP) focuses on computer understanding of human language. It covers part of Data Science, Artificial Intelligence and linguistics. In NLP, computers analyze human language, interpret it and give it meaning so that it can be used in a practical way. Using NLP we can do tasks like automatic text summarization, language translation, relationship extraction, speech recognition. (Cuc et al., 2010).

 1.3 Python programming language

General purpose, different from others such as C or Java as it is simple and readable. It uses fewer lines of code compared to Java or C ++ (5 and 10 times respectively), for task instructions, which makes it practical, saves programming time. Python has libraries for artificial intelligence, natural language processing, such as speech to text conversion, and text to speech, easy to integrate with language engines in several languages in the cloud like google or Microsoft bing (Grado, 2018).

 1.5 Wit.ai Natural Language processing application

Wit.ai is used to interact with the user with voice or natural language processing (Wit.Ai, n.d.). Wit.ai differentiates the entities in a sentence, those that will take a type, which can be int or float, etc., and a value. Based on training, such as a city, a date, etc., and the value will be the field of the word detected, then actions will be executed depending on the recognized entities and values.

 1.6 Virtual Personal Assistants Applied to Different Sectors

In the education sector, a solution for language learning was made, which according to Dokukina and J. Gumanova (Dokukina & Gumanova, 2020) states that the interaction is oral and by text, based on voice recognition (VR) and semantic analysis.

In economics, the Inter-American Development Bank has implemented conversational robots based on NLP, to improve service to taxpayers. The use of different types of solutions helps people to manage the problems presented by the traditional taxpayer service model (Seco & Muñoz, 2019).

In the area of Tourism, a VPA model was applied using NLP, to locate places of food, bars, events, sales, entertainment; where the Assistant provides information services of georeferenced locations of interest through consultations in natural language (Massai et al., 2019).

The Health sector to improve accessibility and fight against info exclusion developed and presented a virtual personal assistant model whose application is to assist the elderly with visual problems, which is based on the recognition of Voice (Hämäläinen et al., 2015).

Ganesh y Itole (Deshpande & Itole, 2019) implemented a VPA for home automation. The product is based on VR and allows you to interact with various electronic devices through a microphone and broadcast the responses through a speaker.

In the health sector, Nova, the Virtual Nursing Assistant, was implemented, which helps people receive data on their symptoms and ailments with which it collects this data and provides information about their ailments. It can also advise or recommend the next steps and with interaction with natural language, it is also at the midpoint between the patient or person and medical experts (Bidve et al., 2023).

In the Health Sector, a research was carried out and the problem between doctor and patient was observed because the interaction seeks to educate the patient, so that the doctor-patient relationship is not the best, therefore the research analyzes the use of virtual assistant to the that is accessed by telephone for plastic surgery patients, they can ask the most frequent questions, so that when they have the appointment with the doctor they can ask the most important questions regarding the surgery. The study concludes that the results were that patients feel satisfied when using virtual assistants based on artificial intelligence and find it a positive experience when asking frequently asked questions before surgery, and the system's responses were highly accurate (Avila et al., 2023).

2. METHODOLOGY

This work was divided into three phases: General review of the IT project management process in the Information Technology office of a public entity and Implementation of the Virtual Personal Assistant prototype. The second phase, Design and Implementation of the Prototype, has been divided into 6 stages: Activation Module, Speech to Text Conversion Module, Query Processing Module, Text-to-Speech Conversion Module, Prototype Test Cases and the Integration and Final Tests of the 3 functionalities of the VPA. In the third phase, the evaluation of the developed Prototype is described.

Activities or functionalities supported by the Virtual Personal Assistant prototype:

1. Create an IT Project (Creates the folder and subfolders of an IT project in a given Projects directory).
2. Generate Record of Consultations and Observations of Suppliers in Market Study.
3. Generate Selection Committee Records.

 2.1 General review of the IT project management process in the Information Technology office of a Peruvian public entity

Government organizations have offices that manage information technologies, known as Office of Information Technology, Systems, IT, IT Organization, among others. These offices cover at least two areas: Helpdesk or Technical Support for the user and the infrastructure area for the technological base of hardware and software where IT systems and services are provided to users and citizens. The IT projects that are carried out in the Public Entity of this work are divided into three large groups:

1. Acquisition of Technological Infrastructure Assets.
2. The hiring of Technological Services.
3. Information Systems Development.

In relation to the Information Structure of IT Projects, after reviewing the IT processes (see Figure 1), the information that is managed is shown.

Figure 1. IT Project Management in the Peruvian Public Entity

Table 1. Information of IT Project Manager in the Public Entity

Proyecto de TI	Se mantiene Información general como el nombre del proyecto, la Dependencia Solicitante, el Solicitante, la Fecha de Registro, Número de Documento, Resumen del Requerimiento, versión de las EETT o TDR final, Denominación del Procesos (Adjudicación Simplificada/Proceso, Gran Compra), Contratista, número de Informe de Conformidad, Nombres Completos del emisor del Informe, Fecha del Informe de Conformidad
Requerimiento inicial	Estudio de mercado	Comité de Selección	Requerimiento de bienes o servicios (RBS)	Convocatoria
Especificación Técnica Inicial o Término de Referencia Inicial -Informe de Necesidad -Informe de Estandarización -Informe de Evaluación de alternativas	-Consultas y Observaciones / Especificaciones Técnicas/Termino de Referencia -Registro de Visitas Técnicas -Validación de Cotizaciones -Datos del Estudio de Mercado (Analista Logístico, Fecha de culminación del Estudio de Mercado, Monto Referencial del estudio de Mercado)	Se refiere a cuantos miembros se aporta para el comité de selección en titular y suplente y quiénes son y las Declaraciones Juradas	Corresponde a la disponibilidad presupuestal para el proyecto	-Postores Participantes -Consultas y Observaciones / Especificaciones Técnicas/Termino de Referencia -Registro de Visitas Técnicas -Validación de Ofertas
Otorgamiento de la Buena Pro	Eventos adicionales	Ejecución de la prestación principal	Ejecución de la prestación accesoria
-Otorgamiento de la Buena Pro / Adjudicación -Orden de Compra -Orden de Servicio -Contrato	-Documentos órgano de Control Interno OCI -Documentos del Organismo Supervisor de Contrataciones con el Estado OSCE	-Adquisición -Servicio	-Informe del contratista -Observaciones al informe del contratista -Informe de Conformidad del Requerimiento emitido por el área técnica	-Postores Participantes -Consultas y Observaciones / Especificaciones Técnicas/Termino de Referencia -Registro de Visitas Técnicas -Validación de Ofertas

 2.2 Development Proposal for the Implementation of the Virtual Personal Assistant prototype

The block diagram (see Figure 2) has been considered for the development of the prototype (Grado, 2018)

Figure 2. Block diagram of the prototype

Activation Module

Used for the detection of the keyword to change the state from asleep to awake, activate the microphone to listen to the user's request.

Speech to Text Conversion Module

Python speech recognition module and google recognition engine were used, which uses deep learning neural network algorithms, to get the text (previously it was voice) in Spanish, with the precision of the country; that is, Peru and be able to be processed.

Query Processing Module

The Wit.ai (open source) Api will be implemented to process the text (previously it was voice). For each phrase, entities with values are obtained to process the request.

Text-to-speech module

The pyttsx3 library was used (Pyttsx3 · PyPI, n.d.) that works in offline mode, to transform a string of text to speech. Parameters such as the language can be customized. Once the operations are finished, the VPA will remain on standby

Prototype Implementation’s Test Cases

For the present work, three test cases were considered:

Case 1, Capture the speech of a person and get the text. The speech-recognition library was used (SpeechRecognition · PyPI, n.d.). Figure 3 shows the code and the console execution of the python program that captures the user's voice and converts it to text:

Figure 3. Speech to text conversion

Case 2, Handling of text for the execution of applications requested by voice. The voice was captured and the application was executed using the subprocess library and they are shown in Figure 4:

Figure 4. Running applications that were requested

Test Case 3, Responding to the user by voice in Spanish, was carried out using the pyttsx3 library that replays the given text to voice. Figure 5 shows the source code and the execution of the browser command and then the response for the user is heard: “explorador abierto”

Figure 5. Source code and runing applications that plays a response by speech

Integration and Testing

The three test cases and the 3 functions of the prototype were implemented:

1. Create an IT Project (Creates the directory and sub-directories of an IT project), for this, the os library was used to manage the Operating System and the makedirs function to create the directory. Figure 6 shows code and the project directory created:

   

   Figure 6. Source Code and Directory created with makedir function

2. Generate Record of Queries and Observations of Suppliers in Market Study, for this, the os library was used to manage the Operating System and the function makedirs to create the directory. Figure 7 shows the directory created from inquiries and observations of the bidder ElectroDa.

   

   Figure 7. Inquiries and observations’s Directory Created

3. Generate Selection Committee Records, the openpyxl library (pip install openpyxl in a power Shell console in the python installation directory) was used to automate excel. Figure 8 shows the code and the generated Excel file:

   

   Figure 8. Source Code and excel file created

 2.3 Prototype Evaluation

Carried out by IT office staff, in the sense of whether the information generated is sufficient and appropriate or similar to the conventional process. The functions that motivated the research were also verified, corroborating the benefits of using the product during IT project management, in the functionalities of the prototype:

1. Create an IT Project (Creates the folder and subfolders of an IT project in a given Projects directory)
2. Generate a record of supplier queries and observations in market research
3. Generate Selection Committee record

3. RESULTS AND DISCUSSION

A prototype was implemented, developed with the Python programming language, on a Microsoft Windows 10 platform, which interacts with voice and natural language and fulfills three functions: Creation of an IT Project, Generate a record of queries and observations from suppliers in market research. , Generate a record of the Selection Committee, optimizing time and resources for these activities. In similarity, there is reference to the implementation of a virtual assistant prototype with the support of artificial intelligence in academic companies, achieving faster and more effective systems (María et al., 2020); likewise, agreement was found with Batura (2019), who considers that APVs, with interaction based on natural language, facilitate processes and management activities and allow us to respond to current speed demands with benefits such as massiveness, simplicity, traceability, among others to automate different activities, such as the functions that it will fulfill, is the prototype developed in this research.

The implemented prototype is based on Artificial Intelligence, and contributes to solving the problems of Project Management of Public Sector IT Offices, especially in relation to time and resources, as referred to by Chen & Hall; (2020) who states that it will contribute with knowledge to solve part of the problems in Public Sector organizations, such as time, making the completion time of projects more efficient (Chen & Hall, 2020) and optimizing customer or user service according to how confirmed in a study of the use of a virtual assistant based on artificial intelligence in the health sector, where patients asked the most frequent questions prior to their plastic surgeries, where it was concluded that patients have high satisfaction and perceive the use with a positive experience. of Virtual Assistants, which provided high accuracy in responses to patient's questions.

In the Virtual Personal Assistant prototype, the operation modules were implemented: voice-to-text conversion, query processing and text-to-speech conversión that facilitate interaction.

The installation process of Python, the recognition libraries and text-to-speech conversion was difficult, due to dependency issues, however, after the learning curve and many error messages were overcome, the correct installation was achieved, using Windows Power Shell running python from the installation directory using .\python.exe -m pip install libraryname.

Once the application was running, there were problems with the voice-to-text transformation, so the adjust_for_ambient_noise parameter had to be adjusted to improve the capture.

The voice accent of the prototype is not Peruvian, therefore it could be adapted using a voice that future work on the matter considers more familiar.

The activation of the Virtual Assistant was done through its own development, so that it recognizes the word “blacky” by listening to the user's request. Once the request is made and responded to by the assistant using voice, the assistant “sleep” or is disabled until a new call is made by the user by pronouncing the word “blacky”.

For text processing, management of entities and values, the wit.ai tool is used, but it is necessary to go deeper in order to achieve mastery in the management of entities and values.

As you can see, some challenges were presented in its implementation, to which a solution was given and the expected operation was achieved according to the objectives of the research, however other types of challenges, difficulties or challenges may arise, as Botes (2023) refers. ), who considers that although the adoption of these technologies is very promising, they present challenges on the individual or collective aspect, as well as in their CONCLUSIÓN implementation and especially in the legal and regulatory frameworks that do not cover what is suffered in the field of its use, such as in the aspects of access to information and protection of personal data.

The transformation of speech to text and the management of entities and values that is done with wit.ai could be improved with offline solutions so that it works without having an internet link or avoid problems with slow connections.

4. CONCLUSION

From the implementation of the APV prototype, the following conclusions are obtained:

The prototype improves the Management of IT Projects of the Information Technology Offices of the Public Sector.

The prototype assists the IT leader in Project management, simplifying and optimizing time and resources in the activities of organizing, capturing and recording information on IT Projects in the IT Offices of Public Sector organizations.

The prototype simplifies and optimizes the functions for which it was implemented: creating an IT project, generating a record of supplier queries and observations in market research, generating a selection committee record, fulfilling them with efficiency and greater productivity.

The prototype works with three operation modules: speech to text conversion, query processing and text to speech conversion.

The developed prototype is a reduced version of a virtual personal assistant and can be enhanced to support greater assistance activities for IT leaders in the Information Technology Offices in order to become a tool with more features and support for the IT Offices.

REFERENCES

Android | The platform pushing what’s possible. (n.d.). Retrieved September 12, 2020, from https://www.android.com/

Avila, F. R., Boczar, D., Spaulding, A. C., Quest, D. J., Samanta, A., Torres-Guzman, R. A., Maita, K. C., Garcia, J. P., Eldaly, A. S., & Forte, A. J. (2023). High Satisfaction With a Virtual Assistant for Plastic Surgery Frequently Asked Questions. Aesthetic Surgery Journal, 43(4), 494–503. https://doi.org/10.1093/ASJ/SJAC290

Batura, A. (2019). Integrating Virtual Assistant Technology Into Omni-Channel. April.

Bidve, V., Virkar, A., Raut, P., & Velapurkar, S. (2023). NOVA-a virtual nursing assistant. Indonesian Journal of Electrical Engineering and Computer Science, 30(1), 307–315. https://doi.org/10.11591/ijeecs.v30.i1.pp307-315

Bloch, M., Blumberg, S., & Jurgen., L. (2012). Delivering large-scale IT projects on time, on budget, and on value. McKinsey & Company, October 2012, 1–11. http://www.qitrmanagement.com/Delivering large-scale IT projects on time on budget and on value.pdf

Chen, B., & Hall, N. G. (2020). Incentive schemes for resolving Parkinson’s Law in project management. European Journal of Operational Research, xxxx. https://doi.org/10.1016/j.ejor.2020.06.006

Cuc, I., De, A. K., & Corporaci, H. M. (2010). Diseño de un sistema multi-agente para monitoreo de redes utilizando JADE y JPCAP. Inge-Cuc, 6(1), 215–226.

Deshpande, N. G., & Itole, D. A. (2019). Personal Assistant Based Home Automation using Raspberry PI. 3, 5398–5401. https://doi.org/10.35940/ijrte.C6165.098319

Dokukina, I., & Gumanova, J. (2020). The rise of chatbots-new personal assistants in foreign language learning. Procedia Computer Science, 169(2019), 542–546. https://doi.org/10.1016/j.procs.2020.02.212

Grado, T. F. De. (2018). Diseño e implementación de un asistente personal inteligente en español basado en reconocimiento de voz empleando Raspberry Pi Carlos Díaz Fernández.

Hämäläinen, A., Teixeira, A., Almeida, N., Meinedo, H., Fegyó, T., & Dias, M. S. (2015). Multilingual Speech Recognition for the Elderly: The AALFred Personal Life Assistant. Procedia Computer Science, 67(Dsai), 283–292. https://doi.org/10.1016/j.procs.2015.09.272

Helode, P. S., Dr. K. H. Walse, & Karande M.U. (2017). An Online Secure Social Networking with Friend Discovery System. International Journal of Innovative Research in Computer and Communication Engineering, 5(4), 8198–8205. https://doi.org/10.15680/IJIRCCE.2017

M Botes, B. T. (2023). Art View of Bridging the regulatory gaps created by Smart and Connected technologies in South Africa.pdf.

María, M., Torres, E., & Manjarrés-Betancur, R. (2020). Asistente virtual académico utilizando tecnologías cognitivas de procesamiento de lenguaje natural. Revista Politécnica, 16(31), 85–96. https://doi.org/10.33571/RPOLITEC.V16N31A7

Massai, L., Nesi, P., & Pantaleo, G. (2019). PAVAL: A location-aware virtual personal assistant for retrieving geolocated points of interest and location-based services. Engineering Applications of Artificial Intelligence, 77(September 2018), 70–85. https://doi.org/10.1016/j.engappai.2018.09.013

pyttsx3 · PyPI. (n.d.). Retrieved January 30, 2021, from https://pypi.org/project/pyttsx3/

Schuetzler, R. M., Giboney, J. S., Grimes, G. M., & Nunamaker, J. F. (2018). The influence of conversational agent embodiment and conversational relevance on socially desirable responding. Decision Support Systems, 114(August), 94–102. https://doi.org/10.1016/j.dss.2018.08.011

Seco, A., & Muñoz, A. (2019). Asistentes conversacionales virtuales en las administraciones tributarias: Principios, modelos y recomendaciones.

SpeechRecognition · PyPI. (n.d.). Retrieved January 30, 2021, from https://pypi.org/project/SpeechRecognition/

Talwar, R., & Koury, A. (2017). Artificial intelligence – the next frontier in IT security? Network Security, 2017(4), 14–17. https://doi.org/10.1016/S1353-4858(17)30039-9

Thirupathur Kailashnathan Vijay Vangara, S. V. (2020). A Survey onNatural Language Processing in context with Machine Learning. Ijaema, XII(1), 1390–1395. https://doi.org/DOI:18.0002.IJAEMA.2020.V12I1.200001.015103

Wit.ai. (n.d.). Retrieved January 30, 2021, from https://wit.ai/