1. Introduction
The inclusion of blind people in all areas of the Society is remarkable, the use of braille text and adaptations can be seen in many places such as banks, bus stops, elevator buttons, etc. Even more and more developers are promoting the creation of technological devices specially designed for blind people (Mausksch, 2021).
For the visually impaired, hearing and touch become their main way of perceiving the world around. With the imminent establishment of technology in everyday life, access to it is more than necessary. Modern technology has several accessibility options that allow disabled people to use it easily, the devices use speakers, sensors, embossed text and other methods to allow orientation of the blind in the environment (Mereu & Kazman, 1997).
The number of blind people in Ecuador is considerable, according to figures from Consejo Nacional para la Igualdad de Discapacidades, there are 56,570 people with visual disabilities, that is 11.66% of the population (Ramstein,1996); being a big community, the opinion of these people is important for the improvement and adaptation of products or services; therefore, it is necessary to develop an evaluation system that allows a quick, easy, intuitive and objective collection of the level of satisfaction that the user perceives.
In Ecuador, academic training for people with visual disabilities has faced challenges that hinder their development, starting with the lack of a continuous and regularized educational system, as well as the absence of specialized materials for learning subjects such as Braille reading, basic mathematics, languages and others. Casvi (Ramstein,1996), is an investigation project which aims to develop a system of mathematics teaching for people with visual disabilities through software developed in Ecuador with a view to being an important tool for the inclusive education system in the country. Casvi developers, like many other product designers, they need to measure the impact software has had on their users and that’s where Devi comes in to help.
The main objective of Devi is to provide a universal qualification tool to quantify the acceptance level of products or services such as Casvi, and to overcome the problems caused by not having a safe and personal voting tool. In this sense, Devi offers the possibility of eliminating conventional printed forms assisted by third parties, guaranteeing that the voter issues his or her judgment objectively and with little or no intervention from a human evaluator.
2. Materials and Methods
Speech synthesizers are software or hardware products that allow written information to be perceived through the ear, being able to reproduce written text through a speaker. To facilitate the use of a computer for blind people, there are programs supported on speech synthesizers capable of narrating the interface of a screen. A critical element in this type of interface is the adequate design of the dialogue established between the user and the computer, otherwise the usability falls to an inoperable level (Geethanjali el at, 2017).
In search of new forms of inclusive evaluation, in 1974, an emotional measurement method called SAM (Self-Assessment Manikin) was developed, which is a non-verbal pictorial qualification technique that directly measures the pleasure and arousal that a person experiences on a wide variety of stimuli which requires 18 different classifications (Geethanjali el at, 2017). The system has come to be considered as a valid qualification tool for products or services; however, the method can present symbolic variants according to its application as can be seen on Fig. 1(Bradley and Lang, 1994). Identification and differentiation become complex; especially when the tool is intended for people with visual impairment or disability.
Figure 1
The Self-Assessment Manikin (SAM) scale used to measure pleasure (IEEE Computer Society,2008).
Source: Own elaboration
When carrying out surveys or evaluations, aspects that allow obtaining results without affecting the psychological state of individuals should be taken into account, thus avoiding subjective responses. The questions must be formulated with the user in mind, in addition, they must have a flow that is easy to interpret and requires the least possible effort. Factors such as evaluation time, type of questions, contextualization of the evaluation are critical to avoid an increase in user stress. The approach to the questions should also take into account that the collection of information must be of high quality and easy to interpret (Geethanjali el at, 2017).
The continuous growth of computing and, with it, the elimination of barriers that communication used to have due to the distance forced the development of a universal symbology to identify and express emotions, this is how emoticons arise, which have definitively taken root in computer culture. The use of emoticons is beneficial because it increases the richness of the message in communication, as well as its efficiency and effectiveness, and also reduces the time of interpretation of the information (Bradley & Lang, 1994). Thanks to its easy use and understanding this type of symbology is a viable option within the development of new validation techniques, facilitating voting options for those evaluated and providing an easy to analyze comparison margin for evaluators.
Devi is an integrated hardware and software system for evaluating processes, activities and products of all kinds, as a result, the analysis documentation of the feedback from the users is obtained; this documentation is processed and the software displays pie charts with ratings on a scale of 1 to 4. The assessment methodology is based on international standards, emphasizing the documentation of the IEEE 829 designed for the documentation of tests (Sneed & Seidl, 2017).
The qualifier architecture is presented as well as the phases in which the questions are based, the methodology used in the evaluation, and finally, the palpable conclusions obtained once the prototype has been tested. A preview version of Casvi was tested on a small group of blind people and its impact needed to be measured; with the help of Devi this was possible and the analysis of Devi’s effectiveness is shown in the last sections.
Devi is a universal qualifier system, it works thanks to the integration of a software developed for Windows and an external qualification panel with USB connection which works only with Devi software. The system has been designed for blind people with or without knowledge of braille language, the buttons have haptic symbols so that they can be easily recognized, the highest rating being a happy face and a sad face for a bad rating.
The software allows the evaluator to input or edit the questions in the administrator section of the program; the questions need to be in three languages: Spanish, English and Portuguese. Devi reads the questions to the user thanks to the text to speech API included on Windows SDK.
During the execution of Devi, it will display a statistical pie chart for each question asked showing the percentage of each rating and providing objective feedback to the evaluator or analyst of the results. Devi generates a file in .xlsx in the computer’s documents folder for storing the rates once the program is closed. If the evaluator prefers, the document can be used for a deeper analysis using the power of Microsoft Excel software (De Souza et al., 2017).
The Devi graphical interface is intuitive and easy to use only requiring a brief introduction of the usage of the Devi tool; the rest of the work will depend solely on the person evaluated, since a new question will be heard only if the last question was already qualified. The questions are spoken when the “question mark” button is pressed.
A. Function of Devi
• Language selection
• Approach and edition of questions oriented to the product or service to be rated through the administration section.
• Connection from the panel to the computer through USB with auto-detection.
• Average statistical display of the survey carried out.
Results file exported in Excel file.
B. Physical components
The design of the device contemplates requirements of comfort and accessibility oriented to the user facilitating their experience; the following aspects have been considered:
• Portability: Its dimensions are 100mm x 100mm, making the equipment easy to locate in any fixed or improvised work area.
• Usability: Its one-handed operation provides ease of manipulation for ambidextrous users, thanks to its only 5 clearly identified buttons, the experience of use adopts a common posture for average users as if it were a matter of holding a computer mouse.
• Inclusive Support: In order to help an increasingly inclusive society, the device has Braille writing next to each button of the board as well as buttons with high relief engravings to identify more or less acceptance when issuing a rating from users.
• Adaptive Design: The device has a micro USB connection and is Plug and Play for maximum compatibility with any Windows computer.
In Fig. 2 you can see the qualification panel which focuses its operation on one hand, providing the necessary facilities for people with a motor or visual disability; in addition to having audible assistance in three different Languages.
• Spanish
• English
• Portuguese
C. Software
Program developed exclusively to work with the physical panel through USB connection to the computer, the minimum requirements for the functionality of the program are detailed below (Hudyjaya & Raidy. 2020):
• Operating System: Windows 8 or superior.
• Available disk space: 30 MB (minimum).
• Ram Memory: 1 GB.
• Screen Resolution: 800 x 600 pixels (minimum).
Figure 2
Devi’s physical device. (a) Top view. (b) 3D view of the design model.
Source: Own elaboration
• Speakers or headphones.
• Installed languages: English, Spanish, Portuguese.
Experiments:
The test documentation was carried out under the IEEE 829 standard regarding software test documentation [5], which establishes ten documents that provide an optimal development of activities evaluation.
The required documents are presented below.
• Master Test Plan (MTP).
• Level Test Plan (LTP).
• Level Test Design (LTD).
• Level Test Case (LTC).
• Level Test Procedure (LTPr).
• Level Test Log (LTL).
• Anomaly Report (AR).
• level Interim Test Status report (LITRS).
• level Test Report (LTR).
• Master test report (MTR).
Criteria selected for the evaluator based on IEEE test documentation The standard suggests each of the documents presented, however, document selection criteria must be managed for each particular case. The first criterion suggests establishing the level of integrity. In search of optimizing resources. The qualifier focuses its operation on three main axes which are presented below in Table. I. (IEEE Computer Society,2008).
Table 1
IEEE-829 STANDARD FOR SOFTWARE TEST DOCUMENTATION
|
Level Test Plan (LTP) |
For each LTP the scope, approach, resources and schedule of the testing activities |
|
Level Test Procedure (LTPr) |
Detailing how to run each test, includ- ing any set-up preconditions and the steps that need to be followed. |
|
Anomaly Report (AR) |
To document any event that occurs during the testing process that requires investigation. |
Source: Criteria documentation (Sidek et al., 2011)
The documentation is also flexible in the choice of documents for the evaluation of a specific software since not all are necessary for the specific job. In search of optimizing resources, the qualifier focuses its operation on three main axes which are:
• Methodology: Level Test Plan (LTP).
• Evaluation Process:Level Test Procedure (LTPr).
• Analysis of Results: Level Test Report (LTR).
Each of these are managed under parameters that are explained in Fig. 3, complementing the use of the qualifying panel with the developed program, converting the votes cast into statistical results shown at the end of the survey.
Figure 3
Devi - Flow diagram of a case of use for Devi.
Source: Own elaboration
A. Methodology
The responsible evaluator must enter the questions related to the product or service provided. Therefore, in the Administer button, each question must be typed in Spanish, English and Portuguese, as shown in the Fig. 4, these questions will be presented in order to the user and narrated through the voice support incorporated into waiting for a grade until completing the total number of questions asked.
Figure 4
Devi - Questions administrator.
Source: Own elaboration
B. Evaluation Process
To carry out the evaluation process, 9 blind people made use of the board, for which an assistant was needed to help the evaluated person, placing his hand on the Devi board, positioning his fingers on each button so that he can use it without any inconvenience. It is expected to find in the level of complexity that the use of the board presents once its operation has been explained, at the same time it is desired to determine the difference that exists when the user manipulates the board with the right or left hand, for which the instructor must position the hand chosen by the user and arrange it as shown in the Fig. 5.
Figure 5
Devi - (a) Suggested usage with left hand. (b) Suggested usage with right hand.
Source: Own elaboration
The disposition of each finger is distributed as follows:
• Thumb: To press the help button with which you will hear the question before casting your vote.
• Index finger: Located on the Excellent button to qualify with the highest score.
• Middle finger: Located on the Good button to qualify with a high score.
• Ring finger: Located on the Medium button to qualify with an average score.
• Pinky: Located on the Excellent button to qualify with the lowest score.
3. Results and Discussion
The use of the Standard for Software Test Documentation was a support for the software development tests whose results show that all the participants were able to correctly manipulate the rating panel and, guided by the program’s voice assistance, were able to faithfully answer all the questions previously asked.
Usage trend
The board turned out to be of versatile use, adapting to the hands of the participants, where the following trend of use was found, as shown in the ”fig. 6”.
Identification and preference guide The device has two guide supports for its use which are:
Figure 6
Devi - Right and left hand usage trend.
Source: Own elaboration
Braille text: Typeface engraved in high relief to be recognized by a user with knowledge of Braille writing. • High relief buttons: Emoticons to relate the level of acceptance in each question.
There is a clear preference for tactile exploration of the board, identifying each button by position, as well as its size and volume, trying to find differences between them.
The identification by braille of each of the options was reduced since only one of the nine participants mastered the written language aimed at blind people, which is an indicator that the educational system for this group of society needs a restructuring.
As can be seen in ”Fig. 7” there is a preference for the assistance of the guide and the voice support provided by the program to understand its operation and together with the buttons in high relief the users were able to manipulate the board efficiently without problem.
4. Conclusions
The physical device is light with adequate dimensions, which makes comfortable to transport and it does not subtract excessive space on the work table. From the Devi software, there are no problems presented in three languages: Spanish, English and Portuguese, which has an attractive visual environment for the evaluator and simple operation. Showing an acceptance level of 88.88% as an alternative to conventional evaluation methods, Devi has proven to be a tool capable of being implemented in various fields at the service of blind people.
Figure 7
Devi - Identification and preference guide.
Source: Own elaboration
The use of buttons with different sizes to ask both the question and to qualify the service and the implementation of haptic reliefs in each of them fulfills its function satisfactorily as reflected in the level of acceptance, since when performing the qualifier tests people who do not they had knowledge of braille they were able to recognize the level of qualification by means of the hollow relief of the expressions of each button, the positioning of the buttons is adequate helping the positioning of the hands that as a result of the evaluation was comfortable to use.
The Devi software helps the evaluators to quickly obtain the evaluation percentage through its graphical interface in real time, allowing to identify the shortcomings of the evaluated service at the moment; in addition, the tool creates a detailed Microsoft Excel’s file backup to be able to make reports which include date and hour registers that can be used to establish a plan for the improvement of the service.
The tests carried out with the software showed a significant reduction in the time spent both in the training of use, and in the time of execution of the test by approximately 30% since the blind person quickly adapted to the device and was capable of completing a survey in less time. In a normal test of ten questions, it was determined that the average speed didn´t exceed one minute for each question compared to the traditional method that consists of listening to the questions and casting their vote verbally.
As it is a universal device, it does not limit age or a specific use of evaluation since it allows to qualify a service, a feeling about an event taking into account that it can be widely used in children with disabilities or not, since due to its design and Colors is eye- catching for children, improving the child’s communication if the child has any problem or impediment.
5. References
De Souza, É. F., De Almeida Falbo, R., & Vijaykumar, N. L. (2017). ROoST: Reference Ontology on Software Testing. Applied Ontology, 12(1), 59-90. https://doi.org/10.3233/ao-170177
Espinal-Loor, J. E. (2022). La inclusión laboral basada en las experiencias de las personas con discapacidad visual de Manabí. Revista Científica y Arbitrada de Ciencias Sociales y Trabajo Social: Tejedora. ISSN: 2697-3626, 5(10), 144-155. https://publicacionescd.uleam.edu.ec/index.php/tejedora/article/view/322
Geethanjali B., Adalarasu K., Hemapraba A., Kumar S. P., and Rajasekeran R., (2017) “Emotion analysis using sam (self- assessment manikin) scale,” Biomedical Research.
IEEE Computer Society S. S. E. S. C.(2008), IEEE standard for software and system test documentation. IEEE.
Jo H. S. and Jo R. S.(2020), Design and Development of Remote Laboratory System to Facilitate Online Learning in Hardware Programming Subjects, 13th International UNIMAS Engineering Conference (EnCon), Kota Samarahan, Malaysia, 2020, pp. 1-5, doi: 10.1109/EnCon51501.2020.9299326.
Mauksch, S. (2021). Being blind, being exceptional: work integration, social entrepreneurship and the
reimagination of blind potential in Nepal. Disability & Society, 38(3), 401-420. https://doi.org/10.1080/09687599.2021.1933387
Mereu S. W. and Kazman R.(1997). Audio enhanced 3d interfaces for visually impaired users, ACM SIGCAPH. Computers and the Physically Handicapped, no. 57, pp. 10–15.
Mejía, P., Martini, LC, Grijalva, F., & Zambrano, AM (2021). Casvi: Sistema de álgebra informática dirigido a personas con discapacidad visual. experimentos. Acceso IEEE, 9, 157021-157034. DOI: 10.1109/ACCESO.2021.3129106
Ramstein C. (1996), Combining haptic and braille technologies: design issues and pilot study in Proceedings of the second annual ACM conference on Assistive technologies, pp. 37–44.
Sidek, R. M., Noraziah, A., & Wahab, M. H. A. (2011). The preferable test documentation using IEEE 829. Communications in computer and information science (pp. 109-118). https://doi.org/10.1007/978-3-642-22203-0_10
Sneed, H., & Seidl, R. (2017). Software Testdokumentation nach den IEEE und ISO Standards. https://dl.gi.de/items/692e4512-0775-4dcf-bd49-55073773f1e8