Technology systems to enable a man with intellectual, sensory and motor disabilities to make verbal requests through simple one-hand signs: proof-of-concept study

Abstract

Purpose

This study assessed two technology systems aimed at enabling a man with intellectual disability, blindness, deafness and motor and tactile discrimination problems to make verbal requests through simple one-hand signs.

Design/methodology/approach

The study was conducted according to an ABAB design. During the B (intervention) phases, the man used the two systems, which included (1) nine mini recording devices fixed on the man’s clothes or wheelchair (i.e. in positions the man touched with his sign movements) and (2) nine tags with radio frequency identification codes (fixed at approximately the same positions as the mini recording devices) and a dedicated tag reader, respectively. Making a sign (i.e. touching a recording device or reaching a tag) led to the verbalization of the request related to that sign.

Findings

During baseline, the mean frequency of signs/requests made was below 2 per session, and only some of those requests were identified/satisfied. During the intervention, the mean frequency of requests made and satisfied was about 10 per session with each of the systems.

Originality/value

The results, which are to be taken with caution given the preliminary nature of the study, seem to suggest that the systems can help translate simple signs into verbal requests.

Citation

Lancioni, G., Alberti, G., Pezzuoli, F., Bruciati, J., Singh, N., O'Reilly, M. and Sigafoos, J. (2024), "Technology systems to enable a man with intellectual, sensory and motor disabilities to make verbal requests through simple one-hand signs: proof-of-concept study", Journal of Enabling Technologies, Vol. 18 No. 1, pp. 50-58. https://doi.org/10.1108/JET-08-2023-0028

Introduction

People with intellectual and multiple disabilities are likely to encounter very serious difficulties in acquiring functional communication skills and particularly expressive communication skills (Holmqvist et al., 2018; Lancioni et al., 2019; Skarsaune et al., 2021). In fact, many of them may not be able to develop verbal abilities and may need to rely on nonverbal strategies for any form of interaction and particularly for making requests (Alzrayer et al., 2017; Ivy et al., 2020). The non-verbal strategies typically employed for making requests involve the use of: picture exchange communication systems (PECS), speech generating devices (SGDs), manual signs, and eye-gaze communication devices (Fleury et al., 2019; Holmqvist et al., 2018; Kunze et al., 2019; Meuris et al., 2015; Rombouts et al., 2018; Treszl et al., 2022). While widely recommended, those strategies might present clear limitations when employed with people with combinations of intellectual, sensory and motor disabilities (Flink et al., 2022; Prynallt-Jones et al., 2018; Ricci et al., 2017).

For example, standard PECS relying on the use of pictures and eye-gaze technology would not be viable with people whose condition also includes blindness. An adaptation of the PECS (i.e. with object cues instead of pictures) would suit people with blindness or deaf-blindness (Bracken and Rohrer, 2014; Ivy et al., 2020; Trief et al., 2013) but only if they have no problems in manipulating and discriminating objects. The use of typical SGD systems relying on the discrimination and activation of images through touch responses may be impossible for people with blindness and/or motor problems interfering with fine hand/finger movements (e.g. movements typically required to select and activate an image on a tablet screen; Ricci et al., 2017). Manual signs might be useable even with people with blindness, deaf-blindness and fine motor coordination problems or tactile discrimination problems (i.e. inability to easily discriminate and use daily objects; McIntyre et al., 2021; Ricci et al., 2017). A disadvantage common to manual signs and object cues is that their use might not always guarantee an effective communication process. In fact, signs and object cues can convey messages/requests only when the intended communication partner (e.g. a staff or family member) is in the proximity of the person using the signs or the object cues and watches the person while this makes the signs or shows the objects (Lancioni et al., 2023).

In light of the above, one may argue that new research is needed to find ways of tackling the limitations of the aforementioned strategies and offer people with intellectual, sensory and motor disabilities the possibility of making successful requests. The objective of this study was to circumvent one of the main limitations of manual signs and so increase their communication effectiveness. Specifically, the study assessed the applicability and potential of two technology systems aimed at making a man’s one-hand signs produce specific verbal outputs, that is, the verbalization of the signs' intended messages/requests. Such verbalization was meant to be audible by the man’s intended communication partners even when these were not in the man’s proximity and/or were not paying attention to him.

Method

Participant and setting

The participant (hereafter referred to with the pseudonym of Burt) was 49 years old and presented with blindness, deafness, motor impairment (i.e. inability to walk and minimal mobility of the left arm) and extreme difficulty to recognize objects due to tactile discrimination problems. Burt’s age equivalents, assessed via the second edition of the Vineland adaptive behavior scales (Balboni et al., 2016; Sparrow et al., 2005), were 2 years and 2 months for daily living skills (personal subdomain), 1 year and 9 months for receptive communication, and 2 years and 3 months for expressive communication. He had a repertoire of about 50 signs that he made through movements of his right arm and hand and mainly used for expressing his feelings, asking or telling about daily events, and making requests to staff and family members when these were in physical contact with him and thus could see and respond to the signs. Requests would typically concern forms of attention and interaction as well as objects for activities or preferred food. When the communication partners were away from him (and had very limited opportunities to see and respond to his signs), he tended to remain passive and refrain from making signs. His level of intellectual disability had been estimated to be in the moderate to severe range by the psychological service of the rehabilitation center he attended. The study was carried out in Burt’s living and occupation room within the aforementioned rehabilitation center.

Ethical approval and informed consent

Burt seemed eager to use the technology systems assessed in this study, which (1) had been introduced to him in advance and (2) were considered suitable to provide him with a new (satisfactory/positive) communication experience. However, he could not give any formal consent for his involvement in the study given his inability to read and sign a consent document. In light of this, his legal representative was required to provide such consent on his behalf. The study complied with the 1964 Helsinki declaration and its later amendments and had been approved by an institutional ethics committee.

Technology system I

The first technology system entailed nine mini recording devices, which were fixed on Burt’s clothes or wheelchair (i.e. in positions/places such as shoulders, legs and wheelchair’s armrest, which constituted the landmarks for the signs Burt used for making requests). Those devices consisted of small cylinders having a 4.5-cm diameter and a 2.3-cm height (Borgione, art. 804,942). Each of the devices contained a recorded message (request) and uttered such message when activated through a light hand touch/pressure. The upper section of Figure 1 provides a schematic representation of the positions/places where those devices were fixed, and Table 1 provides a list of the signs used and the related verbal requests emitted by the mini recording devices in connection with the signs. For example, bringing the right hand to the right shoulder (and pressing/activating the mini recording device fixed there) led to the verbal request for yoghurt. Bringing the right hand to the left edge of the protective abdominal belt (and pressing the mini recording device fixed there) led to the verbal request for an activity involving tennis balls. Bringing the right hand to the edge of the wheelchair’s right armrest (and pressing the mini recording device fixed there) led to the verbal request for an activity involving the use of blocks. Bringing the right hand to a central area of the wheelchair’s right wheel (and pressing the mini recording device fixed there) led to the verbal request for playing an arm-wrestling game with the research assistant or staff member available. Most of the signs were new as Burt did not possess a specific sign for a variety of food items and activities (even though they were familiar and attractive to him). The verbal requests could consist of single-word utterances (e.g. “YOGHURT”) or two-to four-word utterances (e.g. “YOGHURT PLEASE” and “LET’S PLAY WRESTLING”). The use of the different formulations (which were intended to be clear and immediate) was to bring some, presumably desirable, variability in the verbalization of the requests.

Technology system II

The second technology system entailed (1) nine tags of 5.4 x 8.5 cm with radio frequency identification codes (NCF MIFARE RFID, 1K 13,56 MHz; Yarongtech), which were fixed at approximately the same positions as the mini recording devices (see the lower section of Figure 1), and (2) a tag reader linked via Bluetooth to a smartphone. Each tag was programmed to represent a specific request message that was to be recognized by the tag reader and verbalized by the smartphone. The tag reader, which was built specifically for this study, involved a combination of two connected watch-like devices (of 4 x 5.5 x 1.5 cm and of 5 × 5 × 2 cm, respectively; see Figure 2), which Burt wore at the right wrist. The device that was worn on the lower/volar side of the wrist consisted of a near field communication (NFC) module that recognized the tags' identification codes and related messages. The device that was worn on the upper side of the wrist included an electronic circuit, which was linked via Bluetooth to a smartphone. When Burt used his hand to make a sign (i.e. bringing his wrist in the proximity of one of those tags), the tag reader discriminated the tag approached and uttered via the smartphone the related message/request. The smartphone was attached to Burt’s wheelchair or placed on a desk in his proximity. The use of this system did not require any particular adjustment in the sign movements. This was possible given the slight change in the position of the tags compared to the position of the mini recording devices (see upper and lower sections of Figure 1). The only difference between systems was that for the first the arm/hand movement was also to include a little pressure on the mini recording device available in the area whereas for the second no such pressure component was involved.

Experimental conditions and data collection

The study was carried out according to an ABAB design, in which A and B represented baseline and intervention phases, respectively (Barlow et al., 2009). The baseline (A) and the intervention (B) phases included totals of 10 and 61 sessions, respectively. During all phases, data were collected by a research assistant over 20-min sessions. Data collection consisted of recording the requests Burt made and whether those requests were noticed and followed by a response (i.e. were satisfied). During the baseline sessions, staff was to behave as they typically did with Burt (i.e. regarding interactions with him and responses to his requests). During the intervention sessions, the research assistant was in charge of responding to (satisfying) Burt’s requests. This direct involvement of the research assistant was decided for practical reasons (i.e. to free staff from the need of committing to specific session schedules and consequently avoid any possible conflict between their adherence to those schedules and their performance of other daily tasks).

Baseline and intervention

The baseline sessions did not include the technology systems and were intended to provide a picture of typical daily situations. The research assistant’s role during those sessions was limited to data recording. The intervention sessions (1) included the technology systems whose use was alternated across sessions (i.e. following an alternating treatment approach; Barlow et al., 2009) and (2) served to determine whether Burt could use the systems successfully and thus made audible verbal requests that could be readily identified and eventually responded to even by people not in his immediate proximity. The research assistant who, as in the baseline sessions stood a few meters away from Burt, was to collect data and respond to Burt’s requests in place of staff (see above). The first B phase was preceded by seven introductory sessions during which Burt (1) practiced the nine signs to be used during the intervention sessions with the technology systems (see Table 1), (2) received the events (e.g. food items and activity material) requested via the signs as soon as these were emitted, and (3) also familiarized with the systems and the way they worked.

During the intervention sessions, each sign (arm/hand movement leading to the activation of a mini recording device or of the tag reader) was followed by the related verbal utterance/request. Every verbal request was followed by the research assistant’s response. For example, the request for a particular food item led to the research assistant’s delivery of such food item (e.g. a few spoonsful of yoghurt). The request for an activity led to the research assistant’s delivery of the material needed for such an activity (e.g. tennis balls to fill up one or two tennis balls' tubes). The request for staff led to the research assistant helping Burt get in touch with a preferred staff member. If Burt repeated the same request (e.g. for a specific food item) for up to three times, the research assistant responded by satisfying such request each time. Any additional repetition of the same request was not counted/recorded and the research assistant informed Burt that he could choose one of the other request options.

Inter-rater agreement

Inter-rater agreement on data recording was checked in all 10 baseline sessions and 22 of the 61 intervention sessions, through the involvement of a reliability observer in addition to the research assistant. The percentage of agreement (computed by dividing the number of sessions in which the research assistant and reliability observer reported the same scores on requests made and requests satisfied by the total number of sessions in which agreement checks were made and multiplying by 100%) neared 94%.

Results

Figure 3 reports the baseline and intervention data. The empty circles and dots indicate the mean frequency of requests that Burt made and the mean frequency of requests that staff recognized and responded to (satisfied) per session over blocks of baseline sessions. The black triangles and asterisks indicate the mean frequency of requests made and responded to (satisfied) per session over blocks of intervention sessions with the system using mini recording devices and with the system using the tag reader, respectively. The blocks typically include two sessions. Occasional blocks with three sessions (i.e. at the end of the phases) are indicated with an arrow. The verbalization of the requests during the intervention sessions ensured that those requests were easily identified and eventually satisfied and thus requests made and requests satisfied corresponded. The figure does not report the introductory sessions.

As shown by the figure, the frequency of requests Burt made during the baseline phases was relatively low (i.e. a mean of less than 2 per session). Only some of those requests (i.e. a mean of about 1 per session) were apparently noticed and eventually responded to by staff. Staff, in fact, could be at some distance from Burt and pay no visual attention to him when he made a sign.

The first intervention phase showed a clear change. During the sessions with the system involving mini recording devices, Burt had a mean frequency of about 10 requests per session. Those requests were verbalized by the mini recording devices and easily recognized and responded to by the research assistant. During the intervention sessions with the tag reader, the request frequency was similar to that obtained with the mini recording devices. The relatively high request frequencies observed during the sessions were typically due to the fact that some of the requests could be satisfied fairly rapidly (e.g. requests for food and wrestling or pushing games) and could be repeated during the sessions. The data for the second baseline and the second intervention phase were largely consistent with the data observed during the first baseline and first intervention phase.

Discussion

The results, which are to be taken with caution given the preliminary nature of the study, seem to suggest that it is possible to set up technology systems that can help a participant with intellectual disability, blindness, deafness, motor impairment and poor tactile discrimination to produce verbal requests through simple one-hand signs. In light of these results, five basic considerations can be made.

First, enabling people like Burt to make requests that are easily audible and recognizable by staff and caregivers even when they are not in the immediate proximity represents a meaningful achievement. Indeed, it empowers generally marginalized people to have an active (successful) role and conceivably improves their quality of life (Wehmeyer, 2020). It may also help staff and caregivers reduce misinterpretations of the people’s needs and desires (Lancioni et al., 2023).

Second, while the two systems used in this study appear equally effective and user-friendly, a difference exists as to their accessibility and cost. The first system relies on commercially available mini recording devices, which can be directly acquired at a cost of about US$15 per device and can be readily applied (Kulkarni et al., 2022; Mishra et al., 2022). The second system relies on the use of commercially available and inexpensive tags (i.e. US$1 per tag), a basic smartphone and a dedicated tag reader. This reader includes commercially available components as well as specific electronic circuit allowing staff personnel to easily customize/program the system for different participants. Acquiring the tags, the commercially available components, and the smartphone may cost about US$350–400. The design of the electronic circuit is available at no cost from the authors.

Third, both systems are easily transportable and useable across settings with practical advantages for participant and staff or caregivers (Zonneveld et al., 2020). At the same time, both systems might be viewed as relatively intrusive because they require the participant to wear a number of mini recording devices or a number of tags and a tag reader. In spite of this limitation, it may be argued that putting on those technology components is relatively easy and wearing them does not seem to interfere with the participant’s typical actions (Moon et al., 2019).

Fourth, to extend the number of requests possible, two options might be considered. One would entail the use of additional recording devices/tags and related signs. The other (probably more practical and realistic) would maintain the same number of recording devices/tags and signs utilized in this study, but make each device/tag and sign produce a request for two similar events (e.g. two food items and two occupational activities) rather than a single event. The communication partner responding to the request would then ask the participant to indicate which of the two events was specifically intended (preferred) at that particular time.

Fifth, future research might extend the present investigation in different ways. For example, it may replicate the present study with new participants to determine the strength and generality of the present data. It may evaluate ways of enlarging the number of requests manageable. It may also develop new systems that are inexpensive, easily accessible, as well as less visible/intrusive than the present ones.

Figures

Figure 1

Schematic representations with the positions of the mini recording devices and the tags (upper and lower sections of the figure, respectively)

Figure 2

Schematic representation of the electronic circuit and NFC module involved in the tag reader

Figure 3

The empty circles and dots indicate the mean frequency of requests that Burt made and the mean frequency of requests that staff recognized and responded to (satisfied) per session over blocks of baseline sessions, respectively. The black triangles and asterisks indicate the mean frequency of requests made and responded to (satisfied) per session over blocks of intervention sessions with the system using mini recording devices and with the system using the tag reader, respectively. The blocks typically include two sessions. Blocks with three sessions (i.e. at the end of the phases) are indicated with an arrow

Table 1

List of signs available and corresponding verbal requests uttered by the mini recording devices (first system) or tag reader (second system)

Signs	Corresponding verbal requestsa
Bringing the right hand to the right shoulder	Yoghurt
Bringing the right hand to the left shoulder	Pistachios
Bringing the right hand to the edge of the wheelchair’s right armrest	Blocks
Bringing the right hand to the left edge of the protective abdominal belt	Balls
Bringing the right hand close to the wheelchair back at the hip level/side	Pushing
Bringing the right hand to the center of the wheelchair’s right wheel	Wrestling
Bringing the right hand to the left leg	Staff
Bringing the right hand to the right leg	Peanuts
Bringing the right hand to the left arm	Walk

Note(s): ^aOnly the single-word formulations are listed

Source(s): Authors’ own work

Ethical approval: The study was approved by an institutional ethics committee. All procedures performed were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent: The participant’s legal representative provided written informed consent for the participant’s involvement in the study.

Conflict of interest: The authors developed the technology system involving the tag reader and make available at no cost the design of the electronic circuit regulating the system.