Activity ordering task: conceptualization and development of a novel context-based working memory task with a metacognitive facet

Jacob, Nidhi Lalu; Bajaj, Gagan; Rooha, Aysha; George, Vinitha Mary; Bhat, Jayashree S.

doi:10.1590/2317-1782/20242024041en

Acessibilidade / Reportar erro

Brasil

Español English

sumário « anterior atual seguinte »

Sumário

Original Article • CoDAS 36 (6) • 2024 • https://doi.org/10.1590/2317-1782/20242024041en copy

Activity ordering task: conceptualization and development of a novel context-based working memory task with a metacognitive facet

Authorship SCIMAGO INSTITUTIONS RANKINGS

ABSTRACT

Purpose

The aim of the present study was to conceptualize and develop a novel context-based working memory (WM) task which would possess the potential to assess the context-based WM in an Indian scenario and incorporate a task-linked metacognitive facet to understand an individual's self-perception of performance.

Methods

Employing the ADDIE instructional design model, comprising Analysis, Design, Development, Implementation and Evaluation phases, the present study focused on the initial four phases. In the Analysis-phase, the needs and objectives for creating a context-based WM task were identified. Design-phase included task conceptualization and content validation of the conceptualized context-based WM task. In the Develop-phase, the designed context-based WM task was developed in an animated video format. Five content experts and fifteen lay experts were involved in the content validation process. In the Implementation-phase, a pilot study was done on ten adults to assess the feasibility of the novel context-based WM task.

Results

The task obtained good content validation index and understandability scores on the Patient Education Materials Assessment Tool. Preliminary data trends from the implementation-phase revealed the task's potential to detect age-related WM differences. Significantly correlated with established WM tasks, the novel Activity Ordering Task (AOT) effectively measured WM-spans. Observed discrepancies between performance and prediction/postdiction spans during metacognitive facet administration highlighted the AOT's utility in evaluating metacognition.

Conclusion

Addressing the limitations of context in traditional tasks, the AOT appears to be a promising tool that not only measures WM but also assesses metacognition, thereby enhancing its utility beyond an everyday WM assessment.

Keywords:
Everyday Working Memory; Activity Ordering Task; Metacognition; Indian Adults; Context-based Task

Attribute	Participants for Content Validation (Design and Develop Phase)				Participants for pilot study (Implementation Phase)
	Content Experts (N=5)	Lay experts (N=15)			YA (N=5)	MAA (N=5)
	SLP	YA (n=5)	MAA (n=5)	OA (n=5)	YA (N=5)	MAA (N=5)
Mean Age	30±4.24 years	19.6+2.07 years	54.4+5.36 years	71.4+5.68years	22+1.58 years	51.8+7.79 years
Gender	Male: n=1	Male: n=1	Male: n= 3	Male: n=2	Male: n=0	Male: n=2
Gender	Female: n=4	Female: n=4	Female: n=2	Female: n=3	Female: n=5	Female: n=3
Mean LEAP Q score	9.3+0.62	8.3+0.84	8.9+0.84	8.7+0.92	8.1+0.80	8.5+1.01
Mean socioeconomic status score on Modified Kuppuswamy scale	23.4+3.96	22.4+3.29	23.2+2.68	24+3.80	17.8+2.04	21.8+3.96

Pilot study aspects	Probe questions
Comprehensibility of instructions	Did you find the activity instructions provided to the central character lengthy?
Comprehensibility of instructions	Do you think the activity instructions are difficult to understand?
Effectiveness of Voice-over	Did you find the use of voice-over for instructions effective?
	Should voice-over be used for the entire content, or only for the activity description and completion timeline?
	In your opinion, did the voice-over enhance or hinder your understanding of the tasks?
Timing of Instructions	Do you prefer receiving detailed instructions for each trial, or having one-time instructions at the beginning of the first trial?
Necessity of clock display	Do you think the presence of a clock on each slide helpful for keeping track of time?
Necessity of clock display	Do you think displaying a clock on each slide is necessary, or could it be omitted without affecting the task completion?
Overall Task Experience	Please share your overall feedback on the task experience

Sl no.	Shortlisted ideas after Step 1 and Step 2	Refined set of shortlisted ideas after Step 3 and 4	Ranks assigned
1.	Spatial-based Activity Ordering Task	Spatial-based Activity Ordering Task	2
2.	Task Importance-based Ordering	Category-based Word Recall and Ordering	6
3.	Category-based Word Recall and Ordering	Timeline-based Activity Ordering	1
4.	Recipe Ingredient Ordering Task	Daily Use Object Ordering Task	4
5.	List Creation Task	Category-based Object Ordering	5
6.	Store Section-based Object Ordering	Store Section-based Object Ordering	3
7.	Daily Use Object Ordering Task
8.	Timeline-based Activity Ordering
9.	Event Sequencing/Story Sequencing Tasks
10.	Floor-based Activity Ordering

Task elements	Descriptions
Participant instruction	You'll see several scenes from everyday life in which a character is given multiple tasks to complete at different timings in a day. You must help the character order the tasks according to the timeline. The number of tasks given to the character increases as the assessment progresses. Carefully attend to each of the tasks. When the screen flashes 'Help me order,' please respond within 60 seconds. Accuracy is more important than speed; we will evaluate your response based on its correctness, not how quickly you complete it. Remember, the maximum response time is 60 seconds. Press the 'Proceed' button after completing your response.
Themes	Home/Family Errands and Work/Professional Errands
Characters	Family member, Working Parent, Homemaker, Policeman, Vegetable vendor, Railway officer, Gardener, Postman, Doctor, Peon, Bank Manager, Tailor, Milkman, Actor, Chef, Hotel Employee, Professor
Setting	Living Room, Dining Hall, Balcony, Garden, Doorway, Stable, Kitchen, School, College, Hospital, Garden, Shoot Location, Hotel, Bank, Office, Vegetable stall, Railway station, Police station, Tailoring Shop
Example of a 4-span trial token	It’s 9:00 am now and Mr. Mathew is a tailor at work. His employees and family members will approach him, each with a task for him to complete.
	Carefully attend to each of the tasks!
	You are supposed to help Mr. Mathew order the tasks in the sequence that they are to be completed based on the timeline.
	Give your response when the screen flashes “Help me order”.
	1. Sir, I have to deliver the blouse today. Please get the sewing machine fixed by 12:00 pm.
	2. Mathew, a customer will be coming to collect her dress. Please have it ready by 10:00 am.
	3. Son, there will be no electricity at home this evening. Please buy candles by 7:00 pm.
	4. Mathew, I am in your city for a week. Let’s meet today at 10:30 a.m.
Stimuli length of each activity instruction	15-20 words
Inter stimulus interval	1000 milliseconds
Progression of trials	The task goes across multiple spans starting from span 2 with two activities to be ordered and progressing to span 10 with ten activities to be ordered respectively.
Trials at each span	Each span includes two trials, offering a second chance if the first attempt is unsuccessful.
Task familiarization	The participants are initially provided with task instructions to make them familiar with the nature of the task. Subsequently, a maximum of two practice trials is administered to familiarize participants with the task's intricacies.
Response prompt slide	The concluding response prompt slide for each trial instructs participants with "help me order," indicating their goal of correctly sequencing the activities.
Response acquisition window	60 seconds
Scoring of WM span	The participant's WM span is identified as the highest successfully completed span prior to failing both tokens within a particular span. For example, if a participant accurately arranges activities up to span 4 but fails on both tokens of span 5, their recorded span is span 4.
Scoring of the metacognitive facet: Calculation of the estimation discrepancy score	The estimation discrepancy score is the difference between participant’s predicted or postdicted spans and the actual span of their task performance.
Maximum duration of the task	20-30 minutes

Probe aspect	Results
Comprehensibility of Instructions	Participants unanimously reported that the activity instructions were exceptionally clear and straightforward, with none finding them lengthy or challenging to understand.
Effectiveness of Voice-over Instructions	All participants found the voice-over to be effective. They suggested that maintaining voice-over throughout the entire activity instruction was preferable to intermittent use, as this provided a consistent and less distracting experience, ultimately aiding their comprehension significantly.
Timing of Instructions	Regarding the timing of instructions, the majority of participants (8 out of 10) recommended displaying instructions only at the beginning of each trial. They expressed that repeated instructions within the trial could become repetitive.
Necessity of Clock Display	Regarding the inclusion of a clock on every slide, the feedback was mixed, with 6 out of 10 participants suggesting that it helped them maintain a better sense of time context.
Overall Task Experience	All participants reported an enjoyable experience during the pilot study. They noted that the tasks closely resembled activities encountered in their daily lives, allowing them to relate better. Participants reported minimal levels of subjective anxiety during task completion, with all scoring either 0 or 1 on a scale of 10.

Sociedade Brasileira de Fonoaudiologia Al. Jaú, 684, 7º andar, 01420-002 São Paulo - SP Brasil, Tel./Fax 55 11 - 3873-4211 - São Paulo - SP - Brazil
E-mail: revista@codas.org.br

Acompanhe os números deste periódico no seu leitor de RSS

[1] Conflict of interests:

nothing to declare.