Rasch Analysis and Functional Measurement in Post-Hospital Brain Injury Rehabilitation

Rasch analysis is a statistical technique used in determining statistical properties of functional measures for use in research and treatment. The technique was used in the current study to determine the reliability and validity of the Mayo Portland Adaptability Inventory-Version 4 (MPAI-4) for use with three different acquired brain injury samples. Subjects were 777 adults (each group comprised of 259 individuals) with acquired brain injury treated in one of three rehabilitation program types: Neurorehabilitation (NR), Neurobehavioral (NB), or Supported Living (SL). The MPAI-4 was administered to each participant upon admission to program. Rasch analysis was conducted to assess item fit, reliability, and separation statistics for MPAI-4 assessments conducted within each program. Item difficulty values were examined to determine if the MPAI-4 differentiated among groups based on deficit profiles. The results revealed that for each group, fit statistics fell with appropriate levels (0.5 – 1.5) for at least 24 of 29 items. Rasch person reliability statistics were 0.89 for NR and NB, and 0.90 for SL. Item reliability was 0.99 for each of the groups. Item difficulty values accurately differentiated the three groups based on their specific deficit profiles expected. Specifically, NR participants’ greatest deficits demonstrated by the MPAI-4 were within cognitive and physical functions. For the NB participants, the greater deficits demonstrated were within the behavioral and adjustment items. Supported Living participants had the most limitation within the instrumental activities of daily living items. As in prior research findings, the current Rasch analysis supported the use of the MPAI-4 within this heterogeneous, acquired brain injury population. This unique statistical approach translates to treatment priorities that may assist clinicians with identifying treatment goals specific to unique treatment group characteristics (e.g., NR, NB, and SL).


Introduction
Increasingly Rasch analysis is being applied in the development of functional outcome measures to assess performance following acquired brain injury (ABI) (Malec, Kragness, Evans, Finlay, Kent, & Lezak, 2003;Kean, Malec, Altman, & Swick, 2011;Pretz, Kean, Heinemann, Kozlowski, Bode & Gebhardt, 2016;and Malec & Kean, 2016).Historically, ordinal measures have been used to evaluate outcomes for this population (Grimby, Tennant, & Tesio, 2012).However, the limitations of ordinal measurement for outcome assessment have been well documented (Merbitz, Morris & Grip 1989;Kucukdeveci, Tennant, Grimby & Franchignoni, 2011).While ordinal measures are useful for rank ordering individuals based on possessing more or less of a particular attribute, they do not measure how much more or less.The distance between scaling levels on ordinal measures are not equal, and therefore are not appropriate for calculating mean scores for pre-post or between group comparisons.The need exists for precise interval measurement of ABI outcomes to demonstrate treatment effectiveness, which then improves access to needed services with adequate funding.This need has led to a movement toward the use of Rasch analysis to develop interval level functional outcome measures within rehabilitation care (Malec, Kragness, Evans, et.al, 2003;Pretz, Kean, Heinemann, et.al. 2016).The Rasch technique allows for the calculation of the metric distance between test items, and determines the contribution of each item to defining a level of the construct measured, thus producing an interval level parametric measure (Linacre, 2002).
(RSM) to assess outcomes from post-hospital brain injury rehabilitation.Rasch RSM has been defined as a method of Rasch analysis appropriate for transforming Likert type scales to interval level measures.Rasch RSM has established the reliability of the MPAI-4 for measuring functional outcomes within the heterogeneous ABI population (Malec & Lezak, 2008;Male & Kean, 2016;Horn & Lewis, 2016;Lewis, Horn, & Russell, 2017;Lewis, Horn, & Russell, 2017).
Brain injury rehabilitation includes several program types designed to address different symptoms that emerge along the recovery continuum.It would be of interest to determine if the MPAI-4 demonstrates consistent reliability and construct validity when applied to programs along the continuum.Specifically, Rasch RSM would reveal if results differ with item fit statistics, discriminative power, and item difficulty hierarchies as the MPAI-4 is applied to different programs along the rehabilitation continuum.
To address these questions, the present study applied Rasch RSM analysis to determine the reliability and construct validity of the MPAI-4 as applied to three different settings within the post-hospital continuum of care for persons with acquired brain injury: neurorehabilitation (NR), neurobehavioral intensive (NB), and supported living (SL).Additionally, this study examined whether the MPAI-4 identified differences among the groups in item difficulty, item fit, and person discrimination among the three samples.Such an analysis could potentially guide therapists in identifying treatment priorities within each rehabilitation program.

Method
The MPAI-4 was administered to each participant within 30 days following admission to their program.The results of all evaluations were compiled into a national database and combined with participant demographic data.Rasch analyses were conducted on MPAI-4s taken at the time of admission to program.Winsteps version 3.81 was used to conduct Rasch analyses.

Assessment Instrument
The MPAI -4 consists of 29 items rated from 0 to 4 on a 5-point scale, where 0 represents no limitations and 4 represents a severe problem interfering with activity more than 75% of the time.Raw scores on the 29 items are converted to T-scores within three subscales: Abilities Index (physical, communication, and cognitive skills), Adjustment Index (emotional and behavioral skills), and Participation Index (contextual skill application).T-scores have a mean of 50 and a standard deviation of 10.Higher T-scores indicate greater disability.Used primarily in post-hospital rehabilitation settings, the MPAI-4 has undergone rigorous psychometric testing and has proven reliability and validity as determined through Rasch analysis, Item Cluster, Principle Component Analyses (PCA), and measures of concurrent and predictive validity (Malec J., Kragness M., Evans R. et. al. 2003, Malec & Lezak, 2008).For a detailed description of the 29 MPAI-4 items, Indices, and rating scale see Malec and Lezak 2008.

Participant Characteristics
The three study groups were selected from 2,687 neurologically impaired adults (18 years or older) with consecutive discharges from 68 post-hospital programs in 22 states from 2011 through 2016.The neurorehabilitation group was comprised of 259 individuals that were randomly selected using SPSS version 22 from 1,551 participants who met inclusion criteria such as shorter chronicity of injury (e.g., time from injury to admission falling between 6-12 months), greater impairment with cognitive and physical deficits, and receiving services from a specialized neurorehabilitation program with physical, occupational and speech therapies, along with counseling and medical services.Life skill training was emphasized as an extension of therapy training.Treatment was provided 5 days per week, and application of life skills were emphasized continuously throughout the rehabilitation program.The purpose of this level of care was to teach and promote generalization of functional skills (e.g.safe mobility, problem solving, initiation, compensatory memory strategies development, effective interpersonal communication, self-care, and money management) assisting patients with return to their home resuming productive pre-vocational or vocational activities.
Within the neurobehavioral intense group, 259 individuals were selected based on criteria marked by behavioral disruption.In contrast to persons in the neurorehabilitation and supported living groups, these individuals exhibited disabling behavioral dyscontrol characterized by poor impulse control, explosive outbursts, poor planning and judgment, limited or poor self-awareness, verbal and physical aggression.Treatment was provided within one of 6 programs that met the behavioral needs and included a medical director and nursing, counseling, behavior specialists, physical therapy consultation, occupational therapy, speech therapy, cognitive rehabilitation, case management, and life skills teachers.Persons in this group tended to be injured longer than the typical neurorehabilitation patients, with time between onset of injury to admission often averaging greater than five years (Lewis & Horn, 2015;Horn & Lewis, 2016).
The third group consisted of 259 individuals treated within supported living environments.Supported Living programs do not typically provide active restorative rehabilitation services (e.g., physical, occupational and speech therapies) but emphasize quality of life and gradual life skills application in the community.This program type provides a safe living environment, maintenance of patient's health (e.g., glucose and blood pressure monitoring), and prevention of decline through monitoring daily health needs (e.g., cognitive and physical exercise and challenge).These participants also tend to be chronic, with an onset of injury to admission interval often exceeding 7 years (Lewis & Horn, 2015).Demographic characteristics of the three study groups are presented in Table 1.Tumor/disease 18% 19% 16%

Rasch Rating Scale Model Introduction
The Rasch RSM uses a logistical model of probability to describe the relationship among the person tested, item difficulty, and the probability of each response category in a scale being selected (Bond & Fox, 2007).The model accounts for a response to a specific item in relationship to probability of a specific response to other items on the measure.This logistical model enables the calculation of the metric distance between items, and supports reliable differentiation among persons tested on the measure.The Rasch RSM can be used to transform ordinal Likert-type measures to interval level measurement.As applied to the MPAI-4, Rasch RSM reveals the person item interaction by illustrating that as the severity of disability increases the probability of receiving a "no problem" or "mild problem but not interfering" limitation rating on a given item decreases.As severity level lessens, the probability of passing an item (no problem or mild limitation without interference) increases.ABI may disrupt a few, or many, functions to varying degrees of severity.Therefore, a measure such as the MPAI-4 must have a defined range of items from easy to difficult to reliably measure the magnitude of disability experienced by an individual and to adequately discriminate among individuals along the continuum of disability levels (mild to severe).Rasch RSM possesses a hierarchical structure (item ranking based on probability of endorsing or not endorsing), which makes it uniquely suited for evaluation of item fit when a measure requires both "easy and difficult" items to fully understand the population of interest (Waugh & Chapman, 2005).Factor analysis (including principle component analysis) is also frequently used in test construction and validation (Sick, 2010;and Tormakangas, 2011).Both Rasch RSM and factor analysis identify items that "load on" or describe an underlying construct (Wright, 2009).However, factor analysis is a correlational model that requires association among items for inclusion.As a result, low probability items (items infrequently endorsed) weakly correlated to other test items may be excluded from the model even though they are indicative of the latent trait (Sick, 2011).The Rasch approach assesses fit of both low probability (difficult) and high probability (easy) items.The present study employed Rasch RSM rather than factor analysis to better understand differences in item fit and item hierarchy (easy and difficult items) among the three study groups.

Fit Statistics
Rasch analysis determines the construct validity of an assessment tool by comparing expected from the actual values of the items comprising a measure revealing how accurately items capture the latent construct that they are intended to measure.Rasch fit statistics provide an estimate of the distinct contribution for each item in describing the underlying construct and the extent to which they differentiate among people along the continuum of that construct (Linacre, 2002).
As applied to the MPAI-4, Rasch Infit and Outfit statistics illustrate the fit of each item representing unique contribution to a person's level of disability (latent construct).Fit values that are nearest to 1.0 indicate minimal distortion.Values falling below 1 indicate that persons are answering incorrectly when they are expected to answer correctly (i.e., Guttman error).Low fit values on the MPAI-4 suggest that high levels of limitation are observed when low levels would be expected for that person on those items.Values greater than 1 indicate that there is more noise or random variation on an item than would be expected.Fit values falling between 0.5 and 1.5 are considered productive for measurement use (Linacre, 2002).Items that fall outside those parameters may not reliably represent the latent construct being measured.

Reliability Statistics
Other key statistics provided by Rasch analysis are Person and Item Reliability.In general, reliability refers to the reproducibility of results obtained by a measure.Specifically, Person Reliability indicates how well items comprising a measure distinguish among individuals (e.g.those possessing a lot or a little of the construct measured) while Item Reliability refers to whether test items relate to each other in a consistent way in describing a disparate group of individuals.A coefficient of 0.80 or greater is considered acceptable for Person Reliability, while a coefficient of at least 0.90 is optimal for Item Reliability (Wright, Linacre, & Gustafson, 1994).

Separation Statistics
Separation values indicate "the extent to which items distinguishes among people (Person Separation) and are distinct from each other (Item Separation)" (Malec, Kragness, Evans, et.al. 2003).Specifically, Person Separation values indicate the number of performance levels detected by a measure.For example, a Person Separation index of 2.00 means that two levels of performance can be reliably identified.Item Separation refers to the extent to which items on a test are consistently ranked from least difficult to most difficult.Low Item Separation (< 3.00) implies that the item difficulty hierarchy is not reliable, whereas magnitudes exceeding 3.00 indicate greater consistency of item hierarchy.

Item Difficulty Statistics
The Rasch RSM orders item difficulty by identifying the probability of an item receiving a particular rating along the measurement scale (i.e.no limitation to severe limitation).For example, mean item difficulty is the point at which the highest and lowest categories have an equal probability of being observed (Andrich, 1978).In the case of the MPAI-4, the more difficult items would be those that have a higher probability of a moderate to severe limitation being observed than no limitation.Difficulty measures are presented as logits with two decimal points.

Fit Statistics
Table 2 presents Rasch infit and outfit statistics by program type that fell outside the 1.0 + 0.5 parameter established for acceptable fit.Values for each of these items exceeded 1.5, revealing significant unexplained variation in observations and a tendency for outlier responding.In each group the items "paid work" and "unpaid work" (home making, volunteering, school) presented infit and outfit values greater than 1.5.Values for these items ranged from 1.55 (neurobehavioral infit unpaid work) to 2.83 (neurobehavioral outfit paid work).Audition was the next most unstable item, with outfit statistics greater than 1.5 observed in each program.Overall, the pattern of misfit items was consistent across the groups.The neurobehavioral program had the greatest number of misfit items.
Specifically within the Abilities subscale, the Audition item (rarely endorsed as a limitation) was the only misfit variable.
One item on the Adjustment subscale, Family relationships, fell outside of acceptable fit values in neurorehabilitation and neurobehavioral programs.On the Participation subscale, in addition to paid and unpaid work, the other item with a value greater than 1.5 was transportation (neurobehavioral outfit and supported living infit).Overall, the majority of MPAI-4 items accurately contributed to measuring disability after brain injury for each of the three program types.

Reliability
Rasch person reliability coefficients for the three rehabilitation groups were 0.89, 0.89, and 0.90 respectively for the neurorehabilitation, neurobehavioral, and supported living programs.These reliability coefficients exceeded the acceptable standard of 0.80 for person reliability and indicated the MPAI-4 effectively discriminated among those along the disability continuum.
Item reliability coefficients were 0.99 for each of the three groups.These were considered strong reliability coefficients.Therefore, the results suggested that within each of the rehabilitation groups, the items across the disability continuum were stable.There was a high level of agreement in identifying easy through difficult items.

Separation
Rasch person separation values were 2.91 (Neurorehabilitation), 2.85 (Neurobehavioral), and 2.94 (Supported Living).These values indicate that the MPAI-4 revealed three performance strata (e.g.low, medium, and high disability) within each of the program types.The item separation values for each of the programs exceeded 3.00 indicating a consistent item hierarchy from least difficult to most difficult.Specific item separation values were 9.16 (Neurorehabilitation), 10.11 (Neurobehavioral), and 9.61 (Supported Living).Based on the results, there was sufficient item and person separation for purposes of the analysis.

Item Difficulty
Figures 1 through 3 illustrate the item difficulty comparisons among the three program types by MPAI-4 subscales (Abilities, Adjustment, and Participation).Difficulty values of zero indicate that there is an equal probability of observing high or low levels of disability.Values below zero indicate an increased likelihood of observing greater magnitude of disability, while values above zero suggest a lesser magnitude of disability.Figure 1 shows the magnitude of disability within the Abilities subscale items.The neurorehabilitation group presented the greatest level of disability.Specifically, this group demonstrated the greatest magnitude of disability on 8 of 11 items, followed by the SL and NB groups.All three groups showed the greatest disability with the Novel Problem Solving and Attention/Concentration items.Figure 2 illustrates the magnitude of difficulty values for the Adjustment subscale items.As would be expected, participants in the NB programs showed the greatest level of disability on these items.Of particular interest, the three MPAI-4 items which would most characterize a neurobehavioral participant (Irritability, Impaired Awareness, and Inappropriate Social Behavior) were observed as having the greatest disability among these study participants, and were rated as more difficult than for NR and SL participants.Among the Adjustment items, the Impaired Awareness item was also of the highest magnitude of disability for the other two groups in this analysis.The impaired awareness item having a high magnitude of disability in all three groups supports the notion that awareness is a short and long-term impairment following acquired brain injury.Figure 3 presents the magnitude of difficulty values for Participation (i.e., application of skills) subscale items.Clearly Participation items were the most difficult for participants in all three groups as evidenced by the number of items with difficulty values below zero.Difficulty values for those in the NR group were zero or lower on all six items.The Residential (Home skill) and Money Management items were the most difficult for this group.Similarly, item difficulty values for participants in NB were all below zero with exception of self-care.The Money Management and Residential (home skill) items were the most difficult for this group (NB).Similar values were observed for the SL participants.Overall, limitations measured by the MPAI-4 in these application skills, which are needed to live independently, validate placement in residential programs providing various levels of 24 hour support and assistance.

Discussion
The need exists for precise interval measurement of acquired brain injury outcomes to demonstrate treatment effectiveness.When adequate measurement is used, then inferences can be made regarding efficacy with model discovery, which then improves access to needed services with adequate funding.The Mayo Portland Adaptability Inventory-4 has been accepted as a precise measure of treatment outcome in the post-hospital level of care due to treatment efficacy studies and Rasch analysis scrutiny.The Rasch technique allows for the calculation of the metric distance between test items, and determines the contribution of each item to defining a level of the construct measured, thus producing an interval level parametric measure (Linacre, 2002).The focus of this article demonstrated the use and application of Rasch analysis using the Mayo Portland Adaptability Inventory -4 within a post-hospital acquired brain injury population.
Brain injury rehabilitation is complicated where one size does not fit all needs.In fact, several program subtypes have been identified based on differing symptoms that emerge along the recovery continuum.The second focus of this research study was to determine if the MPAI-4 demonstrates consistent reliability and construct validity when applied to programs along the continuum.Specifically, our focus was to determine if Rasch RSM would reveal differing results through item fit statistics, discriminative power, and item difficulty hierarchies as the MPAI-4 is applied to different programs along the rehabilitation continuum.The current research was able to show the overlapping of items that contribute to neurological recovery generally.In addition, the current findings revealed that through item fit, each program level was in fact unique by item response.The NR group showed the greatest magnitude of disability with physical and cognitive items which is consistent with prior clinical research describing specific physical and cognitive deficits.The NB group showed the greatest magnitude of disability with items measuring irritability, self-awareness, and social control.Again, the NB group findings were consistent with previous research on neurobehavioral consequences of acquired brain injury.Finally, the SL group showed less magnitude of disability overall then the other two groups with specific items.Although no particular pattern was clear, no item profile was expected since this group is similar to a clinical control group.This group has the underlying construct of acquired brain injury, but because of the chronicity the magnitude of any one item would expected to be less than the other two groups.The magnitude difference for the SL group was demonstrated by the Rasch RSM technique.
Evidence-based treatment is essential in current healthcare.The current findings revealed that for all three samples measured, the Rasch analysis provided unique information about each level of care.The use of this model in translating the mathematic calculation from conversion of Likert-scale data to interval data for purposes of rehabilitation modeling has been further advanced.In addition, the data derived in this study provides an analysis that can be used for treatment planning and implementation purposes.

Figure 1 .
Figure 1.MPAI-4 Abilities: Item difficulty by program type.Note: this chart provides a comparison of the three programs on items that include physical, cognitive, and communication skills.SL = Supported Living, NB = Neurobehavioral, NR = Neurorehabilitation.

Figure 2 .
Figure 2. MPAI-4 Adjustment Subscale: Item difficulty by program type.Note: this chart provides a comparison of the three programs on items that include mood regulation, irritability, social awareness, and symptom management.SL = Supported Living, NB = Neurobehavioral, NR = Neurorehabilitation.

Figure 3 .
Figure 3. MPAI-4 Participation Subscale: Item difficulty by program type.Note: the Participation Subscale measures Instrumental Activities of Daily Living items.These items are the final common pathway to rehabilitation outcomes for each sample group.SL = Supported Living, NB = Neurobehavioral, NR = Neurorehabilitation.

Table 2 .
MPAI-4 items with infit and outfit values by program type outside acceptable parameters.