• MH/DD/SAS Home

• About UsAcronyms
• A-Z Topics
• Contacts

 

NC SNAP (NC Support Needs Assessment Profile)

History and Development of NC SNAP

The NC-SNAP assessment tool was developed over a 2½ -year period and underwent field-tests on two separate occasions. The initial field-test on the NC-SNAP was conducted in 1997 and compared the NC-SNAP with the Inventory for Client and Agency Planning (ICAP)¹ and the Developmental Disabilities Profile (DDP)². The goal of this research was to determine which of the three instruments most accurately assess level of intensity of support needs for persons served by the North Carolina developmental disabilities service system. The results of the 1997 field-test indicated that the NC-SNAP and the ICAP (with scores combined to yield five levels) had very similar predictive validity; however, neither of these two instruments in their current form was deemed satisfactory in predicting the needs of persons with developmental disabilities for the purposes defined by the North Carolina Policies Work Group. In 1998, the NC-SNAP was revised in an effort to maximize its effectiveness to predict the needs of persons with developmental disabilities. In 1999, a second field-test was conducted on the NC-SNAP to ensure that its reliability and validity were sufficient for the instrument to be confidently used as a statewide assessment tool.

A brief overview of the design and pertinent data gathered from both the 1997 and the 1999 field-tests are presented in the following paragraphs. A more extensive report is in preparation for publication. Additionally, extensive data are being collected on the NC-SNAP through the ongoing look-behind quality assurance effort.

1997 Field-test Design

The NC-SNAP researchers hypothesized that in order to test an assessment instrument's ability to predict the level of intensity of support needs, the assessment tool should predict the current support levels of those individuals who receive good or ideal supports. Therefore, persons who were currently receiving good or ideal supports were selected for participation in a field-test of the instruments. The level of support intensity participants were receiving were identified and categorized. The initial field-test required the researchers to:

1. Find individuals with developmental disabilities who were well served;
2. Determine the participating individual's current support array level; and
3. Administer the three assessment instruments.

Finding individuals who were well served

In order to determine if a person was well served by current DD supports, a five-level survey was administered to the individual or his or her guardian, his or her case manager, and his or her service provider. Individuals were only identified for participation in the study when all three sources agreed that the individual was receiving either good (better than adequate) or ideal services. Initially, 2,332 people receiving services from five area programs in North Carolina were identified for participation in the study. After the survey was conducted, 553 people (24 percent) were selected as participants for the 1997 study.

Determining Current Support Array Levels

To determine a participant's current support level, case managers were asked to
identify the support array received by each person for whom they had responsibility. A participant's support array identified both residential and other types of supports the participant was receiving. Using the support array identified by the individual's case manager, each participant was independently assigned to one of five support array levels by three separate raters. Raters achieved an agreement level of 98% with regard to the assigned support array levels.

Administering the Assessment Instruments

Case managers and Qualified Mental Retardation Professionals (QMRPs) in each of the five participating area programs and at three mental retardation centers were trained to administer all three of the need assessment instruments. Case managers and QMRPs were then asked to complete in random order the three need assessment tools on participants on their caseload. As an additional control procedure, an author or a research assistant interviewed a second person familiar with the participant to complete an inter-rater reliability assessment for 10% of the participant population.

Results

The researchers were interested in the three assessment instruments' abilities to
accurately predict an individual's need level as assigned by the identified support array (predictive validity) and on the ability of different raters to administer the assessment tool and get the same results (inter-rater agreement). The results of the 1997 field-test are given for the NC-SNAP only.

 

• Predictive Validity: How well does the NC-SNAP predict the need level
  determined by the support arrays of persons receiving "good" or "ideal" supports?
  • Percent exact match (between assessment results and identified support array level): 30.4%
  • Percent match within one level: 68.7%
• Inter-rater Agreement: How well did different raters achieve the same need
  level when administering the NC-SNAP?
  • Percent exact match inter-rater agreement: 70.7%
    
  • Percent inter-rater agreement within one level: 98.3%
    
• Other Facts of Interest
  • The mean duration time to administer the NC-SNAP was 15 minutes with a range of 2 to 45 minutes.
    
  • The NC-SNAP had a higher predictive value when used to assess
    individuals with high needs.
  • The NC-SNAP tended to overestimate need.

Design of 1999 Field-test

After the 1997 field-test, the NC-SNAP was judged to be approximately equal across all variables in effectiveness to the next best alternative instrument. The authors of the NC-SNAP were asked by North Carolina's Developmental Disability Policy Advisory Work Group to conduct a comprehensive analysis of the instrument using the field-test data and to make revisions to the instrument with the goal of maximizing the predictive validity and inter-rater agreement of the NC-SNAP. In brief, the NC-SNAP was modified by identifying items associated with errors in predictive validity, and then eliminating or modifying those items to enhance accuracy.

After careful analysis and modification, the re-tooled NC-SNAP was field-tested in 1999 using a stratified sample of 97 participants served by one area program. The design of this second field-test was almost identical to the 1997 field-test except an additional analysis was conducted to identify errors in support array level determination. Results were analyzed based on both the original assigned support array and on a "corrected" support array. That is, the support array was corrected if additional information was obtained indicating that the original support array had been determined using incomplete or erroneous information, or if a change in the individual's status had occurred since the support array was originally determined.

Results

The researchers were interested in the NC-SNAP's ability to accurately predict an individual's need level as indicated by the identified support array level (predictive validity). Results were evaluated for the NC-SNAP's overall scores, at each need level, and when the NC-SNAP was used specifically to assess children from birth to sixteen years. Additionally, the researchers added an element to the 1999 study to measure improvements in predictive validity that could be displayed by correcting inaccuracies in identified support arrays.

Predictive Validity of the NC-SNAP for overall scores and for each need level, for both corrected and uncorrected support arrays, is shown in the table on the following page.

 

The predictive validity of the NC-SNAP when used to assess children from birth to sixteen years, for corrected and uncorrected support arrays, is given in the table below.

 

To assess the inter-rater reliability of the NC-SNAP, the same 101 individuals were re-administered an NC-SNAP by another certified examiner who was unaware of the previous NC-SNAP results. The results of this analysis are shown in the following table.

 

Summary

At the conclusion of the 1999 field-test, the reliability and validity data collected on the revised NC-SNAP indicated that the NC-SNAP predicted the level of need on the participant population at rates considered very good for instruments assessing acuity of need. Based on this data, the North Carolina Policies Work Group determined that, when administered properly, the NC-SNAP is a reliable, valid, and easy-to administer measurement of individual need.

The NC-SNAP was officially adopted by the state of North Carolina in 1999 as the preferred tool for determining an individual's intensity of need for Development Disabilities services and supports.

 

¹ Bruininks, R. H., Hill, B., Weatherman, R. E., & Woodcock, R. (1986). Inventory For Client and Agency Planning. Chicago, IL: Riverside Publishing Co.
² Developmental Disabilities Profile, Albany, New York State Office of Mental Retardation and Developmental Disabilities.

 

Return to top