Welcome to the Schizophrenia Resource Centre

Welcome, this website is intended for healthcare professionals in EMEA with an interest in the treatment of schizophrenia. By clicking the link below you are declaring and confirming that you are a healthcare professional

You are here

Psychosis-like experiences and distress among adolescents using mental health services

Schizophrenia Research, 2-3, 152, pages 498 - 502

Abstract

Although 'psychosis-like experiences' (PLEs) may reflect elevated risk for onset of serious mental illness, many individuals reporting PLEs are not truly at risk for developing clinical psychosis. Interview-based instruments that define and diagnose “clinical high risk” status attempt to distinguish between normative PLEs and attenuated symptoms indicating progression toward psychosis by probing whether such experiences create clinically relevant concerns. Two recently developed self-report measures, the Prodromal Questionnaire—Brief and the Prodromal Questionnaire-16, contain a ‘distress scale’ that helps assessors to gauge distress within a screening format. The aim of the current study is to examine the association of PLEs with distress within a sample of young people seeking mental health care and to investigate the usefulness of the distress scale in differentiating between participants who do and do not meet standardized criteria for a clinical high-risk syndrome. Sixty-six adolescents and young adults receiving mental health services completed the Prodromal Questionnaire—Brief and the Structured Interview for Psychosis Risk Syndromes. The screener was scored in ways that emphasized varying interpretations of respondents' distress ratings. Within this sample, focusing only on PLEs associated with distress yielded improved prediction of clinical high-risk status, and participants meeting high-risk clinical criteria were found to report more distress per PLE relative to participants with other psychiatric disorders. Findings suggest that including a distress scale within a screener aids in identifying a group more likely to meet clinical high-risk criteria. Further, PLEs that respondents describe as neutral or positive do not appear to be relevant for clinical high-risk screening.

Abbreviations: PLE - psychosis-like experience, CHR - clinical high risk, PQ - Prodromal Questionnaire, SIPS - Structured Interview for Psychosis Risk Syndromes, ROC - receiver operating characteristic, PPV - positive predictive value, NPV - negative predictive value.

Keywords: Psychosis-like experience, Attenuated symptom, Clinical high risk, Assessment, Distress, Screening.

1. Introduction

An emphasis on ‘attenuated’ psychosis symptoms has yielded some success with regard to predicting psychotic disorder development in samples of interest (Cannon et al, 2008 and Fusar-Poli et al, 2012). A closely related construct, ‘psychosis-like experiences’ (PLEs; e.g., perceptual anomalies, unusual beliefs, distorted thinking) may also reflect elevated risk for onset of serious mental illness, however, many individuals reporting PLEs are not truly at risk for developing clinical psychosis (van Os et al, 2000 and Kelleher et al, 2012). PLEs are common in the general population (7–8%; Shevlin et al, 2007, van Os et al, 2009, and Gale et al, 2011) and are thought to represent the relatively normal end of the “psychosis continuum” (Strauss, 1969 and Kwapil et al, 1999). For some, PLEs may represent culturally sanctioned religious beliefs, superstitions, or imaginative experiences that are normative within the context of culture, life stress, and/or developmental norms. Although PLEs have shown some association with distress and functional impairments in general population samples ( Armando et al., 2010 ), PLEs are not automatically assumed to have a negative clinical impact; some experiences, such as feeling a divine presence or being “watched over” by loved ones may be neutral or even positive for individuals reporting such experiences.

Despite considerable construct overlap, interview-based instruments that define and diagnose clinical high-risk (CHR) status attempt to distinguish between normative PLEs and attenuated symptoms indicating progression toward psychosis by probing whether such experiences create clinically relevant concerns. For example, the Structured Interview for Psychosis Risk Syndromes (SIPS; Miller et al., 2003 ) emphasizes that symptoms in the ‘high-risk’ severity range will typically cause distress and disruptions to daily life ( McGlashan et al., 2010 ). Alongside patients' level of insight, the intensity and frequency of symptoms, and whether respondents report altering behavior in response to symptoms, distress constitutes one of multiple dimensions considered important within the SIPS for determining whether symptoms are likely forerunners of more serious illness.

Despite interview-based assessments' emphasis on distress, self-report or screening measures aiming to assess psychosis risk status typically focus on whether or not respondents have experienced PLEs, rather than attempting to gauge how stressful respondents find such experiences (e.g., Heinimaa et al, 2003, Miller et al, 2004, Ord et al, 2004, and Loewy et al, 2005). As such, most self-report measures do not possess a mechanism to differentiate innocuous PLEs from potentially pathological attenuated symptoms. The Community Assessment of Psychic Experiences (CAPE) is one measure that attempts to distinguish distressing from non-distressing PLEs. Researchers administering the CAPE to both psychiatric and general population samples found that patient groups indicated more PLEs as well as greater distress associated with these experiences, however, the CAPE was not explicitly designed to identify individuals at CHR ( Hanssen et al., 2003 ). Two CHR-specific self-report measures, the Prodromal Questionnaire—Brief Version (PQ-B; Loewy et al., 2011 ) and the Prodromal Questionnaire-16 (PQ-16; Ising et al., 2012 ), include ‘distress scales’ on which participants are prompted to report the level of subjective distress associated with each positively endorsed PLE. Within validation samples, the PQ-B and PQ-16 appear to be somewhat effective for selecting an enriched group with relatively high likelihood of meeting interview-based CHR criteria (Loewy et al, 2011, Ising et al, 2012, Jarrett et al, 2012, and Kline et al, 2012), and preliminary evidence from a single study suggests that the use of the distress scale adds incremental value toward CHR status prediction beyond the “yes/no” form of the original items ( Loewy et al., 2011 ).

Despite the emphasis on distress in most conceptualizations of the clinical high-risk syndrome, the use of a “distress scale” within screening measures has received little attention in and of itself. Further, the question of whether screening for non-distressing PLEs helps to predict CHR status remains unresolved. The aim of the current study is to examine the association of PLEs with distress within a sample of young people seeking mental health care and to investigate the usefulness of the distress scale in differentiating between participants who do and do not meet SIPS-based criteria for a clinical high-risk syndrome. We hypothesize that consideration of the distress scale will add incremental validity to self-report screening when predicting CHR status, and that individuals who meet interview-based criteria for a psychosis risk syndrome will rate their PLE distress as higher relative to participants with other psychiatric disorders.

2. Methods

2.1. Procedure

Data collection took place through the Youth FIRST research program at the University of Maryland, Baltimore County (UMBC) and the University of Maryland, School of Medicine. All research procedures were approved by both institutions' Institutional Review Boards. Participants were recruited through community clinics and were eligible to participate if they were between ages 12–22, receiving mental health services, and (for minors) had a stable guardian to provide consent. The majority of participants received referrals to the study from community mental health providers who noted concerns about possible ‘prodromal’ or psychotic-like symptoms. Referrals came from a university child and adolescent psychiatry clinic, school-based clinicians, a child psychiatric inpatient unit, and multiple private practice offices in the community. No other preliminary screening procedures were used prior to the first study visit. After providing informed consent/assent, participants completed the PQ-B and completed a SIPS interview with study staff.

2.2. Materials

2.2.1. Prodromal Questionnaire—Brief version (PQ-B; Loewy et al., 2011 )

The PQ-B is a 21-item self-report questionnaire that evaluates the presence of PLEs as well as the distress associated with specific experiences. Participants respond “yes” or “no” to items asking whether they have experienced specific PLEs. For “yes” responses, participants indicate agreement with the following statement on a 5-point Likert scale: “when this happens, I feel frightened, concerned, or it causes problems for me.” Although the phrasing of this follow-up prompt instructs respondents to consider either distress and/or impairment, the PQ-B authors refer to these ratings as a “distress scale.” Distress ratings are scored from one (strongly disagree) to five (strongly agree).

Within the current study, PQ-B screening totals were calculated several ways. “PQ-raw” scores indicating number of PLEs endorsed were summed by totaling the number of “yes” responses. “PQ-total” scores were summed by totaling the Likert scale distress ratings for each endorsed experience. “PQ-distress” scores were tabulated by counting the number of items for which a participant endorsed distress and/or impairment (i.e., indicated “agree” or “strongly agree” on the distress scale). Thus for PQ-distress totals, items that were endorsed but not rated as causing distress were not counted toward participants' total scores.

Participants' “average distress” was calculated by dividing each participant's PQ-total by his or her PQ-raw score, to provide a measure of each participant's average degree of distress per item endorsed. This variable had a potential range of zero (for participants endorsing no PLEs) to five (for participants who marked “strongly agree” on the distress scale for every endorsed PLE).

2.2.2. Structured Interview for Psychosis-Risk Syndromes (SIPS; Miller et al, 2003 and McGlashan et al, 2010)

The SIPS is a semi-structured interview and is the most widely used assessment instrument for evaluation clinical high-risk states. The SIPS scales include a total of nineteen symptom constructs (5 positive, 6 negative, 4 disorganized and 4 general) that are evaluated based on the presence, duration, and severity of specific experiences and behaviors. Each item is rated on a scale of 0 (symptom is absent) to 6 (extreme or psychotic symptom intensity). The positive symptom section of the SIPS emphasizes clinically relevant positive symptoms, which may resemble PLEs, but must also cause impairment or distress in order to count toward a psychosis-risk syndrome diagnosis. The SIPS contains diagnostic criteria for three “psychosis risk syndromes,” schizotypal personality disorder (SPD), and psychosis.

Staff were trained to administer the SIPS by attending a two day training with the SIPS authors at which they were ‘certified’ to use the instrument by achieving 90% agreement with gold-standard scores on practice cases. Those who could not attend the training trained by reading vignettes provided by the SIPS authors, rating taped interviews, observing two or more interviews, and leading at least two interviews while being observed by an experienced interviewer. New interviewers were considered reliable once their ratings and diagnoses matched those of the observing interviewer over at least two cases. Cases were reviewed weekly within team meetings with study investigators to increase agreement on ratings and diagnoses (see Kline et al., 2012 ). Current team symptom reliability is ICC = .76, and agreement for diagnosis is perfect (kappa = 1.0).

Within the current study, the positive symptoms subscale of the SIPS (PSOPS) was used as a continuous measure of psychotic symptom severity. Diagnostic groups were dichotomized based on SIPS diagnoses, with any psychosis-risk syndrome diagnosis representing a positive case. Consistent with the North American Prodromal Longitudinal Study, participants under 18 meeting criteria for SPD were counted as positive cases. Participants identified as having a psychotic disorder at the time of assessment were excluded from the current sample.

2.3. Participants

The current sample includes 66 youth and young adults with a mean age of 16.75 (SD = 3.08) years. Sixty-eight percent of the sample was female (n = 45) and the racial composition of the sample was 3% American Indian, 1% Asian, 39% African American, 43% Caucasian, and 14% multiracial or “other.”

Twenty-six participants (39%) were classified as positive cases with regard to SIPS interview diagnosis. Twenty participants met SIPS criteria for Attenuated Positive Symptom Psychosis Risk Syndrome (APS), one met criteria for Brief Intermittent Psychotic Symptom Psychosis Risk Syndrome (BIPS), one met criteria for Genetic Risk and Deterioration Syndrome (GRD), three participants under 18 years met criteria for SPD, and two met for more than one risk category (APS plus BIPS; APS plus GRD respectively).

2.4. Analyses

Data were screened for normality and outliers. Given a relatively normal pattern of score distributions, Pearson correlations were used to examine the association between the various iterations of the PQ-B and SIPS interview results. Specifically, correlations were used to examine associations among the PQ-raw, PQ-total, PQ-distress screening scores; average distress scores; and SIPS results (PSOPS symptom ratings and dichotomized CHR status). In order to evaluate the incremental benefit of including distress ratings in screener totals, the three screening totals were compared and tested inferentially using a modification of Fisher's z for dependent samples (Dunn and Clark, 1969, Steiger, 1980, and Meng et al, 1992).

Screening totals were entered into a ROC curve analysis predicting CHR status to determine area under the curve (AUC) obtained using each scoring strategy. A statistically “optimized” score threshold for each scoring method was calculated by determining the point on each curve with the minimum distance to the point (0,1), indicating maximum balance of sensitivity and specificity for that measure ( Kumar and Indrayan, 2011 ). Using optimized score thresholds, we calculated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and overall accuracy (percentage of accurately predicted cases) for each scoring strategy.

To investigate the relation between number of PLE endorsements and “average” distress per item endorsed among CHR and non-CHR participants, a regression model was run predicting average distress from participants' CHR status, their PQ-raw totals, and an interaction term representing moderating effects of CHR status on the association of PLEs with distress. To aid interpretation of regression results, PQ-raw scores were centered (i.e., adjusted to achieve a mean of zero) prior to calculating the interaction term and entering predictor variables in the regression.

Missing data accounting for 0.03% of cell values, and was ignored (treated as zeros).

3. Results

The range and distributions of the three iterations of the PQ-B, as well as average distress ratings, are reported in Table 1 .

Table 1 Descriptive statistics for self-report sums (N = 66).

Measure Range Mean (SD) Skewness Kurtosis
Possible Observed
PQ-raw 0–21 0–21 8.67 (6.57) 0.41 − 1.09
PQ-total 0–105 0–81 27.58 (25.09) 0.69 − 0.88
PQ-distress 0–21 0–17 3.83 (4.65) 1.14 0.30
Average distress 0–5 0–4.76 2.73 (1.21) − 0.58 − 0.28

Pearson correlations indicated that all of the three scoring methods were significantly associated with both SIPS-determined symptom severity and psychosis risk status. Correlations between the three screening sums and each interview-based variable (PSOPS and CHR status) were compared inferentially using a modified Fisher's z test for dependent samples to examine whether differences in correlation magnitudes were statistically significant. Results indicated that PQ-total and PQ-distress sums (i.e., scoring methods that account for participants' distress) yielded significantly higher correlations with SIPS results (PSOPS and CHR status) than PQ-raw scores (Table 2 and Table 3).

Table 2 Correlations between screener sums and clinician positive symptom ratings (N = 66).

PSOPS
PQ-raw .45a
PQ-total .54b
PQ-distress .60b

All correlations significant at p < 0.01.

Correlations that share a superscript do not differ in R to Z test of correlation magnitude.

Table 3 Correlations between self-report sums and CHR status (N = 66).

CHR status
PQ-raw .35a
PQ-total .45b
PQ-distress .53b

All correlations significant at p < 0.01.

Correlations that share a superscript do not differ in R to Z test of correlation magnitude.

ROC curve analyses indicated that all three scoring methods yielded significant AUC results; that is, each of the three PQ iterations predicted SIPS status better than chance. The within-sample optimized screening thresholds ranged from 4 to 18 for the various scoring strategies considered. The “PQ-distress” scoring methods yielded the highest AUC and highest overall screening accuracy relative to the other methods of PQ-B scoring ( Table 4 ).

Table 4 Prediction of CHR status using dichotomized screen results (N = 66).

  AUC (SE) Screening threshold Sensitivity Specificity PPV NPV Accuracy
PQ-raw .71 lowast (0.07) ≥ 9 0.69 0.70 0.60 0.78 70%
PQ-total .75 lowast (0.07) ≥ 18 0.77 0.68 0.61 0.82 72%
PQ-distress .79 lowast (0.06) ≥ 4 0.73 0.83 0.73 0.83 79%

lowast p < 0.01.

To examine the relation between quantity of PLEs endorsed, SIPS-determined risk status, and average distress regarding PLEs, a linear regression was conducted predicting participants' average distress rating from their PQ-raw scores (i.e., number of PLEs endorsed), dichotomized SIPS diagnoses, and an interaction term representing a moderation effect of risk status on distress. The overall model was significant (Adj. R2 (3,66) = .28, F = 9.60, p < .001). The interaction term was also significant, suggesting that psychosis risk status moderates the relation between PLEs and distress. For participants meeting SIPS criteria for a high-risk diagnosis, a greater number of PLEs predicted greater average distress. For low-risk participants, the effect of PLEs on distress was non-significant ( Table 5 ; Fig. 1 ).

Table 5 Regression model predicting average distress (N = 66).

  Effects on average distress
B (standard error) β t p
Intercept        
 For low-risk 2.53 (0.17) N/A 15.01 < 0.001
 For high-risk 2.80 (0.22) N/A 12.70 < 0.001
SIPS diagnosis 0.27 (0.28) 0.11 0.99 0.33
PQ-raw ∗ SIPS diagnosis (interaction) 0.09 (0.04) N/A 2.11 0.04
PQ-raw        
 For low-risk 0.05 (0.03) 0.26 1.82 0.07
 For high-risk 0.14 (0.03) 0.75 4.14 < 0.001
gr1

Fig. 1 Risk status moderates effect of PLEs on predicted average distress.

4. Discussion

Consistent with this study's primary hypothesis, incorporating the PQ-B's “distress scale” when scoring the measure yielded more accurate prediction of SIPS-determined clinical high-risk status. By scoring the PQ-B several ways, we were able to examine the benefits of a distress scale in detail. From a correlational standpoint, any incorporation of the distress scale embedded within the PQ-B yielded significantly larger correlations with SIPS symptom ratings and CHR status relative to “raw” scores not incorporating distress ratings.

The current study experimented with two strategies for scoring the information captured by the distress scale. The “PQ-total” score, which reflects the PQ-B authors' intended scoring, rates the distress associated with each endorsed PLE using a scale ranging from “strongly disagree” to “strongly agree.” Thus, a PLE that the respondent interprets as highly distressing yields a higher item score than a PLE interpreted to be neutral or positive; these, in turn, yield a higher item score than a PLE not endorsed by the respondent. This scoring method implicitly assumes that non-distressing PLEs are still useful for screening and potentially clinically relevant. Alternatively, the “PQ-distress” scoring used in the current study dichotomizes each endorsed PLE as either distressing or non-distressing. Within this scoring system, non-distressing PLEs are counted as non-events, i.e., equivalent to a non-endorsement. Only PLEs that cause distress are counted toward the sum score. This method of scoring more closely approximates the PQ-16 ( Ising et al., 2012 ), which asks participants to rate distress using distress anchors “no/mild/moderate/severe,” and does not count items that cause no distress toward the total score.

Among these scoring strategies, the “PQ-distress” scoring method yielded the highest AUC and overall accuracy values with regard to predicting participants' SIPS-determined CHR status. The implication of this finding is that screening for psychosis-risk status is more efficient when assessors focus solely on PLEs that cause participants distress (i.e., attenuated symptoms) and ignore PLEs that are not described as stressful. This view is consistent with previous findings that PLEs are common, especially in younger populations, and that these experiences in and of themselves are not necessarily indicative of current or future psychopathology. When interpreting screening responses, further consideration of only PLEs that cause distress might help to avoid unnecessary stigmatization or pathological interpretations of normative experiences.

The “average” distress rating for a given PLE within this sample was 2.73 (SD = 1.21), which corresponds to a rating between “disagree” and “neutral” with regard to whether participants find a given experience to be distressing. This suggests that many participants endorsed items that they did not perceive to be of particular concern. Although the SIPS emphasizes clinical impairment associated with psychotic-like experiences, it is noteworthy that only a minority of PLEs endorsed by respondents on the screener were associated with distress.

Results from the regression model affirm the function of the screener's distress scale in helping to differentiate CHR from non-CHR participants. Although there appears to be an overall positive association regarding the number of PLEs endorsed and the “average” (per-item) degree of distress described by participants within this help-seeking sample, the interaction between CHR status and number of PLEs endorsed highlights the usefulness of the distress construct for distinguishing CHR participants from other psychiatric consumers. For CHR participants, a larger number of psychotic-like experiences predicts greater distress associated with each experience. For participants failing to meet SIPS high-risk criteria, the relation between number of PLEs and average distress ratings is less clear. Because SIPS criteria emphasize distress, impairment, and changes in behavior within its symptom severity anchors, this finding is not surprising. The significant interaction term highlights a multi-method convergence of interview results with self-report ratings, and serves to further validate the function of the screener's distress scale.

4.1. Limitations

The sample analyzed in the current study was relatively small (N = 66) and represents a group primarily referred to the study for evaluation of suspected attenuated symptoms. Although this is a relevant and important group, results may not generalize to unselected help-seeking or general population samples. Optimal screening thresholds may be unstable in small samples and should be more widely studied to lend confidence to clinical cut-offs. Additionally, the small sample may have led to type-II error in the regression model, leading to uncertainty in the interpretation of group moderation effects.

Future research might focus on whether PLE-related distress helps to predict psychiatric outcomes over time, particularly transitions to diagnosable psychotic disorders. Identifying moderators of self-report accuracy – for example, respondents' degree of insight or beliefs about stigma – would also be a useful next step. The phrasing of the PQ-B's “distress” prompt (“when this happens, I feel frightened, concerned, or it causes problems for me”) confounds the constructs of PLE-associated distress and impairment; future research efforts might attempt to disentangle these constructs to achieve a more precise understanding of respondents' experiences. Findings from the current study support the use of a distress scale within screening measures, and raise questions regarding the relevance of non-distressing PLEs for predicting SIPS-based diagnoses.

Role of the funding source

This work was supported in part by funding from the Maryland Department of Health and Mental Hygiene, Mental Hygiene Administration through the Center for Excellence on Early Intervention for Serious Mental Illness (OPASS# 14-13717G/M00B4400241) and the 1915(c) Home and Community-Based Waiver Program Management, Workforce Development and Evaluation (OPASS# 13-10954G/M00B3400369); Baltimore Mental Health Systems; a Research Seed Funding Initiative (RSFI) grant from the University of Maryland, Baltimore County; the Passano Foundation; and the Johns Hopkins Center for Mental Health in Pediatric Primary Care. The funders were not involved in study design, analyses, manuscript preparation, or decision to submit for publication.

Contributors

Dr. Schiffman oversaw the study design, data analysis, data interpretation and manuscript preparation. Ms. Thompson and Ms. Kline contributed to data collection, data analyses, and manuscript preparation. Ms. Bussell oversaw protocol implementation and contributed to study design. Dr. Pitts served as statistical consultant. Dr. Reeves oversaw protocol development and implementation.

Conflict of interest

The authors have no actual or potential conflicts of interest to report.

Acknowledgments

None.

References

  • Armando et al., 2010 M. Armando, B. Nelson, A.R. Yung, M. Ross, M. Birchwood, P. Girardi, et al. Psychotic-like experiences and correlation with distress and depressive symptoms in a community sample of adolescents and young adults. Schizophr. Res.. 2010;119(1):258-265 Crossref
  • Cannon et al., 2008 T.D. Cannon, K. Cadenhead, B. Cornblatt, S.W. Woods, J. Addington, E. Walker, et al. Prediction of psychosis in youth at high clinical risk: a multisite longitudinal study in North America. JAMA Psychiatry. 2008;65(1):28 Crossref
  • Dunn and Clark, 1969 O.J. Dunn, V. Clark. Correlation coefficients measured on the same individuals. J. Am. Stat. Assoc.. 1969;64(325):366-377 Crossref
  • Fusar-Poli et al., 2012 P. Fusar-Poli, I. Bonoldi, A.R. Yung, S. Borgwardt, M.J. Kempton, L. Valmaggia, et al. Predicting psychosis: meta-analysis of transition outcomes in individuals at high clinical risk. JAMA Psychiatry. 2012;69(3):220 Crossref
  • Gale et al., 2011 C.K. Gale, J.E. Wells, M.A. McGee, O. Browne. A latent class analysis of psychosis‐like experiences in the New Zealand Mental Health Survey. Acta Psychiatr. Scand.. 2011;124(3):205-213 Crossref
  • Hanssen et al., 2003 M. Hanssen, F. Peeters, L. Krabbendam, S. Radstake, H. Verdoux, J. Van Os. How psychotic are individuals with non-psychotic disorders?. Soc. Psychiatry Psychiatr. Epidemiol.. 2003;38(3):149-154 Crossref
  • Heinimaa et al., 2003 M. Heinimaa, R.K.R. Salokangas, T. Ristkari, M. Plathin, J. Huttunen, T. Ilonen, et al. PROD‐screen—a screen for prodromal symptoms of psychosis. Int. J. Methods Psychiatr. Res.. 2003;12(2):92-104 Crossref
  • Ising et al., 2012 H.K. Ising, W. Veling, R.L. Loewy, M.W. Rietveld, J. Rietdijk, S. Dragt, et al. The validity of the 16-item version of the Prodromal Questionnaire (PQ-16) to screen for ultra high risk of developing psychosis in the general help-seeking population. Schizophr. Bull.. 2012;38(6):1288-1296 Crossref
  • Jarrett et al., 2012 M. Jarrett, T. Craig, J. Parrott, A. Forrester, T. Winton-Brown, H. Maguire, et al. Identifying men at ultra high risk of psychosis in a prison population. Schizophr. Res.. 2012;136(1):1-6 Crossref
  • Kelleher et al., 2012 I. Kelleher, D. Connor, M.C. Clarke, N. Devlin, M. Harley, M. Cannon. Prevalence of psychotic symptoms in childhood and adolescence: a systematic review and meta-analysis of population-based studies. Psychol. Med.. 2012;42(9):1857-1863 Crossref
  • Kline et al., 2012 E. Kline, C. Wilson, S. Ereshefsky, D. Denenny, E. Thompson, S.C. Pitts, et al. Psychosis risk screening in youth: a validation study of three self-report measures of attenuated psychosis symptoms. Schizophr. Res.. 2012;141(1):72-77 Crossref
  • Kumar and Indrayan, 2011 R. Kumar, A. Indrayan. Receiver operating characteristic (ROC) curve for medical researchers. Indian Pediatr.. 2011;48(4):277-287 Crossref
  • Kwapil et al., 1999 T.R. Kwapil, L.J. Chapman, J. Chapman. Validity and usefulness of the Wisconsin Manual for assessing psychotic-like experiences. Schizophr. Bull.. 1999;25(2):363-375 Crossref
  • Loewy et al., 2005 R.L. Loewy, C.E. Bearden, J.K. Johnson, A. Raine, T.D. Cannon. The Prodromal Questionnaire (PQ): preliminary validation of a self-report screening measure for prodromal and psychotic syndromes. Schizophr. Res.. 2005;79(1):117-125 Crossref
  • Loewy et al., 2011 R.L. Loewy, R. Pearson, S. Vinogradov, C.E. Bearden, T.D. Cannon. Psychosis risk screening with the Prodromal Questionnaire—Brief version (PQ-B). Schizophr. Res.. 2011;129(1):42-46 Crossref
  • McGlashan et al., 2010 T.H. McGlashan, B. Walsh, S. Woods. The Psychosis Risk Syndrome: Handbook for Diagnosis and Follow-up. (Oxford University Press, New York, 2010)
  • Meng et al., 1992 X. Meng, R. Rosenthal, D.B. Rubin. Comparing correlated correlation coefficients. Psychol. Bull.. 1992;111(1):172-175 Crossref
  • Miller et al., 2003 T.J. Miller, T.H. McGlashan, J.L. Rosen, K. Cadenhead, J. Ventura, W. McFarlane, et al. Prodromal assessment with the structured interview for prodromal syndromes and the scale of prodromal symptoms: predictive validity, interrater reliability, and training to reliability. Schizophr. Bull.. 2003;29(4):703-715 Crossref
  • Miller et al., 2004 T.J. Miller, D. Cicchetti, P.J. Markovich, T.H. McGlashan, S.W. Woods. The SIPS screen: a brief self-report screen to detect the schizophrenia prodrome. Schizophr. Res.. 2004;70(Suppl. 1):78
  • Ord et al., 2004 L. Ord, M. Myles-Worsley, F. Blailes, H. Ngiralmau. Screening for prodromal adolescents in an isolated high-risk population. Schizophr. Res.. 2004;71(2–3):507-508 Crossref
  • Shevlin et al., 2007 M. Shevlin, J. Murphy, M.J. Dorahy, G. Adamson. The distribution of positive psychosis-like symptoms in the population: a latent class analysis of the National Comorbidity Survey. Schizophr. Res.. 2007;89(1):101-109 Crossref
  • Steiger, 1980 J.H. Steiger. Tests for comparing elements of a correlation matrix. Psychol. Bull.. 1980;87(2):245-251 Crossref
  • Strauss, 1969 J.S. Strauss. Hallucinations and delusions as points on continua function: rating scale evidence. JAMA Psychiatry. 1969;21(5):581-586 Crossref
  • van Os et al., 2000 J. van Os, M. Hanssen, R.V. Bijl, A. Ravelli. Strauss (1969) revisited: a psychosis continuum in the general population?. Schizophr. Res.. 2000;45(1):11-20 Crossref
  • van Os et al., 2009 J. van Os, R.J. Linscott, I. Myin-Germeys, P. Delespaul, L. Krabbendam. A systematic review and meta-analysis of the psychosis continuum: evidence for a psychosis proneness–persistence–impairment model of psychotic disorder. Psychol. Med.. 2009;39(2):179 Crossref

Footnotes

a University of Maryland, Baltimore County, Department of Psychology, United States

b University of Maryland, Baltimore, Department of Psychiatry, United States

lowast Corresponding author at: University of Maryland, Baltimore County, Department of Psychology, 1000 Hilltop Circle, Baltimore, MD 21250, United States.