You are here
On the HORYZON: Moderated online social therapy for long-term recovery in first episode psychosis
Schizophrenia Research, 1, 143, pages 143 - 149
Early intervention services have demonstrated improved outcomes in first episode psychosis (FEP); however, recent evidence shows that treatment benefits may not be sustainable over time. These findings have resulted in repeated recommendations for the implementation of longer term treatment programs. An Internet-based intervention specifically designed for young people with psychosis may provide a cost-effective alternative to prevent loss of treatment benefits from early intervention.
Our multi-disciplinary team has developed a highly novel online intervention (HORYZONS) in regular consultation with stakeholders within a specialist early psychosis program. HORYZONS integrates: i) peer-to-peer social networking, ii) individually tailored interactive psychosocial interventions, and iii) expert interdisciplinary and peer-moderation in a coherent platform designed to improve long-term outcomes in FEP. The acceptability, safety and initial clinical benefits of HORYZONS were examined through a 1-month pilot study with 20 participants with FEP.
There were no dropouts during the pilot study. Seventy per cent of participants utilised the system for at least 3 weeks, 95% used the social networking features, and 60% completed at least 3 therapy modules. System usage was high during the study. There were no incidents and the majority of participants reported feeling safe, empowered and more socially connected using HORYZONS. Analysis revealed a significant reduction in depressive symptoms at follow-up.
Our results indicate that HORYZONS is feasible, engaging and safe and may augment social connectedness and empowerment in FEP. These findings have significant implications for the enhancement of specialist FEP services. The potential of HORYZONS to improve long-term recovery is worthy of further investigation.
Keywords: First episode psychosis, Early intervention, Online, Long-term recovery, Schizophrenia, Social networking.
Over the preceding two decades, early intervention for psychosis has emerged as a major international psychiatric reform ( McGorry and Yung, 2003 ). The transformation of psychiatric care has led to the proliferation of specialist first episode psychosis (FEP) services in Australia, Europe, North America and Asia ( McGorry et al., 2008 ). Early psychosis services seek to provide timely entry into treatment and support young people in achieving both symptomatic remission and full functional recovery ( McGorry et al., 1996 ). They adopt a youth-friendly and recovery-focussed model in order to empower young people to become more involved in their own treatment decisions and prevent the development of self-stigma, comorbid symptomatology and functional and social disability ( McGorry et al., 2008 ). Controlled and quasi-experimental research has demonstrated that specialist FEP services improve psychotic symptoms ( Petersen et al., 2005 ), reduce relapse rates (Craig et al, 2004, Petersen et al, 2005, and Alvarez-Jimenez et al, 2011) and comorbid substance misuse ( Petersen et al., 2005 ), and foster functional recovery ( Melle et al., 2004 ), engagement with services ( Craig et al., 2004 ) and patient satisfaction ( Petersen et al., 2005 ), at least during the first 2 years of treatment.
However, the maintenance of treatment effects poses a major challenge to achieving the aims of early intervention (Singh, 2010 and Gleeson et al, 2011). Patients with FEP are responsive to acute-phase treatments, but relapse rates are high ( Alvarez-Jimenez et al., 2012a ). Our group has developed effective cognitive-based interventions for relapse prevention in FEP ( Gleeson et al., 2009 ), but treatment benefits were lost over time suggesting the interventions needed to be maintained over the longer term ( Gleeson et al., 2011 ). More generally, specialist FEP services typically have treatment resources for only 18 months to 2 years, and recent reports suggest that the benefits of early intervention seen at the end of 2 years may not persist at 5 years after patients have been receiving generic follow-up (Bertelsen et al, 2008 and Gafoor et al, 2010). These findings have triggered widespread calls for studies assessing the effects of longer term treatment programs (Bertelsen et al, 2008, Friis, 2010, and Singh, 2010).
The positive effects of specialist FEP services are likely to persist when patients continue to receive specialised treatment (Linszen et al, 2001 and Norman et al, 2011). This view is supported by the evidence that the first 5 years after psychosis onset constitutes a critical period determining longer term outcomes (Crumlish et al, 2009 and Norman et al, 2011). Indeed, the period of risk for relapse extends well beyond the timeframe of specialist FEP services ( Alvarez-Jimenez et al., 2012a ). Furthermore, the termination of the specialised intervention and the transfer of care bring about feelings of loss for the patients ( Bertelsen et al., 2008 ) and significantly derail engagement with treatment services ( Singh, 2010 ), a pivotal element of early intervention programmes.
In this context, there is a clear need to fully realise the aims of early intervention by extending its effectiveness through innovative approaches (Kreyenbuhl et al, 2009 and Singh, 2010). A stepped down level of care which bridges the gap between specialised intervention and standard treatment could provide a cost-effective alternative to prevent loss of treatment benefits and reduce the risk for disengagement from mental health services over the longer term ( Norman and Malla, 2001 ). Internet-based interventions have the potential to realise this goal through providing cost-effective, non-stigmatising, constantly available support to young people suffering from psychosis.
This article aims to: i) provide a rationale for Internet-based interventions for psychosis and their potential to improve long-term recovery in FEP; ii) describe the development of a world-first online psychosocial intervention (HORYZONS) designed to maintain the benefits of early intervention beyond discharge from specialist FEP services; and iii) examine the feasibility, acceptability, safety and potential clinical benefits of HORYZONS in FEP patients through a pilot study.
2. Internet-based technologies and early intervention for psychosis
2.1. Are online interventions for first episode psychosis feasible?
The Internet and mobile technologies are developing at a phenomenal rate and hold great promise for influencing and even transforming treatment delivery in psychosis ( Ben-Zeev, 2012 ). These technologies can be particularly useful for young people with FEP as individuals under 25 are by far the greatest users of Internet resources ( Lloyd and Bill, 2004 ). Indeed, 95% of young Australians use the Internet daily ( Ewing et al., 2008 ) and 96% of people aged 16–24 in the UK have used the Internet, with 86% using it every day or almost every day ( Sharkey et al., 2011 ). Likewise, our data show that FEP patients show positive attitudes toward Internet-based interventions and, in particular, toward online social networking ( Lederman et al., 2011 ).
2.2. What type of Internet-based intervention?
To date, despite their clinical and social potential, only a handful of studies have investigated the effects of evidence-based interventions for psychosis delivered through Internet-based technologies. These include online forums, computer-based psychoeducation, and mobile-based interventions.
2.2.1. Online peer-support groups
The development of peer support or user-led services is based on solid theory as well as promising research findings ( Davidson et al., 2006 ). Peer support rests on the assumption that people who have overcome adversity can provide valuable support, guidance and hope to others facing similar difficulties ( Davidson et al., 2006 ). Moreover, consistent with the helper-therapy principle ( Riessman, 1965 ), being able to be of assistance to others can improve self-esteem and reduce experiences of self-stigma ( Corrigan, 2006 ). Research on user-led services has found robust associations between peer support, empowerment, social networks and recovery in people suffering from psychosis ( Corrigan, 2006 ). Social support has also shown to be a protective factor against psychotic relapse in FEP ( Norman et al., 2005 ). While research on the effects of online peer-support groups for people with psychosis is sparse, preliminary evidence suggests that professionally moderated forums focussed on positive constructs such as self-efficacy, problem solving and social recovery are likely to produce the best outcomes ( Alvarez-Jimenez et al., 2012b ).
2.2.2. Online psychoeducation
Psychoeducation interventions for psychosis have been shown to improve medication compliance, reduce relapse rates, promote social functioning, and increase satisfaction with mental health services ( Xia et al., 2011 ). Preliminary research further suggests that computer-based psychoeducation is acceptable, as effective as face-to-face or bibliotherapy, and preferred by some (Jones et al, 2001 and Walker, 2006). Although more research is needed to ascertain their effective elements, online interventions which provide clear and engaging information and cater for level of insight are likely to produce better outcomes (Williams and Whitfield, 2001, Gonzalez-Blanch et al, 2007, and Alvarez-Jimenez et al, 2009).
2.2.3. Mobile-based interventions
The characteristics of novel mobile devices including their accessibility, portability, affordability, online-connectivity, and ease of use provide an opportunity to widely deliver evidence-based interventions and enable real-time support in psychosis treatment ( Alvarez-Jimenez et al., 2012b ). Indeed, recent research shows that therapeutic interventions employing mobile devices are both acceptable to and efficiently used by people suffering from psychosis ( Depp et al., 2010 ). Moreover, preliminary evidence shows that mobile phones and SMS-based interventions can be successfully used to monitor early warning signs (EWS) of relapse and prevent hospital admissions ( Spaniel et al., 2008 ) as well as significantly improve medication adherence, socialisation and auditory hallucinations ( Granholm et al., 2012 ).
2.2.4. Internet-based interventions and FEP
To date, no study has investigated the feasibility or effects of Internet-based interventions for FEP patients. Consequently, we conducted a focus group with young people with psychosis to examine their views on technology-based tools ( Alvarez-Jimenez et al., 2012b ). Qualitative analysis showed that FEP patients were enthusiastic about the use of Internet-based strategies as part of their treatment program. They clearly preferred a system characterised by social connectivity (i.e. users should be able to contact peers, share personal experiences, and contact clinicians if required). The system should resemble regular social networking programs (i.e. asynchronous, ongoing communication), but provide a private and enclosed social network. In addition, expert moderators should guide, but not censor, the interaction to ensure a safe and supportive environment. Finally, patients indicated that the system should provide useful and updated information relevant to their needs, and optional therapeutic interventions (i.e. cognitive-behavioural strategies). Conversely, concerns were raised about the potential risks of Internet-based strategies when experiencing active psychotic symptoms and about content which made them focus negatively on their condition ( Alvarez-Jimenez et al., 2012b ).
2.3. How to develop an Internet-based intervention for FEP patients?
A number of recommendations have been put forward to increase the likelihood of acceptance, safety and effectiveness of Internet-based interventions in psychosis ( Valimaki et al., 2008 ). These include careful analysis of users' needs and preferences prior to development, active involvement of stakeholders throughout all phases of development ( Valimaki et al., 2008 ), and regular consultations with potential users with regards to all aspects of the protocols (i.e. emergency responding, therapy/psychoeducation content and wording and specific features) ( Depp et al., 2010 ). Careful attention should be paid to clinical safety and emergency procedures ( Sharkey et al., 2011 ). Additionally, the relevance of multimodal induction procedures and pilot testing has been highlighted ( Depp et al., 2010 ). Ultimately, Internet-based interventions for FEP that are specifically designed to supplement existing mental health services and augment traditional relationships through online interaction are likely to be most beneficial ( Alvarez-Jimenez et al., 2012b ).
3. Development of the MOST model and the HORYZONS system
In line with the aforementioned recommendations, we have developed a new conceptual model of on-line behavioural interventions entitled “Moderated On-line Social Therapy” (MOST). The MOST model integrates: i) peer-to-peer on-line social networking; ii) individually tailored interactive psychosocial interventions; and (iii) involvement of expert mental health and peer moderators to ensure the safety of the intervention. The elements of the MOST model have been applied to a world-first on-line system entitled HORYZONS, designed with the purpose of maintaining the clinical benefits of early intervention.
HORYZONS has been developed by a multidisciplinary team of clinical psychologists, computer programmers, health informatics experts, web and graphic designers, and professional writers. In accordance with international recommendations, the online system has been developed iteratively over a 30-month period following participatory design principles ( Schuler and Namioka, 1993 ) through a process of regular feedback and testing with focus groups of stakeholders (i.e. FEP patients and clinicians). HORYZONS includes fully functional versions for computer and Internet-enabled mobile devices (i.e. smart-phones and tablets).
3.1. Characteristics of HORYZONS
The HORYZONS system adopts the MOST conceptual model of online interventions. Thus, HORYZONS comprises a platform for the delivery of a range of evidence-based and interactive psychosocial interventions which are enhanced by a moderated on-line social networking environment. Specifically, HORYZONS integrates: i) peer-to-peer on-line social networking; ii) individually tailored interactive psychosocial interventions; and (iii) expert moderation.
3.1.1. Interactive psychosocial interventions
On entry into the system the patient answers a set of standardised questions that guide the delivery of tailored interactive psychoeducation modules. Modules are informed by evidence-based psychosocial interventions developed by our group (Killackey et al, 2008 and Gleeson et al, 2009) and target key risk factors for psychotic relapse and salient domains in the early recovery process including: (a) psychoeducation, (b) early warning signs of relapse (EWS), (c) depression, (d) social anxiety and (e) stress management ( Table 1 ). Some interactive modules are mandatory (i.e. psycho-education or ‘How minds work’) while others are optional (e.g. ‘Early Warning Signs' (EWS) of Relapse) and their delivery is guided by user preferences. In addition, the content of modules is tailored to individual clinical characteristics. For example, specifically designed “psycho-education” or “EWS” modules have been developed for participants with high insight vs. low insight to foster their engagement with the system ( Alvarez-Jimenez et al., 2009 ). Finally, the therapeutic objectives of the modules are bolstered through discussion with peers in the ‘cafe’ (online environment) under the guidance of ‘coaches’ (expert moderators).
|Snapshot||This module included an interactive evaluation of patient's goals and a number questions assessing level of insight and recovery style, anxiety and depression symptoms and social anxiety and stigma. At the end of the module HORYZONS automatically generated a formulation letter or ‘snapshot’( Ryle and Kerr, 2003 ) which summarised patients feedback and linked their responses to specific HORYZONS modules and social networking features.||90%|
|How minds work||This module provided interactive information on psychosis and the recovery process. Information was provided in a non-stigmatizing and positive manner with particular emphasis on empowerment and social recovery. Different themes were illustrated through 3 fictional characters.||70%|
|Strengths||In this module patients were introduced with the concept of personal strengths. An interactive online card sort game helped users identify their ‘signature’ strengths. Assessment of strengths was informed by positive psychology framework (Seligman et al, 2006 and Rashid and Ostermann, 2009). Participants were encouraged to share their experiences using strengths in the online social networking.||60%|
|Early warning signs (EWS)||This module included an interactive online card sort game to identify potential EWS of relapse and categorize them into early, middle and late signs ( eFig. 4 ). Subsequently, a relapse prevention plan (RPT) was developed interactively in which coping strategies and therapeutic techniques were linked to groups of EWS ( Birchwood et al., 1989 ).||45%|
|The comfort zone||This module included interactive exercises to help patients identify their current level of activity (‘comfort zone’) and promote gradual exposure and increased activity (‘expanding the zone’).||20%|
|Ninja thinking skills||This module included a number of cognitive-based strategies to help patients identify unhelpful thinking patterns (i.e. ruminative thoughts, paranoia) and promote action-focussed, real, concrete thoughts. The concept of flow was also introduced to illustrate helpful thinking patterns ( Csikszentmihalyi, 1990 ).||20%|
|To the HORYZONS||The final module consisted of a general overview of the key aspects of the completed modules. This module emphasized personal achievement, provided recommendations to ‘stay well’ and encouraged participants to use the social networking features and practice personal strengths on a regular basis.||15%|
a All modules included ‘talking points’ where participants were encouraged to share their own ideas and experiences and ‘wrap-ups’ which summarised the ‘take home messages’.
b Percentage of participants of completed the module.
3.1.2. Peer-to-peer on-line social networking (the ‘cafe’)
The ‘cafe’ includes a web feed (or newsfeed) where clients and moderators can post comments, information, upload pictures and videos, and ‘like’ different content. The newsfeed incorporates categories that organise discussion threads into relevant themes (e.g. ‘what's on your mind’, ‘I'm loving right now’, ‘strengths news’, ‘group news’) ( eFig. 1 ). Moreover the system includes a ‘wall’ function displaying the activity of individual users, a ‘homepage’ which shows all the activity and notifications relevant to the user, a ‘network’ (similar to a ‘friends’ function) and a ‘job zone’ where users are provided with information regarding training and vocational recovery and access online to a vocational rehabilitation expert ( eFig. 2 ). HORYZONS further incorporates a number of moderated social networking features that are progressively unlocked as patients advance through the therapy modules. For example, participants are invited to join moderated online problem solving groups. Nominated issues are discussed through structured phases following an evidence-based problem solving framework (McFarlane et al, 1995 and McFarlane, 2002). Offered solutions and users' experiences are stored in a data-base serving as a ‘wiki’ for participants. Moreover, participants are encouraged to share personal coping resources for stressful events (i.e. ‘what works for me’) which are accumulated and categorised for future reference.
Importantly, HORYZONS adopts a strength-based approach ( Seligman et al., 2006 ) through which users are guided and prompted to identify, discuss and exercise key personal strengths within the online environment and in real-life to enhance self-esteem, foster social functioning ( Hall and Tarrier, 2003 ) and reduce depression ( Seligman et al., 2006 ) ( eFig. 3 ). This approach is based on the novel positive psychology model which poses that psychosocial interventions should aim to build strengths, meaning and purpose as well as relieve symptoms (Lee Duckworth et al, 2005 and Seligman et al, 2006). Preliminary research suggests that this model is particularly appropriate for online interventions. For example, FEP patients may not engage effectively in online content which make them focus negatively on their condition ( Alvarez-Jimenez et al., 2012b ). Further, excessive expressions of fear and anxiety online increase stress and reduce quality of life ( Lieberman and Goldstein, 2006 ), while Internet interventions focussed on self-efficacy have produced improved clinical outcomes in patients with psychosis ( Rotondi et al., 2010 ).
3.1.3. Expert moderation
HORYZONS uniquely provides a ‘hub’ in which clinical psychologists and vocational workers collaboratively moderate the system with the purpose of improving both clinical and vocational functioning. Their role is to provide guidance, monitor participant's clinical status and ensure the safety of the social networking environment. HORYZONS moderation follows a theory-driven model known as ‘supportive accountability’ which poses that human support enhances engagement through accountability to a moderator who is perceived as trustworthy, benevolent and having expertise. Accountability involves clear, process-focussed and user driven expectations that take into account patient motivation (i.e. level of support is inversely proportional to the patient's intrinsic motivation) ( Mohr et al., 2011 ). HORYZONS is moderated on a daily basis (i.e. 2 h/day during week-days, and 1 h/day during weekends).
3.1.4. HORYZONS safety protocol
The HORYZONS safety protocol follows best practice in Internet research involving vulnerable people ( Sharkey et al., 2011 ) and is composed of 3 levels of security including system security, online safety, and clinical safety. System security conforms to industry best practice as defined by the Open Web Application Security Project (OWASP) . Online safety is managed in accordance with the guidelines put forward by the Australian Communications and Media Authority (ACMA) .
Clinical safety is ensured through manual and automated procedures. First, information related to clinical risk is screened daily by moderators. Second, HORYZONS incorporates an automatic alert system which detects information consistent with increased risk of relapse or suicide (via regular monitoring of EWS of relapse and self-harm-related terms using previously validated approaches (Goh, 2009 and Huang et al, 2007). Any detected increased risk activates the HORYZONS crisis protocol that includes an initial risk assessment and, where necessary, liaison with suitable emergency services. Details on the safety protocol and ethical issues have been provided elsewhere ( Gleeson et al., in press ).
4. The HORYZONS pilot study
The purpose of the pilot study was to conduct an initial evaluation of HORYZONS regarding its feasibility, acceptability, safety and potential clinical utility for FEP patients. We hypothesised that HORYZONS would be: 1) feasible and favourably received; 2) regularly used; 3) safe; and 4) viewed as a useful tool for long-term recovery and social connectedness.
The sample included 20 patients recruited from the Early Psychosis Prevention and Intervention Centre (EPPIC, Melbourne). Inclusion criteria for the study were: (a) a first episode of a DSM-IV ( APA, 1994 ) psychotic disorder or mood disorder with psychotic features; (b) aged 15–25 years inclusive; (c) ≤ 6 months treatment with an antipsychotic medication prior to registration with the early psychosis service; and (d) remission of positive symptoms of psychosis, defined as scores of 3 (mild) or below on the subscale items hallucinations, unusual thought disorder, conceptual disorganisation, and suspiciousness on the Brief Psychiatric Rating Scale (BPRS) ( Lukoff et al., 1986 ) over that last 4 weeks. Additional inclusion criteria to ensure low level of risk within HORYZONS included: (e) low aggressiveness, defined by a score of 4 or below on the hostility subscale of the expanded version of the BPRS for the month prior to study entry; and (f) low suicidal risk defined as a score of 4 or below on the BPRS suicidality subscale for the month preceding study entry. Exclusion criteria included: (a) intellectual disability and (b) inability to converse in, or read English. Additional exclusion criteria to ensure safety within the online system included (c) a DSM-IV ( APA, 1994 ) diagnosis of either antisocial personality disorder (ASPD) or (d) borderline personality disorder (BPD).
The mean age was 20.3 years (SD = 2.7) with 50% male. Self-reported ethnicity was 45% Anglo Australian, 25% Asian, 15% European, 5% African and 10% Bi-racial. Thirty five per cent of the samples were unemployed with 100% never married. Ninety per cent of the samples were living with their family and 10% living with friends. The study was approved by the Melbourne Health Human Research Ethics Committee. Participants provided informed consent while continuing treatment within the EPPIC programme.
4.2. Design and procedures
HORYZONS was monitored daily (i.e. 2 h/day during week-days, and 1 h/day during weekends) by clinical psychologists and a vocational worker. Moderation integrity was ensured through a detailed moderation manual and weekly group supervision sessions with senior clinical researchers (MA-J and JF-G) from the research team. Participants were assessed at baseline and at 4 weeks follow-up on outcomes described below.
Baseline DSM-IV diagnosis was assessed via the Structured Clinical Interview for DSM-IV (SCID patient version) ( First et al., 1996 ). Feasibility and acceptance of HORYZONS were tracked measuring participants' usage of the online system (i.e. frequency, duration and patterns of use). In addition, a questionnaire and semi-structured interview were developed based on the User Experience (UX) approach ( Bargas-Avila and Hornbæk, 2011 ) which assessed the following themes: 1) helpfulness; 2) easy-of-use; 3) attractiveness; 4) safety; 5) social interaction; and 6) further suggestions. Each domain was assessed through a number of statements which the respondent rated on a Likert scale from strongly agree to strongly disagree as well as open-ended questions. Good acceptability of the system was defined a priori as at least 10 of the 20 participants logging on weekly during the 1-month pilot.
Safety was assessed through the following indicators: 1) adverse events (i.e. psychotic relapse and clinical deterioration related to usage of the system), 2) inappropriate use, and 3) participants' reports on inappropriate content.
Symptom rating measures included the 24-item version of the Brief Psychiatric Rating Scale (BPRS) ( Lukoff et al., 1986 ), the clinician-rated Calgary Depression Scale for Schizophrenia (CDSS) ( Addington et al., 1993 ), and the Beck Anxiety Inventory (BAI) ( Beck et al., 1988 ).
Frequency, duration and patterns of use were tracked in real time. Up-to-date charts were visible on demand to researchers and moderators via the website. Quantitative and qualitative feedbacks from the questionnaire were analysed and reviewed for themes related to usability, perceived benefits, safety, and challenges of online interventions. Paired samples t-tests were conducted and within-group effect sizes reported for statistically significant changes between baseline and post-test in clinical measures.
4.5.1. Feasibility and acceptance
Data on the usage of HORYZONS showed that 60% of participants utilised the system over 4 weeks and 70% for at least 3 out of 4 weeks, with a total of 275 logins during the trial ( Table 2 ). Participants provided positive ratings for the ease of use, perceived utility, and enjoyment of HORYZONS. The majority (75%) reported that they had a positive and constructive experience using HORYZONS, 90% would recommend it to others, and no participants reported negative or challenging experiences.
|Social networking c||208.50||191.99||134.50||0–702||95 d|
|Therapy modules c||69.50||47.60||65.50||7–169||75 d|
a Covers period from February 1, 2012, to 16th of March 16, 2012; participants were recruited into the system from February 1, 2012, to February 17, 2012.
b Percentage of participants with more than 6 logins.
c Number of page reads (page ‘hits’).
d Percentage of participants with more than 30 page hits or posts.
The social networking features were used by 95% of the sample, with a total of 371 postings and 4170 social page views or actions (i.e. posts, ‘like’) during the study. There was a median of 192 (range = 0–702) social pages views/actions per participant ( Table 2 ). Accordingly, 70% perceived social networking as useful, and 90% considered moderation to be supportive. Interestingly, 85% considered that it would be beneficial to include moderators who were previous users of HORYZONS and 90% reported they would like to become online peer-moderators.
With respect to therapy module usage, 95% completed at least 1 full therapy module, 60% completed 3 at least modules, 45% completed 4 modules, and 15% completed all 7 modules ( Table 1 ). There were a total of 1390 therapy-based page views, with a median of 65 (range = 7–169) per participant ( Table 2 ).
No incidents (i.e. adverse events or inappropriate usage) were reported during the study. Similarly, analysis showed no worsening of positive psychotic symptoms at 1-month follow-up ( Table 3 ). One participant experienced a psychotic relapse; however, both the participant and the treating team considered the clinical worsening unrelated to the use of HORYZONS. This is further supported by the low usage of HORYZONS by the relapsing participant, with only 2 logins during the trial. Finally, 100% participants considered HORYZONS to be both safe and confidential, and 90% reported that moderation contributed to the safety of the system.
|Baseline||1 month follow-up||p||D|
BPRS = Brief Psychiatric Rating Scale; CDRS = Calgary Depression Rating Scale; BAI = Beck Anxiety Inventory.
4.5.3. Potential clinical utility
Analysis of symptom rating measures revealed a moderate to large improvement in participants' depressive symptoms at 1-month follow-up ( Table 2 ). Moreover, 60% reported that HORYZONS significantly increased their social connectedness, 55% felt empowered in their own recovery process, and 70% considered the system to be a useful long-term treatment option beyond discharge.
To the best of our knowledge, this is the first study to develop and test an online intervention specifically designed for FEP patients, and is possibly a world-first intervention using the MOST model. HORYZONS has been developed by a multidisciplinary team of experts in constant consultation with stakeholders. It uniquely integrates peer-to-peer online social networking, evidence-based interventions and professional and peer support in a coherent, innovative platform designed to provide ongoing support and prevent disengagement from mental health services. The results of this pilot study demonstrated HORYZONS to be highly acceptable and appealing to young FEP patients. More than two thirds of the sample regularly used the online system and all participants reported feeling safe and supported in doing so. This was further reflected in the highly successful rate of recruitment to the study and the volume of traffic on the site.
Data on system usage and therapy completion strongly supported the feasibility of HORYZONS. There were no drop-outs during the pilot trial and completion of therapy modules was relatively high, with 60% of participants completing at least 3 full therapy modules within one month. This compares well with the standard weekly dosage of psychotherapy provided in individual therapy studies for FEP (Jackson et al, 2005, Jackson et al, 2007, Edwards et al, 2006, Gleeson et al, 2009, and Penn et al, 2010), particularly considering that HORYZONS participants undertake therapy modules independently and without the active prompts of therapeutic sessions.
Our attrition and module completion rates compare very favourably to unsupported, open-access, short-term Internet-based programs (Eysenbach, 2005 and Christensen et al, 2006), and even online interventions incorporating support and tracking by interviewers or counselors (Christensen et al, 2004 and Clarke et al, 2005). Likewise, usage of HORYZONS (i.e. frequency of use in relation to time) compared well to a previous web-based intervention for people with schizophrenia which included online educational materials and a therapy forum ( Rotondi et al., 2010 ). While the short duration of the trial and the characteristics of the clinical sample may partially account for these positive findings, these results suggest that the novel integration of online moderation, peer-to-peer social networking and therapy modules (i.e. the MOST model) provides a promising and engaging platform for delivering online interventions in FEP.
Analysis of clinical variables showed a moderate to large reduction in depressive symptoms at 1-month follow-up ( Table 2 ). Subsequent analysis of individual-level data revealed that 7 of 10 participants with reduced depression scores were regular users of HORYZONS. Whereas the uncontrolled design of the study does not allow causal attributions, the clinical significance of the latter finding should be noted. Depression is pervasive in the early course of psychosis ( Upthegrove et al., 2010 ) and individual interventions for FEP have shown only modest effects in reducing depressive symptoms (Jackson et al, 2005, Edwards et al, 2006, Gleeson et al, 2009, and Gleeson et al, 2011). Taken together, this suggests that possibly the increased social connectedness (as reported by 60% of participants) or the empowering and strengths-based model adopted by HORYZONS (55% of participants reported feeling empowered in their own recovery using the online system) may have contributed to the reduction in depressive symptoms at follow-up. The potential of HORYZONS to improve depressive symptoms in FEP is worthy of further investigation.
The current study has several limitations. First, as noted above, the uncontrolled design precluded any causal inferences about the efficacy of HORYZONS (i.e. reduction in depression may be a function of either time or background interventions), however this was never the intended purpose of the study. Second, the short-term duration of the study precluded examination of the long-term effects and retention of the intervention. Thirdly, the current findings regarding acceptability, safety and potential clinical benefits can only be generalised to remitted FEP patients with low risk of aggression and suicidality. Fourth, the semi-structured interview was administered by an assessor non-blind to the study hypothesis, which may have over-estimated positive findings, and participant responses were subject to demand effects, social desirability and recall bias. That said, considerable efforts were made to minimise these biases. Namely, i) prior to interviews it was emphasised that honest responses were most beneficial for future users and versions of HORYZONS; ii) interviews were conducted showing HORYZONS through an Internet-enabled tablet to minimise recall bias; iii) all interviews were recorded, transcribed and rated by an independent rater; and iv) system usage, qualitative data and responses to the questionnaire were matched for consistency. Notwithstanding these caveats, our pilot study provided “proof of concept” for HORYZONS in terms of its acceptability and potential clinical benefits and warrants a full-scale trial in which these limitations can be addressed.
5.1. Conclusions and future research
The results of this study showed HORYZONS to be a promising online intervention for FEP patients as it yielded high acceptance and usage, low attrition, high satisfaction, and perceived increased social connectedness and empowerment. Participants reported feeling safe and perceived HORYZONS as a useful treatment strategy beyond discharge from a specialised FEP service.
In the near future we will evaluate whether HORYZONS is an effective strategy for the maintenance of specialised treatment effects in FEP. In the meantime the results of our pilot study highlight the potential of HORYZONS to: i) significantly improve the clinical and cost-effectiveness of specialised FEP services; ii) improve access and ongoing engagement of FEP patients in evidence-based preventative interventions; and iii) and transform current models of treatment maintenance in FEP.
Role of funding Source
This study was supported by generous funding from the Telstra Foundation, the Helen MacPherson Smith Trust, the University of Melbourne, the Telematics Trust, The Institute for a Broadband-Enabled Society (IBES), and the Colonial Foundation. The sponsors did not participate in the design or conduct of this study; in the collection, management, analysis, or interpretation of data; in the writing of the manuscript; or in the preparation, review, approval, or decision to submit this manuscript for publication.
MA-J and JF-G supervised the study and wrote the first draft of the manuscript. S-B, R-L, G-W, G-C, S-V and M-L significantly contributed to the development and evaluation of HORYZONS. E-K and PD-M contributed to the design of the study and critically revised the manuscript. All authors contributed to and have approved the final manuscript.
Conflict of interest
The authors report no additional financial or other affiliation relevant to the subject of this article.
The following are the supplementary data related to this article.
The authors wish to thank Christopher Miles, Kathryn Junor and Sylvia Collinety for their contribution to the development of HORYZONS, and the study participants for their valuable contributions to this research.
- ACMA, Australian Communications Media Authority ACMA, Australian Communications Media Authority. www.acma.gov.au
- Addington et al., 1993 D. Addington, J. Addington, E. Matickatyndale. Assessing depression in schizophrenia — the Calgary Depression Scale. Br. J. Psychiatry. 1993;163:39-44
- Alvarez-Jimenez et al., 2009 M. Alvarez-Jimenez, J.F. Gleeson, S. Cotton, D. Wade, D. Gee, T. Pearce, K. Crisp, D. Spiliotacopoulos, B. Newman, P.D. McGorry. Predictors of adherence to cognitive–behavioural therapy in first-episode psychosis. Can. J. Psychiatry. 2009;54(10):710-718
- Alvarez-Jimenez et al., 2011 M. Alvarez-Jimenez, A.G. Parker, S.E. Hetrick, P.D. McGorry, J.F. Gleeson. Preventing the second episode: a systematic review and meta-analysis of psychosocial and pharmacological trials in first-episode psychosis. Schizophr. Bull.. 2011;37(3):619-630
- Alvarez-Jimenez et al., 2012a M. Alvarez-Jimenez, A. Priede, S.E. Hetrick, S. Bendall, E. Killackey, A.G. Parker, P.D. McGorry, J.F. Gleeson. Risk factors for relapse following treatment for first episode psychosis: a systematic review and meta-analysis of longitudinal studies. Schizophr. Res.. 2012; (as)
- Alvarez-Jimenez et al., 2012b M. Alvarez-Jimenez, J.F. Gleeson, S. Bendall, R. Lederman, G. Wadley, E. Killackey, P.D. McGorry. Internet-based interventions for psychosis: a sneak-peek into the future. Psychiatr. Clin. North Am.. 2012;35(3):735-747
- APA, 1994 APA. Diagnostic and Statistical Manual of Mental Disorders. 4 ed. (American Psychiatric Association, Washington, DC, 1994)
- Bargas-Avila and Hornbæk, 2011 J.A. Bargas-Avila, K.P. Hornbæk. Old Wine in New Bottles or Novel Challenges? A Critical Analysis of Empirical Studies of User Experience. ACM CHI Conference, Vancouver (, 2011)
- Beck et al., 1988 A.T. Beck, N. Epstein, G. Brown, R.A. Steer. An inventory for measuring clinical anxiety: psychometric properties. J. Consult. Clin. Psychol.. 1988;56(6):893-897
- Ben-Zeev, 2012 D. Ben-Zeev. Mobile technologies in the study, assessment, and treatment of schizophrenia. Schizophr. Bull.. 2012;38(3):384-385
- Bertelsen et al., 2008 M. Bertelsen, P. Jeppesen, L. Petersen, A. Thorup, J. Ohlenschlaeger, P. le Quach, T.O. Christensen, G. Krarup, P. Jorgensen, M. Nordentoft. Five-year follow-up of a randomized multicenter trial of intensive early intervention vs standard treatment for patients with a first episode of psychotic illness: the OPUS trial. Arch. Gen. Psychiatry. 2008;65(7):762-771
- Birchwood et al., 1989 M. Birchwood, J. Smith, F. Macmillan, B. Hogg, R. Prasad, C. Harvey, S. Bering. Predicting relapse in schizophrenia: the development and implementation of an early signs monitoring system using patients and families as observers, a preliminary investigation. Psychol. Med.. 1989;19(3):649-656
- Christensen et al., 2004 H. Christensen, K.M. Griffiths, A.E. Korten, K. Brittliffe, C. Groves. A comparison of changes in anxiety and depression symptoms of spontaneous users and trial participants of a cognitive behavior therapy website. J. Med. Internet Res.. 2004;6(4):e46
- Christensen et al., 2006 H. Christensen, K.M. Griffiths, A.J. Mackinnon, K. Brittliffe. Online randomized controlled trial of brief and full cognitive behaviour therapy for depression. Psychol. Med.. 2006;36(12):1737-1746
- Clarke et al., 2005 G. Clarke, D. Eubanks, E. Reid, C. Kelleher, E. O'Connor, L.L. DeBar, F. Lynch, S. Nunley, C. Gullion. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J. Med. Internet Res.. 2005;7(2):e16
- Corrigan, 2006 P.W. Corrigan. Impact of consumer-operated services on empowerment and recovery of people with psychiatric disabilities. Psychiatr. Serv.. 2006;57(10):1493-1496
- Craig et al., 2004 T.K. Craig, P. Garety, P. Power, N. Rahaman, S. Colbert, M. Fornells Ambrojo, G. Dunn. The Lambeth Early Onset (LEO) Team: randomised controlled trial of the effectiveness of specialised care for early psychosis. BMJ. 2004;329(7474):1067
- Crumlish et al., 2009 N. Crumlish, P. Whitty, M. Clarke, S. Browne, M. Kamali, M. Gervin, O. McTigue, A. Kinsella, J.L. Waddington, C. Larkin, E. O'Callaghan. Beyond the critical period: longitudinal study of 8-year outcome in first-episode non-affective psychosis. Br. J. Psychiatry. 2009;194(1):18-24
- Csikszentmihalyi, 1990 M. Csikszentmihalyi. Finding Flow: the Psychology of Engagement with Everyday Life. (Harper and Row, New York, 1990)
- Davidson et al., 2006 L. Davidson, M. Chinman, D. Sells, M. Rowe. Peer support among adults with serious mental illness: a report from the field. Schizophr. Bull.. 2006;32(3):443-450
- Depp et al., 2010 C.A. Depp, B. Mausbach, E. Granholm, V. Cardenas, D. Ben-Zeev, T.L. Patterson, B.D. Lebowitz, D.V. Jeste. Mobile interventions for severe mental illness: design and preliminary data from three approaches. J. Nerv. Ment. Dis.. 2010;198(10):715-721
- Edwards et al., 2006 J. Edwards, K. Elkins, M, H., S.M. Harrigan, K. Donovan, O. Athanasopoulos, P.D. McGorry. Randomised controlled trial of a cannabis-focused intervention for young people with first-episode psychosis. Acta Psychiatr. Scand.. 2006;114:109-117
- Ewing et al., 2008 S. Ewing, J. Thomas, J. Schiessl. CCI Digital Futures Report: the Internet in Australia. ARC Centre of Excellence for Creative Industries and Innovation Institute for Social Research. (Swinburne University of Technology, Melbourne, 2008)
- Eysenbach, 2005 G. Eysenbach. The law of attrition. J. Med. Internet Res.. 2005;7(1):e11
- First et al., 1996 M.B. First, M. Gibbon, R.L. Spitzer, J.B.W. Williams. Structured clinical interview for DSM-IV Axis I Disorders—Patient Version (SCID-I/P). (Biometrics Research Department, New York State Psychiatric Institute, New York, NY, 1996)
- Friis, 2010 S. Friis. Early specialised treatment for first-episode psychosis: does it make a difference?. Br. J. Psychiatry. 2010;196(5):339-340
- Gafoor et al., 2010 R. Gafoor, D. Nitsch, P. McCrone, T.K. Craig, P.A. Garety, P. Power, P. McGuire. Effect of early intervention on 5-year outcome in non-affective psychosis. Br. J. Psychiatry. 2010;196(5):372-376
- Gleeson et al., 2009 J.F. Gleeson, S.M. Cotton, M. Alvarez-Jimenez, D. Wade, D. Gee, K. Crisp, T. Pearce, B. Newman, D. Spiliotacopoulos, D. Castle, P.D. McGorry. A randomized controlled trial of relapse prevention therapy for first-episode psychosis patients. J. Clin. Psychiatry. 2009;70(4):477-486
- Gleeson et al., 2011 J.F. Gleeson, S.M. Cotton, M. Alvarez-Jimenez, D. Wade, D. Gee, K. Crisp, T. Pearce, D. Spiliotacopoulos, B. Newman, P.D. McGorry. A randomized controlled trial of relapse prevention therapy for first-episode psychosis patients: outcome at 30-month follow-up. Schizophr. Bull.. 2011;
- Gleeson et al., submitted for publication Gleeson, J.F., Lederman, R., Wadley, G., Bendall, S., McGorry, P.D., Alvarez-Jimenez, M., in press. Successful management of safety, confidentiality and privacy within a moderated online social therapy for young people recovering from first-episode psychosis. J. Med. Internet Res.
- Goh, 2009 T.T. Goh, Y. Huang. Monitoring youth depression risk in Web 2.0. Vine. J. Inf. Knowl. Manag. Syst.. 2009;39(3):192-202
- Gonzalez-Blanch et al., 2007 C. Gonzalez-Blanch, B. Crespo-Facorro, M. Alvarez-Jimenez, J.M. Rodriguez-Sanchez, J.M. Pelayo-Teran, R. Perez-Iglesias, J.L. Vazquez-Barquero. Cognitive dimensions in first-episode schizophrenia spectrum disorders. J. Psychiatr. Res.. 2007;41(11):968-977
- Granholm et al., 2012 E. Granholm, D. Ben-Zeev, P.C. Link, K.R. Bradshaw, J.L. Holden. Mobile Assessment and Treatment for Schizophrenia (MATS): a pilot trial of an interactive text-messaging intervention for medication adherence, socialization, and auditory hallucinations. Schizophr. Bull.. 2012;38(3):414-425
- Hall and Tarrier, 2003 P.L. Hall, N. Tarrier. The cognitive–behavioural treatment of low self-esteem in psychotic patients: a pilot study. Behav. Res. Ther.. 2003;41(3):317-332
- Huang et al., 2007 Y.-P. Huang, T. Goh, C.L. Liew. Hunting Suicide Notes in Web 2.0—Preliminary Findings, Ninth IEEE International Symposium on Multimedia Workshops (, 2007)
- Jackson et al., 2005 H. Jackson, P. McGorry, J. Edwards, C. Hulbert, L. Henry, S. Harrigan, P. Dudgeon, S. Francey, D. Maude, J. Cocks, E. Killackey, P. Power. A controlled trial of cognitively oriented psychotherapy for early psychosis (COPE) with four-year follow-up readmission data. Psychol. Med.. 2005;35(9):1295-1306
- Jackson et al., 2007 H.J. Jackson, P.D. McGorry, E. Killackey, S. Bendall, K. Allott, P. Dudgeon, J. Gleeson, T. Johnson, S. Harrigan. Acute-phase and 1-year follow-up results of a randomized controlled trial of CBT versus befriending for first-episode psychosis: the ACE project. Psychol. Med.. 2007;:1-11
- Jones et al., 2001 R.B. Jones, J.M. Atkinson, D.A. Coia, L. Paterson, A.R. Morton, K. McKenna, N. Craig, J. Morrison, W.H. Gilmour. Randomised trial of personalised computer based information for patients with schizophrenia. BMJ. 2001;322(7290):835-840
- Killackey et al., 2008 E. Killackey, H.J. Jackson, P.D. McGorry. Vocational intervention in first-episode psychosis: individual placement and support v. treatment as usual. Br. J. Psychiatry. 2008;193(2):114-120
- Kreyenbuhl et al., 2009 J. Kreyenbuhl, I.R. Nossel, L.B. Dixon. Disengagement from mental health treatment among individuals with schizophrenia and strategies for facilitating connections to care: a review of the literature. Schizophr. Bull.. 2009;35(4):696-703
- Lederman et al., 2011 R. Lederman, G. Wadley, J.F. Gleeson, A. Spiteri-Staines, M. Alvarez-Jimenez. Supporting young people with psychosis in the community: an ICT-enabled relapse prevention tool. Proceedings of the Pacific Asia Conference on Information Systems (PACIS), Brisbane (, 2011)
- Lee Duckworth et al., 2005 A. Lee Duckworth, T.A. Steen, M.E. Seligman. Positive psychology in clinical practice. Annu. Rev. Clin. Psychol.. 2005;1:629-651
- Lieberman and Goldstein, 2006 M.A. Lieberman, B.A. Goldstein. Not all negative emotions are equal: the role of emotional expression in online support groups for women with breast cancer. Psychooncology. 2006;15(2):160-168
- Linszen et al., 2001 D. Linszen, P. Dingemans, M. Lenior. Early intervention and a five year follow up in young adults with a short duration of untreated psychosis: ethical implications. Schizophr. Res.. 2001;51(1):55-61
- Lloyd and Bill, 2004 R. Lloyd, A. Bill. Australia Online: How Australians Are Using Computers And the Internet. 2001 Australian Census Analytic Program. Publication 2056.0. (Australian Bureau of Statistics, Canberra, 2004)
- Lukoff et al., 1986 D. Lukoff, R.P. Liberman, K.H. Nuechterlein. Symptom monitoring in the rehabilitation of schizophrenic patients. Schizophr. Bull.. 1986;12(4):578-593
- McFarlane, 2002 W.R. McFarlane. Multifamily Groups in the Treatment of Severe Psychiatric Disorders. (Guilford Press, New York, 2002)
- McFarlane et al., 1995 W.R. McFarlane, E. Lukens, B. Link, R. Dushay, S.A. Deakins, M. Newmark, E.J. Dunne, B. Horen, J. Toran. Multiple-family groups and psychoeducation in the treatment of schizophrenia. Arch. Gen. Psychiatry. 1995;52(8):679-687
- McGorry and Yung, 2003 P.D. McGorry, A.R. Yung. Early intervention in psychosis: an overdue reform. Aust. N. Z. J. Psychiatry. 2003;37(4):393-398
- McGorry et al., 1996 P.D. McGorry, J. Edwards, C. Mihalopoulos, S.M. Harrigan, H.J. Jackson. EPPIC: an evolving system of early detection and optimal management. Schizophr. Bull.. 1996;22(2):305-326
- McGorry et al., 2008 P.D. McGorry, E. Killackey, A. Yung. Early intervention in psychosis: concepts, evidence and future directions. World Psychiatry. 2008;7(3):148-156
- Melle et al., 2004 I. Melle, T.K. Larsen, U. Haahr, S. Friis, J.O. Johannessen, S. Opjordsmoen, E. Simonsen, B.R. Rund, P. Vaglum, T. McGlashan. Reducing the duration of untreated first-episode psychosis: effects on clinical presentation. Arch. Gen. Psychiatry. 2004;61(2):143-150
- Mohr et al., 2011 D.C. Mohr, P. Cuijpers, K. Lehman. Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J. Med. Internet Res.. 2011;13(1):e30
- Mueser and Drake, 2005 K.T. Mueser, R.E. Drake. How does a practice become evidenced-based?. R.E. Drake, M.R. Merrens, D.W. Lynde (Eds.) Evidence-Based Mental Health Practice: a Textbook (W.W. Norton Company Ltd, New York, 2005) 217-241
- Norman and Malla, 2001 R.M. Norman, A.K. Malla. Duration of untreated psychosis: a critical examination of the concept and its importance. Psychol. Med.. 2001;31(3):381-400
- Norman et al., 2005 R.M. Norman, A.K. Malla, R. Manchanda, R. Harricharan, J. Takhar, S. Northcott. Social support and three-year symptom and admission outcomes for first episode psychosis. Schizophr. Res.. 2005;80(2–3):227-234
- Norman et al., 2011 R.M. Norman, R. Manchanda, A.K. Malla, D. Windell, R. Harricharan, S. Northcott. Symptom and functional outcomes for a 5 year early intervention program for psychoses. Schizophr. Res.. 2011;129(2–3):111-115
- OWASP, The Open Web Application Security Project OWASP, the Open Web Application Security Project. www.akamai.com
- Penn et al., 2010 D.L. Penn, S.R. Uzenoff, D. Perkins, K.T. Mueser, R. Hamer, E. Waldheter, S. Saade, L. Cook. A pilot investigation of the Graduated Recovery Intervention Program (GRIP) for first episode psychosis. Schizophr. Res.. 2010;125(2–3):247-256
- Petersen et al., 2005 L. Petersen, P. Jeppesen, A. Thorup, M.B. Abel, J. Ohlenschlaeger, T.O. Christensen, G. Krarup, P. Jorgensen, M. Nordentoft. A randomised multicentre trial of integrated versus standard treatment for patients with a first episode of psychotic illness. BMJ. 2005;331(7517):602
- Rashid and Ostermann, 2009 T. Rashid, R.F. Ostermann. Strength-based assessment in clinical practice. J. Clin. Psychol.. 2009;65(5):488-498
- Riessman, 1965 F. Riessman. The ‘helper-therapy’ principle. Soc. Work.. 1965;10:27-32
- Rotondi et al., 2010 A.J. Rotondi, C.M. Anderson, G.L. Haas, S.M. Eack, M.B. Spring, R. Ganguli, C. Newhill, J. Rosenstock. Web-based psychoeducational intervention for persons with schizophrenia and their supporters: one-year outcomes. Psychiatr. Serv.. 2010;61(11):1099-1105
- Ryle and Kerr, 2003 A. Ryle, I.B. Kerr. Cognitive analytic therapy. Br. J. Psychiatry. 2003;183:79
- Schuler and Namioka, 1993 D. Schuler, A. Namioka. Participatory Design: Principles and Practices. (Routledge, Taylor & Francis group, Hillsdale, New Jersey, 1993)
- Seligman et al., 2006 M.E. Seligman, T. Rashid, A.C. Parks. Posit. Psychother. Am. Psychol.. 2006;61(8):774-788
- Sharkey et al., 2011 S. Sharkey, R. Jones, J. Smithson, E. Hewis, T. Emmens, T. Ford, C. Owens. Ethical practice in internet research involving vulnerable people: lessons from a self-harm discussion forum study (SharpTalk). J. Med. Ethics. 2011;37(12):752-758
- Singh, 2010 S.P. Singh. Early intervention in psychosis. Br. J. Psychiatry. 2010;196(5):343-345
- Spaniel et al., 2008 F. Spaniel, P. Vohlidka, J. Hrdlicka, J. Kozeny, T. Novak, L. Motlova, J. Cermak, J. Bednarik, D. Novak, C. Hoschl. ITAREPS: information technology aided relapse prevention programme in schizophrenia. Schizophr. Res.. 2008;98(1–3):312-317
- Upthegrove et al., 2010 R. Upthegrove, M. Birchwood, K. Ross, K. Brunett, R. McCollum, L. Jones. The evolution of depression and suicidality in first episode psychosis. Acta Psychiatr. Scand.. 2010;122(3):211-218
- Valimaki et al., 2008 M. Valimaki, M. Anttila, H. Hatonen, M. Koivunen, T. Jakobsson, A. Pitkanen, J. Herrala, L. Kuosmanen. Design and development process of patient-centered computer-based support system for patients with schizophrenia spectrum psychosis. Inform. Health Soc. Care. 2008;33(2):113-123
- Walker, 2006 H. Walker. Computer-based education for patients with psychosis. Nurs. Stand.. 2006;20(30):49-56
- Williams and Whitfield, 2001 C. Williams, G. Whitfield. Written and computer-based self-help treatments for depression. Br. Med. Bull.. 2001;57:133-144
- Xia et al., 2011 J. Xia, L.B. Merinder, M.R. Belgamwar. Psychoeducation for schizophrenia. Cochrane Database Syst. Rev.. 2011;6:CD002831
a Centre for Youth Mental Health, The University of Melbourne, Australia
b Orygen Youth Health Research Centre, Australia
c The Department of Computing and Information Systems, The University of Melbourne, Australia
d Orygen Youth Health, Australia
e Australian Catholic University, School of Psychology, Melbourne, Australia
Corresponding author at: Centre for Youth Mental Health, The University of Melbourne, Orygen Youth Health Research Centre, 35, Poplar Road, Parkville 3054, Victoria, Melbourne Australia. Tel.: + 61 9342 2805, + 61 401772668 (mobile).
© 2012 Elsevier B.V., All rights reserved.