Mental health outcomes following a four week online training on social emotional and ethical learning for public school teachers

This was a controlled study, with a randomized controlled subsample, followed by semi-structured interviews. Elementary and high school teachers from public schools across Brazil were invited to participate in the SEELOT program through collaborations with municipal education departments and educational networks. After electronically endorsing the Informed Consent, teachers were divided into two groups: some teachers were included in the SEEL classes (SEELOT) immediately after the initial evaluation (T0) and others belonged to a Wait-List Control group- WLC (this group received the training after the completion of this phase of the study). Qualitative interviews were scheduled after the end of SEELOT one month training in T1 (Fig. 1).

Participants flow chart. Number of participants that completed each step of the study, from signing informed consent to completing the questionnaires and being included in the analysis, as well as participating in the qualitative assessment. T0: assessment before training or waiting period; T1: assessment after training of waiting period.

Ethical approval was obtained from the ethics committee of Hospital Israelita Albert Einstein (CAAE 48284621.9.0000.0071), and the study was registered on Clinical Trials under NCT05234073 (first release 10/02/2022). The consort checklist was not followed in this study because only a subsample of the participants were randomized. All research was performed in accordance with relevant guidelines/regulations, and the informed consent was obtained from all participants and/or their legal guardians. The research was performed in accordance with the Declaration of Helsinki.

Data was collected in six waves, meaning that at six different points in time, the SEELOT program was offered and the participants who signed up were invited to participate in this research. In each wave we had the participants attribute either to SEELOT and WLC groups. This attribution was class dependent (all the participants in a given class were either allocated to SEELOT or to WLC). In the first four waves, teachers chose the time and date of the training without knowing if they were choosing the SEELOT or WLC groups. However, the groups were predefined in a way that the WLC was always the class that started later. So, although the teachers were not directly choosing which study group they would be included, they indirectly did this by selecting the sooner or later class, purely based on their availability. In the last two waves, the teachers were randomly assigned to the times of the course and, therefore, to the SEELOT or WLC groups, without choosing the preferred class time (referred to as the Randomized Subsample). This was performed to be able to control for the effects of the choice from the teachers during their application. As this was expected to minor or null and we realized that most teachers would not comply with the training if the time and month of their classes were imposed on them, we decided to have this randomization only in a subsample. Due to the pandemic, we decided on 100% online training and assessments.

SEE learning online training

The one month training program consisted of both synchronous elements (two-hour meetings twice a week) and asynchronous elements, including handouts with pedagogical structure and course curriculum for use in classrooms, assigned readings and videos sent via email and WhatsApp to prepare students for upcoming classes. Additionally, audio files with recorded relaxation practices and meditation, aligned with SEEL curriculum standards, were provided for students to practice four days a week¹⁵.

Context, framework, and pedagogical model

The SEE Learning framework builds on innovative work with attention training, focus on ethics, trauma-informed approach, and systems thinking development and, it is built on three dimensions (awareness, compassion, and engagement) and three domains (personal, social, and systems).

Resourcing for emotional self-regulation and dealing with difficult situations and dilemmas

Teaches students to contrast their bodies’ sensations and processes when recalling their resources to how their bodies feel when they are stressed or dysregulated.

Grounding

Involves bringing attention to any physical contact that creates a sense of support, security, safety, or well-being.

Conceptual foundations

Seeks to promote engaged citizenship, by means of understanding and managing one’s emotions, feeling and develop empathy, developing healthy relationships and making responsible decisions.

Shift and stay practices

If one notices an unpleasant sensation in the body, one can move the attention to another part of the body that feels better (shift) and then keep attention there for a few moments (stay).

Help now! Strategies

Tools for bringing the body’s autonomic nervous system into a more regulated state, such as attending to sights or sounds, drinking a glass of water, or pushing against a wall.

Tracking

Attending to sensations in the body has been given various names, including “mindfulness of sensations,” “tracking,” and “reading the nervous system.”

Resilient zone

Being in the resilient zone is contrasted with being in stuck in the high zone or low zone, where one does not feel in control and where one’s decisions and actions are less likely to be productive.

Attention training

Structured methods for learning to attend to one’s feelings, thoughts, and impulses without being carried away by them or being distracted by outside stimuli.

Body literacy

Gaining awareness of sensations felt in the body and the words to describe them.

Mindful dialogue

A protocol for reflection and sharing in which one student asks a series of questions and listens attentively, without comment or interruption to another student, switching roles after 60–90 s.

Activities to foster kindness, empathy, and compassion

SEE Learning are based on personal insight and understanding, cultivating compassion and kindness. This includes the ability to discern what will bring about theirs and others’ long-term well-being.

Interdependence

Interdependence is a key characteristic of systems, meaning one part of a system can impact several other parts through chains of causal relations.

Data collection

Questionnaires and scales

The REDCap platform^16,17 was used for online questionnaires and scales in the two moments of the study: before intervention or waiting period (T0) and after intervention or waiting period (T1).

The questionnaires and scales that were applied are detailed below:

Positive and negative affect scales (PANAS)

The Positive and negative affect scale (PANAS) is a self-report questionnaire consisting of two 10-item scales to measure positive and negative affect. Each item is rated on a 5-point scale from 1 (not at all) to 5 (a lot). The measure is used primarily as a research tool in group studies, with possible use in clinical and non-clinical populations. Clinical and non-clinical studies have shown the PANAS is reliable and valid for assessing positive and negative affect¹⁸.

World health organization 5-item well-being index (WHO-5)

WHO-5 a short questionnaire consisting of 5 simple, non-invasive, and positively formulated items to measure subjective well-being. Participants rate each of the 5 statements on a 6-point Likert scale, from 5 (all of the time) to 0 (not at all) indicating how they have felt during the last 2 weeks. The raw score theoretically ranges from 0 (absence of well-being) to 25 (maximum well-being), but it is advisable to multiply the result by 4 to translate it into a percentage scale from 0 to 100, with higher scores meaning better subjective well-being¹⁹.

Perceived stress scale (PSS-10)

It was developed by Cohen et al., 1983²⁰, to assess how stressful individuals perceive their life situations to be. PSS-10 items verify how unpredictable, uncontrollable, and overloaded respondents consider their lives; it also includes the number of items that inquire about current levels of stress experienced. PSS-10 is a general scale, as the content of its questions is not specific to any subpopulation or age group; however, it is targeted at participants who have completed high school. The 10 questions address the frequency of participants’ feelings and thoughts about events and situations that occurred during the previous 30 days. A total of 6 questions are negative (1, 2, 3, 6, 9 and 10) and 4 are positive (4, 5, 7 and 8). Question is rated on a 5-point Likert scale from 0 (never) to 4 (very often)²⁰.

Perception of stress, well-being, attention, frustration, motivation in the last week

We assessed perceived stress, well-being, attention, frustration, and motivation before and after the last week using an online sliding percentage scale from 0 (absent) to 100 (maximum). Here, simple questions were presented: (1) What is your stress level in the last week? (2) What was your level of well-being in the last week? (3) What was your level of attention in the last week? (4) What was your frustration level in the last week? (5) What was your level of motivation in the last week? The first two questions were tested and published in²¹.We performed the reliability study of the present scales using data from this study.

Questionnaire on social and emotional learning literacy

It is an evaluation of 10 questions about the acquired knowledge about SEELOT (they were prepared by educators specialized in social and emotional learning). For better verification of results, three tests with different questions, called alpha, beta and gamma, were randomized and distributed via REDCap to teachers at the time of research. It is a test, in the format of an ‘exam’ with 10 questions regarding social and emotional learning, each one with 4 multiple choices.

Qualitative approach

The aim was to obtain a range of reports about general perception, practical tools, life skills in personal life and career, and implementation of SEELOT in the context of Brazilian education involved in the experience.

SEELOT participants were invited to attend an online semi-structured interview, which is a method that “combines closed and open questions, in which the interviewee can talk about the topic in question without being attached to the formulated question”²². These interviews were conducted after the completion of the training course. We offered the participants various schedules to participate in the qualitative interviews. As a result, they were able to choose the time and date that best suited their routines. We allowed them to select the optimal time/date to encourage more participants to contribute. The following questions were proposed in sequential order:

• What is your general perception of the training program?

• What are your comments on the practical tools shared during this training course and their contribution to your classes?

• What is your perception of the life skills shared during this training and how they contribute to your personal life and your career?

• Choose one word to best describe your experience with this training course.

• Describe your plans on how to implement SEEL in your classes.

Participants were organized in small groups of 4 to 6 volunteers. Their objective was to know the opinions and behaviors of each member of the group, in a psychosocial context, in which individuals share perceptions about the intervention. It is an interesting technique due to its richness of data and the fact that moderators can informally encourage participants to reveal important details about their perception, interest, and meaning of the intervention. The data was recorded and fully transcribed, constituting the corpus of analysis²³.

Thirty-four teachers who participated in SEELOT were interviewed, in 5 groups, for approximately 1 h and 15 min per group. Interviews were video recorded, transcribed, and conducted by a trained interviewer, with a script. Transcriptions and categorizations were validated by 3 researchers.

Data analysis

The quantitative analysis of data was executed in software environment R²⁴ using the R Studio integrated development environment²⁵ Allaire²⁶. The threshold of the significance level was p < 0.05 throughout the analysis.

As a pre-processing step, incomplete data at T0 and T1 was removed. Complete questionnaires were evaluated, except for the literacy test, which was not applied to the first wave of data collection (the literacy test was not ready to apply at that moment). For the SEELOT group, only participants certificated by the program were included. To be certificated, participants had to attend at least half of the lectures and complete all the course assignments.

To assess the impact of SEELOT, a trimmed means ANOVA²⁷ of mixed effects was used, evaluating the effects of time (T0 or T1), group (WLC or group SEELOT), and their interaction. This method is an alternative when there are violations of ANOVA assumptions, such as normality. The hypothesis of a significant training effect would be represented by a significant interaction effect in the ANOVA – the groups should differ only at T1, and only the SEELOT group should present a better result at T1 compared to T0. Significant ANOVA results were accompanied by post hoc trimmed mean tests evaluating the differences between times within each group (paired tests), and the differences between groups at each time (two-sample tests), using Bonferroni correction for multiple comparisons within each type of test for each variable (as there were two levels for each effects, the p-values were multiplied by two). A robust version of Cohen’s d effect size was estimated for the trimmed means post hoc analyses, to provide information about the relevance of the observed differences²⁸. This analysis was applied to the general and the randomized samples.

We used Content Analysis to analyze qualitative data, defined as “a set of techniques for analyzing communication processes through systematic and objective procedures for describing the content analysis of messages that allow the interpretation of knowledge about the conditions under which these messages were produced”²⁹ Bardin. We performed the analysis in the following steps:

• Pre-analysis – “float” analysis of each core; selection of documents to be submitted to analytical procedures; formulation of hypotheses and objectives; and category design and coding.

• Material exploration – during this phase, data are categorized according to similarities and differences.

• Content interpretation – discussion of categories and topics correlated with theoretical frameworks.