2012 TALKS Mentoring Evaluation Study
Executive Summary
The TALKS program was implemented in four schools in Jackson-Madison County Schools (JMCSS) during the 2011-2012 academic year. Three middle schools (Rose Hill, North Parkway, and West Middle) and one elementary school (Lincoln) implemented the program. A total of 46 students in grades three through eight participated in TALKS, but not all TALKS students remained in the program the entire year.
The TALKS program consisted of TALKS mentors meeting with TALKS students in a separate space within the school for a total of 30-45 minutes each week. Male TALKS mentors met with male TALKS students and female TALKS mentors met with female TALKS students. The TALKS program included students reading a TALKS-supplied book, discussion of the book, and further discussions about behavior in school, academic achievement, and other topics necessary for success in school.
Students were chosen to participate in the TALKS program by school administrators and/or faculty members.
Students were chosen based on their grades, behavior, and personal life situation. Some TALKS groups were
composed of both high and low achieving students while other schools focused only on students who were most in need of the TALKS program.
For the current evaluation study, TALKS students were asked for their perceptions of TALKS, its impact on their lives,
and any changes they might recommend. Overall, students were positive about the TALKS program and mentioned the positive impact it had on their lives. Some TALKS students felt it helped their grades and helped them better
regulate their emotions and behavior. Other TALKS students stated that the TALKS program made little or no
difference in their lives.
Teachers had limited knowledge of what occurs in a TALKS session and expressed an interest in observing at least one session. Teachers were positive about the effect of the TALKS program, but cited the difficulty of determining
the effects of the TALKS program versus the impact of other activities in the school. Principals and instructional coaches were also positive about the TALKS program. Their knowledge of what occurs during a TALKS session was limited to what students mentioned in classes, but generally, these administrators believed TALKS had a positive impact on students. They mentioned that TALKS students still get into trouble at school, but
the TALKS mentors provided a good role model for students.
TALKS students and their classmates completed the TALKS Survey during the fall of 2011 and again in the spring of
2012 (pretest and posttest). Additionally, student achievement scores were gathered for all students in the study. A quasi-experimental evaluation design was used to address four evaluation questions. Results indicated that the TALKS Survey was significantly correlated with student achievement as measured by state-mandated achievement tests. The four TALKS Survey factors attaining significance in this analysis included a positive self-perception, a negative self-perception (inversely related), calm response during conflict, and interaction with authority figures. Two of these factors were also significantly predictive of future academic achievement; Behavior (interaction with authority figures) and Negative self-perception (inversely related).
The next evaluation question focused on differences between TALKS students and their classmates.
Results indicated that TALKS students began the year with lower TCAP scores in all four subject areas than did their classmates. They also had significantly lower perceptions of their behavior than did their classmates. These results
confirmed principal and instructional coach statements regarding their selection criteria for TALKS students.
Finally, comparisons of the TCAP scores of TALKS students with their precision-matched peers indicated significantly higher Reading/Language Arts scores for TALKS students, but no significant differences for the remaining TCAP tests. Given that the TALKS students began the year with lower TCAP scores, it is important to note that the TALKS students made sufficient gains to become essentially equivalent to their precision-matched peers in three TCAP subject areas and to score significantly higher than their peers in Reading/Language Arts.
These findings are suggestive that the TALKS program may have had a positive impact on the academic achievement of TALKS students as well as on their self-perceptions. Given that the TALKS program is held only once per week for approximately 30-45 minutes, it is possible that increased exposure to the TALKS mentors
(e.g., increased number of sessions or increased length of sessions with TALKS mentors) may provide even more measurable gains in academic achievement and self-perception for TALKS students. It is recommended that the TALKS program continue in Jackson-Madison County schools and the number of TALKS sessions for students should be increased to at least two sessions per week or additional time be allocated for each TALKS
session. Additional consideration should be given to expanding the TALKS program to additional schools and grades.
A complete copy of the 2012 TALKS Mentoring Evaluation Study and Validation Report for JMCSS may be obtained from Thomas J Turner Program Director.
Executive Summary
The TALKS program was implemented in four schools in Jackson-Madison County Schools (JMCSS) during the 2011-2012 academic year. Three middle schools (Rose Hill, North Parkway, and West Middle) and one elementary school (Lincoln) implemented the program. A total of 46 students in grades three through eight participated in TALKS, but not all TALKS students remained in the program the entire year.
The TALKS program consisted of TALKS mentors meeting with TALKS students in a separate space within the school for a total of 30-45 minutes each week. Male TALKS mentors met with male TALKS students and female TALKS mentors met with female TALKS students. The TALKS program included students reading a TALKS-supplied book, discussion of the book, and further discussions about behavior in school, academic achievement, and other topics necessary for success in school.
Students were chosen to participate in the TALKS program by school administrators and/or faculty members.
Students were chosen based on their grades, behavior, and personal life situation. Some TALKS groups were
composed of both high and low achieving students while other schools focused only on students who were most in need of the TALKS program.
For the current evaluation study, TALKS students were asked for their perceptions of TALKS, its impact on their lives,
and any changes they might recommend. Overall, students were positive about the TALKS program and mentioned the positive impact it had on their lives. Some TALKS students felt it helped their grades and helped them better
regulate their emotions and behavior. Other TALKS students stated that the TALKS program made little or no
difference in their lives.
Teachers had limited knowledge of what occurs in a TALKS session and expressed an interest in observing at least one session. Teachers were positive about the effect of the TALKS program, but cited the difficulty of determining
the effects of the TALKS program versus the impact of other activities in the school. Principals and instructional coaches were also positive about the TALKS program. Their knowledge of what occurs during a TALKS session was limited to what students mentioned in classes, but generally, these administrators believed TALKS had a positive impact on students. They mentioned that TALKS students still get into trouble at school, but
the TALKS mentors provided a good role model for students.
TALKS students and their classmates completed the TALKS Survey during the fall of 2011 and again in the spring of
2012 (pretest and posttest). Additionally, student achievement scores were gathered for all students in the study. A quasi-experimental evaluation design was used to address four evaluation questions. Results indicated that the TALKS Survey was significantly correlated with student achievement as measured by state-mandated achievement tests. The four TALKS Survey factors attaining significance in this analysis included a positive self-perception, a negative self-perception (inversely related), calm response during conflict, and interaction with authority figures. Two of these factors were also significantly predictive of future academic achievement; Behavior (interaction with authority figures) and Negative self-perception (inversely related).
The next evaluation question focused on differences between TALKS students and their classmates.
Results indicated that TALKS students began the year with lower TCAP scores in all four subject areas than did their classmates. They also had significantly lower perceptions of their behavior than did their classmates. These results
confirmed principal and instructional coach statements regarding their selection criteria for TALKS students.
Finally, comparisons of the TCAP scores of TALKS students with their precision-matched peers indicated significantly higher Reading/Language Arts scores for TALKS students, but no significant differences for the remaining TCAP tests. Given that the TALKS students began the year with lower TCAP scores, it is important to note that the TALKS students made sufficient gains to become essentially equivalent to their precision-matched peers in three TCAP subject areas and to score significantly higher than their peers in Reading/Language Arts.
These findings are suggestive that the TALKS program may have had a positive impact on the academic achievement of TALKS students as well as on their self-perceptions. Given that the TALKS program is held only once per week for approximately 30-45 minutes, it is possible that increased exposure to the TALKS mentors
(e.g., increased number of sessions or increased length of sessions with TALKS mentors) may provide even more measurable gains in academic achievement and self-perception for TALKS students. It is recommended that the TALKS program continue in Jackson-Madison County schools and the number of TALKS sessions for students should be increased to at least two sessions per week or additional time be allocated for each TALKS
session. Additional consideration should be given to expanding the TALKS program to additional schools and grades.
A complete copy of the 2012 TALKS Mentoring Evaluation Study and Validation Report for JMCSS may be obtained from Thomas J Turner Program Director.