Our Blog RSS

The latest news and occasional commentary about what’s happening at the Foundation and around our great state. 

Rhode Island Foundation Blog


We are proud to be part of the Rhode Island community! Follow our blog for the latest news and occasional commentary about what’s happening at the Foundation and around our great state.

Evaluation guides improvements to new program
By Katie Murray / March 2, 2018 /   Loading Disqus...

In a previous evaluation and learning story, we highlighted Housing First Rhode Island whose findings of impact were supported by an outcomes evaluation led by third-party researchers. In this post we feature a different type of evaluation – formative evaluation – also led by external researchers. It was used by Foster Forward to help refine and improve their youth employment program, Works Wonders.

In 2011, Foster Forward was one of four projects funded nationally through the Children’s Bureau to support older youth as they transition from foster care. Foster Forward’s response was Works Wonders, a 12-week program that provides employment support for young people who are in – or who have recently aged out of – state foster care. In Rhode Island, youth age out of care at age 18, and upwards of 60% of them have never had an employment experience. Works Wonders aims to fill this employment experience gap with tailored career readiness skill building, individualized job coaching, and on-site job training for participants.

The Children’s Bureau required an evaluation as part of the award, which was envisioned as a random control trial (RCT) to compare outcomes between a group of youth that received the full complement of services (treatment group) and youth who did not receive the services in the same timeframe (control group). RCTs are considered the gold standard of evaluation because they enable a comparison between two groups to identify differences that result from a program. RCTs are also complicated and expensive and typically conducted with established programs. Crucial to RCT design are sufficient numbers of participants in the treatment and control groups for meaningful data analysis.

Led by researchers from Rhode Island College and the Columbia School of Social Work, Foster Forward designed the RCT and was able to sufficiently fund it through the course of the project. However, after a year of concerted effort by all parties, Foster Forward realized that they needed to change the evaluation primarily due to the newness of the program and insufficient sampling populations. With approval from the Children’s Bureau, the evaluation was reimagined as a formative evaluation, which is commonly used during the development of new programs. Formative evaluations answer questions such as:

  • What is and is not working?
  • Are the intended services being delivered to intended beneficiaries?
  • Are services being provided in ways that are consistent with known effective models or practices?
  • Are beneficiaries satisfied with the services?

As noted by Kat Keenan, Deputy Director of Foster Forward, this was the right direction for the evaluation because it provided necessary feedback. It was also particularly conducive to the involvement of youth. At every step of the way – from design to implementation, recruitment to evaluation – teens helped make decisions on how the program was structured and implemented. This process provided unique opportunities for participants to build skills in decision making, team work, and communication, even while being paid as consultants. The development, implementation, and feedback processes, core to formative evaluation, were greatly enhanced by this participation and ensured that Works Wonders was grounded and relevant to the needs of the beneficiaries.

While formative, the evaluation still included a focus on outcomes. Employment was key among them. It increased substantially from 38% to 52% among the 180 youth that participated in the program during this time. Foster Forward has continued tracking employment and has maintained a 50% success rate.

 Kazeem, who now lives independently in Bristol, is 19, and is working at Protech Automotive Services in Johnston. He entered the DCYF system at age 13, with foster parents in Bristol, where he went on to graduate from Mount Hope High School. Kazeem got involved with Foster Forward through the Real Connections Mentoring Program in 2014, and then Works Wonders in 2016. 

The evaluation also revealed significant increases in the proportion of youth who stayed engaged in activities such as career exploration and work-readiness skill development following completion of the formal program.

One such pathway is through Foster Forward’s Real Connections program that connects foster youth to an adult mentor. Watch the brief video about Kazeem and his mentor Peter to learn more about how Foster Forward’s programs are making a real difference for young people as they leave foster care.


blog comments powered by Disqus

Featured Post

Read More »


LocationOne Union Station
Providence, RI 02903


(401) 274-4564

E-News Sign Up