PROJECTS

ASSESSING AND DEVELOPING WORKPLACE ESSENTIAL SKILLS WITH ESAT – PHASE I & II    

May 2021

 

      

After being awarded a seed grant from the Future Skills Centre, an initial study into our Employability Skills Assessment Tool (ESAT) conducted by Futureworx and Blueprint allowed us to better understand how ESAT is used, how it is perceived by users, and the types of outcomes that would be expected and could feasibly be measured. The initial research, Phase 1, found that:  

  • Staff perceive positive outcomes for participants, especially related to improvements in self-awareness of social and emotional skills (S&E skills), S&E skills, and well-being.  
  • Staff identify several implementation factors as key to success: the importance of having staff buy-in, resourcing additional capacity for staff to dedicate toward ESAT use, including more than one perspective in the staff-assessment process, integrating S&E skills content across programming, and debriefing results with participants. 
  • There is ample variability in ESAT use and implementation, with some programs following Futureworx’ recommended approach, but many others that do not. 
  • Staff agree on many of ESAT’s core strengths for their participants, including that it is easy to use, helps staff establish a common language around employability skills, and eases tension during potentially difficult conversations with participants. 

 Based on the results of Phase I, The Future Skills Centre provided funding to conduct Phase II. 

 LEARNING GOALS: The learning goals of Phase II are: 

 Clearly define the minimum implementation conditions to maximize ESAT’s effectiveness 

  • Strengthen the current version of ESAT by applying some of staff’s suggestions from Phase I, and perform some light-touch validity testing   
  • Examine how ESAT is implemented across partner organizations to understand readiness factors, organizational barriers, and whether/why adaptations are made to implementation  
  • Examine which target populations are being reached and whether participants are satisfied 
  • Gather externally generated evidence for post-program outcomes (e.g., whether participants have better labour market outcomes) to compare these with ESAT scores; and 
  • Understand whether the ESAT is a valid measure of S&E skills 
  • Contribute to the evidence base on how to assess S&E skills, and how to validate a S&E skills assessment tool within a workforce development context 

TIME FRAME: Completion Sept 2023 

ACTIVITIES: The project activities include: 

Part A: Prepare for evaluation 

The approach to evaluation proposed for Phase II has several components and will therefore involve several preparation activities before evaluation activities can begin: 

  • Clearly define ideal conditions for implementation: Building on the ESAT User Guide, Futureworx will clearly outline the minimum implementation conditions for PDPs. 
  • Partner selection and onboarding: Futureworx will recruit and onboard program delivery partners (PDP) that are interested in taking part in the research and that meet all project requirements.  
  • Support refinements to the evaluation plan: Once partners are onboarded, Futureworx and Blueprint will work with partners to refine and adapt the evaluation plan laid out below.  

Part B: Strengthen ESAT 

Before ESAT is used within this research project, Futureworx will work with Blueprint to apply lessons learned from Phase I alongside some preliminary light-touch validity testing to improve self-assessment questions and exemplar descriptions. This will include the following activities:  

  • Create a Theory of Change (TOC): Ensure that the theories and assumptions underlying ESAT align with findings from Phase I and can be used to strengthen the tool.  
  • Support preliminary light-touch validity testing: Provide Blueprint with the ESAT assessment data required to test the correlation between individual items for each skill. 
  • Revise content of ESAT self-assessment questions and exemplar behaviours: Incorporate recommendations from Blueprint (based on validity testing and findings from Phase I) to revise the ESAT tool. 

Part C: Evaluate ESAT 

Three components of the evaluation will happen in tandem over the course of Phase II. 

Part C.1: Implementation supports and evaluation  

Futureworx and Blueprint will engage an external consultant with expertise in implementation science to support partners in tackling the delicate balance between delivering the program according to implementation requirements (i.e., fidelity of the model) and adaptations that are required for their context. Activities to be led by the implementation consultant and supported by both Futureworx and Blueprint will include: 

  • Readiness Assessment: Identifying and assessing factors that affect a PDP’s readiness for implementation  
  • Barriers and Facilitators Assessment: Assessing PDP’s perceptions of potential challenges and success factors that could be proactively mitigated or leveraged to improve evaluation results  
  • Tracking Fidelity and Adaptations: Understanding how ESAT is implemented across sites including whether and why adaptations are made 

Part C.2: Outcomes Evaluation  

This part of the evaluation will help us understand what outcomes are achieved by participants of each ESAT-using program and whether there is a relationship between ESAT scores and outcomes. Futureworx will lead or support Blueprint with the following activities: 

  • Define outcomes of interest: Identify the key participant outcomes to be tracked during the evaluation (e.g., improved skills, self-awareness, employment outcomes, education outcomes).  
  • Review data collection strategy and tools: Ensure that research instruments are accurately tailored to PDPs by providing feedback to Blueprint on surveys, interview protocols, etc.  
  • Support deployment of data collection strategy: Liaise between delivery partners and Blueprint to support the roll out and sequencing of all data collection.  

Part C.3: Process Evaluation   

The final component of the evaluation will help understand ESAT users’ experience with the tool, focusing primarily on the perceptions of participants and employers. Futureworx will support Blueprint with the following activities: 

  • Support design and deployment of a data collection strategy: Liaise between delivery partners and Blueprint to support interviews with partner staff, participants, and employers.  

Part D: Validity testing 

The final component of Phase II will involve a second, more robust round of validity testing, aiming to understand whether ESAT provides a valid measure of participants’ employability skills. Data collected during the evaluation will provide information on the necessary outcomes to make inferences about validity. Futureworx will support Blueprint as required with the following activities: 

  • Concurrent validity testing: Testing the extent to which ESAT is related to an existing, previously validated measure.  
  • Predictive validity testing: Testing the degree to which ESAT scores accurately predict desired outcomes.  
  • Inter-item reliability testing: Testing whether each revised question is aligned with the skill it seeks to measure.  

 PROGRAM DELIVERY PARTNERS: Futureworx is inviting existing and new ESAT users to participate in this project as program delivery partners (PDPs).  Funding supports and incentives will be provided to PDPs. Existing ESAT site managers can apply in ESAT. New users can apply by contacting Futureworx. 

CONTACT: For more information about, or to express interest in participating in this project, please Contact Paul Brinkhurst, Futureworx, 587-226-7800, paulb@futureworx.ca. 

______________________________________________________________________________________________________________________________________________________

Futureworx is proud to release its report – “Building a PAN CANADIAN SOFT SKILLS FRAMEWORK”

This project was funded to explore the need for, and how best to create, a pan-Canadian soft skills framework to support the development of a productive,
adaptable, globally competitive workforce capable of meeting the rapidly changing needs of work and the workplace.

Funded by the Government of Canada’s Adult Learning, Literacy and Essential Skills Program.

REPORT: Soft Skills Framework Report – PDF (5MB)

We welcome your feedback and questions – simply CONTACT US.

 


 

INTEGRATING SOFT SKILLS INTO THE ESSENTIAL SKILLS FRAMEWORK – BUILDING A NATIONAL CONSENSUS



OBJECTIVE: The objectives of this project are:

  • To develop a plan to create a common assessment and development approach for ‘soft’ skills in Canada;
  • To propose their integration into the existing essential skills framework;
  • To identify and develop common terminology.

These objectives will be achieved by:

  • Identifying existing ‘soft’ skills taxonomies, frameworks, efforts, assessments and resources being used and/or supported; and
  • Developing a plan for moving forward on a collaborative impact effort to expand the essential skills framework to include ‘soft’ skills.

TIME FRAME: Completion Nov 2018

ACTIVITIES: The project has four main activities:

  • Conduct a literature review of work on ‘soft’ skills across Canada to capture current thinking and efforts on ‘soft’ skill development;
  • Develop and implement a survey targeting a broad set of stakeholders across Canada (e.g. employers, service providers, educators, trainers, governments) to understand current practices, language, and views on ‘soft’ skills development;
  • Select a representative group of stakeholders (15-20), to be involved in a planning session to explore possibilities for the creation of a Canadian “soft skills” framework and its integration with the existing essential skills framework (stakeholders to be trained in consensus-based decision making and Collective Impact as part of the process); and
  • Write a report that summarizes the results from the project and providing recommendations for  reaching consensus on the development of a Canadian “soft skills” framework.

OUTPUTS: Principal outputs for the project include:

  • A summary of the existing ESDC Competency-based “soft” skills taxonomy;
  • An initial draft set of skills suited for initiating consensus discussions;
  • Recommendations for integrating “soft skills” into the existing essential skills framework;
  • A plan for developing a national consensus on ‘soft’ skills development and assessment;
  • Recommendations on how to render the taxonomy of practical use to front line application; and
  • A final report to ESDC.

FINAL REPORT: Final Report – Sharing Perspectives on Soft Skills Survey 06-11-2018 – PDF (427KB)

CONTACT: For more information about, or to express interest in participating in this project, please Contact Paul Brinkhurst, Futureworx, 587-226-7800, paulb@futureworx.ca.

Nova Scotia Works

Futureworx is now a Nova Scotia Works Employment Services Centre. We help job seekers and employers navigate a range of programs – from career planning, job searches and on-the-job development to recruitment, planning and HR support.

Employability Skills Assessment Tool

The Employability Skills Assessment Tool (ESAT) provides a framework and quantitative assessment methodology for the development of nine employability skills: motivation, attitude, accountability, time management, stress management, presentation, teamwork, adaptability and confidence.

8 hours ago

Futureworx
Give a Smile. Get a Smile.Here at CCOA, we are very lucky to have some pretty incredible people come to see us. During Smile Cookie week, Futureworx came to give us a hand! We are so thankful for all of their help and hard work!💙 ... See MoreSee Less
View on Facebook

6 days ago

Futureworx
The participants in our Connex New Glasgow program are enhancing their computer literacy skills this week with Lucas Technology & Analytics.This 11 week program equips participants with the supports, resources, and knowledge to enhance their employability, develop new skills and explore possible career paths.For more information on this program and other offerings, check us out online at futureworx.ca ... See MoreSee Less
View on Facebook

2 weeks ago

Futureworx
Today is World Suicide Prevention Day. This year’s message is “creating hope through action.” Action starts with talking about it. Reach out for help. Let people who are struggling know that they are not alone.If you or someone you know is in immediate need, please contact 911 or the Nova Scotia Mental Health Crisis Line — 1.888.429.8167 #suicideprevention #mentalhealthmatters ... See MoreSee Less
View on Facebook

2 weeks ago

Futureworx
Wishing all of the teachers a great first day back into their classrooms this week. 🍎📚#backtoschool #teaching #learning ... See MoreSee Less
View on Facebook

2 weeks ago

Futureworx
Happy Labour Day Weekend!Please note that all Futureworx locations, including our Nova Scoti Works Centres will be closed on Monday. ... See MoreSee Less
View on Facebook

▪️ Long-term care facilities closed to visitors effectively immediately
▪️ Public schools closed for two weeks following March Break; to be reassessed
▪️ Regulated childcare centres to be closed Mar 17 - April 3 then will be reassessed
▪️ March break camps cancelled
#COVID19NS

October is National Disability Employment Awareness Month and Futureworx (ESAT) is pleased to be one of the keynote speakers for the Distinctability Webinar series on Innovation and Assistive Technology. #innovation https://distinctability.ca/webinar-series/

What is it that a #CDP does? This #careermonth, take a moment to meet a #careerdevelopment professional in your area, and really feel supported as you strive to achieve your #career goals. https://careermonth.ca via @careermonth

First DEAM (Disability Employment Awareness Month) EVENT in Nova Scotia. Thank you @markwafer for speaking to our community partners this morning in Truro. @TEAMWork1997 #Inspire #Educate #InclusionMatters

Load More...

NSW Offices:
Truro - 902-893-6859
Elmsdale - 902-883-7010
Tatamagouche - 902-657-0180

Head Office:
14 Court Street, Suite 305
Truro, NS B2N 3H7

Tel: 902-843-4290
Fax: 902-843-4294

Hours:
Mon - Fri 8:30 am - 4:30 pm