Seeking Feedback on High-quality Career & Tech Ed. Framework Pilot

The Association for Career and Technical Education’s (ACTE) High-Quality CTE Program of Study Framework pilot was just released, as part of ACTE’s High-quality CTE Initiative. This phase of the initiative is a pilot test of the Framework’s validity and utility when used in program self-evaluation, and needs those working across CTE to participate. 

Who can participate?

Any CTE educator, administrator or other significant stakeholder can participate in the pilot by choosing one of the CTE Programs of Study they work with to evaluate. For example, a technology education teacher might choose to evaluate their CISCO networking academy program of study, a community college administrator might choose to evaluate their automotive technology program of study, or the CTE director in a district might choose to evaluate their district-wide health sciences program of study. It is important that only a single program of study be evaluated using the instrument during the pilot. In many cases, it might be appropriate for multiple stakeholders to provide input into completing the evaluation, but we need one individual to be our point of contact and be responsible for submitting the evaluation during the pilot process. Please do not consider only your “highest-flying” programs, as a sample of only the best programs in the country will make it difficult for us to determine the validity of the instrument.

What is required for participation?

Pilot volunteers will be asked to engage in a two-step process.

First, the Framework evaluation instrument will be completed as it relates to the chosen program of study. This instrument asks the respondent to rate their chosen program on the 102 separate indicators included under the 12 elements of the High-quality CTE Program of Study Framework. A scale of zero to three is provided, with zero representing the absence of that criterion in the program, and three representing substantial achievement of the criterion throughout the entire program of study.

Once the evaluation instrument has been completed and submitted, participants will be asked to complete a short feedback form about the experience and their thoughts on the usefulness of the instrument and evaluation process. In addition, participants must share program-level Perkins performance indicator data (in percentage form) for the program in question. Before you begin the first step of the pilot process, please make sure you have this data available for the most recent year completed or can get it from your state or district. Keep in mind that publicly reported Perkins performance data is at the district or institution (for postsecondary) level, and so must be disaggregated by program for it to be appropriate for this pilot project. If you don’t have this data or aren’t sure how to get it, please contact us before beginning and we can help provide direction!

Both the evaluation instrument and feedback form will be provided electronically through SurveyMonkey, and a hard copy of the evaluation instrument will be made available as well in order to facilitate thoughtful completion before submitting electronically.

How much time will it take to participate?

We ask participants to set aside enough time to thoughtfully consider the 102 criteria that are included on the evaluation instrument. This may involve asking other colleagues for information, reviewing course materials, or discussing answers as part of a team. It might be appropriate to schedule a meeting with all the program of study’s stakeholders to complete the instrument together, although it is designed in a way that it could be done individually as well. We think between one and two hours should be enough time to complete the evaluation instrument, although larger groups with more discussion might need more time, or it could take longer if the individual completing the form is not familiar with all aspects of the program and needs to gather a significant amount of the information required. The feedback form is a much shorter survey and should be able to be completed in less than 10 minutes.

How will information collected during the pilot be used?

All identifiable information collected during the pilot will be kept confidential, including program performance data and the names of individuals and programs participating. Results will only be released in aggregate form so that information is not personally identifiable. Data obtained from the evaluation instrument will be correlated with Perkins performance data to determine the instrument’s validity as a measure of program quality, and data from the feedback questionnaire will be used to address questions about the instrument’s utility. A few participants will be selected after this part of the pilot to engage in in-depth interviews about their experiences, and this data will supplement what is collected through the survey process. Overall, the pilot will help us determine any necessary changes to the Framework, and provide insights into how ACTE can best use the Framework and make it available to the CTE community going forward.

Why should you participate?

While this pilot is completely voluntary, we hope you will participate! Your participation will help ACTE ensure that this Framework is a valuable tool for program of study self-evaluation and program improvement and will be an important contribution to the field of CTE. In addition, once you have submitted your pilot materials, we will provide you with an individual score report that can be used in your own program improvement efforts, as well as an Excel version of the Evaluation Instrument that can be shared with others or used for other programs you work with. We will also add you a priority list for future information about the project.

Thank you very much for considering participating in the pilot. If you would like to move forward, please email ACTE Director of Public Policy Alisha Hyslop at ahyslop@acteonline.org. You will then be provided with the links to the survey documents and the hard copy of the evaluation instrument. We hope to have all data collected by the end of January.