College of Education and Human Development - George Mason University

2021 CAEP Annual Reporting

Reporting Measure  
1. Impact on P-12 Learning and Development
2. Indicators of Teaching Effectiveness

AY19-20 presented a unique challenge for data collection in regards to completer impact and effectiveness. In March 2020, to reduce the spread of COVID, school divisions around the Commonwealth switched from face-to-face to virtual classrooms. This dramatic switch required flexibility and responsiveness, and raised new pedagogical and technological challenges. In the past, the EPP had reached out to local school divisions to collect anonymized and aggregated Teacher Performance Evaluation data on EPP completers. The EPP did not feel it was appropriate to request additional actions of the school divisions during this time.

On the Virginia Department of Education website, an FAQ page was created (VDOE: School Closure FAQ (virginia.gov)) to provide guidance on the many challenges that COVID created. In regards to the Teacher Performance Evaluation, “Pursuant to the Code of Virginia, evaluation of instructional personnel is the responsibility of the local school division.” The EPP reached out to local school divisions to inquire about the status of AY19-20 Teacher Performance Evaluations. In the context of this pandemic, evaluations were conducted with additional supports and modifications. School divisions provided coaching opportunities and used reflective documentation to enhance the AY19-20 review. The EPP expects to re-establish the data sharing in the future, with the hope that its collaborative work within VEAC will streamline the request and sharing for all Virginia EPPs and their partner school divisions.

3. Satisfaction of Employers and Employment Milestones
4. Completer Satisfaction

As noted in previous Annual Reports, in the Commonwealth of Virginia, there is no clear mechanism for collecting and sharing data across the state education agency, EPPs, and P-12 school divisions. Stakeholders in Virginia consistently identify CAEP Standard 4: Program Impact as the most challenging standard to meet based on current data practices.

Also as previously reported, several members of the 36 Virginia EPPs formed the Virginia Education Assessment Collaborative (VEAC) to create a state-wide system for data collection. In particular, to build a common set of assessment measures, tools, and activities that all Virginia EPPs may use in response to CAEP requirements.

One of the first developments of VEAC was the creation and implementation of completer and employer surveys in response to CAEP Standard 4.4 and 4.3, respectively (https://projectveac.org/veac-documents). VEAC worked with 17 piloting EPPs to create these state-wide surveys. This group also created the VEAC Instruction Guide which outlined how these surveys met sufficiency on the CAEP Evaluation Framework for EPP-Created Assessments, and how these surveys assessed both the InTASC Standards and the Virginia Uniform Performance Standards (see VEAC Survey Instrument Guide at https://projectveac.org/veac-documents).

In AY 19-20, VEAC launched the pilot completer survey and employer surveys. For this pilot, a total of 2,375 completers from these 17 participating EPPs were contacted, with 832 full and partial responses. Based on the completer survey responses, a total of 1,902 employers were contacted, 488 full and partial responses. Based on this state-wide collection, VEAC then provided individual EPP and benchmarking results on completer and employer satisfaction to each EPP in the individualized reports. All EPPs were also provided access to their EPP raw data through a password-protected website.

Completer Satisfaction Data
As noted in the Completer Survey VEAC Report 2019-2020, of “…the 682 VEAC pilot completer survey respondents, 96.63% were either somewhat or extremely satisfied with their preparation from their institution of higher education.” To find the average overall satisfaction, responses were coded from 1 to 5, higher values indicating more satisfaction, and lower values indicating more dissatisfaction. The average rated satisfaction by program completers for all EPPs in 2019-2020 was 4.58. The report also broke down overall satisfaction averages by EPP size, private/public, endorsement category, and undergraduate/graduate programs on each survey item. In each individual EPP report, VEAC provided tables that included average overall satisfaction of the individual EPP and all EPPs for each survey item.

As noted in the Completer Survey VEAC Report 2019-2020: George Mason University, the average overall satisfaction of Mason completers matched the average overall satisfaction at 4.58. Below are the results by licensure area and overall as frequencies. Only program data that met the data threshold is presented. Note that the question for completers was “Overall, how satisfied are you with your preparation from [Field-Institution]?” and this was on a 5 point scale (extremely dissatisfied to extremely satisfied).

COMPLETER SURVEY OVERALL – 15 extremely satisfied, 7 somewhat satisfied, and 1 neither
Completer Survey Elementary Education – 4 extremely satisfied, 3 somewhat satisfied
Completer Survey Secondary Education – 3 extremely satisfied, 1 somewhat satisfied

For George Mason University, the EPP had a 14% response rate on the VEAC Completer Survey based on the total number of contacts submitted to VEAC minus the number of failed/bounced emails. Note that VEAC and George Mason University worked to correct the number of failed/bounced emails via contact list revisions, but the rate of response remains low.

Employer Satisfaction Data
As noted in the Employer Survey VEAC Report 2019-2020, “…[o]f the 457 VEAC pilot employer survey respondents, 90.4% were rated as either Fully Ready or Mostly Ready to impact student learning...” To find the average overall readiness, responses were coded from 1 to 5, higher values indicating more satisfaction, and lower values indicating more dissatisfaction. The average rated readiness by program completers for all EPPs in 2019-2020 was 4.49. The report also broke down overall satisfaction averages by EPP size, private/public, endorsement category, and undergraduate/graduate programs and completer year for each survey item. In each individual EPP report, VEAC provided tables that included average overall satisfaction of the individual EPP and all EPPs for each survey item.

As noted in the Completer Survey VEAC Report 2019-2020: George Mason University, the average overall satisfaction of Mason completers scored lower than the average overall satisfaction at 4.26. The question for employers was “Based on your experiences with this teacher, what best describes the extent to which they were ready to meet the needs of students in your school?” Respondent employers could respond [5] “Fully ready (able to have an immediate impact on student learning), [4] Mostly ready (able to successfully meet the needs of most students,” [3] “Moderately ready (in order to be successful, needed additional training, support, and coaching beyond what is typically provided to beginning teachers),” [2] “Minimally ready (limited success meeting the needs of students and improving outcomes even with additional supports)” or [1] “Not ready (unable to meet the needs of students even with additional supports).” Only data that meets the data threshold is presented.

EMPLOYER SURVEY OVERALL – 34 fully ready, 19 mostly ready, 8 moderately ready, 3 minimally ready, 1 not ready
Employer Survey Early Childhood – 4 mostly ready, 2 moderately ready, 1 minimally ready
Employer Survey Elementary – 17 fully ready, 8 mostly ready, 2 moderately ready, 1 minimally ready, 1 not ready
Employer General Curriculum – 1 mostly ready, 2 moderately ready, 1 minimally ready
Employer Music Ed – 3 fully ready, 1 mostly ready, 1 moderately ready
Employer Secondary Ed – 5 fully, 2 mostly, 1 moderately

These reports were widely shared with the faculty to review and inform their program goals and improvement.

In previous years, the advanced licensure faculty have expressed concern about survey distribution to advanced license holders who may choose to remain in their initial licensure employment, and to their employers. There was a concern that those surveyed would not be able to clearly differentiate between skills gained in the initial licensure program as opposed to the advanced licensure program. Quantitative data was collected, but did not satisfy the need for relevant and actionable data that may be more readily available from a qualitative approach.

Based on these findings, the EPP advanced licensure programs instead identified open-ended qualitative questions that were shared with their advisory councils. The responses of the advisory councils – made up of stakeholders such as completers and employers – will be assessed for applicable use.

In AY19-20, the advanced licensure programs were arranging their advisory council meetings as pandemic-related closings were initiated. Some were able to shift online; others postponed until AY20-21.

VEAC is currently developing an Advanced Survey which will be potentially piloted in spring 2022. The EPP plans to work with its EPP colleagues in VEAC to participate in this pilot, or the subsequent final process.

5. Graduation Rates Due to the structure of the EPP programs, the most accurate measure is the Title II Licensure completer rate as described below in Outcome Measure #6 “Licensure rate.”
See 6. Ability of Completers to Meet Licensure
6. Ability of Completers to Meet Licensure

Data show:
Number of completers in programs leading to initial teacher certification or licensure: 250

Number of completers in advanced programs or programs leading to a degree, endorsement, or some other credential that prepares the holder to serve in P-12 schools (Do not include those completers counted above.): 260

7. Ability of Completers to be hired Initial and Advanced Programs: Mason’s One Year Out Career Survey collects information about employment status and further education up to one year after obtaining a degree from Mason. This survey is administered to degree recipients once per year in early summer. Results from the 2019-20 survey show an overall 20% return rate. Out of those who responded under the Curriculum and Instruction degree (the majority of our licensure programs), 93% were employed in their field; 6% were not employed but seeking employment; and one percent was not employed and not seeking employment. Three completers of Health and Physical Ed BSED responded that they were employed. For completers of the Education Leadership, or Mathematics Education Leadership program, 97% reported that they were employed in their field, while the remaining 3% were not seeking employment. These results are comparable to the results for completers of the Mason graduate Nursing program, which showed an employment rate of 95%, with 5% seeking employment. The ongoing shortage for educator professionals in Virginia leads to many opportunities for employment of educators in Virginia (http://www.doe.virginia.gov/teaching/workforce_data/). It is important also to note that although the EPP is provided with limited employment data from VDOE, we make every effort to gather completer information.
8. Student Loan Default Rates and Other Consumer Information Initial and Advanced Programs: Link to Student Consumer Information:

Average GPA for Program Completers

Traditional Program Group Degree Level GPA
George Mason University (017) All program completers, 2017-18 Bachelor’s 3.52
George Mason University (017) All program completers, 2017-18 Master’s 3.96
George Mason University (017) All program completers, 2016-17 Bachelor’s 3.48
George Mason University (017) All program completers, 2016-17 Master’s 3.97
George Mason University (017) All program completers, 2015-16 Bachelor’s 3.44
George Mason University (017) All program completers, 2015-16 Master’s 3.97

Institutional Licensure Pass Rates

Traditional Program Group Number
taking tests
Number
passing tests
Pass rate (%)
George Mason University (017) All program completers, 2017-18 334 334 100
George Mason University (017) All program completers, 2016-17 337 337 100
George Mason University (017) All program completers, 2015-16 375 375 100

Institutional Report Card