Skip Navigation LinksHome » Georgia Educator Preparation Reform » Preparation Program Effectiveness Measures (PPEMs)

Preparation Program Effectiveness Measures (PPEMs)

Purpose

Beginning in academic year 2015-16, the Georgia Professional Standards Commission (GaPSC) will use Preparation Program Effectiveness Measures (PPEM) to evaluate educator preparation programs. Specific measures included in Teacher Preparation Program Effectiveness Measures (TPPEM) and Leader Preparation Program Effectiveness Measures (LPPEM) are designed to provide data on program quality, as well as program impact on teaching and learning in P-12 schools. Use of PPEM data by the GaPSC and other state agencies, as well as by education program providers (EPPs), will accomplish several purposes:
  • Hold EPPs accountable to high program standards;
  • Apply a consistent set of state-determined effectiveness measures across all teacher and leader preparation programs;
  • Improve the effectiveness of teacher and leader preparation programs;
  • Inform the citizens of Georgia about preparation program quality; and
  • Improve teaching and learning in P-12 schools.
The new program evaluation system will be piloted for two years before it becomes consequential. Please see the timeline on page 2 for additional information.

This document is intended to provide an overview of the PPEM measures, as well as the GaPSC plans for implementing the new program evaluation system. As it will be a dynamic document, readers are encouraged to frequently check the website for updates. Revision dates will be included in the footer.

In addition to links to other useful resources, you will find included in this document:
  • The PPEM Implementation Timeline
  • A graphical illustration of the TPPEM and LPPEM measures with percent weightings of each; and
  • Descriptions of each TPPEM and LPPEM measure (data elements, scales, status, collection time/window, reporting period).

Timeline

Academic Year Phase of Work Description
2012-13 Planning and Development Policies, implementation and evaluation plans, timelines, systems, and procedures necessary to pilot and implement will be developed.
2013-14 Pilot Phase I A limited set of PPEM data from a sampling of traditional and non-traditional education program providers (EPPs) will be used to pilot data collection, analysis, interpretation, and reporting procedures. Data will include: GACE content assessment pass rates, program completers, yield, and results of piloted statewide surveys of completers and employers. Limited TEM/LEM data may be available and if so, will be incorporated into the pilot data collection where and when appropriate.

Procedures will be evaluated and refined to inform Pilot Phase II.
2014-15 Pilot Phase II All available data from all program providers will be used to for a comprehensive pilot of data collection, analysis, interpretation, and reporting procedures. In addition to the data piloted in Phase I, limited edTPA data and limited TEM/LEM data may be available and if so, will be incorporated.

Procedures, as well as the measures themselves, will be evaluated and refined to inform full implementation.
2015-16 and beyond Full Implementation PPEMs will be fully implemented for all EPPs. All PPEM components will be collected, analyzed, and reported to EPPs, and shared with all stakeholder agencies. Limited data may be shared with the public in the form of a dashboard report, with carefully worded explanations regarding each measure and the fact that this is the first year of full implementation.

PPEM data will be incorporated into the GaPSC approval process.

A regular/ongoing cycle of implementation evaluation will begin for the purpose of continued improvement of PPEM data collection, analysis, interpretation, and reporting procedures. Adjustments may be made in measures and/or procedures in 2016-17 and beyond.

Measures


Teacher Preparation Program Effectiveness Measures (TPPEMs)

TPPEM Data Elements Scale Status Collected Reported
Teacher Effectiveness Measure (TEM) 1-4 pt. scale with 4 being the highest rating, based on the 4 levels reported by DOE. Lagging data Dynamic Summer (at the end of each academic year) Annually
Success at Induction 1-5 scale, based on ranges of percentages. State retention and attrition data will be used to determine a benchmark (3). Lagging data Static At the end of the third year after cohort completion year In the third report after cohort completion year
Content Knowledge GACE Content Assessment 2-pt. scale: 2 points for passing at professional, 1 point for passing at induction, 0 points for failing. Static At the end of the *reporting year (Aug. 31) Annually
edTPA TBD Preliminary Plans: 1-5 pt. scale (The summative score of the 15 rubrics will be reported on a scale of 15 to 75). Static Non-Traditional: second semester of employment Traditional: At program completion Annually
**GaPSC Annual Program Performance Data Completion Rates Completion rates will be reported on a 1-5 scale with 5 being the highest score. The state average completion rate will be used to determine a benchmark (3).

Retention data will be reported on a 1-5 scale with 5 being the highest score. State retention and attrition data will be used to determine a benchmark (3).

Yield data will be reported on a 1-5 scale with 5 being the highest score. State employment data will be used to determine a benchmark (3).

Inductee Survey (formerly named the Completer Survey) data will be reported on a 1-5 scale with 5 being the highest score.

Employer Survey data will be reported on a 1-5 scale with 5 being the highest score.
Static Non-Traditional: second semester of employment Traditional: At program completion Annually
Retention Rates Dynamic At the beginning of the second and third years of employment (fall CPI data) In the second and third reports after cohort completion year
Yield Dynamic Annually beginning in the first year after cohort completion year Annually
***Inductee Survey Static At the end of the first year of employment Annually
Employer Survey Static At the end of the first year of employment Annually
* The GaPSC reporting year will change from the current fiscal year (July 1 – June 30), to (September 1 – August 31). As the first two GACE assessment attempts are counted for PPEM, the revised reporting year enables EPPs to grant eligibility to test later in the program while giving candidates more opportunities to test for the second time if needed during the summer after program completion. (All candidates must attempt the GACE content assessment at least once prior to program completion). In addition, the new reporting year gives EPP candidates additional GACE assessment opportunities to be counted toward the “best attempt” score required by Title II. The new reporting year will become effective 2014-2015.

** Data elements in the GaPSC Annual Program Performance Data measure will be weighted as described below:
  • 20% - Completion Rates
  • 10% - Retention Rates
  • 10% - Yield
  • 30% - Completer Survey
  • 30% - Employer Survey
*** Two surveys of program graduates/completers will be administered:
  1. First, a survey of recent graduates will be conducted to gather perceptions of preparation, as well as intent data. In addition to survey items related to preparation, intent items might include questions such as:
    1. a. Do you intend to seek employment in a Georgia public school or public charter school?
    2. b. Do you intend to seek employment in a private school, other education organization, or in another industry (not education)?
    3. c. If you intend to seek employment in a Georgia school, to how many school systems have you applied?
    4. d. Have you applied for teaching jobs in the field that you were prepared?
    5. e. Have you applied for teaching jobs in a field you added by passing the GACE content assessment?
    Completion of this survey will be tied to the certification process so that it must be submitted before a certificate will be issued.

    While data resulting from this survey will not be included in PPEMs, these data will be useful in explaining yield and will be included in PAAR, where the EPP will be able to use them to respond to positive or negative trends.f

  2. A second survey will be conducted after completers are hired in a Georgia public/or a Georgia public charter school. Formerly referred to as the Completer Survey, the Inductee Survey will be administered only to those completers employed in a Georgia public /charter school where the state-mandated evaluation system is used.

Leader Preparation Program Effectiveness Measures (LPPEMs)

LPPEM Data Elements Scale Status Collected Reported
Leader Effectiveness Measure (LEM) 1-4 pt. scale with 4 being the highest rating, based on the 4 levels reported by DOE. Lagging data Dynamic Summer (at the end of each academic year) Annually
Content Knowledge GACE Content Assessment TBD Static At the end of the ****reporting year (Aug. 31) Annually
*****GaPSC Annual Program Performance Data Completion Rates Completion rates will be reported on a 1-5 scale with 5 being the highest score. The state average completion rate will be used to determine a benchmark (3).

Retention data will be reported on a 1-5 scale with 5 being the highest score. State retention and attrition data will be used to determine a benchmark (3).

Yield data will be reported on a 1-5 scale with 5 being the highest score. State employment data will be used to determine a benchmark (3).

Inductee Survey (formerly named the Completer Survey) data will be reported on a 1-5 scale with 5 being the highest score.

Employer Survey data will be reported on a 1-5 scale with 5 being the highest score.
Static At program completion Annually
Retention Rates Dynamic At the beginning of the second and third years of employment (fall CPI data) In the second and third reports after cohort completion year
Yield Dynamic Annually beginning in the first year after cohort completion year im Annually
******Inductee Survey Static At the end of the first year of employment Annually
Employer Survey Static At the end of the first year of employment Annually
**** The GaPSC reporting year will change from the current fiscal year (July 1 – June 30), to (September 1 – August 31). As the first two GACE assessment attempts are counted for PPEM, the revised reporting year enables EPPs to grant eligibility to test later in the program while giving candidates more opportunities to test for the second time if needed during the summer after program completion. (All candidates must attempt the GACE content assessment at least once prior to program completion). In addition, the new reporting year gives EPP candidates additional GACE assessment opportunities to be counted toward the “best attempt” score required by Title II. The new reporting year will become effective 2014-2015.

***** Data elements in the GaPSC Annual Program Performance Data measure will be weighted as described below:
  • 20% - Completion Rates
  • 10% - Retention Rates
  • 10% - Yield
  • 30% - Inductee Survey
  • 30% - Employer Survey
****** Two surveys of program graduates/completers will be administered:
  1. 1. First, a survey of recent graduates will be conducted to gather perceptions of preparation, as well as intent data. In addition to survey items related to preparation, intent items might include questions such as:
    1. a. Are you currently employed as an assistant principal or principal in a Georgia public school or public charter school? If NO, are you employed as a teacher or other school professional in a Georgia public school or public charter school?
    2. b. Do you intend to seek employment in a Georgia public school or public charter school?
    3. c. Do you intend to seek employment in a private school, other education organization, or in another industry (not education)?
    4. d. If you intend to seek employment in a Georgia school, to how many school systems have you applied?
    5. e. Have you applied for assistant principal/principal jobs?
  2. 2. A second survey will be conducted after completers are hired in a Georgia public/or a Georgia public charter school. Formerly referred to as the Completer Survey, the Inductee Survey will be administered only to those completers employed in a Georgia public /charter school where the state-mandated evaluation system is used.

Links to Related Documents

PPEM FAQs

Georgia Preparation Program Effectiveness Measures Framework (Final Report of PPEM Task Force, July 2012)