Smaller Learning Communities, Career Technical Education (CTE) programming and other approaches to secondary education often overlap with each other and with other efforts to improve student achievement. PW has conducted multiple evaluations of these kinds of school improvement efforts including partnership academies and career pathway approaches for high school students. PW has evaluated larger initiatives intended to bring about broader change at the secondary level such as smaller learning communities, school and community partnerships and institutional connections to workforce initiatives. Within this work, PW considers the broader context in which the evaluation is situated and develops an approach to evaluate particular initiatives within the broader K-12 and postsecondary education systems.

Evaluation of US Department of Education Smaller Learning Communities Supplemental Grant to the Los Angeles Unified School District (LAUSD) and San Bernardino County Superintendent of Schools, Center for the Advancement of Smaller Learning Environments.  PW served as the third party evaluator for seven Cohort VI high schools and three Cohort VIII high schools in LAUSD and 10 Cohort VI in SBCSS. The supplemental grant was awarded to enhance the timely collection and reporting of data on enrollment in postsecondary education, advanced training, or registered apprenticeship program by graduates of schools included in the approved application. Data collected and analyzed for this project was incorporated in the findings in the USDE ALC Annual Performance Report.

Evaluation Planning, Alliance for Education, San Bernardino County Superintendent of Schools. PW assisted in the development of an Evaluation Design for the Alliance for Education, a partnership of business, labor, government and education for a qualified workforce. Services included gathering proposals and grant reports written by the Alliance; convening an Evaluation Group to assess goals and objectives; interviews with stakeholders, partners and staff; and producing and presenting a proposed evaluation strategy to the Evaluation Group. For the Alliance for Education, PW examined STEM and ABLE programming. PW evaluated STEM outcomes by collecting rosters of students participating in STEM programming from schools and matching that information to student outcomes. PW also conducted a follow-up survey of seniors after graduation from selected schools.

Evaluation Planning of the Los Angeles Police Academy Magnet Schools and Police Orientation Preparation Program LAPAMS and POPP. PW has provided technical assistance and guidance in the development of a comprehensive master evaluation plan, which focused on improving the preparation of students for careers in the LAPD, other law enforcement, and related fields for the Irvine Foundation.  PW provided leadership in the development of the evaluation design and methodology and worked with other evaluators to ensure the final written plan was complete and met the highest standards of evaluation design.

Evaluation of Training to Work, Department of Labor (DOL), Homeboy Industries. PW evaluated the planning and implementation 18-month Career Pathways model focused on adults with felony convictions. The program offered case management, mentoring, education and employment services in four pathways—Culinary, Business, Manufacturing, and Construction Trades. PW assisted in the collection of data related to participants, analysis of quarterly reports, and providing information on the integration of the DOL program into the overall organization. Each participant had an Employment Development Plan (EDP) with goals related to training and placement.

Regional Youth Resource Mapping, San Diego Workforce Investment Board. PW in partnership with New Ways to Work, assisted the San Diego Workforce Investment Board in a mapping project to identify the resources and services available for youth across San Diego County and to gather opinions from youth, parents, employers, educators and community organizations about current education and employment services for youth ages 14 to 21. The Regional Youth Resource Mapping Project incorporated five components:  a qualitative report which reflected the collection and analysis of opinions gathered from stakeholders regarding the issues, needs, assets and gaps related to the county’s youth workforce preparation system; a demographic report which provided an analysis of the county’s youth population including trends and forecasts for the future and inventories of various services.

Evaluation, Feasibility Study, Junior Achievement of Upper Midwest, Inc. PW conducted a feasibility study to expand JA Exchange City, serving approximately 10,000 students from 97 schools to a second location. Evaluation activities included interviews and focus groups with stakeholders, development and administration of written and online surveys of stakeholders, observation of the program, observation of volunteer training, analysis of funding streams, analysis of geography and analysis of market capacity.

UNITE-LA, Review of Organizational Accomplishments and Capacity. Under contract to UNITE-LA, an intermediary organization committed to improving the education and workforce development of youth in the Los Angeles area, PW prepared a summary of organizational accomplishments and capacity to be used in leveraging additional support and partnerships. PW evaluated UNITE-LA’s activities and work in the prior 11 years through: a) review of prior evaluation reports; b) analysis of publicly disseminated documents; c) interviews with staff, clients, and partners. Based on these activities, PW documented accomplishments in the areas of advocacy, partnerships, functional expertise, fiscal and program management, and overall effectiveness. Under an additional contract, PW assisted in developing a framework and outline for data measures, collected data from student outcome measures identified by the LA Compact, analyzed data and provided graphs and narratives describing the process of each measure.

Evaluation of California Partnership Academies (CPA) in PUSD. PW assisted the ROP, Adult Education, and High School Academies Office at PUSD to complete the required reporting requirements for continued funding under the Carl D. Perkins Vocational and Technical Education Act and the Partnership Academy program allocated by the California Department of Education, High School Initiatives Office. Based on student demographic and outcome data, PW reported qualitative and quantitative data elements to the state as required by Perkins and the Partnership Academy guidelines.

International Trade Education Programs (ITEP) Evaluation. PW conducted an evaluation of the ITEP Academies. Building on the success of the original International Trade Careers Academy (ITA) and a school wide federal grant to implement smaller learning communities, Banning High School was organized into four smaller schools, each with a different curricular and thematic focus. The ITA Academies served over 700 students and were supported by the International Trade Education Programs (ITEP), a non-profit organization supporting the development of curriculum and connections to the employer and education community that are relevant to the focus areas of the Academies.

Youth Systems Capacity-Building Through Regional Coordination, Sonoma County STC Partnership.PW prepared a regional gap analysis related to a comprehensive youth system including the availability of support for academic excellence, career development, youth leadership and youth support for the counties of Sonoma, Mendocino, Lake, Humboldt and Del Norte. Methodologies included coordination of county leads and data collection teams from each county, administration of a comprehensive survey of youth-serving organizations and youth-led focus groups of young people from each county.

Sonoma County School-to-Career Partnership Data Collection System, Sonoma County STC Partnership.  PW supported the implementation of a data collection system that allowed the Sonoma STC partnership to collect information about partnership activities and implementation. Through the facilitation of data teams comprised of key partners, this system provided information for coaching schools and partners on how to collect and report on student outcome information.

Evaluation of the Sonoma County Employment Literacy Certificate, Sonoma STC Partnership.  PW evaluated the implementation of a student-led Employment Literacy Certificate designed by the Sonoma County STC Partnership.  Evaluation activities included the administration of a pre- and post- assessment of students assessing familiarity with job seeking and general employability skills, a focus group of teachers involved in the pilot, and an analysis of the assessment design.

Evaluation Report for the Intermediary Network (INet), New Ways to Work. PW provided evaluation support to INET, a membership organization that was an outgrowth of the school-to-work-funded Intermediary Project, an effort to build support for the work of intermediaries, bring together a collegial network of professionals with similar challenges, and for professional development in school-to-work partnerships throughout the country.

Evaluation of the UNITE-LA STC Partnership, Los Angeles Unified School District (LAUSD), Los Angeles Community College District (LACCD) and the City of Los Angeles. PW conducted a three-year evaluation of the UNITE-LA STC Partnership.  In the first year of the evaluation, PW conducted a baseline study of the different components of the evolving STC system in Los Angeles. For the baseline study, PW and its partner WestEd focused on documenting the extent of STC implementation, stakeholder support and involvement in STC initiatives and the impact of STC on student outcomes.  Research methods included an intensive study based on a sample of 11 LAUSD high schools representing all of the regional consortia served by UNITE-LA, as well as 3 community colleges and 3 JTPA youth programs. Exemplary STC programs were chosen and student data and qualitative program information was collected and analyzed through site visits, surveys and focus groups with a follow-up study in the subsequent year.

Core Case Study and Plus Evaluations for the State of California STC Evaluation.  On behalf of multiple partnerships, PW was selected to participate in the Core Case Study evaluation conducted by the State of California to evaluate the status of school-to-career throughout the state. Two evaluation organizations, WestEd and MPR Associates, Inc., directed the evaluation under contract to the State. The Core evaluation consisted of a study of the school-to-career partnership as a whole and included a variety of research methodologies including a survey of all K-12 schools, interviews of employers and labor organizations and interviews of community college administrators involved in school-to-career. PW also participated in the second component of the case study evaluation on behalf of the partnerships. The Plus study focused on evaluating whether STC participation affected student preparation for postsecondary education and career entry. Partnerships evaluated by PW included:

• Unite-LA
• Sonoma County STC Partnership
• Verdugo STC Partnership

Survey of High Schools for the Solano County STC Partnership, Solano County Office of Education. PW conducted a written survey of all high schools in the Solano County STC partnership based on the National School-to-Work Progress Measures Survey.  The survey provided information to the partnership about the extent of STC offerings such as career majors, integrated curriculum, and work-based learning.  This survey also provided information about activities that are undertaken at the local school level—some with the support of the STC grant, others from local initiative. PW tabulated survey responses and provided an analysis and write-up to the partnership.

Statewide Performance Evaluation of Tech-Prep Local Consortia and Tech-Prep Targeted Use Grants, Chancellor’s Office, California Community Colleges (COCCC). PW conducted a three-year management evaluation using both qualitative and quantitative methodologies to provide the COCCC information about local program delivery by reporting on the status of local consortia regarding program implementation through document reviews, a survey of all consortia and site visits to a subset of local consortia. PW assessed the status of implementation, barriers encountered and the degree to which local factors promote certain elements over others. In the second year of the evaluation, activities focused on implementation of Industry Sector and Innovative Practices and Programs grants. The third year of the evaluation focused on overall consortia implementation and a comparison between the first and second year reports.

Evaluation of Communities and Schools for Career Success (CS²) Initiative, New Ways to Work.  PW supported New Ways staff and the four sites involved in a statewide initiative aimed a fostering systemic change in career and youth development programs. Evaluation strategies included the use of data teams in each community to fine tune data collection and analysis and to develop internal expertise to evaluate and monitor progress. Evaluation methodologies included the analysis of trends in long-term student outcome data and examination of interim progress based on data collected by the sites and site visits.

Evaluation of Countywide Connections of Education and Workforce Development, UNITE-LA/Los Angeles Unified School District and Los Angeles County Office of Education.   PW conducted phone interviews and a written survey with Workforce Investment Boards, One Stops, Youth Councils, Youth Providers, School-to-Career Partnerships, and representatives of Districts and Regional Occupational Programs to determine youth programs and services offered, industry involvement, cross training, partnering of organizations and best practices.  The evaluation focused on the connections and disconnections of workforce development programs and education systems in Los Angeles County.

Evaluation of Career Pathways, Verdugo STC Partnership.  PW evaluated a supplemental STC grant used to fund career pathways (nursing, law enforcement, multi-media, and xerographics) in the Glendale Unified School District.  Evaluation methods included surveys of teachers regarding student classroom performance as well as the collection of baseline data on student achievement among students enrolled in the pathways.  PW also conducted site interview and focus groups with students, teachers, and industry partners as part of the preparation of a final report for the State.

Workforce Gap Analysis for the North Bay Technology Roundtable (NBTR). PW conducted a workforce gap analysis titled Balancing the Sonoma County Technology Workforce Gap: An Analysis of Workforce and Education Needs. The report included data collected from employers, education and government institutions, educators and students. The report also included a series of recommendations used by the NBTR to prioritize and develop an action plan for future activities. Methodologies that were used in the analysis were in depth interviews of human resources personnel and executives in Sonoma County’s high-tech clusters, interviews of administrators at local high schools and postsecondary programs and a study of a sample of local high schools that included focus groups of students and teachers and a survey of seniors at these schools.

Evaluation of a Pilot Special Education Project, Los Angeles Unified School District (LAUSD). PW evaluated a pilot joint venture at Los Angeles Mission College that provided work-based learning and career education for special education students 18-21 years old and classified as Mentally Retarded Moderate (MRM) or Specific Learning Disability (SLD). PW examined the implementation of the Center for Advanced Transition Skills Program in terms of its ability to provide participating students with opportunities to acquire knowledge and skills related to a potential career and place them on a job related to classroom instruction.  Interviews with program staff, employers, and students were the primary methods used. In the second phase of the evaluation, PW evaluated student records and outcomes of program completers in the areas of basic and social skills through the analysis of test scores, transcripts and Individual Education Plans (IEPs) and examined barriers to student employment.

Evaluation Design for the National Assessment in Vocational Education, U.S. Department of Education. PW developed a conceptual framework to conduct research on state’s responses to the accountability requirements in the Perkins III Act. The paper provided a typology for categorizing states in the four required indicator areas, which included (1) academic and occupational competency attainment, (2) completion, (3) placement and retention, and (4) enrollment and completion in nontraditional areas. The paper provided the context for vocational education by describing the tensions between purposes in workforce development and school reform and centralized and decentralized governance structures. The paper was used to develop an RFP process for a contractor to research Perkins III accountability systems.

Evaluation of the Orange County Coalition Vision 2020 Project, Orange County Office of Education and the Orange County Private Industry Council.  PW evaluated the first-year implementation of Vision 2020, a federally funded comprehensive STC project, through the examination of student outcomes and program activities in 10 Model Collaboratives and 100 Local Integrated Projects. Evaluation efforts focused on assessing the effectiveness of the Coalition’s collaborative governance structures and collecting baseline data on school-to-career efforts in 12 participating school districts. The evaluation included a 12^th grade student survey and employer survey as well as intensive evaluation efforts in individual Model Collaboratives.

Career Technical Assessment Projects

State of Massachusetts Certificate of Occupational Proficiency (COP) Assessment Research and Design.Conducted research on promising national and state assessment models including the implementation and results of the COP implementation to date. In order to inform the state’s assessment design, the project examined the following dimensions of assessment including assessment types (written, performance, simulations and portfolio); issues of reliability, fairness and validity; feasibility; scoring, test management and reporting; performance specifications; and costs. Methods included a review of promising state assessment and industry-based certification models, National Skill Standards Board projects and a survey of Massachusetts high schools to investigate the level of portfolio usage.

Technical Assistance in Developing a State System of Vocational Certification at the Secondary Level, Utah State Department of Workforce Development. PW assisted Utah in the development and implementation of a statewide assessment system that provided accountability information on programs as well as signals to employers and educators that students passing the exam possess a set of skills assessed in the tests. Activities included facilitating the development and revision of standards, multiple-choice test items connected to standards and performance assessments. PW also researched and identified electronic databases to provide test layout, scoring, item analysis and reporting and provided guidance on technical quality issues related to validity, reliability and fairness. In the second phase of the project, PW provided technical assistance in the implementation of new item bank software and scoring and reporting to districts and teachers.

Technical Assistance and Training to Support Utah Applied Technology Education Perkins III Accountability, Utah State Department of Workforce Development. PW developed a training guide and a self-assessment tool to collect information for comprehensive assessment of vocational and technical education programs as required by Perkins III.  PW delivered a series of one-day program quality initiative trainings with groups of eligible recipients including school districts, technology centers and community colleges. The meetings resulted in a Program Quality Improvement Plan for each recipient and a completed self-assessment instrument. A final report addressed the requirements of the accountability sections in the legislation and assessed Utah’s vocational and technical education programs using state-adjusted levels of performance.

Technical Assistance in Developing a State System of Vocational Certification at the Secondary Level, Kentucky State Department of Secondary Education. PW assisted Kentucky in the development and implementation of a statewide assessment system in 17 industry areas that provided accountability information on programs as well as signals to employers that students passing the exam are skilled workers. Activities included facilitating the development or refining industry skill standards, facilitating the development of multiple-choice test items, written or problem-based scenarios, and performance assessment; researching electronic databases that provide assessment layout, scoring, item analysis and reporting; and providing guidance on technical quality issues related to validity, reliability and fairness.

Technical Assistance and Strategic Planning Projects

Development of a Guide to the Workforce Investment Act (WIA), National Skills Standards Board. PW produced training guide for WIA practitioners in the use of skill standards and occupational certifications to support connections to workforce needs, improve programs and build accountability systems. The guide was written in a workbook format and included a section for trainers to be used in training the materials. The guide included information on WIA legislation, how to incorporate a skill standards system in WIA programs, curriculum and policies; standards-based assessment; establishing a work readiness certificate; and using existing occupational certifications to improve WIA programs.

Technical Assistance to States to Implement Accountability Requirements and Integrate School Reform and Workforce Development Initiatives, National Center for Research in Vocational Education.  PW provided technical assistance to state agencies in the areas of accountability, using data for program improvement and standards-driven curriculum and assessment. PW also provided technical assistance to state agencies in the implementation of accountability requirements included in the Carl D. Perkins Vocational and Applied Technology Education Act of 1998 (Perkins III). A report documenting best practices across states was developed to spur innovation and information sharing around improving accountability in vocational education. A second document was developed with profiles of all states involved in the project including their accountability system developed under Perkins III.

Technical Assistance to the All Schools Access Pathways (ASAP) School-to-Work Partnership, Pasadena Unified School District (PUSD). PW served as the coordinator of the state STC partnership grant in the Foothill region called All Students Access Pathways (ASAP). The partnership included the communities of Pasadena, Altadena, Sierra Madre, Alhambra, Duarte, South Pasadena, Arcadia, Monrovia and San Marino. Coordination activities included conducting all meetings related to the partnership, coordinating the development of technology centers at each high school and middle school, managing grant activities, providing professional development and developing materials to support implementation.

Technical Assistance to the Regional Occupational (ROP), Adult Education, High School Academies Office, Pasadena Unified School District (PUSD).  After originally authoring the PUSD plan to implement vocational education under the Carl D. Perkins Vocational and Technical Education Act of 1998, PW assisted PUSD in completing reports due to the California Department of Education to maintain Perkins funding.  The reports reflected quantitative data on student enrollment and student outcomes for the nine PUSD Partnership Academies.  PW also collected qualitative and quantitative data as required by California Partnership Academy guidelines including District matching funds and support, business partner matching funds and/or contributions, student demographics, and student outcome measures.

Technical Assistance to New Ways to Work (NWW), California Workforce Investment Board (CAL WIB).  Under subcontract to NWW, PW developed a matrix of Federal, State, and private foundation funding sources available for serving at-risk youth.  The programs and grants identified in the matrix center on assisting youth in meeting basic needs, completing educational goals, and securing employment as a resource for local youth service providers that were looking to fund programs and services for youth.

Strategic Planning and Technical Assistance, Sonoma County STC Partnership.  PW provided technical assistance to the Sonoma County STC Partnership in the area of sustainability by identifying potential funding sources to sustain partnership activities, organizing sustainability strategies for the director and providing grant writing technical assistance to schools and other partners. PW also facilitated the development of a three-year business plan and other promotional materials for the partnership through strategic planning sessions with the advisory board, business plan subcommittee members and the partnership director.

Strategic Planning for Career and Workforce Development, Los Angeles County Office of Education (LACOE).  PW facilitated directors of programs administered by LACOE as part of the joint strategic plan to better serve preK-adult clients. By bringing together administrators of the Greater Avenues for Independence Now (GAIN) and Regional Occupation Program (ROP) (pre-reorganization included Headstart and Career and Family Services), these strategic planning sessions focused on improving the coordination and implementation of LACOE’s mission in the areas of instructional technology, fund development, program development, infrastructure, marketing, communication, and product development.

Technical Assistance to the All Schools Access Pathways (ASAP) School-to-Work Partnership, Foothill Private Industry Council (FPIC). PW wrote the state STC partnership grant in the Foothill region called All Students Access Pathways (ASAP). The partnership included the communities of Pasadena, Altadena, Sierra Madre, Alhambra, Duarte, South Pasadena, Arcadia, Monrovia and San Marino. PW organized and facilitated kick-off meetings with all stakeholders.

Technical Assistance on Making Linkages to Education Institutions, Laborers-Associated General Contractors (AGC). PW worked with six regions throughout the U.S. to capture effective practices in the area of education linkages and developed a user-friendly guide on how to improve linkages. The guide included support for educators and counselors, including strategies for encouraging industry experience, developing curriculum units, raising exposure and awareness of industry, obtaining work experience and working with advisory boards. Workshops were offered at both national conferences and within the regions at individual sites to improve existing linkages.

Development of Construction Standards, Laborers-Associated General Contractors (AGC). PW developed academic and industry standard crosswalks intended to improve classroom training conducted by Laborers-AGC conducted site visits and interviewed workers in the areas of Concrete, Open Cut Pipe Laying, Lead Abatement and Hazardous Waste.  The evaluation included site visits and interview with workers in each of these occupational areas.  Information from the site interviews was used to guide teachers in the improvement and implementation of new training strategies that better align to industry need. Laborers-AGC standards were aligned to National Academic Standards to assess the level of math and science embedded in construction standards.

Technical Assistance to the City of Los Angeles Job Training Partnership Act (JTPA) Youth Consortia, Kip Stottlemyer Associates.  Kip Stottlemyer Associates and PW trained representatives of JTPA youth partnerships to improve practices in youth training and employer recruitment using SCANS Competencies and Foundation Skills. Training topics included assessment, mentoring, work-site supervisor training, marketing, integrated curriculum and employer recruitment.  Strategies utilized included a pre/post survey, facilitation, training, interviews and a best practice.