Insights into Community College Development and Student Learning Metrics

Richard Alfred Center for Center for Community College Community College Development Development Strategic Horizon Network Colloquium June 4, 2012 Keystone, Colorado 1

A few reasons Inside Source of feeling and pride Contributes to employee satisfaction and morale Shapes convergence between individual and institution Attaches people to the center of the organization / holistic function Outside Adds to/detracts from institutional visibility and reputation Brands the organization Attracts/deters students from enrolling, persisting, and completing Attracts/retains and repels/attrits quality personnel Influences the flow of revenue from public and private sources A few reasons 2

Arum and Limited Learning Almost half significant gains in learning during first two years of college More than one-third (36% improvement over four years Survey data and pre/post test results on Collegiate Learning Assessment (CLA) obtained from 2,300 undergraduates enrolled in two dozen four-year colleges and universities CLA measures gains in critical thinking, complex reasoning, and written communication Arum and Roksa s Limited Learning on College Campuses (2010) half of college students did not demonstrate Roksa s Academically on College Campuses (2010) Academically Adrift: Adrift: 36%) did not show significant 3

Rankings: World Report s college and university rankings. EU athletic teams are consistently ranked in the AP Top Twenty in major sports (football and basketball) Grade Data: reveal that more than half of EU s students received a grade of B or better in most of their courses Instructors: and course grades are an accurate index of what they are learning Students: its real world currency / teaching methods do not match their learning preferences Rankings: EU is ranked in the Top Ten of U.S. News and Grade Data: Academic grade data between 2007 and 2011 Instructors: Feel that most students master course content Students: Question the value of what they are learning and 4

Course Evaluations: More than two-thirds of EUs Course Evaluations: students express satisfaction with course and instructor in end-of-semester course evaluations Student Outcomes Data: results students experience varying degrees of success in transition to work and further education Employers: EU s graduates indicate that one-third to one-half express dissatisfaction with graduates because of inadequate soft skills (written and verbal communication, problem-solving, and customer care) Student Outcomes Data: Inconsistent pattern of Employers: Follow-up surveys with employers hiring 5

Is Exemplary Universitys overall performance good, bad, or mixed? What is the basis for your answer? Which Which way of looking at EU s performance is best way of looking at EU s performance is best? ? Numbers grades student survey results value-added (CLA pre- and post-test scores) student outcomes data Numbers Rankings & Ratings U.S. News college rank and program rankings EU s standing in athletic polls Rankings & Ratings Perceptions student instructor employers Perceptions Something else? Something else? If you had to pick one, what would be your choice as to the best measure of institutional performance? Why? 6

Performance operates in three domains: Objective numbers and value-added Subjective rankings and ratings (valence applied to numbers) Cognitive feelings, beliefs, and perceptions of organizations people develop through direct interaction with them, observation, and exposure to information about them Performance operates in three domains: Objective Subjective Cognitive 7

Performance is What organizations do, produce, and accomplish for the constituencies they serve Actual outputs or results measured against a preset standard, its goals, or intended outputs A measurement The result of activities given period of time The carrying into execution, achievement, or accomplishment A process environment aimed at achieving a goal Performance is: : do, produce, and accomplish outputs or results of an organization as measurement of output or behavior result of activities of an organization over a execution, achievement, or accomplishment process shaped by stakeholders in a competitive 8

Performance is comprised of three constructs that contribute to a working definition Outputs and Results Goals and Objectives Stakeholders three constructs Working Definition -- Performance is: The an organization and the value assigned to them by The sum product of results produced by an organization and the value assigned to them by stakeholders sum product of results produced by stakeholders 9

10

Bare facts not influenced by personal opinion Numbers Results that are objective influenced by personal feelings or prejudice enrollment, degree completion, transfer rate, placement in jobs following graduation, etc. Q Quantitative way things are/ what organizations actually do Value The leveraging or enhancement that students achieve as a result of the educational experience measured through comparison of pre determine the direction and extent of change Numbers objective insofar as they are not uantitative data data that provide tangible evidence of the Value- -Added Added pre- -and post and post- -test scores test scores to 11

Numbers are not sufficient themselves to explain performance. They lack meaning without interpretation. not sufficient in and of 12

The valence people assign to numbers Ratings A standing or position assigned on a scale Example: The Princeton Review College 99 for academics, admissions selectivity, financial aid, fire safety, quality of life, green, total enrollment, ACT composite Rankings A position or standing through assignment Example: An institution s position in the U.S. News College Rankings class size, expenditures per student, graduation rate performance, etc. Ratings scale Ratings - Scale of 60- Princeton Review College Ratings Rankings assignment U.S. News College Rankings determined by criteria such as acceptance rate, 13

(continued) Winning An act of obtaining something generally through competition A reward for achieving a favorable outcome loss D Differentiation and influence, our place in the world C Competitive competitors, but not having to win in order to claim success P Pursuit primary goal no matter what the cost Winning competition favorable outcome and aversion of ifferentiation, proof of being the best, attainment of space ompetitive motivation motivation performing toward a goal with ursuit of winning of winning besting a competitor whose defeat is the 14

Rankings, ratings, and winning add to our understanding of performance by putting a valence magnifying their impact. They do not, however, explain the role of cognition in performance. valence* on numbers, thereby * high or low rank, good or bad rating, winning or losing 15

The an organization through a process of Valuation The assignment of meaning to results through direct interaction information about The feelings, beliefs and impressions people form of an organization through a process of valuation Valuation The assignment of meaning to results through direct interaction with organizations information about them feelings, beliefs and impressions people form of valuation with organizations, observation and exposure to them , observation and exposure to Social Context Past Experience Identity Salient Cues Social Context Past Experience with the organization Identity Salient Cues reinforcement of organizational results Enactment basis of consistency external forces affecting judgment with the organization sense of self self reinforcement of organizational results reinforcement or rejection of cues on the basis of consistency external forces affecting judgment sense of self self- -selection, interpretation, and selection, interpretation, and Enactment reinforcement or rejection of cues on the The cognitive domain of performance is entered when impressions formed by stakeholders of organizational actions and results become decoupled from numbers and initial subjective judgments. 16

Cognition is the single most powerful driver of performance. what the numbers say. of people believe that an institution is low performance embedded in numbers become subservient to impressions. Cognition is the single most powerful driver of performance. It doesn t matter what the numbers say. If a critical mass of people believe that an institution is low- -performing, facts about performance embedded in numbers become subservient to impressions. It doesn t matter If a critical mass performing, facts about 17

Questions to consider What drives performance in your institution? Do you think broadly about performance? Are you measuring too much? Are you measuring things that count? Do you challenge convention? Is mediocrity acceptable? Questions to consider 18

Think broadly about performance Embrace multiple conceptions of organizational success Reduce and simplify what is measured Measure things that count Challenge convention Aim for high performance 19

Institutional Goals and Priorities Increase headcount enrollment, FTE, and credit hours by 3-5% annually Enhance student success by increasing retention and degree completion Improve service as measured by increased levels of student satisfaction Improve financial efficiency and employee productivity Achieve financial stability by locating additional sources of funding Institutional Goals and Priorities Key Stakeholders *Enrolled Students, *Prospective Students, *Employers, *Community Organizers, *Taxpayers, *K-12 schools, *4-year colleges Key Stakeholders 20

OBJECTIVE DOMAIN OBJECTIVE DOMAIN NUMBERS Indicator Headcount enrollment Fall to Fall retention Graduation rate (3-year) Transfer rate NUMBERS Current Performance 22,160 students 55% 12% 30% of entering cohort $40 million 85% classroom space Performance Goal 25,500 students 65% 20% 40% of entering cohort $50 million 95% classroom space Gap 3,340 students 10% 8% 10% Service area economic impact Facilities utilization $10 million 10% VALUE ADDED Indicator Success in subsequent related coursework: % of students receiving a grade of C or better in regular credit courses following remedial course completion VALUE ADDED Current Performance 40% Performance Goal 60% Gap 20% Mastery of discipline: Students demonstrating skills specific to pre-determined outcomes Licensure and certification test performance: Pass rate Percentage of career program graduates obtaining job related to curriculum Mean GPA of transfer students at senior institution 70% 90% 20% 90% 95% 5% 60% 80% 20% 2.4 2.8 0.4 21

SUBJECTIVE DOMAIN Current Performance None SUBJECTIVE DOMAIN Current Performance Indicator Compared to peer institutions Athletic conference championships Annual enrollment growth: Fall to Fall headcount High school penetration rate: % of h.s. graduates enrolling in CCC Adult market penetration rate: % of adult population enrolled annually Visibility in media: Media articles focused on college Indicator Compared to peer institutions National rank on associate degrees produced National rank by size of enrollment Indicator Performance Goal Basketball/baseball Performance Goal Gap Gap 2 championships 2% 4% 2% WINNING 21% 25% 4% 8% 10% 2% Episodic Weekly 1 article a week Indicator Current Performance Top 50 Current Performance Performance Goal Top 25 Performance Goal Gap Gap RANKINGS 25 positions Top 20 Top 10 10 positions Indicator Compared to peer institutions Recognition by foundations and government agencies Granting agency selection Indicator Current Performance Not recognized Current Performance Performance Goal Selected as Top 100 College Frequently selected for grant prgrm participation Consistently recognized through awards Achievement recognized, celebrated, and publicized Performance Goal Gap Gap AWARDS & RECOGNITION Recognition Periodically selected for grant prgrm participation Awards are episodic Frequency Awards for academic excellence Consistency Recognition received by faculty and students Achievement not recognized Recognition/pu blicity 22

COGNITIVE DOMAIN COGNITIVE DOMAIN VALUATION Indicator VALUATION Current Performance 60% highly satisfied 65% indicate goals achieved 65% favorable Moderate # of articles/some critical Performance Goal Gap Student satisfaction Student goal attainment 75% highly satisfied 90% indicate goals achieved 85% favorable Significant media coverage/decidedly favorable 15% 25% Community perceptions Media treatment 20% More and better media coverage 23

High performance is as much a function of organizational of exceeding preset or competitor outcomes measures of performance in objective, subjective, and cognitive domains. High performance is as much a function of organizational distinctiveness of exceeding preset internal standards or competitor outcomes on salient measures of performance in objective, subjective, and cognitive domains. distinctiveness as it is internal standards on salient as it is 24

High Performance through Accretion Running strong numbers (enrollment, retention, completion) Playing the rankings game (Aspen Institute) Securing and retaining a visible leader Hiring top-notch credentialed personnel Winning in athletics Brand imaging through marketing Leveraging traditional prestige Building on strength/ centers of excellence 25

High Performance through Deviation Disruptive innovation Positive deviance from normative institutional behavior Adrian College (student centeredness) Gettysburg College (employee focus) Lorain County Community College (partnerships and collaboration) University of Toledo (program innovation) University of Alaska, Anchorage (student success) Abundance 26

Working in colleges, develop a Performance Canvas for your institution using principles of design thinking 27

Design Thinking Ideation Step 3 Create Design Prototypes Understand Design Constraints Step 4 Step 2 Synthesize Insights Abstract Gather User Feedback Concrete Perform User Observation Iterate Design Step 1 Step 5 Final Expression Identify the Real Challenge Conception of Performance

Small Groups Identify the Challenge User Observation Ideation Prototyping Group Feedback Who are our stakeholders and what do they want? Create a Performance Canvas Present Prototype What is our conception of performance ______? Develop a new idea Describe the challenge it poses What are we trying to accomplish? For whom? What are our performance objectives?