Teras PSPTNTeras PSPTN Meluaskan akses dan meningkatkan ekuiti; Menambah baik kualiti pengajaran dan pembelajaran; Memperteguh penyelidikan dan inovasi; Memperkasakan institusi pengajian tinggi; Mempergiatkan pengantarabangsaan; Membudayakan pembelajaran sepanjang hayat; dan Mengukuhkan sistem penyampaian KPT.
Aspects of quality in higher education Aspects of quality in higher education Three aspects: Client quality – what students and employers want from the service. Professional quality – whether the service meets needs as defined by professional providers and whether it carries out techniques and procedures which are believed to be necessary to meet clients needs. Management quality – the most efficient and productive use of resources within limits and directives set by higher authorities. Ovretveit (1992)
Judging quality in higher education Judging quality in higher education Three categories: Academic standards of courses – what the students demands of the students, the extent it meets staff needs. Teaching quality – staff training, appraisal systems, teaching evaluation, reports from external examiners, patterns of employment of graduates, students feedback. Student achievement – completion rates, class of degrees obtained. O’Neil (1994)
Issues to be addressed The need for competent staff Management responsibility Quality policy Quality manuals Quality planning Issues to be addressed
Performance cultures in higher educationHEIs worldwide have undergone reform to improve quality HEIs have implemented systematic and formalized quality assurance processes to achieve greater efficiency and accountability Establishment of quality models and organizations to audit and review university performance Institutional and national quality models and performance indicators are vital components to raise the standard of HEIs Quantitative performance indicators are used to provide international comparisons Performance cultures in higher education
Rationale for performance indicatorsRationale for performance indicators To ensure education provided by HEIs equips students for employment and provide the country with a highly skilled workforce that support economic growth. To contribute to educational, social, and political values.
Purposes of performance indicators in HEIsTo monitor own performance for comparative purposes. To facilitate the assessment and evaluation of institutional operations. To provide information for external quality assurance audits. To provide information to the government for accountability and reporting purposes (Rowe, 2004). Purposes of performance indicators in HEIs
The use of performance indicators in HEIsEnsure accountability for public funds Improve the quality of higher education provision Stimulate competition within and between institutions Verify the quality of new institutions Assign institutional status Underwrite transfer of authority between the state and institutions Facilitate international comparisons Ref: Chalmers (2008) The use of performance indicators in HEIs
Defining performance indicators Simple indicators – expressed in absolute figures and are intended to provide an unbiased description of a situation or process. Performance indicators – imply a point of reference; for example, a standard, objective, assessment, or comparator, are relative rather than absolute in character. Involve value judgements. General indicators – externally driven and are not indicators in the strict sense; they are frequently opinions, survey findings or general statistics. Ref: Chalmers (2008, quoted from Hanney, Henkel & Kogan, 1997) Defining performance indicators
Defining performance indicators Currently there is no common definition of performance indicators. PI cannot be considered as facts, but are goal, value and context laden, and utilized in different ways depending on the performance model being used. PI are defined as measures which give information and statistics context; permitting comparisons between fields, over time and with commonly accepted standards. They provide information about the degree to which teaching and learning quality objectives are being met within higher education sector and institutions. Ref: Chalmers (2008, p.3) Defining performance indicators
Types of performance indicators Input Process Output Outcome The types can be categorized as quantitative indicators and qualitative indicators. Ref: Chalmers (2008, p.3) Types of performance indicators
Quantitative Indicators define as those associated with the measurement of quantity or amount, and are expressed as numerical values. Input indicators Human, financial and physical resources in supporting institutional programmes, activities and services. Output indicators Output reflects the quantity of outcomes produced, including immediate measurable results, and direct consequences of activities implemented to produce results. Do not demonstrate quality of education, but quantities of outcomes. Ref: Chalmers (2008) Quantitative Indicators
Qualitative Indicators associated with observation based descriptions, rather than an exact numerical measurement or value. Relate to or involve comparisons based on qualities of non-numerical data such as policies and processes for assessing students’ learning, the experience, the content of a mission statement. Outcome Indicators Focus on the quality of educational program, activity and service benefits for all stakeholders. Insightful, meaningful and accurate since they are related to the objectives of higher education. Students are treated as customers. Ref: Chalmers (2008, p.5) Qualitative Indicators
Qualitative Indicators Process Indicators include the means used to deliver educational programmes, activities and services within the institutional environment. qualitative information on teaching and learning such as policies and practices. Considered as most practical, useful and appropriate measures of quality teaching and learning. Ref: Chalmers (2008) Qualitative Indicators
Research conducted in AustraliaLook at 13 process indicators Mission, Vision and Objectives Teaching and Learning Plans and Policies Teaching and Learning Indicators Internal and External Performance Funds for Teaching and Learning Organizational Unit Review Curriculum Review Assessment and Feedback Policies Graduate Attribute Statement Student experience Ref: Chalmers (2008) Research conducted in Australia
Research conducted in AustraliaLook at 13 process indicators Professional Development Appointment and Promotion Criteria Review of Academic Staff – performance Recognition of Excellence in Teaching and Enhancing Student Learning Experience Ref: Chalmers (2008) Research conducted in Australia
Research in Australia – Quality TeachingLook at four dimensions of teaching practice Institutional climate and systems – commitment to the enhancement, transformation and innovation of learning. Measure student experience and level of satisfaction. Diversity – relates to ethnic, cultural and socioeconomic diversity as well as students’ abilities, talents and learning approaches. Ref: Chalmers (2008) Research in Australia – Quality Teaching
Research in Australia – Quality TeachingLook at four dimensions of teaching practice Assessment – the assessment tasks of student enrolled in the program of study. Look at the design, delivery and administration, provision of feedback, moderation, and review of assessment. Engagement and learning community – student engagement, i.e. the student commitment and engagement with their own education. Also includes staff engagement. Ref: Chalmers (2008) Research in Australia – Quality Teaching
Study in Hong Kong and ChinaStudy in Hong Kong and China Seven factors contributing to the quality of education. They are: 1 course content; 2 concern for students; 3 facilities; 4 assessment; 5 medium of instruction; 6 social activities; and 7 people.
Course content itemsCourse content items The chance that your study fulfils your personal needs The appropriateness of requirements for your course The chance to develop your abilities and prepare for your career The quality of material emphasized in course The usefulness of the module components offered in your career development The usefulness of the module components in fulfilling your personal needs
Concern for studentConcern for student The chance that your study fulfils your personal needs The appropriateness of requirements for your course The chance to develop your abilities and prepare for your career The quality of material emphasized in course The usefulness of the module components offered in your career development The usefulness of the module components in fulfilling your personal needs
FacilitiesFacilities The availability of quiet places to study in the university The cleanliness of most facilities used by students The amount and availability of library facilities The places provided for students to relax and lounge during the day The amount and availability of computing facilities The amount and availability of sports and recreational facilities
AssessmentAssessment The chance that you do well if you work hard, The appropriateness of the standard of modules offered, The appropriateness of the assessment system, Detailed lecture notes are distributed, The amount of work required in most modules, The amount of time you must spend studying to get a passing grade, The likelihood of getting distinction if you work hard.
Medium of instructionMedium of instruction Lectures be conducted in Language understood by students Tutorials be conducted in language understood by students
Social activitiesSocial activities The activities and clubs you can join in the university The social events that are provided for students in the university
PeoplePeople The friendliness of students and the opportunity to make close friends. The chances to meet people with the same interests as you have.
Conclusion of the study by Kwan & Ng, 1999Conclusion of the study by Kwan & Ng, 1999 Hong Kong and Chinese students are very practical and only focus on study-related matters rather than social life in campus. Students in the States are more interested in campus life but the importance of facilities has not been mentioned. It seems that Hong Kong and Chinese students regard university education as an investment and thus stress course content and facilities.
DISCUSSIONDISCUSSION Jelaskan elemen yang dipilih oleh anda sebagai pelajar untuk dinilai dalam konteks kepuasan pelanggan. Apa kriteria yang diguna untuk penilaian indeks kepuasan anda sebagai pelanggan?
Higher Education in Malaysia Challenges Peranan universiti dan ahli akademik (University’s and academics’ roles) Perkembangan kurikulum mengikut keperluan pasaran (Curruculum development according to market needs) Penyelidikan, pembangunan dan pengkormesialan dalam sistem inovasi kebangsaan (Research, development and commercialization in the national innovation system) Kaedah pengajaran dan pembelajaran (Teaching and Learning Methods) Perluasan akses dan mobiliti pengetahuan (Accessibility and knowledge mobility) Higher Education in Malaysia Challenges
Slide63Pekeliling Kemajuan Pentadbiran Awam Bilangan 2 Tahun 2005 Garis Panduan Bagi Mewujudkan Petunjuk-petunjuk Prestasi Utama Atau Key Performance Indicators (Kpi) Dan Melaksanakan Pengukuran Prestasi Di Agensi Kerajaan
Terminologi Terminologi Petunjuk Prestasi Utama (KPI) ialah salah satu kaedah bagi mengukur prestasi perkhidmatan agensi-agensi Kerajaan Perkhidmatan Teras Bidang tanggungjawab agensi sejajar dengan visi (core business): dan misi agensi; Proses Utama Fungsi-fungsi di bawah perkhidmatan teras yang (core process): perlu dilaksanakan bagi menghasilkan perkhidmatan untuk pelanggan; Key Performance Petunjuk-petunjuk prestasi utama yang ditentukan Indicators (KPI): sebagai asas mengukur prestasi;
NKRA NKRA Education Crime (Public Safety) Corruption Low Income Households Rural Basic Infrastructure Urban Public Transportation
Higher Education in Malaysia Challenges Globalisasi dan piawaian melalui pemeringkatan dan penarafan (Globalization and standardization through development and accreditation) Peluang guna tenaga (Resource utilization opportunities) Higher Education in Malaysia Challenges
Malaysia - Current scenario20 public universities 21 polytechnics 37 community colleges >400 registered private colleges 21 private universities and university colleges 11 local university branch campuses + 5 foreign university branch campus Areas for indicators: Academic staff Educational programs Student selectivity Educational resources Governance The method used will be peer review. Malaysia - Current scenario
How are universities ranked?Different ranking approaches: League table – each university is assigned a specific rank. Higher ranks indicate higher quality, lower ranks indicate lower quality. Quality criteria and indicators are used in this ranking methodology to assess universities. Each indicator such as research impact as the number of citations per faculty in the Thompson Scientific Database or teaching quality as in THES are given weight. This approach are applied to all universities assessed. How are universities ranked?
How are universities ranked?Different ranking approaches: A ranking of individual disciplines or departments instead of whole institutions. A multidimensional concept of university quality instead of a ‘one-size-fits-all’ approach, taking into account the diversity of academic institutions, missions and goals as well as language and cultural specifics. A separate measurement and presentation of single indicators. A presentation of ranking results in rank groups (top, middle, bottom groups) instead of league tables. How are universities ranked?
Purposes of ranking of HEIRanking serves several purposes: Responds to demands from consumers for easily interpretable information on the standing of HEIs. Stimulates competition among universities. Provides some rationale for allocation of funds. Helps to differentiate among different types of institutions and different programs and disciplines. Contributes to the definition of ‘quality’ of HEIs within a particular country. (source: Berlin Principles of Ranking of Higher Education Institutions, 2006 http://www.che.de/downloads/Berlin_Principles_IREG_534.pdf) Purposes of ranking of HEI
Berlin Principles of Ranking of Higher Education InstitutionsPurposes and goals of rankings: Be one of a number of diverse approaches to the assessment of higher education inputs, processes, and outputs. Be clear about their purpose and their target groups. Recognize the diversity of institutions and take the different missions and goals of institutions into account. Provide clarity about the range of information sources for ranking and the messages each source generates. Specify the linguistic, cultural, economic, and historical contexts of the educational systems being ranked – should be aware of possible biases. (source: Berlin Principles of Ranking of Higher Education Institutions, 2006) Berlin Principles of Ranking of Higher Education Institutions
Berlin Principles of Ranking of Higher Education InstitutionsBerlin Principles of Ranking of Higher Education Institutions Design and weighting indicators: Be transparent regarding the methodology used for creating the rankings. Choose indicators according to their relevance and validity. Measure outcomes in preference to inputs whenever possible. Make the weights assigned to different indicators (if used) prominent and limit changes to them.
Berlin Principles of Ranking of Higher Education InstitutionsBerlin Principles of Ranking of Higher Education Institutions Collection and processing of data: Pay due attention to ethical standards and the good practice recommendations articulated in these Principles. Use audited and verifiable data whenever possible. Include data that are collected with proper procedures for scientific data collection. Apply measures of quality assurance to ranking processes themselves. Apply organizational measures that enhance the credibility of rankings.
Berlin Principles of Ranking of Higher Education InstitutionsBerlin Principles of Ranking of Higher Education Institutions Presentation of ranking results: Provide consumers with a clear understanding of all of the factors used to develop a ranking, and offer them a choice in how rankings are displayed. Be compiled in a way that eliminates or reduces errors in original data, and be organized and published in a way that errors and faults can be corrected.
Purposes of Quality Indicator SystemPurposes of Quality Indicator System Colorado State, USA: Encouraging continuous improvement by institutions in achieving high levels of performance. Measuring institutional performance and accountability. Determining funding recommendations and the funding distribution for the higher education system. Build public support for increased funding for higher education.
Some Quality IndicatorsSome Quality Indicators Baccalaureate graduation rates Achievement scores of graduating students on various comprehensive examinations, tests, and /or professional specific licensure or certification examinations Graduates employed or continuing their education Institutional support expenditures – administration expenditure, expenditures per student Undergraduate class size Faculty teaching workload
Current scenarioCategories of institutions APEX university Research intensive General The structure of Malaysian Qualifications Agency (MQA) One-stop center for institutions for registration and accreditation of courses Current scenario
Accountability and QualityAccountability and Quality The concept of accountability and quality assessment in higher education is an international phenomenon In America, many regions are moving toward ‘performance incentive funding’. In Europe and Australia, the central government is directly involved in establishing quality mechanisms through: Quality control, Quality audit Quality assessment The agencies involved are like the Higher Education Quality Council and the Higher Education Statistics Agency (HESA)
Accountability and QualityThe focus of attention in performance indicators in the U.S. has been cost efficiency, scientific and technical education, management of intellectual property produced at universities, and undergraduate education. Less attention is paid to graduate education and research. Categories of costs for higher education: Education and general: faculty and staff salaries, equipment, libraries, administrative and basic academic computing, and certain capital or such as rent. Cost for sponsored research Costs of student living: room, board, clothing, laundry, entertainment, and etc. Cost of foregone earnings: While disengaged from the productive work force. Accountability and Quality
The Movement in Setting PerformanceIndicators in Higher Education(U.S.A.)The Movement in Setting Performance Indicators in Higher Education(U.S.A.) 1980s Era concerned with growth in enrollments and access was over Emerging issues include: Public accountability Quality Productivity Undergraduate education In 1986, all 50 states and the District of Columbia had developed initiatives to improve the undergraduate education
Slide8Shift from growth funding (formula funding) toward funding outcomes, results, and performance These efforts paralleled developments in Europe and Australia 1990s The development of performance indicators differs from that in 1980s From voluntary institutional improvement to a system of mandated public accountability By 1994, 18 states had developed indicator systems Greater centralization of authority Issue raised: Will the federal government assume greater centralized control of higher education through areas such as accreditation and financial aids by using a set of national goals and performance standards?
The Future of Higher Education The White Paper 2003The Future of Higher Education The White Paper 2003 Higher education must expand to meet rising skill needs The social gap among those entering university remains too wide Many of our economic competitors invest more in higher education Universities are struggling to employ the best academics Funding per student fell 36% between 1987 and 1997 Universities need stronger links with business and industry
Reports of Institutional Effectiveness (EOIE) Virginia’s Public Institutions of Higher LearningReports of Institutional Effectiveness (EOIE) Virginia’s Public Institutions of Higher Learning Annual report to provide meaningful information on the academic quality and operational efficiency of Virginia’s public institutions To provide evidence of institutional effectiveness – the extent to which institutions accomplish their missions and students achieve their educational goals.
Structure of the Reports (Five Points)Structure of the Reports (Five Points) Institution’s mission The mission statement sets a vision for the institution and defines how it will get there. Institutional profile In-depth views of enrollment and projections of future enrollment. System-wide measures Include 14 performance measures focused on operational efficiency and factors associated with academic quality: Example: Classroom and laboratory space utilization, percentage of professionally accredited programs and etc.
Slide12Institution-specific measures Represent unique aspects of the mission that the college or university chose to highlight Core competency reports Explore student general education assessments in written communication and technology/information literacy.
Performance Indicators of California Higher Education, 2001Performance Indicators of California Higher Education, 2001 Describes the scope of the current set of indicators reported by the Commission, and highlights recent trends based on current information related to these indicators. This report are divided into five categories: Population Context, Fiscal Context, Student Preparation, Student Access, and Student Outcomes
Key Performance Indicators (KPIs)Key Performance Indicators (KPIs) Key Performance Indicators are quantifiable measurements, agreed to beforehand, that reflect the Critical Success Factors of an organization. They defer depending on organization KPIs must: Reflect the organizational goals Be key to its success quantifiable
Curtin’s efficiency and effectiveness Key Performance Indicators (KPIs)Curtin’s efficiency and effectiveness Key Performance Indicators (KPIs) Teaching and learning Effectiveness indicators Quality of graduates Quality of teaching Student progress and achievement Input Efficiency indicators Teaching and learning expenditure Student progress and achievement
E FFICIENCY &EFFECTIVENESSE FFICIENCY &EFFECTIVENESS EFFICIENCY means: saving TIME, MONEY or EFFORT Efficiency measures the resources used to attain a certain level of output EFFECTIVENESS means how well the the job gets done. i.e. the quality of the output. Effectiveness measures the extent to which outcomes have been achieved
Transforming Quality7th Quality in Higher Education InternationalSeminar (Melbourne) Oct. 2002Transforming Quality 7th Quality in Higher Education International Seminar (Melbourne) Oct. 2002 Three main themes: 1.To reconceptualise how higher education engages with access, employability and funding issues 2.What constitutes a high quality learning process and outcomes 3.How might quality evaluation be transformed to help improve the quality of the experience and of the learning?
8th Quality in Higher Education InternationalSeminar (Sheffield) May. 20038th Quality in Higher Education International Seminar (Sheffield) May. 2003 Two major themes: 1.How does student feedback inform quality?To what extent do institutions need to adopt new procedures to make student feedback effective? 2.What does the White Paper encourage a closer link between quality and learning?
Indicators of Research Quality in Higher EducationIndicators of Research Quality in Higher Education The vast majority of discoveries have been made in higher education environment (Dill, 1986). Review of literature on research productivity highlighted several indicators which include: Productivity dollars Productivity publications Peer evaluation
Slide22Productivity dollars The number of dollars generated by research was the most often cited measurement of success Those universities that are ranked higher, their faculty have are adept at obtaining research grants Productivity publications The number of publications is frequently used as an indicator of quality in research The research that is published is taken as an indication of its quality The types of publication which determine its quality: Journal articles, monographs, chapters, books Quality: reputation of publication in discipline, distribution of publication, refereed vs. non-refereed journals, invited chapters/papers
OUTPUT & IMPACT FACTOROUTPUT & IMPACT FACTOR Output versus Impact factor of publications Output refers to how prolific the the research is producing acceptable articles/books Impact was determined by checking citations of the articles over a period of years
PEER EVALUATIONPeer Evaluation Assemble a group of peers to review the research efforts and make a determination of the quality of those efforts. The concerns of such approach include: The visiting group doesn’t fully understand the work of the unit/individual being reviewed (especially when it is multidisciplinary) PEER EVALUATION
Efficiency and Effectiveness Key Performance Indicators (KPIs)Efficiency Indicators: Research Performance Index Research Funding Research Publications Effectiveness measures the extent to which outcomes have been achieved Efficiency measures the resources used to attain a certain level of output Efficiency and Effectiveness Key Performance Indicators (KPIs)
Publications and Number of Ph.D. GraduatesPublications and Number of Ph.D. Graduates 2001 76 2002 89 2003 112 Publications 2000 1,864 (319 international journals, 212 in local journals) 2001 1,815 (303 international journals, 204 in local journals) 2002 2,507 (496 international journals, 328 local journals) Malaysia was ranked #63 by MASTIC (Pusat Informasi Sains dan Teknologi Malaysia) in terms of production referred journals
Critical Success factors (Research)Critical Success factors (Research) Wide academic base and facilities Graduate students Screening of IRPA application and monitoring Evaluation of research and innovation Research culture and administrative support Strong research networking Research cluster development Incentives & in-house competition as incubator
Steps to be Taken by Institutions toPromote Excellence in TeachingSteps to be Taken by Institutions to Promote Excellence in Teaching Define what they mean by excellent in teaching Having well-defined criteria about excellent teaching and standards for weighting and rating of teaching/research/service Weigh teaching more heavily Increase sophistication of teachers Promote excellent teaching, not just excellent teachers Not treat promotion as a separate issue (Gibbs, 1995)
Theoretical Definition for Excellent TeachersTheoretical Definition for Excellent Teachers The character of the professor Values, personality, and social intelligence The knowledge of the professor Disciplinary and pedagogical understanding The actions of the professor Problem-solving behaviors The responses of the students Learning outcomes
Quality Teaching in Higher EducationQuality Teaching in Higher Education Flexibility in approaches to teaching and learning (including assessment) Good organization of subject matter and course, including relevance and coherence of content and planned teaching/learning activities Effective communication Knowledge and enthusiasm for subject matter and teaching Facilitation of learning through student interaction and active experience Respect for and positive attitude toward students Critically reflective orientation to teaching including effective use of feedback to guide learning and improve teaching Appropriateness and fairness in assessment and grading Reeders, E, & Marshall, H. 1996
GOOD TEACHING: THE TOP TEN REQUIREMENTSBy Richard Leblanc, York University, Ontario , 1998. GOOD TEACHING: THE TOP TEN REQUIREMENTS By Richard Leblanc, York University, Ontario , 1998. 1.Good teaching is as much about passion as it is about reason. 2.Good teaching is about substance and treating students as consumers of knowledge. 3.Good teaching is about listening, questioning, being responsive, and remembering that each student and class is different. 4.Good teaching is about not always having a fixed agenda and being rigid, but being flexible, fluid, experimenting, and having the confidence to react and adjust to changing circumstances
Slide336. Good teaching is also about style This is very important -- good teaching is about humor. 7. Good teaching is about caring, nurturing, and developing minds and talents. 8. Good teaching is supported by strong and visionary leadership, and very tangible institutional support -- resources, personnel, and funds. 9. Good teaching is about mentoring between senior and junior faculty, teamwork, and being recognized and promoted by one's peers. 10. At the end of the day, good teaching is about having fun.
Quality in College Teaching:A Research ApproachQuality in College Teaching: A Research Approach Flexibility in approaches to teaching and learning (including assessment) Good organization of subject matter and course, including relevance and coherence of content and planned teaching/learning activities Effective communication Knowledge and enthusiasm for subject matter and teaching
Slide35Facilitation of learning through student interaction and active experience Respect for and positive attitude toward students Critically reflective orientation to teaching including effective use of feedback to guide learning and improve teaching Appropriateness and fairness in assessment and grading