Standard 7: Institutional Assessment

The institution has developed and implemented an assessment process that evaluates its overall effectiveness in achieving its mission and goals and its compliance with accreditation standards.

Rutgers long history, with campuses and individual academic units experiencing unique paths of development, has led to a strong decentralized structure in its administrative and academic organization. Equally powerful historical forces have contributed to Rutgers emergence as the sole public comprehensive university in the state of New Jersey. Rutgers development into a major public research university has been accompanied by a complex and multilayered organizational structure through which an active process of institutional assessment is pursued. This assessment does not occur in a vacuum and is guided and informed by issues and forces on both the national and state levels. The result is a broad-based and extensive process of institutional assessment.

Relative to this standard, an accredited institution is expected to possess or demonstrate the following attributes or activities:

documented, organized, and sustained assessment process to evaluate and improve the total range of programs and services; achievement of institutional mission, goals, and plans; and compliance with accreditation standards that meets the following criteria:

1a. a foundation in the institution’s mission and clearly articulated institutional, unit-level, and program-level goals that encompass all programs, services, and initiatives and are appropriately integrated with one another (see Standards 1: Mission and Goals and 2: Planning, Resource Allocation, and Institutional Renewal);

1b. systematic, sustained, and thorough use of multiple qualitative and/or quantitative measures that:

1b1. maximize the use of existing data and information;

1b2. clearly and purposefully relate to the goals they are assessing;

1b3. are of sufficient quality that results can be used with confidence to inform decisions;

1c. support and collaboration of faculty and administration;

1d. clear realistic guidelines and a timetable, supported by appropriate investment of institutional resources;

1e. sufficient simplicity, practicality, detail, and ownership to be sustainable;1f. periodic evaluation of the effectiveness and comprehensiveness of the institution’s assessment process;

The state’s long range plan, A Blueprint for Excellence - New Jersey’s Long-Range Plan for Higher Education Update 2005, provides the broad parameters for New Jersey’s colleges and universities to pursue their respective goals in the context of their missions. The plan, developed by the state’s Commission on Higher Education (CHE) in cooperation with colleges and universities, sets systemwide objectives for higher education in the state; the pursuit of many of these objectives is measured with specific indicators that institutions have identified as consistent with their mission and own set of institutional goals and objectives. Institutions are individually measured on their selected indicators, which are evaluated on an annual basis and provide the basis for periodic assessment of the success to which the state is achieving its higher education objectives and the degree to which individual colleges and universities, including Rutgers, are contributing to that success.

As was documented in standard 2, Rutgers has worked closely with the state on other higher education planning activities. The most notable has been the extensive work over the proposal to restructure the research sector of New Jersey higher education. (see standard 2, element 2) This effort included an extensive assessment of the university’s functions and processes to determine the feasibility of merging and possibly realigning the state’s three research universities.

An important assessment function played by the state is through its review of the university’s annual budget request to ensure that the university is efficiently and effectively utilizing its resources (see standard 3, element 1). The university also submits an annual Accountability Report to the state that contains evaluative data on the university’s performance in areas such as student access, educational outcomes and instructional delivery.

At the university level, institutional assessment at Rutgers is guided by the institution’s mission and is operationalized through the President’s strategic goals for the university (see standard one, element one). Areas of emphasis are annually selected by the President and his senior administrators, which are put forth in his annual address and other speeches and addresses to the university community. These areas are annually assessed through progress reports (Report on Progress Toward 2006-2007 Strategic Goals; Report on Progress Toward 2005–06 Strategic Goals; Report on Progress Toward 2004–05 Strategic Goals). This assessment is reinforced by ongoing review by the Board of Governors through its standing committees on policy and planning, finances and facilities.

Assisting the President and his administration in the effort to evaluate the effectiveness of university policies and actions is the University Senate. The Senate’s duties and powers include being concerned “with all academic and non-academic matters pertaining to the mission of the university” and advising “the President on matters of broad educational and research policy.” (see section 50.2.2 of the UPL). In its role, the senate undertakes a wide range of studies that contribute to the ongoing assessment of university policies and functions. A listing of many of the senate studies and reports can be found here.

Campuswide faculty councils also provide an advisory role to the President and the senior administration of the university. (see standard 4, element 1)

While the President’s strategic goals provide a broad approach to institutional assessment at Rutgers, more specific assessment activities crucial to the effectiveness of the university in achieving its institutional goals are provided, through the President’s leadership, at the provostial, vice presidential and collegiate levels of organization.

A fundamental mechanism for moving the university along the road to achieving these goals is the All Funds Budgeting (AFB) process adopted by the university in 2004-05 academic year. Under the direction of the Executive Vice President for Academic Affairs and the Vice President for Budgeting, AFB supports the transparent distribution of institutional resources. It is consistent with Rutgers’ tradition of decentralized organization of its academic units and provides these decanal units with enhanced managerial tools that effectively relate planning, resource allocation and assessment. See the description of the AFB process under element 2 of standard 2.

Because AFB is a heavily tuition-driven model it is important for deans to have access to data that accurately tracks their enrollment and course credit data (see diagram of how tuition dollars are allocated). These data are critical planning elements for the units and are contained in the data warehouse maintained by Institutional Research. The following are examples of the data used (view1; view2). The Office of Budget and Resource Studies works with unit deans to provide relevant budget information that assists the units in the development of their unit priorities, revenue projections, and faculty hiring projections. These plans are developed, discussed and assessed in regular meetings with the Executive Vice President for Academic Affairs.

The Center for Organizational Development and Leadership (ODL) provides a critical dimension to assessment activities at Rutgers. The center offers consulting services and workshops to assist academic, administrative, and service departments in organizational assessment, goal clarification, measures and effectiveness evaluation of their core programs and services (including focus groups and web-based surveys), and in using results in planning. As noted in standard 2, element 6 ODL has played a critical role in the development of the Excellence in Higher Education (EHE) model in higher education. This model of organizational assessment has been applied to institutions of higher education and stresses both assessment and the use of the assessment process for institutional improvement. Forty, or so administrative and academic units at Rutgers have participated in the EHE process (see also EHE at Rutgers).

Another major effort of institutional assessment has been the effort to assess and better understand the way the university is perceived by its many different constituencies. The first step in this process was the University Relations Constituency Study, which provided the basis for a process of continual assessment of how Rutgers is presenting itself to its varied external constituencies, including the Rutgers Visual Identity Manual.

A Matrix of Education Support Technology provides an overview of goals for education support technology at Rutgers and a summary of assessment methods and plans across the university community.

The Office of Institutional Research and Academic Planning (OIRAP) gathers, analyzes, and uses data to inform institutional planning, policy development and decision-making. The Office provides reporting, assessment, benchmarking, planning, and public information services to support institutional effectiveness and to respond to the needs of the university community and the citizens of New Jersey. See the office overview of its structure, mission, and the range of its activities. The office compiles and maintains the University Fact Book which provides a ready and comprehensive source of historical and current information on students, faculty, finances, facilities, and instruction. It also includes substantial benchmark data on Rutgers and its peer group, the public members of the Association of American Universities. OIRAP also has developed dashboard indicators that are used by senior administrators and the Board of Governors to assess how the university is meeting its institutional goals.

As an active member of the American Association of Universities Data Exchange, Institutional Research collaborates with respective offices of other member schools of the Association of American Universities (AAU) to develop a set of comparative indicators for both undergraduate and graduate education. The development of these indicators will be used by the AAU and its member schools to not only facilitate internal assessment of institutional effectiveness, but provide critical information about institutional performance to external stakeholders. Many of these indicators are planned to be ready for use by the individual AAU institutions in spring 2008.

A selection of regular and ad hoc reports used for institutional assessment and produced by OIRAP include studies on student economic and racial/ethnic diversity; student retention and graduation rates; periodic Economic Impact Statements; reports to the NCAA and Department of Education concerning athletics for each campus (New Brunswick, Newark, and Camden); Title II – Annual Questionnaire on Teacher Preparation Programs; and Legislative District Summaries.

When opportunity presents itself, the office works closely with outside organizations engaged in researching important topics in higher education. Two recent efforts have resulted in efforts to comparatively evaluate the success of Rutgers in recruiting, retaining, and graduating students. The first is a follow-up study by the Mellon Foundation to their original College and Beyond project to which Rutgers has agreed to participate. The second is a study by The Pell Institute titled, “Demography is Not Destiny: Increasing the Graduation Rates of Low-Income College Students at Large Public Universities.”

Through its data warehouse, OIRAP is able to provide information to central administrators and the many decanal units of the university and conduct special analytical studies on a need to know basis. The data warehouse contains diachronic and current information on students, faculty, and staff. The information included in the data warehouse comes from various university legacy data systems, such as the Student Records Database (SRDB), Human Resources, Course Scheduling, and the Course Analysis System (CAS), as well as the state's Student Unit Record Enrollment (SURE) database. Other sources of data include student placement data from the College Assessment Research System (CARS) and faculty information from the Faculty Survey. Data from the new Student Financial Aid system and data from the Office of Research and Sponsored Programs is also planned to be incorporated into the system. Entry into the data warehouse can occur through different access points (direct querying, On-Line Analytical Programming (OLAP), etc. – see linked diagram. Examples of the type, level and detail of the data that are available from the data warehouse can be found in the linked file of selected data views. The data warehouse also contains the source data for the all funds budgeting system (see standard 2, element 2).

OIRAP collects information on faculty activities through its management of the Rutgers Faculty Survey. The Faculty Survey provides faculty with a single point of data entry for all their required data reporting needs such as tenure and promotion forms and grant application forms. Reports of faculty activity are available to deans and other administrators (see link for an example of a report at a Dean’s level). In addition, OIRAP collects and produces reports on instructional workload activity at Rutgers. These reports are available to central and decanal administrators through data views produced by OIRAP staff and placed on the office’s reporting server.

OIRAP engages in an active survey research program, which provides a rich source of primary data on students’ assessment of their academic and student life experiences. The office regularly administers the Upper Division and Graduating Student Surveys. These and other surveys (see listing of undergraduate student surveys) have informed decision-making activities including the Transforming Undergraduate Education initiative, student services improvements, the proposed revision of the student code of honor, etc. Graduate student surveys including the Ph.D. exit survey are used by graduate programs as an essential assessment tool. Surveys of faculty and staff have also been administered to help inform decisions on a variety of topics including faculty service, undergraduate research and learning outcomes, global and international initiatives, academic integrity, and campus climate concerns.

Other institutional assessment-related activities of the office include:

· overseeing the development of new programs, which has recently been revised to include specifications for assessing learning outcomes in the development of new programs

· providing staff support for the Committee on Academic Planning and Review (see standard 2, element 2)

· administering the testing and placement program of incoming students and reporting on the effectiveness of the program functioning as the university liaison to the Association of American Universities Data Exchange, an organization that collects and exchanges benchmark data on institutional effectiveness among member institutions (a description and list of data exchange items available to participating members can be found here)

· coordinating the university’s participation in national surveys such as the Cooperative Institutional Research Program (CIRP) and the National Survey of Student Engagement (NSSE)

· managing university reporting to national agencies such as the U.S. Department of Education (e.g., IPEDS ); state agencies such as the Commission on Higher Education and the Department of Education ; NCAA – athletic progress reports (New Brunswick, Newark); and college guides (e.g., U.S. News, CDS, Peterson’s)

evidence that assessment results are shared and discussed with appropriate constituents and used in institutional planning, resource allocation, and renewal (see Standard 2: Planning, Resource Allocation, and Institutional Renewal) to improve and gain efficiencies in programs, services and processes, including activities specific to the institution’s mission (e.g., service, outreach, research); and

3. written institutional (strategic) plan(s) that reflect(s) consideration of assessment results.

The incorporation of assessment results that reveal their discussion with constituents in written plans and improvement efforts can be found in standard 2, element 2.

Roadmap Home