The BA in Contemporary Asian Arts, a fictitious program, is being used in this module to demonstrate the creation of all the elements which are used in the assessment process.
We will review each of them so you have a good idea of what these elements in the assessment process should look like. Before we do that, we will set the stage by describing this fictitious program in more detail so you have a feeling of what it is about.
The practice of Chinese and Japanese calligraphy has been considered for centuries as an aesthetic, spiritual and contemplative practice and the highest art form of China, Japan, and Korea.
- Marlow Brooks, Naropa faculty & Artist
Background
From the Song Dynasty to the early 21st century, the practices of calligraphy, poetry and ink painting have been intertwined. Painting, poetry and calligraphy are known as the ‘Three Perfections’.
This program seeks to preserve the knowledge of these contemplative art forms and to serve as the means for training dharma arts educators.
The significance of the traditional forms is not lost on contemporary artists, who are increasingly marrying past practices with a drive to subvert expectations and communicate multiple meanings in works which appropriate, reinvent, re-contextualize, and reconsider the past.
This program is seated in the School of the Arts and was developed in collaboration with faculty in the School of the Arts and the Departments of Education and Wisdom Traditions.
Grounded in the study of traditional Asian art forms, the program trains students in contemplation and perception to sharpen artistic intelligence. Students develop technical proficiency and confidence in their creativity, and manifest their personal vision informed by scholarly thinking, investigation, and invention. Students are prepared to be scholar artists and to continue on to practice or teach the contemporary application of the dharma arts.
The 36-credit major in Visual Arts includes studio electives in Chinese brush painting and calligraphy, sculpture, pottery, and photography. There are required courses in art history, contemplative and studio practice, poetry, flower arranging, teacher education, and arts education. Students are mentored in Advanced Studio Practice, Warrior Artist, and Portfolio and Gallery Presentation. The final project included a digital portfolio, artist's statement, teaching internship, and an exhibition in the Nalanda Art Galleries.
Graduates of the program go on to work as practicing artists and arts educators. They may also choose to continue their education at Naropa by applying to study in the MA Education or MA Transpersonal Counseling, Art Therapy programs.
We will now take a look at a mission statement for this program.
A mission statement should be concise and memorable. It should reflect the work of the program and be aligned with the university mission.
___
A good general formula for a mission statement is:
The mission of _________________ is to ___________________ by providing___________________ to ________________.
The mission statement for this program is:
The mission of the BA Contemporary Asian Arts is to prepare students to be confident, technically proficient studio artists and arts educators by drawing on the disciplines and practices of the dharma arts and contemplative teacher education.
Once the mission is set, the next thing to consider is goals for the program.
There should be 3 - 5 general goals that derive from the mission of the program. Because of their general nature, goals are not measured.
Goals focus on what faculty members do to create the educational setting within which students thrive.
Goals start with the statements like: to provide, to nurture, to promote, to prepare, to enhance, to structure, to encourage, to establish, to serve. They are written broadly and are the basis from which more specific, measurable Program Learning Outcomes are derived.
Goals for this program are:
- To provide a developmentally sequenced curriculum of studio and theory courses that train students in the major modes and genres of the dharma arts.
- To promote contemplative experiences that develop students' perception and ability to focus.
- To train students as art educators.
- To prepare students for successful careers as artists and/or arts educators by providing them with training in essential skills in business, marketing, and teaching.
After goals have been set, Program Learning Outcomes can be written.
Program Learning Outcomes state what a graduate of a program can do at the end of their studies that they could not do at the beginning - the lasting results of their studies.
They should be aligned with the program mission statement and goals.
Program Learning Outcomes (PLOs) are a broad, general type of student learning outcome. They are a developmental achievement: students build up to their attainment of a Program Learning Outcome through multiple courses in the curriculum. This is the principle behind the undergraduate curricular arc and curriculum mapping (discussed in other modules).
As the program's high level learning outcomes, PLOs are a concise, high-level expression of what students learn in the program. More- specific, detailed learning outcomes are stated as Student Learning Outcomes on individual course syllabi.
Program Learning Outcomes must be measurable, meaning there is a way to look at evidence of student learning to see how well students are attaining them.
An appropriate number of Program Learning Outcomes is between 5 - 9.
Each academic program should have a plan to assess all their program learning outcomes within 2 years.
For some programs, it may be challenging to narrow down their Program Learning Outcomes to this number. Keep in mind that:
- PLOs focus on what a graduate can do after completing their degree.
- The focus is on integrative capacities, critical abilities and knowledge and skill developed over time.
- They state broad expectations of student performance, and describe what a student can do with what he/she knows.
- They focus on lasting results of courses, programs, and missions.
- They include cognitive abilities, psychomotor abilities, competencies and proficiencies.
- They are measurable.
Ideal Program Learning Outcomes start with the phrase, Students verb.... such as:
- Students demonstrate knowledge of...
- Students interpret...
- Students create...
- Students synthesize and translate...
The Program Mission, Goals and Learning Outcomes worksheet posted on MyNaropa gives many more useful examples of verbs to use when writing outcomes.
Following these guidelines, the Program Learning Outcomes for the BA in Contemporary Asian Arts are listed below.
Program Learning Outcomes
- Students demonstrate knowledge and skill in the practices of the contemporary Asian arts.
- Students make works of art that reflect critical thinking, competent technical execution, and experimental combinations of traditional and contemporary media.
- Students articulate how contemplative practices are related to their art.
- Students demonstrate an artistic and visual vocabulary and the artistic skills of a working professional artist.
- Students integrate the arts with contemplative practices in the teaching/learning environment.
- Students engage in important art-related business skills.
- Students create a professional ePortfolio.
While writing Program Learning Outcomes, think about how they will be measured.
Program Learning Outcomes are measured with evidence or artifacts of learning, also called measures of learning.
What are artifacts or measures of learning?
They are objects created by students during the course of instruction. To be considered an artifact or measure, an object needs to be lasting, durable, public, and materially present. Through creation of these objects, students' knowledge is made visible. The creation and display of artifacts or measures of learning should provide students with opportunities for engagement, revision, and feedback,
There are different types of artifacts or measures of learning and they are classified as direct and indirect measures.
Direct Measures
Direct Measures of learning are artifacts like:
- Quizzes and tests
- Portfolios
- Writing samples
- Capstone projects
- Research papers
- Master's papers
- Presentations
- Theses
- Exhibitions
- Performances
- Team/group projects and presentations
- Oral exams and warrior exams
- Online course discussions
- See the module Measures of Learning for more examples.
The strength of using direct measures is that you are capturing a sample of what students can do, which can be very strong evidence of student learning.
A possible weakness of using direct measures is that not everything can be demonstrated in a direct way, such as values, perceptions, feelings, and attitudes. For this reason, you may want to use some indirect measures of learning.
Indirect Measures
What is an indirect measure of learning? It is a measure based upon a report of perceived student learning.
Examples of indirect measures of learning are:
- Number of graduating students pursuing their education at the next level
- Reputation of graduate or post-graduate programs accepting graduating students
- Surveys, questionnaires, open-ended self reports dealing with current students perceptions of their own learning
- Focus-group or individual interviews dealing with alumni's perception of their own learning or their career satisfaction
- Surveys or questionnaires dealing with faculty, staff, or internship supervisors' perceptions of student learning
- Honors, awards, scholarships or other forms of public recognition earned by students and alumni
- Employment or placement rates of graduating students into appropriate career positions.
- Number or rate of students involved in faculty research, collaborative publications and/or presentations, service learning, or extension of learning in the larger community
- See the module Measures of Learning for more examples.
The strength of indirect measures is that they can assess qualities of student learning such as values, feelings, perceptions, and attitudes, from a variety of perspectives. The weakness of this approach is that, in the absence of direct evidence, assumptions must be made about what exactly a self-report means and how well perceptions match the reality of actual achievement.
That is why it is good idea to use a combination of direct and indirect measures of learning.
It is a best practice to use the measure that most closely fits the outcome being assessed.
The BA in Contemporary Asian Arts has selected to use the following artifacts as direct measures of learning:
- Digital portfolios
- Capstone project/exhibition
- Research papers
- Artist's statement
- Warrior exam
- Student teaching
They also selected the following indirect measures to assess how well students attained their Program Learning Outcomes:
- Satisfaction survey of current students
- Alumni survey re: satisfaction and employment in desired careers
The measure selected should be an appropriate measure of each learning outcome being assessed.
Achievement Targets
After measures are selected, faculty set targets for the level of desired performance of their students on those measures of learning. There should be a meaningful performance target for each measure.
Some achievement targets set by the BA in Contemporary Asian Arts are:
- 80% of students will score a B or higher.
- An average of 90% of students will complete a satisfactory digital portfolio.
- 80% of students will score a 4 or above in every category of the performance rubric.
- 75% of responding alumni will report success as in the arts business or art education.
Assessment Findings
Assessment findings is a term that refers to information about how well students are achieving the program learning outcomes and meeting the learning targets that are set. Once you have collected artifacts or measures of learning, you can analyze them.
After you analyze assessment findings, you can then determine where a different or new approach or emphasis may be useful. It is at this place that you will create action plans to improve student performance and/or attainment of program learning outcomes.
Action Plans
Based on their assessment findings, the faculty in the BA in Contemporary Asian Arts determined that they needed to create action plans to improve the quality of student installations, to enhance data collection for digital portfolios, and to improve assessment of the internship.
An action plan should also include a plan for implementation of the changes. Implementation plans should consider the responsible person or group and any additional resources that may be needed.
Note that in this context, the term action plan refers to a specific programmatic or curricular change to improve student performance of a program learning outcome. Besides a curricular change, an action plan might contain any of the following changes:
- A course revision or addition
- A change in pedagogy
- The development of a rubric to assist students in understanding the required performance level
- Personnel changes related to core or adjunct faculty, academic support staff, administrative, or GA support
- An assessment plan change
- Equipment or software
- Academic support services such as improved advising, coaching, or internship support
- Faculty development/training
Once you have finalized your plan, you need to implement the changes.
Assessment Plan
As we are using the term at Naropa, an assessment plan is the comprehensive plan for assessing your program. It contains the following components:
- The year in which each of your program learning outcomes will be assessed
- The measures used to assess student learning
- Student achievement targets
- Your action plans and which program learning outcomes they are meant to improve, along with the academic year in which they will be put into effect.
- The year in which your Program Learning Outcomes will be reassessed to determine whether the changes had a positive effect on student learning.
Continuous Improvement
As you can see, when a program engages in an assessment process, there is a container and structure to look at student learning and determine how it can be improved - so there is a rolling cycle of continuous improvement.
We are also striving to continuously improve our assessment practices at Naropa so they are easy to apply, clear, and useful. There is an Assessment Plan & Tracker worksheet available to help you walk through, consider, and complete all the elements of the assessment process. There is a blank version, and to support you, a completed sample outlining the assessment plan for the fictitious BA in Contemporary Asian Arts, used in this case study.
These and many other assessment resources are available on MyNaropa under Adminstrative Depts: Academic Affairs: Assessment.
If you ever need assistance with assessment or have any questions or feedback, please contact the Faculty Assessment Committee or Janine Ibbotson, the Program Manager for Institutional Effectiveness in the Provost's Office at 303-245-4813 or jibbotson@naropa.edu.
Credits:
Created with images by Kanko* - "greeting / 年賀状" • fourcotts - "stripes" • Brett Jordan - "Calligraphy on Red" • Kanko* - "筆と筆巻き" • Kanko* - "inkstone and brush / 硯と筆" • lungstruck - "My name スコット spraypainted on crate" • John Rabbit - "kaki goori - shaved ice" • TANAKA Juuyoh (田中十洋) - "Japanese old style house interior design / 和室(わしつ)の内装(ないそう)" • TANAKA Juuyoh (田中十洋) - "Japanese house traditional style interior design / 和室(わしつ)の内装(ないそう)" • philographia-jou - "_DSC0063" • Jason. P. - "Chinese Calligraphy" • Steve Snodgrass - "Chinese" • Steve A Johnson - "Magnetic Fridge Poetry" • Muffet - "carp on floral" • The Wandering Angel - "Incomplete Canvas" • Dominic's pics - "Reply - SAWAMURA Sumiko 沢村澄子" • angelhsu - "毛笔酥 ｜ Chinese calligraphy brush crisp" • lydia_x_liu - "Calligraphy Brushes" • Wonderlane - "Statue of a red Buddha holding a begging bowl, blue and yellow aura, pink, orange, and blue lotus, stupa niche, vajra, Swayambhunath, Kathmandu, Nepal" • philographia-jou - "_DSC0060" • gill_penney - "beijing 3 294" • clayglazepots - "Box" • harrisdeller - "Untitled" • cbaskin99 - "Pasta Tower" • shaire productions - "Chinese Calligraphy Patterns" • stephaniecraig.net - "capsula" • boblloydb - ""VASE"" • summonedbyfells - "REFLECTIVE COLOURS TOAD" • anathea - "refrigerator poetry" • Roger Wagner - "Angel Island Poetry on Wall" • Fæ - "Landscape LACMA 82.4.3 (4 of 6)" • Empress of Blandings - "雪 xuě" • Empress of Blandings - "时 shí - first attempt" • Empress of Blandings - "时 shí" • Empress of Blandings - "博 bó - first attempt" • Empress of Blandings - "博 bó - corrected!" • Empress of Blandings - "回 huí - first attempt" • Empress of Blandings - "塗鴉 túyā" • Conveyor belt sushi - "Japanese flower arrangement 34, Ikebana: いけばな" • Alan Cleaver - "Coloured pencils" • Swallowtail Garden Seeds - "Ukiyo e woodblock print of peonies and a canary by Japanese artist Katsushika Hokusai (1834)" • LoboStudioHamburg - "cherry blossom japanese cherry blossom spring" • Conveyor belt sushi - "Japanese flower arrangement 48, Ikebana: いけばな" Training module created by Janine Ibbotson.