Blog Viewer

I recently presented an overview of the existing global learning programs in my Mechanical and Aerospace Engineering (MAE) Department and STEAM’D programs at West Virginia University (WVU) at the recent NAFSA Colloquium on “Global Learning in STEAM’D: A Cross-Disciplinary Approach” that was held on May 31, 2017 in Los Angeles, California. Over my career, I have taught mechanical and aerospace engineering students for the past 43 years. However, it is only within the past three to four years that I have become involved in global learning programs. In 2014-15, I worked with colleagues at the University of Rome Tor Vergata to renew our existing MOU and the study abroad student exchange agreement between the two universities. I also worked at UTV in May-June of 2015 to teach WVU students there. Since that time, I have worked to improve the course credit transfer process because this was the most common  suggestion from our students where they saw a need for improvement in the program. During the 2015-16 academic year, I was able to get the MAE-UTV course equivalencies formally approved by my departmental faculty curriculum committees and a table of these course equivalencies has appeared in the WVU online catalog since June of 2016. I also have served for the past two years as advisor to our WVU students who are interested in participating in this study abroad program. Through these experiences, I have become convinced of the significant value of study abroad programs.

My primary reason for attending the NAFSA Annual Conference and the STEAM’D Colloquium was that I had become convinced that both my department and WVU would benefit by expanding and better institutionalizing our assessment of the outcomes of these types of programs, and I was hoping to get guidance on how we could establish some type of formal assessment program in MAE. One goal of this assessment would be to help inform future work to improve the programs. Also, I could see that if this assessment program could document concrete benefits to the WVU MAE students who participate in study abroad programs, this would help to make me personally more comfortable in my role as advisor and an advocate specifically for the UTV program. At the NAFSA Colloquium, I was able to get several ideas of how to establish a formal assessment program for the MAE global learning programs. But since I believe that more information is always better, I am seeking additional input as to current best practices to assess learning outcomes for global learning programs through this blog post.

From reading the materials that were distributed to attendees of the STEAM’D Colloquium (especially Learning Across Cultures, edited by Gary Althen, 1994), I have learned that several instruments to assess cultural and global learning exist, such as the Beliefs Events and Values Inventory (BEVI), the Global Perspective Inventory (GPI), the Intercultural Development Inventory (IDI), and the Global Learning/Intercultural Knowledge and Competence VALUE Rubrics. However, at WVU it unfortunately does not appear that there is funding available to utilize these instruments; over the past three academic years our state appropriations have been reduced by 10%, 6%, and 4% (anticipated), and we were not particularly well-funded to begin with. Thus, it appears that here at WVU we will be developing our own in-house global learning assessment plan(s). I have also begun reading about the best practices in assessment of global learning outcomes as developed at other colleges and universities. One obvious but key consideration is that one must first clearly define what the goals of the assessment plan are to be, before moving on to selecting or developing the assessment methods. A consensus says that it is best to use multiple assessment methods; e.g., use both quantitative (surveys, etc.) and qualitative (interviews etc.) methods. Further, I have now learned about several simplified concepts, such as “culture shock”, “culture bump”, and the ”U-curve” of adjustment, all of which will help me when I am counseling and advising our students who are considering participating in a study abroad experience.

After returning to WVU from the NAFSA Annual Conference, I also got information about WVU Education Abroad Program current practices relating to program assessment from Program Coordinator Jessica Yerkovich and Global Affairs Advancement Coordinator Henry Oliver. Currently, the WVU Education Abroad Office provides a pre-departure orientation program, organized by geographic region, for all participating students. This program will be moved online in the near future, and the students will then be required to pass a quiz over the program content before they leave for their study abroad experience. All students also provide pre-departure feedback via a questionnaire that they complete as part of their study abroad application. Faculty-led study abroad programs are also assessed after the students return to WVU through the use of the online course evaluations (“Student Evaluation of Instruction”, or “SEI” surveys) that are administered for each course. One additional staff person has been added the WVU Education Abroad Office whose responsibilities include the development of a reliable data base of all WVU study abroad alumni, and implementation of a more complete study abroad program assessment plan. This will allow the development over time of longitudinal studies that could help to document program benefits to our study abroad alumni.

I also recently met with Statler College of Engineering and Mineral Resources Associate Dean for Academic Affairs, Dr. David Wyrick, to propose adding questions about study abroad experiences to the existing graduating student survey that is administered to our students during their final semester just prior to their graduation from WVU. A meeting between Dr. Wyrick, myself, and the Statler College Coordinator of Corporate Relations and Career Assistance, Lloyd Ford, has been scheduled for later this July to develop a plan to move forward to establish such a survey assessment tool. To that end, I have written several candidate questions for the survey, as shown below. I plan to suggest that we meet with all Statler College faculty who are involved in existing study abroad programs, to actively seek their participation in the development of the assessment survey, by first inviting each of them to also develop their own lists of survey questions, and then to ask them to help to organize the total pool of questions into a coherent assessment tool. It is hoped that some guided open-ended questions will also be developed for the survey.

It is hoped that this initial meeting will move both my MAE Department and the Statler College towards a start in the assessment of our global learning programs, with goals of developing more of a community and sharing of best practices among the various study abroad program leaders, and helping us to learn what works best for the students here at WVU who choose to participate in our study abroad programs.

My current list of suggested “straw-man” survey questions, in no particular order:

1. My study abroad experience will help me in my future career. (Strongly agree àStrongly disagree, on a 5-point scale)
2. My study abroad experience was extremely enjoyable. (Strongly agree àStrongly disagree)
3. My study abroad experience was extremely stressful. (Strongly agree àStrongly disagree)
4. I believe that I made several new lifelong friends during my study abroad experience. (Strongly agree àStrongly disagree)
5. My study abroad experience has made me more flexible or adaptable to new situations. (Strongly agree àStrongly disagree)
6. My technical coursework during my study abroad experience was quite challenging. (Strongly agree àStrongly disagree)
7. My technical coursework during my study abroad experience was good preparation for the subsequent courses. (Strongly agree àStrongly disagree)
8. My instructors during my study abroad experience were excellent. (Strongly agree àStrongly disagree)
9. My study abroad experience has made me more aware of differences among different cultures. (Strongly agree àStrongly disagree)
10. My study abroad experience has made me more aware of similarities among different cultures. (Strongly agree àStrongly disagree)
11. During my study abroad experience I feel that I learned a great deal about the culture and the people of my host country. (Strongly agree àStrongly disagree)
12. Additional help upon my arrival at my study abroad destination would have greatly reduced my initial level of stress. (Strongly agree àStrongly disagree)

Your input and suggestions of how to begin to construct a formal assessment program for MAE global learning and study abroad programs will be extremely valuable in this process, and will be greatly appreciated.

(Written by John Kuhlman, Professor at West Virginia University)