Assessment Details
Academic Year: 2020-2021 Level: Graduate
Campus Department: Morrissey College of Arts & Sciences [UG and Grad]
Program Type: Major [UG] / Program [Grad]
Program Name: Geology/Geophysics MS (Link)
Description of Data Collection:
a. All MS and PhD students are required to submit a preliminary draft thesis proposal as the final paper for the Earth Systems Seminar (EESC6691). This class is required of all incoming MS students and PhD students who have not completed an MS degree. The Earth Systems Seminar is taken in the Fall semester of their first year.
b. All MS and PhD students who have not completed an MS degree must submit a final thesis proposal during the third semester of the program.
c. All MS and PhD students who have not completed an MS degree must take an Oral Qualifying exam during the third semester. A faculty committee of three administers the exam. The student’s academic advisor serves as the chair of the committee. At the conclusion of the exam, the chair provides the student with a written summary, including an assessment of how well they have achieved the outcomes above at this stage of the program. A copy is provided to the department graduate committee, and is placed in the student’s file.
d. Placement and publication data for graduate students are recorded annually in the department office and reviewed by the Graduate Program Committee.
e. The Graduate Program Director reviews student evaluations of Teaching Assistants.
f. A Graduate Student Research Colloquium is held every spring at the end of the semester. All graduate students make presentations and the faculty evaluates the students on a scale of 1 to 5 with respect to Outcome #5 above (i.e., speak effectively to professional and lay audiences about issues in the field). This year due to Covid the colloquium was held virtually by Zoom.
Review Process:
The Graduate Program Committee meets each year to review the above data and make recommendations to the whole department for improvement. The department discusses these recommendations at our annual faculty retreat in late May.
Resulting Program Changes:
Oral Exams: To encourage shorter time to finish the program, the date for the comprehensive exam and thesis proposal were moved to the third semester. The deadline had formerly been in the fourth semester. The change was implemented in AY13-14.
The Oral Qualifying exam assessment form (attached) was used for the first time during AY15-16. Overall the faculty is pleased with the results (typically between very good and excellent). In general, we are seeing improvement (or no difference, but good) for all outcomes. Students are typically ranked highest for learning outcomes 1,3 and 4 and lower for outcomes 2 and 5 (see Table 1). The committee concluded that the Oral Qualifying exam was not necessarily the best place to evaluate outcomes 2 and 5 due to the high stress nature of the exam. To address this, the committee now summarizes overall student outcomes for 5 based on discussion following the student colloquium. The change was implemented in spring 2017. Students continue to score on average lowest on outcome 2. Improving this outcome was a topic of this year’s faculty retreat. One idea was to modify our breadth requirement slightly to ensure students are getting a good distribution of courses. The current breadth requirement dates back 10+ years and no longer reflects the faculty in the department. Over the next year, we plan to reassess the breadth requirement to (1) ensure that it aligns with our current faculty, and (2) helps to better prepare students for outcome 2.
In AY19-20 we had an anomalously high number of students (2 of 7) who did not pass the Oral Qualifying exam on their first attempt (both students passed on the second attempt). In order to help better prepare students for the Oral Qualifying this year, the graduate committee met with all 2nd-year (and most 1st-year) students in September 2020 and went over what to expect on the Oral Qualifying exam. This appeared to help, reducing stress associated with the exam and allowing the students to prepare more effectively—resulting in successful outcomes on all 5 Oral Qualifying exams taken in Fall 2020. We plan to continue these informational meetings to help prepare students in the early fall of each year.
Results from Oral Qualifying exam assessment, student colloquium and teaching evaluations are summarized in the tables below. [see attachment]
Evaluation of TAs: We implemented a new online TA evaluation form in Spring 2020 that can be more easily analyzed to collect statistics (see example form below). Overall, our TAs are receiving ‘excellent’ and ‘very good’ for the vast majority of questions, though the 2020 and 2021 scores are slightly lower than before we went to the online evaluation form. Responses with 3 and lower are pulled and examined more closely and discussed with the grad student.
We attribute the lower scores since spring 2020 to the fact that our TAs were often working remotely and did not have the same opportunity to interact face-to-face with the students. It is also possible that the online nature of the evaluations has led to a difference in scoring. We plan to continue to monitor these evaluations as we move back to a more traditional in person teaching mode next year.
In addition, based on feedback from the TAs, the Graduate Program Committee plans to compile a short Roles & Responsibilities document that can be circulated to all TAs and instructors at the beginning of each semester. The purpose of this document is to help manage expectations and ensure consistency in TA workloads across the department.
Placement: Of the five students who completed their Masters degrees from summer 2020 to May 2021, one received a NOAA Knauss Fellowship, one is working at the environmental geotechnical company Wilcox & Barton, one is working for the Isosceles Group (an environmental, health & safety company in Boston), one is tutoring math & science, and one is currently applying to PhD programs. In addition, one of our current MS students is transitioning directly into our new PhD program, while another has been accepted to a PhD program at Princeton University. Our impression is that the vast majority of students (if not all) who have sought employment in the field or continuation to a higher degree have been successful. We have taken a more formalized approach to evaluate this recently by asking students what their plans are when they complete the Masters and following up on an annual basis with all our former students. We have fairly good information for the last four years.
New PhD Program: This academic year (Fall 2020) we welcomed our first incoming class of PhD students. Leading up to the inaugural PhD class, the EES faculty developed a new curriculum, evaluation metrics, and learning outcomes for the PhD program. These include the addition of an Oral Comprehensive exam that will be taken during the 3rd year of the PhD program and a new Broader Impacts course that will be required for all PhD students. The Broader Impacts course will be taught for the first time next academic year (Spring 2022). Note that like our current MS students, PhD students will take an Oral Qualifying exam in their 3rd semester in the program. In addition, PhD students will be expected to orally present and defend their PhD thesis before graduation. The full description of the PhD program and its requirements can be found in the Department of Earth & Environmental Sciences Graduate Student Regulations and Procedures Handbook. Because the PhD program began this year, no students have yet to progress to taking their qualifying or comprehensive exams. However, we were very pleased with the performance of our PhD students at the spring colloquium, where they received an average rating of 4.8/5 for their research presentations (Table 1b).
Date of Most Recent Program Review:
Oral Exams: To encourage shorter time to finish the program, the date for the comprehensive exam and thesis proposal were moved to the third semester. The deadline had formerly been in the fourth semester. The change was implemented in AY13-14.
The Oral Qualifying exam assessment form (attached) was used for the first time during AY15-16. Overall the faculty is pleased with the results (typically between very good and excellent). In general, we are seeing improvement (or no difference, but good) for all outcomes. Students are typically ranked highest for learning outcomes 1,3 and 4 and lower for outcomes 2 and 5 (see Table 1). The committee concluded that the Oral Qualifying exam was not necessarily the best place to evaluate outcomes 2 and 5 due to the high stress nature of the exam. To address this, the committee now summarizes overall student outcomes for 5 based on discussion following the student colloquium. The change was implemented in spring 2017. Students continue to score on average lowest on outcome 2. Improving this outcome was a topic of this year’s faculty retreat. One idea was to modify our breadth requirement slightly to ensure students are getting a good distribution of courses. The current breadth requirement dates back 10+ years and no longer reflects the faculty in the department. Over the next year, we plan to reassess the breadth requirement to (1) ensure that it aligns with our current faculty, and (2) helps to better prepare students for outcome 2.
In AY19-20 we had an anomalously high number of students (2 of 7) who did not pass the Oral Qualifying exam on their first attempt (both students passed on the second attempt). In order to help better prepare students for the Oral Qualifying this year, the graduate committee met with all 2nd-year (and most 1st-year) students in September 2020 and went over what to expect on the Oral Qualifying exam. This appeared to help, reducing stress associated with the exam and allowing the students to prepare more effectively—resulting in successful outcomes on all 5 Oral Qualifying exams taken in Fall 2020. We plan to continue these informational meetings to help prepare students in the early fall of each year.
Results from Oral Qualifying exam assessment, student colloquium and teaching evaluations are summarized in the tables below. [see attachment]
Evaluation of TAs: We implemented a new online TA evaluation form in Spring 2020 that can be more easily analyzed to collect statistics (see example form below). Overall, our TAs are receiving ‘excellent’ and ‘very good’ for the vast majority of questions, though the 2020 and 2021 scores are slightly lower than before we went to the online evaluation form. Responses with 3 and lower are pulled and examined more closely and discussed with the grad student.
We attribute the lower scores since spring 2020 to the fact that our TAs were often working remotely and did not have the same opportunity to interact face-to-face with the students. It is also possible that the online nature of the evaluations has led to a difference in scoring. We plan to continue to monitor these evaluations as we move back to a more traditional in person teaching mode next year.
In addition, based on feedback from the TAs, the Graduate Program Committee plans to compile a short Roles & Responsibilities document that can be circulated to all TAs and instructors at the beginning of each semester. The purpose of this document is to help manage expectations and ensure consistency in TA workloads across the department.
Placement: Of the five students who completed their Masters degrees from summer 2020 to May 2021, one received a NOAA Knauss Fellowship, one is working at the environmental geotechnical company Wilcox & Barton, one is working for the Isosceles Group (an environmental, health & safety company in Boston), one is tutoring math & science, and one is currently applying to PhD programs. In addition, one of our current MS students is transitioning directly into our new PhD program, while another has been accepted to a PhD program at Princeton University. Our impression is that the vast majority of students (if not all) who have sought employment in the field or continuation to a higher degree have been successful. We have taken a more formalized approach to evaluate this recently by asking students what their plans are when they complete the Masters and following up on an annual basis with all our former students. We have fairly good information for the last four years.
New PhD Program: This academic year (Fall 2020) we welcomed our first incoming class of PhD students. Leading up to the inaugural PhD class, the EES faculty developed a new curriculum, evaluation metrics, and learning outcomes for the PhD program. These include the addition of an Oral Comprehensive exam that will be taken during the 3rd year of the PhD program and a new Broader Impacts course that will be required for all PhD students. The Broader Impacts course will be taught for the first time next academic year (Spring 2022). Note that like our current MS students, PhD students will take an Oral Qualifying exam in their 3rd semester in the program. In addition, PhD students will be expected to orally present and defend their PhD thesis before graduation. The full description of the PhD program and its requirements can be found in the Department of Earth & Environmental Sciences Graduate Student Regulations and Procedures Handbook. Because the PhD program began this year, no students have yet to progress to taking their qualifying or comprehensive exams. However, we were very pleased with the performance of our PhD students at the spring colloquium, where they received an average rating of 4.8/5 for their research presentations (Table 1b).
Attachments (if available)