Usage

The completion prediction is available immediately for use for each student, including first-year students, without restriction. At an aggregate level, the model applies at any time, across the student's entire time at the institution. There is a slight improvement of the prediction after one year, but, overall, this prediction tracks well when comparing the prediction to actual results.

Yes, individual classes can be removed. Contact your Civitas team to have a particular class hidden for you.

No. Course Insights only includes each student once. For students who have retaken a course, their most recent attempt is included in the calculation. This ensures that each student who completes the course is only counted once for the purpose of calculating insights.

Course Insights displays letter grades, including A, B, C, D, F, and W. These are whole letter grades only: there is no differentiation in mapping for +/- grades. For example, a B+ would be mapped as a B and all insights are determined using these letter grades.

Why? Distributing the historical student population among additional grades would reduce the statistical significance of the course insights.

Students who passed a class as a pass/fail student are excluded from the data set for that course. Students who failed a class as a pass/fail student are included in the data set with students who received an F in the course.

Yes, a single course can have multiple signals. If it has multiple signals, an icon will not display on the chart. This is the hierarchy of signals that would appear:

Yellow Flag
Challenge
Late Hurdle
Qualifier

Because of the historical nature of the signal detection, Course Insights works with filters that won't change too frequently over the course of a student's academic career (major changes notwithstanding).

Not currently. The capability is designed to understand the relationship between student grades and student outcomes (persistence and graduation) for a particular course, not a section or instructor.

If there is a crosswalk between old sections and new sections, and new courses are generally structured the same as older courses (no major changes to curriculum, expectations, etc.), this can be accounted for in the deployment process. If there is no crosswalk, or a course changes significantly, it will take 6-10 years of data on the new course to show persistence and graduation data.

All courses on the bubble chart are top influencers of lift for either completion or persistence (depending on which you're looking at). They have an outsized impact on completion or persistence rates, even if they don't fit one of the four course signals. The list on the right is a ranking of completion/persistence lift for all classes, no matter which year it's generally taken. The higher it is on the list, the higher it is on the y-axis of the chart.

Calculations

The average course is determined by identifying the top 25 courses (those with the largest enrollment) that are typically taken in the same year as the selected course for the filtered group of students. To see a list of all 25 courses included in the calculation of the average course Key Insights, click the Avg Course column header on the Course Details page.

For each letter grade, we look back at historical graduation rates for students who get each grade. The lift is the average change in graduation between each letter grade. In other words, we look at the boosts to graduation if students were to move from an F to a D, D to a C, C to a B, and B to an A, and average those deltas to calculate the graduation lift.

For each letter grade, we look at historical persistence rates for students across each grade. The lift is the average change in persistence between each letter grade. In other words, we look at the gains in persistence if students were to move from an F to a D, D to a C, C to a B, and B to an A, and average those deltas to calculate the persistence lift.

The sum of the differences between each grade is then divided by the total number of students who persisted in that course: students who would have persisted with a higher letter grade divided by the actual number of students who persisted.

The year typically taken is determined by the year in the academic career during which most students take the course, also known as the mode year.

Important:

"Year" in Courses is calendar year, not academic year. When a course is in "Year 3" it means "taken in the third year at the institution" not "taken as a Junior".
"Year" does not mean the year a majority of students take the course, but the year it is taken most often.

We use up to 10 years of historical data provided by the institution, excluding years outside of active completion windows and excluding terms for which outcomes aren't yet known. Non-degree-seeking students are excluded.

Graduation — Only uses data for students who have had a full 6 years to graduate. This means that, if given 10 years of historical data (e.g., 2006-2016), only the first 4 years of data have students meeting the 6-year threshold.
Persistence — Only uses the most recent 4 years of data provided by your institution.

Yes. For example, if it's May 2023 with a 4-term year, then the following terms are used for insights:

Graduation Insights (based on the last 10 years in which students had 6 years to finish):

2013 - Winter, Spring, Summer, Fall
2014 - Winter, Spring, Summer, Fall
2015 - Winter, Spring, Summer, Fall
2016 - Winter, Spring, Summer, Fall

Persistence Insights (based on the most recent 4 years of course outcomes):

2019 - Winter, Spring, Summer, Fall
2020 - Winter, Spring, Summer, Fall
2021 - Winter, Spring, Summer, Fall
2022 - Winter, Spring, Summer, Fall
2023 - Winter (outcomes are not yet known for the other terms)

The earliest term included in the full term set is Winter 2013. This will be the earliest term until December 31, 2023. On January 1, 2024, the earliest term included will be Winter 2014.

There is no set minimum. If at least one student took a course, then that data is in the student enrollment data set.

The Degree Program Alignment compares a student's academic career (course credits) to those of students who graduated successfully in that same program of study. This score quantifies how well they're progressing compared to your institution's historical data.

How it works:

For each course per degree program (including accepted transfer credits), we calculate the percentage of graduated students who received course credit (earned_course_percent). It captures: How many graduates took that particular course?
For each student and graduate of a degree program, we calculate the average earned_course_percent of all courses that they earned credits from. This is the degree program alignment score.
The degree program alignment score compares each student against the average alignment score value of graduates of the same degree program.

What to expect:

Because earned credits change once the term completes, expect alignment scores to hold steady during a term.
Students with high numbers of earned course credits do not see a strong effect on their alignment score, because the score is an average across all they took.
Students with low numbers (who only completed a few courses) could have higher alignment scores than students who are about to graduate.
New students (without course credits) will have no degree program alignment score (null).

In addition to key factors such as the student's GPA, completed credits, and degree program alignment score, these are common variables driving the completion prediction model:

Is transient — Whether the student is taking courses at your institution with the intention of transferring those courses to another school where they are seeking a degree. Transient students are non-matriculating.
Is full-time — Whether the student is considered full-time in the current term. Full-time and part-time designations are institution-specific.
Sections attempted — Number of sections being attempted in the current term.
Sections attempted next term — Number of sections registered for the next term, weighted by the fraction of other students who have registered sections for that term.
Failure Ratio (cumulative) — The student's ratio of failed sections to attempted ones, across their entire academic career.

Important Which variables are most impactful varies by institution.

No, they are based on all students in the last 10 years who fall within the group as you have currently filtered.

In the CSV export of Courses data, the Score for a course is a calculation of persistence impact based on the historical persistence of students who received grades in that course. The Score calculates the impact on persistence or graduation if the student's grade for that course would have changed to a higher value, say from B to A. That impact is derived from actual persistence data at your institution. For each letter grade (A, B, C, D, F, W, and so on), the score works with these counts:

Total number of students who received that grade.
Number of students receiving that grade who persisted.
Number of students receiving that grade who did not.

You can then focus on improving grades for certain courses knowing that it improves their overall persistence/graduation rate.

It ranks courses based on outcome potential, to surface high-impact courses for intervention. When you drill into a course's details, you can select to see the potential boost for either persistence or graduation. This score calculates the impact of a single letter grade increase for one student to project the potential lift in rates across the entire filtered student group, which reveals the possible lift in persistence and graduation rate for the average student taking that course if they had achieved the next higher course grade. That lift is shown next to the lift for the average (typical) course at your institution, to give you a relevant comparison.

Important The "boost potential" score projects how much higher persistence or graduation rates would have been if the average student's final grade in one course term were one letter higher. It is not calculating a time range over how many terms it would take for the graduation rate to increase.

Formula: Only students who have taken that course are included, not the entire population. The formula estimates the persistence rate change if the grade had been higher, with extra checks to ensure that better grades do yield better outcomes and that there are enough students to run this:

Boost = Persistence grade signal × (% of filter students who took course ÷ Avg # of terms a student enrolls in)

You may see few or no results for "Potential to boost persistence rates is highest?" even if you get results for a boost in graduation rates. This can happen when the number of students who persist is much higher than the number who graduate. In this case, a grade change will have less effect because it's a larger population (than only those who graduate).

Because the data sets are different. Persistence and graduation insights use their own data sets: persistence uses the most recent 4 years of data, and graduation uses earlier years of data. Within those different data sets, which courses have the highest rates of Ds, Fs, and withdrawals will differ.