Source: New York Times

By Joseph Treaster, February 2, 2017

(Mirko Illic)

At Georgia State’s nursing school, the faculty used to believe that students who got a poor grade in “Conceptual Foundations of Nursing” probably wouldn’t go on to graduation. So they were surprised, after an analysis of student records stretching back a decade, to discover what really made a difference for nursing students: their performance in introductory math.

“You could get a C or an A in that first nursing class and still be successful,” said Timothy M. Renick, the vice provost. “But if you got a low grade in your math courses, by the time you were in your junior and senior years, you were doing very poorly.”

The analysis showed that fewer than 10 percent of nursing students with a C in math graduated, compared with about 80 percent of students with at least a B+. Algebra and statistics, it seems, were providing an essential foundation for later classes in biology, microbiology, physiology and pharmacology.

Georgia State is one of a growing number of colleges and universities using what is known as predictive analytics to spot students in danger of dropping out. Crunching hundreds of thousands and sometimes millions of student academic and personal records, past and present, they are coming up with courses that signal a need for intervention.

A little less than half of the nation’s students graduate in four years; given two more years to get the job done, the percentage rises to only about 60 percent. That’s no small concern for families shouldering the additional tuition or student debt (an average of more than $28,000 on graduation, according to a 2016 College Board report). Students who drop out are in even worse shape. Such outcomes have led parents and politicians to demand colleges do better. Big data is one experiment in how to do that.

Health care companies and sports teams have been working with predictive analytics for years. But the approach is in its early stages on campuses. A handful of companies have sprung up in the last few years, working with perhaps 200 universities. They identify trends in the data and create computer programs that monitor student progress and alert advisers when students go off historically successful pathways.

Dr. Renick uses Amazon to suggest how predictive analytics work. “When Amazon looks at all your choices of books and makes predictions that you’ll like other books, it doesn’t need to know why you’ll like the third book,” he said. “Our big data doesn’t need to know exactly why a student gets a bad grade. It just happens thousands and thousands of times. We’re looking at a pattern.”

Different courses at different universities have proved to be predictors of success, or failure. The most significant seem to be foundational courses that prepare students for higher-level work in a particular major. Across a dozen of its clients, the data analysts Civitas Learning found that the probability of graduating dropped precipitously if students got less than an A or a B in a foundational course in their major, like management for a business major or elementary education for an education major. El Paso Community College’s nursing hot spot was a foundational biology course. Anyone who got an A had a 71 percent chance of graduating in six years; those with a B had only a 53 percent chance.

At the University of Arizona, a high grade in English comp proved to be crucial to graduation. Only 41 percent of students who got a C in freshman writing ended up with a degree, compared with 61 percent of the B students and 72 percent of A students.

“We always figured that if a student got a C, she was fine,” said Melissa Vito, a senior vice provost. “It turns out, a C in a foundation course like freshman composition can be an indicator that the student is not going to succeed.” The university now knows it needs to throw more resources at writing, specifically at those C students.

At Middle Tennessee State University, History 2020, an American history course required for most students, has been a powerful predictor. The most consistent feature for those who did not graduate was that they received a D in it. “History is a heavy reading course,” said Richard D. Sluder, vice provost for student success, “so it signifies a need for reading comprehension.”

Before predictive analytics, Dr. Sluder said, many of the D’s went unnoticed. That’s because advisers were mainly monitoring grade-point averages, not grades in specific courses. “You take a student who’s getting A’s and B’s and you see a C in this one class,” he said, “and you’d say, ‘He looks fine.’ But, really, he was at risk.”

Such insight may revolutionize the way student advising works.

Among the historic data that the education tech companies analyze are SAT and ACT scores; personal and demographic information; courses that students are taking and grades they are getting; and behaviors like how frequently they are seeing advisers and tutors and how actively they are engaging in the campus networks where professors post homework assignments, lecture notes, comments and grades.

The analytics programs know the paths that successful students have followed. When a student veers off that path, like getting a low grade in a predictor course or taking a course out of sequence, advisers get an alert, a signal to reach out to the student and offer suggestions.

One perennial problem is that students register for the wrong classes. “Students think they’re signing up for a particular lab for chem majors,” Dr. Renick said, “but it turns out to be for nonmajors. In the past we might not learn about that until the student was already in the class or even after he had finished the course. Now we’re getting an alert and we get the student into the right class. We had 2,000 students like this last year.”

The advisers are linked so that their notes and meetings with students are available for colleagues, just as doctors in hospital systems share notes on patients.

Most universities have not been using big data long enough to see improvement in four- or six-year graduation rates. But some evidence indicates a positive impact. The schools that have seen some success have combined analytics with significant investment in other initiatives. Georgia State has more than doubled its advising staff, to about 100, and now meets the national standard of one adviser for 300 students. “It’s a heavy load,” Dr. Renick said, “but in the past we had 750 to one.” In 2016, Georgia State’s four-year graduation rate rose 5 percentage points, to 27 percent, and the six-year rate rose 6 points, to 54 percent.

Georgia State is sharing technology with 10 other universities in a coalition called the University Innovation Alliance. The alliance has pledged to graduate an additional 68,000 students in the next eight years.

Middle Tennessee State, which like Georgia State has many low-income, minority students, began working with the big data company EAB in fall 2014. Since then, it has more than doubled its advising staff, to 78, expanded free tutoring to 200 subjects from about 20, redesigned 27 foundation courses and started monitoring student progress weekly.

“All of this is important,” said Dr. Sluder. “But the technology is crucial. Take out all the technology and you just don’t get that end result.” Freshman retention at Middle Tennessee has risen 5 percentage points, to 76 percent, in just two years.

A few universities are developing their own programs. In September, students at Stanford began using a digital tool based on 15 years of data that helps them in the daunting task of choosing from among some 5,000 undergraduate classes.

“No single adviser, however wise and alert, can possibly be aware of all the instructional opportunities,” said Mitchell L. Stevens, an associate professor of education who led in the development of the program.

Freshmen at the University of Michigan began using a tool in the fall that mixes their personal and academic data with data on how earlier students did, advice from them and study guides from professors. After students complete a survey about their expectations for success in a course, the program digs into the data to see if those expectations are realistic. When they sign on, it takes them through questions and answers in what feels like a conversation.

“Hello, Kate. You told us that the grade you want to receive is a B. This is a good goal for you. You should be more confident that you can achieve this! What if you aim for an even higher grade?”

Tim McKay, the professor of physics, astronomy and education who created the application, called ECoach, said thousands of students who regularly used the application over six years of testing often earned a third of a letter grade higher — from a B- to a B, for instance — than those who did not use it.

At the University of Arizona, Sudha Ram, the director of Insite: Center for Business Intelligence and Analytics, has been experimenting with tracking freshmen — the category of students most likely to drop out — as they swipe their identification cards to go to the library or gym, pay for a meal in the cafeteria or buy a sweatshirt in the bookstore.

“We are measuring social interaction,” Dr. Ram said. “How many people do they tend to hang out with for different activities, and is their hanging out dropping off week by week or getting stronger? A lot of theoretical work has been done on this.”

The findings are put into algorithms to predict who is in danger of not making it to sophomore year.

“Most of the predictive-analytics people are looking at grades,” Dr. Ram said. “A lot of times it’s not the grades but whether they feel comfortable and socially integrated. If they are not socially integrated, they drop out.”

Dr. Ram has tracked nearly 30,000 students over the last three years. Matching her findings against actual dropouts, she said, she has an accuracy rate of about 85 percent, but her project is still in the testing phase. She says identities are kept private.

That’s a major concern about big data: that student details could become public. It is not the only issue. Martin Kurzweil, a program director at Ithaka S + R, an education research organization, worries that students whose performance is setting off alarms could be discouraged from following their passion. “Algorithm is not destiny,” he said. “It’s important that human judgment is never removed from the process and that there is always an opportunity for a student to appeal a pathway that’s being plotted for them.”

Another concern: the temptation to weed out at-risk students to improve a school’s ranking. Could it happen? Mr. Kurzweil cites the president of Mount St. Mary’s University, in Maryland, who famously wanted to “drown the bunnies” who struggled.

“There are risks,” Mr. Kurzweil said. “But I think the people in predictive analytics are mostly white hats, and they’re doing it because they really believe they’re helping students.”

Still, he described the field as a little like the Wild West: “They’ve pushed ahead quickly without setting up the kinds of standards and governance that will mitigate risk.”

In June, Ithaka S + R and a team from Stanford brought together 73 specialists from universities, analytics companies, foundations and the Department of Education for three days of discussion on developing standards and ethical guidelines for big data on college campuses. They met at the Asilomar Conference Grounds on the Monterey Peninsula, where 41 years earlier scientists started work on the first guidelines for genetic research.

Some universities are staying on the sidelines. The financial commitment — a half-million dollars or more over the course of a three-year contract that includes data analysis, software and training — is significant. Some are simply uneasy about the concept, Mr. Kurzweil said, or are not convinced predictive analytics really works.

Laura Mercer, chief of staff to the president of Sinclair College, a two-year school in Dayton, Ohio, tells a story to counter that view. At Sinclair, a C in general psychology or in “Foundations of Business” was found to be a sign that students majoring in those subjects won’t make it. She decided to take a look at the records of a few students who had taken the psychology course.

One woman who was planning to major in psychology had taken it and three other courses as a freshman in the fall of 2014. She earned three A’s and a C.

“It was a pretty decent start,” Ms. Mercer said. “But guess what? The C was in Psych 1100.” In the spring of 2015, the student signed up for five classes. She withdrew from one. The next semester she withdrew from three of her five classes. This fall she took four classes and withdrew from all of them.

“It was just what the analytics had predicted,” Ms. Mercer said. “I tend to be a little skeptical. It wasn’t until I dove into the records and I saw, ‘Yes, indeed, this is a problem.’ ”