Crunching Numbers
Lee Coffin:
From Hanover, New Hampshire, I'm Lee Coffin, Dartmouth's vice president and dean of admission and financial aid, and this is Admissions Beat.
(music)
Every year, college admissions spits out lots of stats—so many stats, so many numbers, that I sometimes think, "Oh my God, I am swimming in data." And the nerd in me likes the data because I get to model things and understand the behavior of seventeen-year-olds and see how predictive it is and interesting. They are predictive in some bizarre way, when you boil them down to some data. However, on the outside of college admission, that data is catnip.
Whether it's a news story that talks about acceptance rates and app volume and changes in both, whether it's the rankings, whether it's a guidebook, whether it's the College Board, BigFuture where you could dive in to all sorts of different topics related to you, and plug in your numbers, and it tells you a story. And so, I thought, let's take a beat and explain the numbers. And give you all a data toolkit to be an informed consumer of admissions statistics. So, when we come back, we will have a conversation about admissions data, when it matters and when it doesn't. We'll be right back. So, we welcome Jacques Steinberg back to Admissions Beat. Hello, Jacques.
Jacques Steinberg:
Hello, Lee. Glad to be back.
Lee Coffin:
Nice to have you back. And for listeners, Jack is former higher ed reporter with The New York Times, author of The Gatekeepers and The College Conversation, and just an expert on all things admissions. And Jack, I've often teased you as one of the founding fathers of the media admissions beat. I think in that former role, you were a purveyor of data. You would report it and share all sorts of trends, and want to talk about that with you.
Jacques Steinberg:
Yeah, and in the spirit of this episode, I tried, as a journalist, to help parents and students and counselors, focus on the data that matters, keep it in context, but also jettison those distracting figures that they needn't worry about. So, I'm really pleased to have this conversation and especially pleased by our special guest.
Lee Coffin:
And our special guest is returning friend of the pod, Karen Kristof, the dean of admissions at Colorado College. Hello, Karen.
Karen Kristof:
Hey, Lee. So glad to be here. Hope I can help unpack this. You've promised to make data fun, so we will make data fun. Let's do this.
Lee Coffin:
Well, so, for this conversation, Jack is going to be the moderator and Karen and I will be the deans answering his quiz about all things admission data. So, Jack, take it away.
Jacques Steinberg:
The first question doesn't necessarily focus on the fun, but it does focus on the anxiety. I put myself in the category of those who, sometimes, can find data and math a little daunting, a little stress-producing. So, before we dig into some data sources here and do's and don'ts and user guide, you both deal with data as part of your day jobs. What advice do you have for those who find data triggering, just in general?
Karen Kristof:
Yeah, I think it's always trying to understand data in context. And I think I love the way that we've become more transparent about what we do and how we do it. And so, you can look at something like the common data set and learn a lot about the demographics of the class that Colorado College just admitted, but you don't really know everything, right? And trying to keep it all in perspective, it doesn't tell the whole story. And it also uses words that can, sometimes, be a little bit inaccessible. Sometimes, when I have conversations, especially with parents who are quite anxious about this, I'd rather talk about the big picture. So, engaging with the humans after engaging with the data, I think, is good advice.
Jacques Steinberg:
One step, in terms of lowering the temperature a little bit, lowering the blood pressure, is to put the data in context and to know that it's only one part of the process. And the process is called holistic admissions, which deals with the whole human. Lee, what's a tip you have, for parents in particular, who find a sea of admissions data points to be anxiety-producing?
Lee Coffin:
We live in a data-centric world. You want to buy a car, you go to Consumer Reports and you do the research on what's the best car. You want to buy a house, you go to Zillow and you do the same thing with the square-foot price of the house. As consumers, we break things down into digits and I think that instinct carries over into this work as well, where families that are looking at the very selective part of the admission rainbow know the odds are long and they're trying to crack the code. And I think that's what causes the data to become a bit of a Frankenstein that hops up off the desk and it's like, "Ha, I've got a life of my own." And it doesn't always deserve to be up and running around your living room. And I think something Karen just said is, my other caution is, "The data is very prevalent."
It's in the spirit of transparency. A lot of us put a lot of stats on our websites, but you don't know the context of the data. And I think that caution from Karen is the right one. It's everything doesn't mean what you think it does.
Jacques Steinberg:
I love that image, Karen, of Lee's image of data getting up and running around the living room. It really puts us in families' homes. So, I see this conversation as a little bit of a user's guide, road map, glossary, all of the above, and a temperature reducer. Let's start with those data sources that are family-facing, parent- and student- and counselor-facing, particularly as pertains to the discovery and search portion of the process. And then, let's have a conversation about how you all use data as admission themes, which may have a little bit of a "Don't try this at home" side to it. Good to understand it. Probably not numbers you need to know, but important to understand. So, let's start with a data source that I suspect is familiar to many listeners and if it isn't, it will be.
And that's Naviance. Let's assume no knowledge of Naviance. Karen, can you just give a definition for the uninitiated?
Karen Kristof:
Sure. So, it is a tool that high schools use to do everything, from track letters of recommendation, get them submitted electronically to colleges, to ingesting some data about who's been admitted to which places. And so, it gives you an opportunity as a student or parent to see, "What are my chances, roughly?", right? It's grade point average. It's standardized test scores if they're available. It looks at a long history, a number of years. And, "If I put my numbers in, what are my chances of getting into a particular institution?" I mean, there's a lot of guardrails, I would say, about this, because it doesn't give the name. So, it wasn't that Lee Coffin was admitted to Colorado College.
It's got a little X, that this student was admitted with this particular grade point average and this particular standardized test score, but we don't really know much about the individual. So, you see a lot of outliers. For example, you can see a number of students that got admitted with these particular stats, and then these outliers of students that might've gotten admitted for other reasons. They might've been a recruited athlete. They might have served another kind of institutional priority. So, it's a little bit like saying, "There's a 10% admit rate." So, if you line up 10 high school students, one of them's going to get in, admitted. So, "If it's a 10% admit rate from my high school, then one of the 10 of us, me and nine of my friends, one of us is going to get admitted."
Of course, that's not exactly how the admit rate works. So, it seems to suggest there's an easy way to crack the code. It's one way to start, but the outliers could drive you mad because you don't always know who those outliers are and the high school can restrict the amount of data that you're seeing. There's a lot there and I would say it's a single tool in the toolbox. You make a lot of assumptions about who did and didn't get it. When you just look at the raw numbers in the chart.
Jacques Steinberg:
So, Lee, as a student, is engaging in the self-discovery part of this process, and then embarking on the search as parents and other adults are seeking to support them on that journey? What would your counsel be on how to use Naviance as a helpful tool and where to be careful?
Lee Coffin:
Yeah. And I'll expand. Naviance is one very common platform. There are multiple others, Score being another popular one. But whatever your high school uses is a very preliminary data review. And I think the other danger of something like a scattergram is it plays into the narrative of, there are really two pieces of the data that count more than anything else, your GPA and your test score. And because that's all that's plotted on that, it says, "This is what counts." And at some places, it is what counted because it's a larger university and they're not screening for the same holistic qualities because that law of averages are going to fill in that class with lots of different types of people. The value of it is to say, "Okay, let's look at the list I'm getting from my counselor."
It's been roughly categorized into things like likelies and probables, and aspirational or reaches, whatever the school calls them. And you get a reach, and you look at the scattergram and you say, "Oh, I understand why this is a reach. People with my data profile don't always get in or rarely get in. And that's because the place has a 9% acceptance rate." So, there's another piece of data that you have to overlay onto your own data is, what's the competitive framework of that place? A place with a 75% admit rate, which could be a terrific place, might have a lot of people getting in. And you need to understand the way the data lives in the local context.
We say that to students when we read their transcript, we're looking at the environmental factor of their high school and conversely, parents and students, you have to look at our data in our local context too because every college is filling different parts of itself year by year by year. And if you're looking at... Cornell was a guest on a recent episode, you're looking at a scattergram of Cornell. It doesn't break it out by the eight colleges of Cornell. It just says Cornell. And you don't know, is this the School of Agriculture or is this the School of Arts and Sciences or is this the School of Engineering? And those schools have very different admission processes and data profiles that get masked by this really flat representation of competitiveness. And I think that's my caution.
I'm glad you started with Naviance, Jack, because I think it is the gateway drug to admissions data. It's the first thing families are shown and it's like, "Woohoo, look at this." A friend of mine, who's a former admission officer, has a kid in high school and she texted me the other day, and she said, "I just was introduced to Naviance. I'm drunk," because as an admission officer, she was seeing all sorts of things that we don't normally see from the school side, but she said, "It's a rabbit hole. I can't get myself out of it now."
Karen Kristof:
And it makes sense, right? Because it feels like an overly complex process. So, if you can break it down into a couple of data points, that seems to make some sense, but I think it also takes away, particularly for students and parents who are looking at more selective places, it takes away the opportunity to have a conversation about what else matters. I wish you could say, "It's this plus this, x and y-axis, and these are your chances of getting in." But there is a huge other way to think about it, that you don't just take all of the students with As and 1500s because you might decide to take a student that doesn't quite have that shiny of a profile, but there are other reasons why they are important to that institution, why they match the ethos or values of the institution.
Representing a small liberal arts college in particular, for me, it often is... There are times when we don't exactly set aside the numbers, but we say, "These are the numbers. This gets you in the conversation. But what's more interesting is what gets you beyond the start of the conversation, like what you will bring to a community." And so, I know it would be nicer if we could put those into nice, neat data packages or nice, neat numbers, but they don't fit very well in nice, neat numbers. I have told many stories over the year about my... Years about my brother's wife, and there was one year, the youngest was going through the process, so you would think, "Two down, one to go," but she was, in fact, more anxious for the third.
I couldn't quite figure out why my sister-in-law was so anxious, but I was talking to my nephew and I said, "Oh, I'm really interested in my institution and the high school that you go to, to see what I can see in Naviance." So, he added my institution, which was a single-sex college at the time, and within an hour, his mother said, "David, get down here. Why did you put Smith College on your list? You can't go to Smith." And I was like, "Why are you monitoring Naviance so closely, that within 60 minutes of your son just trying one out for size just because I was curious, you knew," like, "What are you doing?" But the panic of what those numbers do and don't say and what they may or may not mean for the search.
And the other thing that kills me is many families have such good school counselors or very well-intentioned, want-to-get-it-right, yet the [parents] look at the numbers and they've decided they know it better than anyone else does. That really also misses out on seeing, trusting the sources in front of you, trusting us on the college side, but also trusting school counselors, trusting those who are assisting students and parents in the process. It doesn't seem to lower the flame in the way that I would hope it would.
Jacques Steinberg:
Yeah, I mean we all know that feeling of getting a medical test back, that drops in an app, and using language that we, as non-physicians, or at least in my case, don't understand, and the temptation to jump to conclusion versus wait to consult the medical professional about, "Hey, what does that mean?" I wonder if there's a little bit of an analogy here.
Karen Kristof:
It is a really good way to think about it, right? And yet, I think many of us who are nonmedical people understand that that is more mysterious, but somehow college admission comes down to a couple of things, and that the average lay person should know exactly how it rolls and without... I don't want to take away from it anyone's process or suggest that they don't know what they're talking about, but in so many cases, we really need the trusted sources, the reliable sources, the folks that are in this day to day, to really lead the conversation.
Jacques Steinberg:
So, let's add a second data source, which is very much in the lives of parents and students and counselors too. And those are rankings, including those from sources like U.S. News and World Report or Princeton Review. So, I want to talk about how to be careful with those rankings and ways not to use them, but also, I would argue some numbers that are components of those rankings that may be very helpful. Let's start about with the aggregate of those families who just want to, quite literally, look at the top 10 U.S. News colleges and apply to those, or wonder why a college "fell two spots in a particular year or rose three spots". Why should they be cautious of treating colleges almost like cars or refrigerators?
Lee Coffin:
Because we're not a car or refrigerator. A campus is this really messy, multi-layered, human place. That's what makes a campus so interesting. Even the places we think we know really well, you don't know everything about the place. And so, that's part one. Part two is, if it's a guidebook or ranking or any outside source helping you to do some research, you just have to understand, all of those guidebooks and rankings have an editorial point of view that may not be your editorial point of view. It doesn't mean the point of view is wrong, just may not align with what you think matters. To talk about U.S. News for a second, they have multiple fields of data that goes into calculating the overall rank. Why does someone go up or down a point or two a year?
They change the categories or the way those metrics are assessed and that's fine, but it doesn't mean that someone going from 12 to 10 or from 12 to 20 somehow got better or worse. It just means the information was sorted in a new way. But the real question underlying a ranking is, "Do I care about these categories?" So, many of them in U.S. News are financial. Finances are a signal of institutional health, of course. But if you don't see that as a really urgent part of your college choice, that ranking isn't really telling you anything. Or you might say, "I care a lot about campus climate and community." That's hard to measure. You're not going to get that element in U.S. News because how would they do it? Something as important as the campus vibe is missing from some of these rankings.
I think that's where the guidebooks fill in the blank as a bit more narrative. Even those can be out of date. Periodically, someone hands me the Dartmouth section for one of the rankings, I'm like, "When was this written?" And it sounds like it was from 1972 and maybe it was, but it doesn't, to me, seem like the place where I work, but I didn't write it and I don't always have input. So, it's the thing to remember about all of these info sources, is, point of view matters.
Karen Kristof:
So, when I was very young, in this profession, I heard Mel Elfen speak, who was the original editor of U.S. News and World Report. And the one takeaway that I still have, this is probably... Oh gosh. ... at least 20 years ago, maybe 25 years ago, was, "I don't know, we're just trying to sell magazines." And I thought, "Oh my gosh, you were whipping us all up into a frenzy." Parents, students, college admission people, and you're trying to sell magazines, right? No doubt that was the most popular U.S. News and World Report magazine. I'm not even sure that this generation knows that that was actually a Time magazine or a Newsweek. It was a really... It wasn't just the college magazine, it was like it had 51 other weekly issues.
The other thing that I thought was interesting about it, the first time that I left the US to recruit students, people thought it was a government index because U.S. News and World Report. It really sounds like something, and in many countries, particularly East Asia, it comes from the government. It's to be... It's a very reliable source. It is trusted, it is double checked, it is fact-checked to within an inch of its life. And really, that's not what it is. It is, is as Lee says, an assemblage of facts, impressions, ways to think about, "Okay, well, faculty salaries are weighted 15% this year and 20% next year. Has the importance of faculty salaries changed on college campuses, right?" Many days, I think this is not the way to go about a college search and I don't know what people are thinking.
And the other days, I think people are trying to unpack a really complicated decision-making process with what they feel like is very little information. And so, it's not that people are silly and grasping at straws, they're just trying to do what they can. They're just trying to get enough research. They don't necessarily want to hear from the admission office. Of course, we all love our institutions, we're all happy every day. The sun is always shining, all the publications are pretty. But I think in a way, you go to the other extreme by taking something that really is based on how an editor or a series of editors or a publication wants to talk about institutions in particular ways, and not realize there might be some flaws in that particular logic.
Jacques Steinberg:
So, if you dig down a little deeper from that overall score that School X is ranked number whatever, and you look at some of the things that go into the U.S. News and World Report formula, I'm hearing you both say, I think, "That there may be some data points that could be helpful to search and discovery." For example, first year retention rates, the percentage of students who move on from the first year to the second year. As students and parents are looking at colleges, U.S. News puts some value on that variable. Should students and families as well?
Lee Coffin:
Yes.
Karen Kristof:
Oh, for sure, right? It is not that some of the data points on their own aren't helpful, it's how it's packaged. I think that's where Jack, you're describing this as a, "Let's pull it apart." And I think that's exactly the way to do it. Let's figure out what went into this particular rating and how things count and in what measure. So, there are some things in there. I think first-year retention is an incredibly important point. And if first-year retention has dropped, that's a much more interesting question to ask than, "Why did your rankings drop?", right? If you're struggling with first-year retention or you have goals around first-year retention or new programs around first-year retention, I'll spend the day talking to students and families about that. But why that particular percentage had this impact.
And just that whole psychology of, "If it's the top 10, they must be great. And number 11, that must be just a terrible institution." It's so arbitrary, in a way, and that's what worries me, about ending the conversation there. Maybe you start the conversation there, but that's where the humans could be helpful in framing the priorities.
Lee Coffin:
And this question of the first-year retention. And if you unpack that one more step, why might the first-year class start to decompose as it moves into its second year? Could be that they were academically unprepared and they left because the curriculum in that place was not one in which they could thrive? It could be they transferred to another institution, so they enrolled and they said, "This isn't my place for whatever reason," and they left. Or it could be the financial aid change between first year and second year. And that prompted people to leave as well. So, any of those or all of those are a really important thing for a family to know. And I've always said to our faculty wherever I've worked, "The proof point of the admission process is the number of students who come back for year two."
And for places that do a really thoughtful admission process and give financial aid that meets full need, you usually see high retention rates, but not everywhere. And that's a good example, Jack, of a data point that I don't think gets a lot of play in a lot of family minds. It's important.
Jacques Steinberg:
Here's another one that U.S. News values, and I'll be curious what your guidance would be to listeners, a student to faculty ratio. So, first of all, let's first define that. What is the student to faculty ratio?
Karen Kristof:
Sure. It's just a very rough number of how many faculty members are teaching and how many students do you have. It's a simple arithmetic solution. I hear a lot of colleges talk about between 9:1 to 12:1. There's this... Apparently, all of us are in that range. Do you have a maximum class size, right? What is the average class by department? What is the average class by first year classes versus senior classes that often get smaller? Again, it's like if you just go, "9:1, 10:1, cool." I mean, I think the assumption sometimes is, "My student will only be in a class with nine other peers." And of course, that's not exactly how 10:1 works, right? Because there may be larger classes at the beginning, there may be smaller classes toward the end of a college career.
So, I'm... As much as we all put that down as one pride point, the numbers seem to sound the same to me, but the more interesting answers would be related to how big could a class grow and how much does it change over the course of a major? And if it's a university, in particular, majors are their larger classes, right? So, what could you say about the first undergraduate sequence in biology, for example, how big could that class grow? Could that be 150-person class, a 500-person class? Because you could have 150-person class and still have a 10:1 student faculty ratio. So, it's the numbers behind the numbers or the way to describe the size of the classroom as a student navigates through four years. That, to me, is much more telling of the experience.
Jacques Steinberg:
So, listeners, one thing you can do is what I did, which is to Google "What does U.S. News consider as part of its rankings?" Look at some of the data points or categories that speak to you, and then maybe, do your own rankings or at least your own data dive by looking at the websites of the colleges themselves. And so, let's pivot Lee and Karen as examples, your two institutions, websites and those of others. What are the sorts of data points that colleges list and how should students and families use those as part of their own search and discovery?
Lee Coffin:
I think all of us post data on the previous year's admission cycle. So, how many applications did we receive? How many did we accept? How many did we enroll? I mean, that's an Admissions 101, map of selectivity, is what would fall under. But even there, the acceptance rate is not as transparent as people think it is. So, you see a number and you think, "15%. This place admitted 15%." Of whom? That's one thing. We often post class profiles. So, a portrait of the class as it enrolls, demographics, geography, academic profiles to give applicants a sense of the Naviance in another way to say, "The average profile of a student who enrolled is this."
And then, you could look and say, "That looks like me," or, "Oh, guidance counselor said this is a reach. That looks stronger than me," or, "Oh, this is a likely. All of my stats are better than that." That's why it's a likely.
Jacques Steinberg:
So, we share that kind of information. Karen, what's on your site?
Karen Kristof:
Very similar kinds of things. I think if you really want to do a deep enough dive, you could find almost anything. There are ways in which we have to adhere to federal guidelines about how things are counted, who gets counted. Things like something called the Common Data Set, which is pages and pages and pages of data. Data on financial aid, data on how need is assessed on the financial aid side, information about... If test scores are required or not required. And then, looking at a middle 50% range of the students who were admitted. There is so much that you can find out about where students are from, who are the students who are admitted. We don't do admit rates by, for example, states.
So, I don't know the admit rate from Texas versus the admit rate from Massachusetts, even though I think some people would love to know that, that's not a very helpful stat because it assumes that all the students from those two states are exactly the same, same with high schools. I do think one thing that we do that's perhaps a little controversial, is we publish rates of admission. So, what percentage of students are admitted by round? Meaning, who do we admit early decision? What percentage is that? Who do we admit early action? What percentage is that? Who do we admit regular action? It's a very popular part of our website and it turns out that students behave in ways we would want them to, based on that, right? That early action is easier than regular action.
So, lots of students are going to apply early action. So, there are also ways in which data that you more prominently display or data that is a little bit more granular potentially are things that people pay attention to. But mostly, you could find anything if you were really motivated to do it. I'm not so sure you'd get a lot of value out of spending hours on a college website looking at data. But certainly, we're not hiding anything. And we do have regulatory rules that require us to make a lot of these data public.
Jacques Steinberg:
So, Lee mentioned at the outset, for those listening who don't know me, that I'm a former journalist, and journalists bring a lot of skepticism to their craft, and also are very concerned with the source of facts and figures. And so, Lee and Karen, as families set out onto the internet, onto social media, data points are presented where their sourcing is not quite clear or the credibility of the site is not quite clear, what would your counsel be to them, in terms of being skeptical, double-checking, what have you?
Lee Coffin:
I think it's really critical to trust your source. I'm not going to name them, but there are a lot of websites out there where data is trafficked in all sorts of unverified ways. And you go there and you get yourself alarmed, engaged, smitten, and you don't know who posted the numbers or what they're saying, but they're there. And I think as a consumer, I would advise caution on third-party sources like that, that don't always have a guarantee that what you're seeing by way of data is accurate.
Karen Kristof:
There are not a lot of guarantees. And so, when you see guarantees, that is where I start to get skeptical. And it's somewhere between it's not as mysterious as some parents and families think, and yet, it's not as simple as there is a formula, right? Particularly, as you start to look at places that are more selective than less selective. For me, a trusted source is somebody who works in this field, right? Again, you probably wouldn't get your medical advice from the plumber who's really good at fixing the leak in your sink, but may not be so good at assessing a human being and their diagnosis. So, really thinking about who are the people who are trusted in this work, people that do this work day to day. On the school counseling side, school counselors that talk to admission people.
We have a professional association, we have conferences, we talk to one another, we share information. The other thing that I really worry about is families who believe the more that they pay for advice, the better the advice is. And that is simply not the case either, right? Because the challenge now is there's so much advice out there. And so, Jack, you're absolutely right. The way in which you say, "Could that really be true? Or could that really be the case?" You now know that there's data sources where you can go back and check that. If it says, "Oh, there's this percent admit rate at this school, I can go back and look at the website. I can go back and look at these other sources."
But I think what concerns me the most is the way in which people are pay to play, pay to have access to the secrets that exist, when in fact, a lot of these things are knowable. And a lot of these things are unpredictable because when you're talking about lower admit rates, you're talking about institutional priorities that colleges are looking for particular kinds of students. You'd be better off having a conversation with a member of the admission staff or a conversation with your school counselor than relying on something you see on Instagram. I want to throw my phone every time I see admission advice on Instagram, but of course, I've got to be careful to swipe up so that the algorithm is not recording that I loved.
I spent a whole 10 seconds on that college admission advice, and then delivers more to me because then, I really would to throw my phone out the window, so-
Jacques Steinberg:
I want to flip as we bring this conversation to a close, to just briefly stepping inside the admissions office with you both imagining a behind the closed door of an admissions committee or an admissions office with a file. I want to underscore Lee's point. Please don't use data to try to be an amateur admissions officer. Leave that to the professionals. You can't possibly be privy to all the different variables, including so many softer variables like essays and interviews and recommendations. But there's two data points that I would like to have you unpack a bit that you all work with throughout the process. And the first is standardized test scores. Lee, you talked about local context and including looking at a test score in the context of a high school.
What makes a good score? And as you look at one in the process, and I should say, "Dartmouth is a school that requires the SAT or the ACT."
Lee Coffin:
All scores should be evaluated in the local context. And I think one of the historic protocols a lot of us have followed for decades has been we report the class average and the middle 50% range. So, X to Y. I don't think that works in a contemporary moment for a couple of reasons. One, you got a lot of test optional places where those ranges don't incorporate people who didn't submit testing. So, you have an upward skew to your data distribution. And then, for those of us with testing, as we've seen our applicant pool and class become more heterogeneous. So, just as an example, Dartmouth has an entering class this last fall of 1,205. And they came from over 1000 high schools, in almost every state, 60 countries. So, that wide array of high schools speaks to this point about testing.
So, each of those schools has a different testing profile. And so, what we've started doing during the pandemic, a little before the pandemic, but definitely during it, when data was getting harder and harder to find, we started leaning into this idea that, "Let's anchor every score, when we have one, in the environmental context of place." So, we look at what's the average score in that senior class. What's the 75th percentile of test takers in that high school, and how does this student compare to both of those data points? So, we've been tracking, so T-seventy-five for us is the 75th percentile and 92% of the class that just enrolled last year were above the 75th percentile of their high school.
Sometimes, that meant the student had a 1510 at a school where the average was a 1500 and sometimes, it meant the student had a 1210 at a school where the average was an eight 90. It's a score produced in a really different place. And that score that's 300 points higher than the norm is a very illuminating data point. And again, that's where, just saying to you, "My average SAT is X." It's too flat. It doesn't help us know all the different places and people that go into this profile. I think testing contextualized is really valuable. Testing in a really flat way isn't helpful. There are some kids who go to big suburban public high schools where everybody has super juiced testing, lots of test prep. I'm not saying that's bad, it's just the norm in that place. So, we know it when we read.
And it doesn't mean that all those scores are better than the rural school 20 miles away where maybe 20% of the kids go to college. So, only 20% took the SAT. So, that's environmental factor, as it relates to testing, but it's also how we evaluate transcripts. The GPA of a 4.0 doesn't tell you very much unless you know what curriculum produced those grades. What did the high school offer by way of understanding what the student took. So, if the rigor of a high school curriculum is abundant, we expect that abundance to be on the transcript. If that rigor is limited, then that's all that valedictorian could take, and we know that. And that data, that GPA, that curriculum rating, are locally sourced.
Jacques Steinberg:
Karen, can you talk about how Colorado College uses SAT and ACT scores and compare-contrast from what Lee said?
Karen Kristof:
So, we became test optional just before the pandemic in 2019. And so, it gave us an opportunity to level set what we wanted to do when we thought about testing. One of the things, two things that we do differently than other test optional schools is we ask all students after they enroll to send their testing so that we don't have this way of having curated testing before we're reporting our middle 50%. And we often have these moments of looking at the bottom of the range of students who we so happily admitted and may have been modest testers. And that, for us, just reinforces what Lee said, really, is that in context, these were students who were doing very well academically and were bringing something really special to the table for us.
We moved into what I would call test-optional 2.0, 2 years ago, and that is really helping students to understand this context piece. So, we would often have students from schools where that 1210 was above the 75th percentile. They were submitting scores and that was hurting them in our process. And then, students who were not submitting scores unless they were 1500 or above, that was helping them in the process. So, we moved to a test optional where what we will do is look at what the student has submitted, and if we think the score helps them in the process or is neutral in our evaluation, we will use the score.
And if the score that's submitted actually doesn't help them in the process, for those students who come from schools where a 1210 might be a perfectly good or even extraordinary score, we will look at the academic performance more closely. So, that was the way of solving what I think is the biggest dilemma in the test optional world is, "Should I or shouldn't I?" The worst-case scenario is that it's neutral, in terms of our evaluation of your academic performance and potential.
Jacques Steinberg:
Really helpful. So, two final questions from me. Lee mentioned grade point average. For schools that do rank students within their high school class, can you each talk very briefly about how you use class rank and how you don't?
Lee Coffin:
Class rank is one of those data points that is going extinct. So, I think when Karen and I were baby admission officers, class rank was a much more common data point in every file. It's more and more rare. When it's there, it's a way of understanding as someone's stands amongst their peers. For my pool, it's really helpful to see like, "Oh, this student has an academic standing at the top of the class," and we're being so precise in our review that that helps. But again, environmental context matters because there's some high schools where they're pretty deep and you could go well beyond the top 5, 10, 15 kids and go to top 15, 20, 25, 30% and find really talented high school seniors. We admit them.
It's helpful when we have it, but the difference between being number eight and number 14 and number 25 isn't hugely meaningful, unless it's a class of 50. And then, being 25 out of 50 is instructive. But also the other thing that I notice myself doing as a reader is when I'm reading a file from a school that sends less than 50% of its class to a four-year college, I pay a bit more attention to class standing then, because I want to make sure that that under-resourced school has really pushed this person to be prepared for our rigor and pace. And class rank is helpful, not in and of itself, determinative.
Karen Kristof:
Yeah, I would agree and I'm fine with it, as Lee says, going extinct. I find most high schools produce a 1-4-page document called a school profile that goes out with every transcript. That's a much better, more vibrant, richer description of the school, the community, the expectations for students. And so, I like the context of, "Where would you put this student? Is this student near the top of the class, near the middle of the class, near the bottom of the class?" That's enough for me, in terms of context.
Jacques Steinberg:
So, I think in this conversation, you both have gone a long way toward illuminating, demystifying, clarifying, defining. For each of you, and starting with you Karen, is there a data point that we've missed that you think should be important to parents and students, particularly in the discovery and search portion of the process?
Karen Kristof:
Yeah, I think I would lean more, these days particularly, on understanding a college's financial resources. What is an endowment? What does it mean? What are the financial pressures that we're all facing given new federal guidelines, new threats to college financing, alongside the questions around admission. I think there's questions around affordability and value. And where will my student be in the four years that they're there? What are the ways in which institutions can help make up for some of the things that are happening that are taking away resources? What are the ways in which institutions are going to be nimble and responsive? And to what extent can we all be focused on the student experience?
Regardless of our size, regardless of where we're located, whether you're a university or a college, whether you have undergrads and grad students. What are the ways in which an institution is really committed to students from a resource standpoint?
Jacques Steinberg:
And of course, what's it going to mean to me as a parent or a student, financially, in terms of what's it going to cost? What am I going to have to borrow?
Karen Kristof:
Mm-hmm, for sure.
Jacques Steinberg:
How long is it going to take me to pay those loans back, what have you? Lee, what have we missed in terms of data points that are important and that we have not spoken about?
Lee Coffin:
My recommendation is one that I have just started to track for Dartmouth, and that is, what percentage of the cost of attendance is covered by the average scholarship? Dartmouth's all-in cost for this year is $95,000. Our average scholarship is almost $75,000. So, that scholarship covers 78% of the cost. I think that's a really helpful metric. I don't know that every college looks at it that way, but that's one of those places you can take your phone out and type in your own numbers, say, ""What's the cost of attendance? What's the average scholarship? What percentage is covered by that?" And it will give you a way of understanding the financial aid policies because a place that doesn't have loans is going to have more scholarships.
So, you start to see... We can all meet need or meet 100% of need, but we could do it differently. And as a consumer who's really focused on affordability, it is one of those places where colleges start to sort themselves out pretty quickly, into tiers of resources that they can dedicate to need-based financial aid. And I've worked at places that don't have those resources and they were wonderful places. I work for one now that has the resource and I see the difference between A and B. And so, I think for a family that's looking at cost as a non-negotiable factor, this is a data point that's really valuable.
Jacques Steinberg:
Should we close by referencing the net price calculator as a source and resource? And if so, just a very quick definition and how best to use it.
Karen Kristof:
Yeah. So, it's a tool on every [college] website, and there's a number of different kinds of tools, including a new one that I really like called an instant net price calculator. So, there's at least one on every site, sometimes two or three. And the idea is to put in some information about a family's asset, resources, income, and get a little bit of a feel for what you might expect from that institution in terms of need-based financial aid, even sometimes merit-based financial aid. But using their philosophy of how they assess need, getting a feel for, if it's $91,000 at Colorado College and you've put your numbers in, maybe it's a couple minutes, maybe you've used the longer one that's more like 30 minutes. It gives you a sense for what you might anticipate paying.
So, it is not a financial aid award, it is not the end of the conversation, but at least, it gives you a sense for what a college actually costs. What is the net cost versus the cost that is advertised on the website? Some families get to the end of that, and then have a ton of questions. And I think that's where you can dig into other resources that help you understand affordability and what might be available. But the idea is to really get a feel for, "This is the published cost. This is what my family might likely need to contribute to the cost of that education."
Jacques Steinberg:
So, Lee, in one sense, I like to think of the Admissions Beat as a really highly tuned, high-performance race car. And thanks for tossing the keys to me. I'm going to toss them back to you to drive us home.
Lee Coffin:
Drive us home. Well, as we pull into the garage, I'm glad we had this conversation. Admissions produces a lot of data. It doesn't mean it's all valuable data to a student and parents. It doesn't even mean it's always valuable to me as a dean, but I generate a lot of information and I'm reminded again of a quote, I thought it was Albert Einstein, but I'm learning it wasn't. The quote is, "Not everything that can be counted counts and not everything that counts can be counted." And our friend Google tells me, "often misattributed to Einstein," but actually written by sociologist William Bruce Cameron in 1963. Whoever said it, I love it because so many people get caught in this admission conversation, asking what counts, and they value things that can be counted.
That's for class rank. It's a way of counting. You can sort your classroom one to the bottom, and that means something, but the things that count that can't be counted, like curiosity or citizenship or collaboration. Karen, do you care about collaboration citizenship and curiosity?
Karen Kristof:
Of course.
Jacques Steinberg:
Of course.
Karen Kristof:
It's hugely important.
Lee Coffin:
Yeah, but that's not going to show up on your Naviance scattergram. There's no way for me to say on the first day of school, "The curiosity quotient of the class of 2030 is a 9.7." So, the data, the high-level data that everybody sees and talks about, tells a story. I think the admit rates tend to be in the volume... The pool went up, the pool went down. Get people distracted. The difference between a 5%, an 8%, a 10%, a 15% admit rate are just, "We're saying no to more people." It's not that the people at the 11th percentile and the 15th and the 21st aren't terrific. That's the peril of admission data. So, listeners, try and keep your wits about you. When you see numbers, focus on the data you can control. You can control your grade point average.
You can, to a degree, control your test scores. You can control the rigor of your curriculum. That's it. You can't control all the other admission numbers that swirl around this process. So, don't lose sleep. Karen, thanks for coming back on Admissions Beat, was this fun?
Karen Kristof:
This was more fun than I anticipated. So, it turns out, we can have fun with data, particularly if maybe we think of it as data informed, right?
Lee Coffin:
Yup, yup.
Karen Kristof:
So, the humans are still there, the stories are still there. Having a great process regardless of where you get in, knowing that, keeping in perspective that where you go to college is really important, but there's lots of other life to live, lots of other things that are equally impactful. So, yeah, I had fun. This was great.
Lee Coffin:
Jack, thanks as always for coming on to moderate. Next week, we will have an episode that looks at social media and the way Gen Alpha uses things that their parents never thought about as a way of discovering college options. So, that's next week. For now, this is Lee Coffin from Dartmouth College. Thanks for listening.