In late June, ProPublica released its Opportunity Gap news application—an interactive feature that allows users to search for and compare schools and school districts on multiple factors, including total enrollment, percent of students taking at least one AP course, percent of students enrolled in subsidized lunch programs, the ethnic makeup and more.

ProPublica released its own analysis based on this data with more stories expected in the coming months, but we wanted to make sure other people could use the information to localize the story and give parents a sense of how their schools are performing. ProPublica education reporter Sharona Coutts, director of computer-assisted reporting Jennifer LaFleur and news applications developer Al Shaw hosted the call, offered their insights and answered questions about the app.

You can listen to their conversation and read an edited transcript on this page. You can also see more about how to use the app here.

EDITED TRANSCRIPT

Mike Webb: Hello, everyone, and welcome to the ProPublica "Opportunity Gap" education call. Our goal today is to provide you with some tips on how to look at the project so that you can do some reporting on your own based on the data. Before we begin, I'd like to advise you that if you're near a computer, please go to our Opportunity Gap site so that you can follow along. The web address for that is projects.propublica.org/schools.

Leading the call on ProPublica's end is our chief education reporter, Sharona Coutts, and she will give you an overview of the project. Then our director of computer-assisted reporting, Jennifer LaFleur, will discuss how we put the data together. And then one of our news applications developers, Al Shaw, will show you how to share the findings via Facebook and other tools on the site. With that, here's Sharona Coutts.

Sharona Coutts: Thanks, Mike. Hi, everyone. Thanks for joining us. As Mike was saying, our goal is to show you how to use our news application, which we'll call "the app," in your own reporting. This is a very interactive feature, and we hope you'll find ways of using it that we haven't thought of and that involve your readers in the reporting process. For those of you who haven't had a chance to look at the app, I just want to give you a quick description. It allows you to do three main things:

1. Quickly see how your state compares with others at providing equal access for poorer and wealthier students to high-level courses that have been shown to be related to academic success.

2. Quickly see basic statistics like total enrollment, enrollment by race, percentage of free and reduced-price lunch students and things like the number of inexperienced teachers.

3. Generate your own customized comparisons of schools and districts. And they pop up like baseball cards you can shuffle and rearrange. Al will talk a lot about that feature because he built in ways you can integrate that with social media like Facebook, and his work there has been pretty innovative.

So, we're going to talk about the data, and then we're going to take questions. This  project is based on a new set of data recently released by the federal Department of Education. Its Office for Civil Rights collected this data for the 2009-10 school year—it's the most recent available data, but the numbers may have changed since then, so you probably want to check them. And it's the first part of two sets of data.

The civil rights office collected the data because, as it told us, it can use statistics about what opportunities are provided, broken down by race, to identify schools and districts where there might be a problem from the perspective of civil rights.

While statistics alone, the office told us, wouldn't be enough for a complete investigation, they provide a clue. So, we were pretty keen to dig into it. Now, Jennifer LaFleur our director of computer-assisted reporting, will explain what was in the data and what she did with it.

Jennifer LaFleur: Thanks, Sharona, and thanks again, everybody, for being here today. Our analysis included data for all schools and districts with at least 3,000 students. So, sometimes you may see that a district is not in there, because it was below that threshold. And that's what we have because that was the basis of the Office of Civil Rights' survey. As with any database, we had to account for some problems we found in the data; so, unlike what will be available from the Department of Education, we've actually corrected some data in cases where there were extreme outliers or impossible values that got either entered wrong by a school district or something else happened. We realize that there may still be things in the data that we haven't caught, so we're welcoming folks to let us know if they see anything.

We basically used a statistical tool called linear regression to look up the data to see if there was any relationship between all of the variables in the database. We tested lots of things, but in the end, we found some relationship between the proportion of minority students at schools and access to these advanced programs. But we found the strongest relationship, which is fairly common in education analysis, with the percent of students getting free and reduced-price lunches.

That variable is not in the data that OCR provided. We added that from the National Center for Education Statistics. We found that in some states more than half the change in enrollment in certain programs could be explained by the increase in the percent of poor students, and basically, that means that higher-poverty schools do not have the same access to those programs. One example: In Oklahoma, we found that for every 10-point increase in the percentage of low-income students, the percent of students enrolled in AP courses dropped by three points on average.

In many states, we saw little relationship. Florida was one that Sharona wrote about where they've actually put in special policies to increase AP enrollment, and that's why we didn't see the same kind of relationship there. And again, in other places, there also were specific policies that may have affected whether the programs were widespread or isolated.

Sharona Coutts: Thanks, Jen. So, for folks who've got questions about data, you can direct those to Jen, and I'm now going to walk you through certain parts of the app, and then Al's going to speak a bit about interactivity. Then we'll turn it over to questions. So, just a note on what Jen was just saying. The reason it was important for us to add poverty data is that the Office of Civil Rights didn't look at that. That's why Jennifer included it, and it really ended up telling us quite a strong story.

OK, so, to the app. Those of you who are in front of your computers at the moment, if you can pull up the Opportunity Gap, I'll take you through some of the ways that you can use it. Reporters with a particular interest in, for instance, education of students with disabilities, we're really hoping you use that specialized knowledge and mash whatever data you have with ours. We're really hoping you actually see this as a tool.

There's a lot in here, but, for instance, there are folks on this call who are in places where there's pretty good outcome data for schools, so you might be able to view their data and then include things like graduation rates or test scores, and then you might wonder, looking at the opportunity analysis, does it really make a difference? When you've got a lot of, for example, inexperienced teachers, do we see a pattern?

That might be one way you can use it. If you are on the home page of the app, you'll see that on the left-hand side there's a menu with a list of states, and above that, you'll see that there are five variables. One is AP classes. If you click on it, you'll see that the order of the states actually shifts, and Oklahoma jumps to the top of the list.

And you'll see that for AP classes the indicator box next to it is a darker red. It means there's a strong relationship. Where there is this strong relationship between AP classes, in this case, and low-income schools, that means that in Oklahoma, schools that have a high percentage of kids who are eligible for free and reduced-priced lunches are less likely to be enrolled in AP courses. So, to put it in a simplified way, kids at poorer schools are less likely to be enrolled in those courses. Kids at wealthier schools in Oklahoma are more likely to be enrolled in those courses.

So, you can get into the app that way by first looking at, how is my state doing in physics, how is my state doing in chemistry. Then, you can actually do something like hit the state name itself and you can click through. So, let's go to Maryland. That brings up the state page for Maryland, and here is where you can very quickly see the total number of students in Maryland who are in the survey, the number of teachers, the number of districts, the number of schools. We had some key findings based on a few interviews that we did. You can look at the overall percentages of free and reduced-price lunches, and you can see how Maryland compares to the national average so that there's that brown-gray sort of bar.

So, you can see, oh, when it comes to overall enrollment in at least one AP course, Maryland exceeds the national average. However, in Maryland, there is a strong relationship between poverty and enrollment in AP courses. So, although on average a lot of students in Maryland are enrolled in at least one AP course, there seems to be quite a bit of inequality in that state. Depending on your state, you can get into it that way and take a look at the numbers and those sorts of comparisons.

The next thing you can do is compare districts. So, say you choose Baltimore City Public Schools and hit that link. The app is going to bring up a profile of the district. So, that'll tell you, again, things like the free and reduced-price lunches It's going to give you the racial breakdown of the districts. And it's got a list of the schools right next to it. And I am going to turn this over to Al at this point because this is the part where his really innovative cool design starts kicking in and allowing you to actually start doing your own custom-made comparisons. So, here's Al Shaw.

Al Shaw: Thanks, Sharona. And thanks everyone for joining. So, if you're on a district page, let's kind of pick up where she left off, you can actually compare that to other districts by this large blue menu on the left. And you can either choose nearby districts, high- and low-poverty examples, or any district at all in the state. So, if I'm in California and I'm in Los Angeles Unified, I could type in, say, Hawthorne Unified or the district comes up as Hawthorne, and then hit return in the auto complete, and then I'll get a new page made up of these baseball cards and then what you can do actually is sort these baseball cards by a bunch of different criteria.

So, if I want to sort by inexperienced teachers or AP classes or AP enrollment, it will actually shuffle the baseball cards in real time based on those criteria. This is really useful when you want to do an analysis of local schools or districts, all these apply to schools themselves, too, so not just districts. So, if I drill into Los Angeles Unified and I go into any school in that list, I can do the same thing. So, if I go into one of these schools I can either press the “nearby schools” button, which will bring up six of the nearby schools in this baseball card viewer which I can then sort, or I can choose high- and low-poverty examples. Or I can actually type in my own and create a whole custom dashboard of schools.

You can only show six at a time, so if you want to get rid of the school on the page, you click this little "x" at the top of any of these baseball cards to get rid of it and then add more schools. So, once you kind of have a view that you like that you might want to share with your readers, there are a couple of ways to share that. The first is anytime you change a card or change the sort on that page, the URL on the top bar will change. So, you could actually just copy this URL at the top and it’s a really long URL because it includes all the schools, and you can paste that right into your stories, and then anyone that goes to it will see the exact same view you see. You could even use a URL shortener like bit.ly to make that URL shorter.

But an even easier way to share it would be to sign in with Facebook. There's a little Facebook link in the bottom left, and if you click that and you authenticate, you can actually post this custom view of schools to your wall and get another custom URL with your comments next to it that you can share with anyone. Now, your wall has to be public for those comments to actually show up to anyone you give the link to; otherwise, they're going to show up as an anonymous user.

But you'll still be able to get out that link, and people will be able to see this custom array of schools or districts on your dashboard, and I can answer individual questions later during the call. One thing to know: On these district and school baseball cards, if there are these green and yellow lines on each bar graph, those are the state and district averages. So, if you're on a school dashboard, you'll see yellow and green lines.

If you're on a district view, you'll only see yellow lines, meaning those are the state averages and how each of these districts compare to the state average. So, there are a lot of different ways to slice through this data. I can take individual questions later if you have them.

Mike Webb: OK, everyone, we're going to open it up to questions. Remember, hit star one and you will go into a queue, and we'll take your questions as they come up. However, I'm going to ask the first question because it's the question that we most often get, and that is: Where does my school rank? Since I'm from Ohio, I'd like to know: Where does Ohio rank among all the states?

Sharona Coutts: I think Jen and I will collaborate on answering that one. Yeah, that is one we've been getting quite a lot and, basically, in terms of ranking states, it's not possible to give an overall ranking. You can't just say that there's a certain level that any one state has overall. And the reason for that is a state that does really well in one indicator, like AP courses, might do very poorly when it comes to another indicator like gifted and talented, and what I mean when I say “well” or “poorly” I guess should be more specific What I mean is, let's take Florida. That state has no significant relationship between high- or low-income schools and the number of kids in those schools that are enrolled in AP courses. The reverse is true when it comes to gifted and talented in Florida. So, in Florida, there is actually a strong relationship between whether you're at a poor or wealthy school or whether you're likely to be enrolled in gifted and talented. Jen, did you want to …

Jennifer LaFleur: Yes. I mean the other thing is it's because it's a statistical analysis that number wouldn't be that meaningful, but if you use that state's graphic on the front of the site, you can really see what's on each variable, which states tend to have a very strong relationship and which do not have a relationship at all, and I think that's probably the best guide. I wanted to jump in on one other thing. I see we've got a couple of Twitter questions about some additional districts or schools not being included. I just wanted to mention that we did not include alternative schools, and we did not include charter schools, because of specific enrollments and other reasons in our analysis those schools act a little bit differently.

So, if you have a question about a school being left out that's a charter school or an alternative school, we do find a few cases where a state actually designated something as an alternative school that turned out to not be the case and we've corrected that, but that may be why certain schools are not in the app.

Mike Webb: OK, we're going to start taking questions. Remember, if you want to tweet a question to us, use the hashtag #edugap. First question. Operator, can you open it up to Judy Walton from the Chattanooga Times?

Operator: She has disconnected.

Mike Webb: OK. Let's go to the next one. That's from Becky Vavilla from the Chicago News Corp.

Becky Vavilla: Am I on?

Mike Webb: Yes.

Becky Vavilla: OK. Sorry. Thank you so much for taking my question and I am curious if there is any possible way to do, say, more than six schools, say take all of the high schools in Chicago and compare them among each other and see how many AP courses they have, how many inexperienced teachers they have and submitting that in the district where there are more opportunities than in other areas. Is that something that would be possible or would it have to be done manually?

Al Shaw: At this point, using the dashboard interface, you can only add up to six schools through that one interface, so you'd probably have to go to the Chicago page, the Chicago district page and find smaller, I guess, smaller sets of schools to analyze. I guess that was mostly just for space reasons since it'd be hard to shuffle around that many schools on a single view.

Becky Vavilla: OK.

Mike Webb: OK. We have a question from Lindsay Casner from San Antonio.

Lindsay Casner: Hi. I was just wondering what you mean by “relevant students."

Sharona Coutts: Is there one of us in particular that said “relevant students”?

Lindsay Casner: At the top of each of the little baseball cards it says percentage of relevant students, and I'm just wondering what …

Jennifer LaFleur: Right. Right. Yes. Yes. Mainly, that's to just let you know that the denominator is used just for schools that would apply there, for example, AP courses are not offered in elementary schools, and so we only did that percentage out of high school students. So, it would be those for whom that variable applies.

Lindsay Casner: OK. Thank you.

Mike Webb: We have a question from Erin Richards. Erin, are you there? OK. We're waiting for another question. Remember, just press star one, and we can get your questions. You can also email Sharona.Coutts, C-o-u-t-t-s, @ProPublica.org. Do you have one, Jen?

Jennifer LaFleur: I don't have a question. I just wanted to let everyone know that if you do want to work further with the data there is an opportunity on the district pages to download the schools as a database as a CSV file that you can work with further, so there are ways to download the data if you want to work with it more.

Mike Webb: OK. We are—if you want to tweet a question, the number is hashtag #edugap. Next question is from Chris Dobbs.

Chris Dobbs: I was wondering why the AP classes were science and then advanced math because I'm wondering if schools have a tendency to offer more history, more AP English, those types—why the two science and advanced math?

Jennifer LaFleur: That's actually—let me explain those variables again. Those were actually separate. The way the AP data is in the education department's data is it's students taking at least one AP course, no matter what. It could be history, it could be, you know, AP French or anything.

That's a separate variable. For advanced math, they have a variable on the number of kids taking advanced math and then, for some reason, they did—they separated out all of the science courses. There wasn't one for taking advanced sciences. For the advanced math included things like trigonometry and calculus and things that were the higher level math as well. And, after the call, we can actually send out a couple links. I know I've had some folks ask for the actual survey and what it looked like from the Department of Ed which kind of gives you a better sense of what districts were providing, but we'll send that out in a link following the call.

Chris Dobbs: Sure.

Mike Webb: OK. Thank you. Is Neil Morton there?

Neil Morton: This question, I believe, will be directed to Jen. I was wondering if in the link that you could send everyone perhaps including the NCES database or tables that did list the free-and-reduced lunch totals. A lot of them were inaccurate for the area I live in, and I was just hoping to see what NCES might be reporting from what year, or what survey they had.

Jennifer LaFleur: Yeah, the data we actually used in the app was the NCES data, but I'd be happy to send that out as well.

Neil Morton: Thank you very much.

Jennifer LaFleur: Uh, huh.

Mike Webb: Next up is Lisa Schneiker? … Schnyder?

Lisa Schanker: Schanker.

I don't know how it is for the other states, but when I was looking at Utah, I noticed only a little more than half the districts are included in this site. What's the reason behind that?

Sharona Coutts: Hi. Thanks for calling in. The reason is that this data collection is not every single school in the country, it’s a lot of them. Every school in the district with at least 3,000 students was included. In addition to that, they did make a different kind of survey that included a sample of the smaller districts, but we didn't include that in our analysis because it just was not the right thing to do. Statistically, Jen can talk about that in more detail if you'd like, but we did this analysis based on using all the schools in the district where there are at least 3,000 students. Yeah, in a smaller state for sure, there are some districts that are missing, which is a shame. We wish that they were all in there, but they're not.

Lisa Schanker: OK. All right. Thank you.

Sharona Coutts: That's all right.

Mike Webb: The St. Louis Beacon, Dale Singer. Are you there, Dale?

Dale Singer: I'm here. For those of us who live in a bi-state or tri-state area, is it possible to make comparisons across state lines?

Jennifer LaFleur: Not on the app at this moment, but if you were to download the data for the state, you would be able to do that. Hi, Dale.

Dale Singer: Hi there. Hi, Jen. Thank you.

Mike Webb: And how do you download the data?

Al Shaw: On every district and state page, there's a link at the bottom left-hand corner that says something like “expert view,” let me see, yes, it says "Reporters: see a table of all results,” so if you click on that and then it'll say “school districts in (that state)” and then you hit “download CSV,” agree to our terms of service and then you can download a CSV of everything in that area.

Mike Webb: OK. We're going to move on. Jaclyn Cosgrove from the Oklahoma Watch.

Jaclyn Cosgrove: Yeah, I had a question kind of similar to the reporter in Utah. Oklahoma also has a lot of districts that aren't included, and I wasn't sure if I understood you properly. Someone had said something about the Office of Civil Rights doesn't ask those smaller districts these questions. I wasn't sure who said that or whether I have misunderstood.

Jennifer LaFleur: Sharona mentioned that earlier, but I'll just add to that. What they did, and this is—they've done similar surveys to this in the past and it's really the most extensive they've done and they did so by making sure that they got every single district with at least 3,000 students. At enrollments below that, they pulled the statistical sample of schools that they could then use to develop national projections, but because they pulled the sample below the 3,000 level, it didn't make sense for us to use that because we wanted to show information about individual schools, so we just decided we were going to use just the ones where they had every school, which is what we did.

Jaclyn Cosgrove: OK. Thank you.

Sharona Coutts: So, I've got a few questions coming in through Twitter now that I think I'll just go through and anyone who's listening, you can tweet at us—we're using the hashtag #edugap.

The question here is: I'm covering New York City. Can I get this data to break down by borough or district area? We would caution against using this data for an analysis of New York City at this point. Jen, do you want to talk about that a little bit?

Jennifer LaFleur: Yeah, I can mention this. We are still working with the New York City schools to answer some of these questions. There were about four schools which are fairly high academic schools that did not have the numbers one might expect. One example is Bronx Science. So, we've actually gone back as we did in many cases across the country to try to figure out if what they submitted is accurate. So, we're still working on that. That doesn't mean the entire district necessarily has problems, but I would just be real cautious with that district because we do know there's some problems in the data there.

Sharona Coutts: I don't think we can break this down by borough, can we, basically because New York City is one district, so unfortunately, not.

OK, here's a question. What's the survey's definition of inexperienced teacher? The survey asked for the number of teachers in the school that had either one or two years of experience, so that was the definition of inexperienced. Basically, two or less.

Mike Webb: Well, we have one from Ed Wakimmie that says we have high participation in AP, but only half get credit. Shouldn't this matter in your result?

Jennifer LaFleur: Yeah, it does matter in our results, and we looked at two things. We looked at both the total participation in certain programs and then we also looked at the relationship between poverty and enrollment in those programs. One state that kind of stood out for us that tends to have very high enrollment overall was Maryland. But it has a very strong relationship between poverty and enrollment and so that, that's definitely a distinction between high-poverty schools and low-poverty schools when it comes to enrollment in those programs. So, we did look at both those things. The chart on the front of the page—that is based on the statistical analysis looking at participation versus poverty. But, as you dig down into the state pages, we provided both of those pieces of information.

Sharona Coutts: I'm also seeing a few questions here that are asking about AP, I guess, from the perspective of acknowledging that in some places advanced placement courses have been controversial and we didn't shy away from that. In fact, if you're a reporter working in an area, either a state or a district, where there has been some controversy over the value of advanced placement courses, we hope you can use this data to assist your reporting on that.

Hopefully, that's a way that you can actually get some specifics about the number of courses and whether those courses are being—whether enrollment reflects—that it’s done in an equal way amongst lower-income and higher-income students.I think that, you know, if people want to look at a very local level and see other variables are interesting in their area there's nothing to stop you from doing that sort of thing. So, we definitely didn't shy away from that. Any other questions coming in on the call?

Mike Webb: Yes. Tom Webber, are you there?

Tom Webber: Actually, two very quick ones. Is there any allowance—it doesn't look like you looked into IB at all, International Baccalaureate. With the downloading, if I'm on a state, I can see that I can download an Excel for all the districts, but can I ever download an Excel for all the schools in the state, a compilation of all the district schools. And then the final one is just talk more about how you decided who was red, who was pink, who was white. Were you looking for a benchmark? Schools with a free-and-reduced over 75 percent and AP less than 10 percent? Or were you comparing them to each other districts in the state, kind of more about the methodology there, if you don't mind.

Jennifer LaFleur: OK. Gosh, that’s a lot of things to remember. Hope I remember all your questions. If not, please let us know.

Tom Webber: OK.

Jennifer LaFleur: On the IB issue, we actually did do an analysis of that in our original work, but what we found in this data is that it was cast a little bit different in how they calculated IB. It was enrollment in IB, so AP was a little bit different that way and it was not actually listed for very many schools. I think the total number of schools in the whole 72,000 we had was a few hundred with IB programs which is why we kind of decided to not include that nationally.

But if you're in a state that actually does do a lot of IB, it would definitely be worth looking at. So, and on the statistical analysis, I'll have Al answer the question of downloading information. What we did was a linear regression to look at how much of the change in the percentage of kids enrolled in each of these programs can be explained by the percentage of poor kids. And, so, as it's redder at the top that means there's a stronger relationship between those variables. Does that make sense?

Tom Webber: Yes. Oh, I see … on the top of the charts.

Jennifer LaFleur: You can actually sort that, too. If you click on chemistry, for example, or physics. It re-sorts it based on the darkness of the relationship, and the darker means it’s a strong relationship—that poorer schools tended to have less access.

Tom Webber: OK.

Al Shaw: I think the last question was how you download all the districts in a state, is that it?

Tom Webber: All the schools within all the districts. We can do all the districts in a state, but what about all the schools in the state?

Sharona Coutts: Yeah, you know, I actually just got an email from someone asking the same thing.

Al Shaw: Right now, that is not actually possible. You have to get all the districts in the state. In fact, that might be something we should look into. Yes. If you want more data, you can always just email us, and we can probably get you custom spreadsheets.

Tom Webber: Cool. Thanks.

Sharona Coutts: We just got a question here by email asking whether we have Puerto Rico's data. No. We don't have that. Sorry, Laura Condalas, we don't really have that data.

Mike Webb: Next up is Sandy. Sandy Conteos, are you there? OK, let's skip ahead to John Burns. John Burns?

John Burns: Yes. I was wondering if you had any questions looking at the poverty data, whether you saw any results with the gifted and talented or whether that data was just kind of skewed all over the place and testing had something to do with it?

Jennifer LaFleur: We actually did look at that as well, and at some places did find a relationship between poverty and the percentage of kids enrolled in gifted-and-talented programs. Ironically, in Florida where we found no relationship in any of the advanced classes and AP, we did find a relationship with gifted and talented. So, it tended to be not quite as strong as some of the other things we found, but we did find a relationship there and it would be worth looking at.

Mike Webb: All right. Cynthia White?

Cynthia White: This is Cynthia White. So, I am looking at the section where when you're looking at schools or districts you can compare to high poverty and low poverty in neighboring schools. There's an automatic generation that gives you, like, one high and one low, for that comparison, and then you have the options printed by school. I'm just wondering how that's generated and if there's a way of if, for example, you don't want that school district but you want a similarly high- or low-poverty school district, whether there's any way of getting a second generation.

Al Shaw: For the low- and high-poverty schools, it takes the representative low- and the high-poverty district or school compared to the school that you're currently on or the district you’re currently on. It's pretty much just the one comparison for every school in the district that you could ever be on and that's not related to the nearby schools, but those are two totally different things. “Nearby schools” is purely geographic, so if you're in a high school, it'll show you nearby high schools. If you're in a middle school, it'll show you nearby middle schools. And if you're in an elementary school, it'll show you nearby elementary schools.

Cynthia White: All right. So, I understand that they’re separate, but I'm just wondering if when you're saying “nearby,” is it the nearest and when it's high- or low-poverty, are you taking the highest and lowest, or is it ... and, when you say it's representative, does that mean that it's not the highest and lowest then?

Sharona Coutts: So, Al, would the nearby schools be the ones with the same grade range?

Al Shaw: The nearby schools will give you the closest geographic schools that have the same grades. So, if the school I started out with has ninth, 10th, 11th and 12th grades, it would show you all the nearest schools that have ninth, 10th, 11th and 12th grades. The low- to high-poverty schools button gives you the schools with the highest and lowest poverty in the state that are in the same grade ranges. So, if my starting school is ninth, 10th, 11th and 12th grades, it'd show you the highest- and lowest-poverty schools of those grade ranges.

Mike Webb: OK, let's go back to Twitter. I've got a question from @KnowTheory and he writes, I'm interested to see how this data could be tied in with university enrollment. Any ideas as to how #edugap could tie this in?

Sharona Coutts: What a great question. I haven't really thought about how to do that. I guess you could. It depends on whether you wanted to … what scale you wanted to do it on. Nothing really springs to mind immediately. I would say if you wanted to do it on a local level you might be able to make some requests to a local institution. I'm not sure how you would, what question you’d be trying to answer. I can think of a few possibilities, but Jen, do you have any thoughts?

Jennifer LaFleur: Well, getting that data would be a tough thing, first of all. But if local agencies are gathering it for their own use, that would probably be the best source of it. I know there are a few academic studies out there that have looked at this, and I'd be happy to email links to those after the call.

Sharona Coutts: Yeah, I mean, I know of certain studies that have been done in some states where they tried to actually track these kinds of thing using unique numbers for students so that they don't give away individual identities, but I don't know—get back in touch with us on that one because we'll try to help if we can.

Mike Webb: OK. Elizabeth Chu, do you have a question for us?

Elizabeth Chu: Yeah, I was wondering if there is no data for a certain category like AP, how I might find out why that's the case?

Sharona Coutts: Are you looking at something or you've run into that problem?

Elizabeth Chu: Yeah, I'm having that issue with my district I'm working at, Montebello Unified.

Sharona Coutts: Where is that?

Elizabeth Chu: It's in California.

And it just says not applicable, but I know that they do have AP classes.

Jennifer LaFleur: There were some places where we kind of made our own decision in correcting the data that if we could not verify that information with the district—and it's cases where there might be more kids enrolled in AP than exist at the school, so we knew that was not a possible number—then we N/A’ed that out, so that may be the reason for that.

Elizabeth Chu: All right. Thanks.

Mike Webb: OK. How about Francisco Vara-Orta? From the San Antonio Express.

Francisco Vara-Orta: OK, great. I have two questions, actually. I will cram them in. The first one being about the free-and reduced-price lunch data set. I was looking at some of my districts which are some of the lowest-income, predominantly minority  districts, and for Hollandale Independent School District, which is my largest district, I see that the free-and-reduced lunch is at 5 percent, and I just find that kind of low. I was wondering what data set that comes from and if you could tell me a little bit more about who qualifies for that.

Jennifer LaFleur: That is from the National Center for Education Statistics, and it is aggregated up from the school level. So, it would be aggregated from all of the schools in our data. So, any school that was not an alternative school or a charter school. But, yeah, that definitely seems like it might be something we just want to double-check. But it may be that one school, for example, reported something really low or really high on the extreme end, and that's kind of messing up the average.

Sharona Coutts: And to the second part of your question, did you want us to talk about what it means to be eligible for free-and-reduced-price lunches or do you understand?

Francisco Vara-Orta: I did understand that. I actually had a second question about a correlation that I noticed as well with my districts which tend to be in the southern part of the City of San Antonio, and they're lower income, working class. I see the level of experienced teachers, a percentage of them, is higher in those than in what are considered two of the best districts here—Alamo Heights and Northside ISD. They're about 4 to 8 percent. In my districts, all on the south side of the city, it's about 18 to 22 percent. I was wondering if that’s something … I'm sorry?

Sharona Coutts: You're saying that's the inexperienced?

Francisco Vara-Orta: Yes. The inexperienced teachers, the percentage. So, I was wondering if that's something you noticed around the country, if that just kind of goes hand in hand. It seems like a bit of, the situation—it wouldn't be a shock for most educational reporters that the most inexperienced teachers are going to some of the poorest districts, but I'm wondering if that's anything you noted in observing the whole data.

Sharona Coutts: Well, I'll let Jen answer the question about whether she noticed that looking across the country, but I just wanted to make a couple of points bouncing off of that question. First of all, yeah, I didn't think that would come as a total surprise to a lot of educational reporters because, you know, it's been something that's been talked about for years in education that lower-income districts generally don't have the same amount of money to pay teachers so there tends to be a pattern where the less experienced teachers go there and then teachers who are recognized as talented and experienced often will take jobs in districts that are able to pay them a little bit more.

That said, a lot of this data I think is really about actually quantifying on the larger scale and in school-by-school numbers what until now has often been sort of learned through academic studies. Also one of the things that I've sort of looked at when I've noticed places that have high numbers of inexperienced teachers is – that's actually where Teach For America is sometimes. I mean, I'm talking about a couple of examples where I've walked into a particular school and said, oh, OK, that's because you've got Teach For America there. Maybe that's actually something that's worth making a couple of phone calls about to see what's going on. Maybe a little story could come out of that, I don't know. But as for whether you looked at that as a national pattern, could you …

Jennifer LaFleur: Well, I didn't yet. Everything we did was actually split out by state, and I did find some states where there was a relationship with poverty and the less experienced teachers it just wasn't strong enough to jump out for everything we'd hoped we were looking at.

Mike Webb: OK. Next up is George Sharpley. George?

George Sharpley: All right. You used a lot of information about the poverty level and so forth, and I'm wondering how that information is gathered. Is that based on the lunches or is it documented by the actual income of the parent? How reliable is that? We're basing an awful lot on that determination.

Jennifer LaFleur: It’s data from the National Center for Education Statistics, which is reported by schools and districts to the Department of Education, and it is the basically the percentage of kids that qualify for either free-and-reduced-lunch programs, and those are calculated based on a measure of poverty of the family.

George Sharpley: There's been a lot of talk about people just signing up without proving their level of the poverty. Is that still going on?

Jennifer LaFleur: I'm not directly familiar with that, but I don't know how schools verify that, so it's possible that it's pretty much a consistent variable that most education researchers use for doing these kinds of analyses.

George Sharpley: Say that again, where it came from?

Jennifer LaFleur: The National Center for Education Statistics. I'll send a link out, as well, after the call.

George Sharpley: OK, great. Thank you.

Mike Webb: OK, we have a Twitter question from Kayla Webley of Time Magazine, and she asks, how can national publications best utilize this data? Any tips? I don’t cover just one state or district. That's a tough one, right?

Sharona Coutts: Well, hi, Kayla. You saw what we did. That's the approach we took. Let me think about these. What else would I write if I were going to give away my next story, I'm sorry, that's a tough one. We were thinking of how to push these and provide a resource and a tool for more local reporters, but we would love national reporters to also use this, but I'm a little reluctant to kind of go into every other story idea that I am contemplating at the moment as a national reporter.

Jennifer LaFleur: That said, I do think it would be kind of a great tool to, especially some of the tools that Al’s developed to actually find schools around the country to go do further reporting on. You know, schools that may be very much in contrast in a very closed area and actually have the data to tell you where to go on the ground and talk to people.

Sharona Coutts: Yeah, that's very true, actually. Like if you wanted to—one way you could do this if you—whatever story you're working on you could use this to find like outliers. If you're looking for a school to illustrate, you know, another Garfield High in East L.A., if you're looking for a school that has extremely high poverty and seems to have very high levels of AP enrollment and you want to tell a story about a school that's doing something like that, then you can actually find those schools in our app. You can find them and that way you can illustrate your story.

Mike Webb: Our next question is from Fabien Pepper. Fabien, are you there?

Fabien Pepper: Yeah, yeah. I was wondering whether when you all were gathering data if you came across any information or numbers on teenage parents and how well they were faring in different schools? Are they staying in school and that kind of thing?

Sharona Coutts: No. I tell you what. We would have loved to have had that. I mean, this data is magnificent in a lot of ways because it really does allow you to get a lot of so much granular information about schools, but there’s also a lot, now that we've worked with it, that we wished was there.

For example, another thing that we don't have and I wish we did is not just the overall percentage of students who are enrolled in AP but which students, you know, because the experts have pointed out to me, OK, that this doesn't account for things like in-school inequality, so we don't have student-by-student level data in this. I don't know, I mean it sounds like this is an area of interest for you. Do you know of any other sources of data about teenage mothers?

Fabien Pepper: No. I just heard a lot about it from individual schools, you know, some schools that really make an effort to keep teen mothers in through graduation and things like that. It is interesting to know whether those things are spread throughout districts or what.

Sharona Coutts: No, it sounds like a really interesting thing to look into, but the other thing that I would say is that the second set of civil rights data—the Department of Education is hoping to have that ready sometime in the fall, and it's not gonna get at your question, but for yourself and everyone listening just to know, it does have some really fascinating other stuff that I think we’ll probably be looking at, and that has more to do with school funding, some test score results, graduation, I believe, and some discipline stuff. But on the teen mother thing, I'm sorry I can't help you on that one at the moment.

Fabien Pepper: That's all right. I've got another small question. It's kind of a follow-up to what someone else asked but just to understand. … So, in terms of qualifying for free-and-reduced lunches, that's not like a fixed, say, family income across the country, right? That kind of varies by district probably—you know, which families or what income families would get those?

Jennifer LaFleur: Actually, not. It's based on national poverty guidelines. And I can send a link out to everyone for what those guidelines are around the country. Even though this is a pretty good measure and is used by a lot of research, when it comes to poverty, you know, there are issues as the previous gentleman mentioned, there are also issues on the opposite direction. Sometimes kids in high school tend to be somewhat less likely to request free lunch programs, and I've actually done local projects which are doable on a state or local area where you've looked at the feeder patterns from the elementary school through the middle school to the high school to really give it a different sense of what those poverty measures might be. But I'll send a link out to all of that after the call.

Mike Webb: OK, Erin Richards you've been very patient.

Erin Richards: Yes. I had two here. I found out when I moved down from the district level to school-by-school data, I lose those racial percentages. Are those available in, you know, do you guys have them, it's just not listed or can you only look up the ratio percentages when you compare districts and not individual schools that are in the district?

Al Shaw: You can see them in the individual schools if you click through to the individual school pages from the baseball card titles. Or you can also get them by downloading the CSV or looking at the expert view which has a link on the bottom left of the district and state pages. You can print them in the table view or on the individual school pages.

Erin Richards: OK, I'll go find that. And then, secondly, is there a way to sort the data by, for instance, looking at the top 10 districts in Wisconsin that have the highest number of kids taking at least one AP class, or the highest percentage of kids taking at least one AP class? Is it possible to sort in that way?

Al Shaw: I think you'll have to download the CSV and do that in a program like Excel. I don't think there's a way to do that in the app itself.

Erin Richards: OK. Great. Thank you.

Mike Webb: OK. I have a couple more tweets. Uhh @MPReports asks: Did you guys look at diversity comparisons? It seems to me after working with this most schools are very homogenous.

Jennifer LaFleur: I guess I'll take this one. We did not specifically look at a diversity measure such as the diversity index or desegregation index. We did look at the percentages of various rates against the outcome data, well, against the access to programs data we were looking at. But, no, I did not do that, but it would definitely be something worth doing if you're interested in looking at diversity.

Mike Webb: OK, and @JNBorchardasks: Most Wyoming numbers are off. Enrollment in a 12,000 district is inflated by 1,000. How was the OCR info verified?

Jennifer LaFleur: I had to email about Wyoming earlier today, and we did verification on extreme cases where we knew that the data could not possibly be correct. We had one school district, for example, that offered more than 22,000 separate AP courses. We knew that was probably not likely.

Sharona Coutts: … because there are only 37 of them in existence.

Jennifer LaFleur: And things like where there were more teachers than students or more students enrolled in a certain program than existed in the school, things that we knew just couldn't be the case, we either fixed if we could contact the district or the school and get correct information or had to N/A out.

The one problem with fixing just one number is that all of the data we have is broken down by category. So, all of the enrollment data and our data are broken down by race, by disability, by new English learners, and so to verify just the enrollment, we would actually have to re-verify all of that information, which we have done in some cases, but I actually just rechecked Wyoming against the original data we got from the Department of Education, and it is exactly what we got from them. So, that definitely would be something that we need to look into further.

Sharona Coutts: And to the extent that folks are finding stuff that doesn't seem right or more strongly than that you actually think is wrong, please let us know. We would love to be able to make those corrections. You know, that said, this is the data the government is using. This is the data that the federal Department of Education is using to make decisions. We did a lot of spot checks as well. And called hundreds of schools, and there were errors that we found that we corrected. Overall it seemed to be reliable, but yeah, that's not to say that there can't be certain districts where there are sort of a higher number than average of errors. If you pick that up, please let us know. We would like to fix it.

Jennifer LaFleur: In some cases where it appears to be an under-count, that could be simply because we did not include charter schools, we didn't include alternative schools, which might have made up for that difference. We also did not include—because everything aggregated from this school up—if there's a group of students that are not counted at a school level but are counted at a district level, we would not have captured those students.

Mike Webb: OK. We're going to begin to wrap things up. I've got two more questions via Twitter. I think you guys have done a great job. Two more questions. One being, do you foresee adding charter school network data in the near future? Will we be able to obtain that?

Jennifer LaFleur: We actually have the data, but we found, statistically, that it acted very differently from our other data, and after talking to a lot of experts, talking to the Department of Education, they said that to really have a similar group of schools for your analysis it was probably best to leave those out since some are designed for specific groups of kids. The same with alternative schools, which is why you don't find those in our statistical analysis.

But we do have charter school data if anyone really needs it. We did not do any updates or changes to the charter school data, so actually the data you can get from the Department of Education would be the same thing we would have on charter schools.

Mike Webb: And the last one is from Travis Pillow. Any suggestions for sources of educational outcome data that we can use for comparison.

Sharona Coutts: Yeah, you can wait for the fall when the next set of this actual survey data is going to come out, which I think could be really, really interesting. If you don't want to wait that long, there are some ways that you can go about this, and I guess we can add those links as well. I mean, there is outcome data that doesn't match up perfectly but might still tell you something like if you try and compare states as we did, you can look at states that have an opportunity gap and then you can look at how they're performing sort of compared to other states on national standardized tests, but it's not a school-by-school comparison, so there are some things you can do. I don't think any of them at this stage are perfect. But there is that next set of data that's coming that could be really interesting.

Jennifer LaFleur: Also, if you're local and covering the local level or state level, your state test score data or other outcome data your state might have or your local district would be another way to do that. And they should actually be able to have the ability to match that up to this data if they’re using the standardized Department of Education school codes.

Sharona Coutts: Yeah. I mean the main thing is we can't compare tests in one state to another state. But for local reporters, I mean, if you are staying within the same area and it's the actual same tests, it seems like that would be something that would be possible. There are also some other news apps floating around in certain places that have some outcome data on schools that you could sort of look at. I know the Chicago Tribune has a pretty cool app.

Mike Webb: All right, everyone, thank you so much for participating. We really look forward to seeing your stories. If you're able to do something, please send it to us at [email protected] and we'll share it. If you have any further questions, offline things, you can email Sharona or Jen, and Jen will also send around some additional links when we're done. And finally, we will put up the recording of this conversation on our web site. Thank you very much, and we hope to hear from you. Bye-bye.