This story was also published in The Seattle Times on Sunday.
The worst foreclosure crisis in the nation's history might have been detected earlier if the government had a better tracking system for mortgage data, experts say.
High levels of risk became apparent only after the housing market collapsed and loan defaults skyrocketed.
"It's like air traffic control without a radar screen," said Keith Ernst, director of research for the Center for Responsible Lending, a nonprofit research organization on financial lending with offices in Washington, D.C., and Durham, N.C. "If air traffic control sees 20 planes coming in at once, they can do something about it. Otherwise, there will be serious problems."
Susan Wachter, a real-estate and finance professor at the University of Pennsylvania's Wharton School, said borrowers and investors were both hurt by a lack of real-time data on loan delinquencies and a lack of transparency in the world of mortgage-backed securities.
"If we had that, we would have seen the crisis in 2006," she said.
Even now, federal agencies from the Federal Reserve to the Government Accountability Office have to rely on mortgage data they purchase from private companies.
"There really wasn't and still isn't comprehensive data in the public domain that gives us a good handle to analyze it," said Bill Shear, GAO director of financial markets and community investment.
Congress, consumer advocates and researchers are pushing for expanded public mortgage data, such as loan terms, as well as the age and credit score of borrowers, but all without names, specific addresses and other personal identifiers.
Experts say such faceless data could reveal, for instance, a particular lender making high-interest loans to borrowers with high credit scores, an indicator of deceptive practices. But the mortgage industry is pushing back.
Expanding current mortgage data means "you're essentially broadcasting people's credit scores," said Bill Himpler, executive vice president of the American Financial Services Association.
Outdated Law Tracks Mortgages
A 35-year-old law dictates the home-mortgage data that lenders must report, but it has fallen far behind today's mortgage reality.
The Home Mortgage Disclosure Act (HMDA), passed in 1975, was intended to stop banks from discriminating against minority borrowers. Lenders were required to disclose such things as loan amounts, reasons that a loan was denied, and the race or ethnicity of the borrower.
Not reported: key loan characteristics, such as teaser rates, balloon payments, fees and penalties, or borrower attributes, such as first-time homebuyers, age and their debt level. HMDA also leaves out major players -- smaller banks and mortgage brokers have no obligation to report.
Recognizing the need for better data, regulators have been playing catch-up. But while Congress has called for agencies to collect more information, there still are critical limitations, both in what the government collects and what it provides to the public.
The U.S. Treasury Department began requiring the nine largest national bank-mortgage servicers to submit loan data on a monthly basis beginning in early-2008, and in June of that year, it issued its first quarterly Mortgage Metrics report.
Loan servicers -- mostly big banks -- must report 108 different pieces of information on millions of loans to the department, including interest types and rates, credit scores and foreclosure data.
But those metrics cover only about 65 percent of first mortgages. And loan-level data is not available to the public, not even with names and identifying information stripped out. Treasury doesn't plan to make it public, citing concerns over privacy and banks' proprietary information.
Congress Mandates New Reports
This July, President Obama signed the Dodd-Frank Wall Street Reform and Consumer Protection Act, which among other things, calls for increased reporting under HMDA and adds a financial-research division at Treasury.
As a result, the Federal Reserve Board is revamping what lenders must report, including credit scores, ages and ratios used to assess borrowers' ability to pay.
The financial overhaul also directed the federal Housing and Urban Development Department to develop a database on defaults and foreclosures, and make the information available to the public. Such data might alert regulators of changes in lending patterns before they get out of hand.
The project could take as long as three years, said Brian Sullivan, a HUD spokesman, but consumer advocates say it's worth the wait.
"Homeownership remains our primary mechanism for economic mobility," said Ernst of the Center for Responsible Lending. "Understanding how that's working is pretty critical."
Editor's note: The Sandler Foundation is a major funder of ProPublica and the Center for Responsible Lending, which operate independently from each other.
Inform our investigations: Do you have information or expertise relevant to this story? Help us and journalists around the country by sharing your stories and experiences.
Get ProPublica's investigations and major stories delivered to your inbox.