The federal government’s announcement last week that it would begin releasing data on physician payments in the Medicare program seems to have ticked off both supporters and opponents of broader transparency in medicine.

For their part, doctor groups are worried that the information to be released by the Centers for Medicare and Medicaid Services will lack context the public needs to understand it.

"The unfettered release of raw data will result in inaccurate and misleading information," AMA President Ardis Dee Hoven, MD, said in a statement to MedPage Today. "Because of this, the AMA strongly urges HHS to ensure that physician payment information is released only for efforts aimed at improving the quality of healthcare services and with appropriate safeguards."

On the other hand, healthcare hacker Fred Trotter has raised concerns about CMS’ plan to evaluate requests for the data on a case-by-case basis. That isn’t much of a policy at all, he wrote, giving federal officials too much discretion about what to release.

So, how is this all going to shake out?

Three recent examples offer some clues.

The first involves the Wall Street Journal and the Center for Public Integrity. The news organizations sued the government in 2009 to obtain records on physician claims in Medicare. They received the information  as part of a legal settlement, but had to agree not to publish physicians’ names in most cases. They never got a complete set of Medicare payment data. Instead, they received a 5 percent sample of the Carrier Standard Analytic File, which includes records of Medicare Part B (outpatient) billings and payments.

That in itself was huge: In 2008 alone, it had about 42 million rows, each with 612 variables. It was about 38 gigabytes even before being imported into a database, data journalist Maurice Tamman wrote in a legal declaration. At the time, Tamman was a Wall Street Journal news editor.

The second example is the project that my colleagues at ProPublica and I have been working on to examine how doctors and other health professionals prescribe medications in Medicare’s drug program. Instead of seeking individual medication claims, we sought aggregate records for each prescriber, grouped by drug. We gave up some information we wanted, such as characteristics of the patients, but we also were not subject to any limits in terms of our ability to name doctors.

The result is our Prescriber Checkup news application that lets consumers look up their doctors and see how they compare to others in the same specialty and state. Our stories identified examples of risky prescribing, high rates of name-brand prescribing and patterns that suggested fraud.

Even though we did not have individual details on every drug claim filled—more than 1 billion a year—the files we had were also vast: more than 70 million rows of data on the drugs prescribed by 1.6 million providers in 2011 alone. In cases in which a provider wrote fewer than 11 claims for a particular drug, the data were redacted.

Finally, healthcare hacker Trotter obtained data from Medicare on referrals to and from providers within Medicare. He received statistics on the number of patients who saw one doctor (Doctor A) within 30 days of seeing another doctor (Doctor B). He’s created DocGraph to show these referrals visually.

According to his website, Trotter received nearly 50 million pairs of referring parties involving about 1 million providers in 2011. Like the data ProPublica received, Trotter did not receive information on referrals in which fewer than 11 patients were involved.

Some takeaways:

1. Medicare is far more likely to release aggregate information than data on individual claims. This is mostly to protect patient privacy, but also because officials have grown increasingly comfortable writing programs to aggregate the data (as was the case with ProPublica and Trotter).
2. Expect redactions. It’s safe to assume that Medicare will redact data in which fewer than 11 patients are involved. 
3. Medicare likely will not create a glamorous news application in which consumers can view the data. When the government released information on hospital charges last year, it released a big spreadsheet and left it to news organizations and others (see here and here) to come up with clever ways of displaying it.
4. Medicare, likewise, is unlikely to put together tip sheets and other context for interpreting the data. While the program should—and probably will—release basic information about what is being released, officials probably won’t tell consumers how much weight they should give it.
5. There will be far more requests for Medicare physician data than there will be Medicare staff assigned or available to fulfill them.
6. Those wanting every morsel of Medicare data to be released will likely be disappointed. This is a massive, immensely complicated program with many interrelated parts. More information may be released each year, but it won’t happen overnight.
7. Finally, few news organizations or research groups are equipped to deal with such large data sets and produce meaningful content quickly.

All that said, let the data releases begin.

Editor’s Note: This post is adapted from Ornstein’s “Healthy buzz” blog.