Seven and a half years ago, as a new reporter here, I filed a Freedom of Information Act request for all reports of misconduct by federal air marshals.

It had been several years since the U.S. government rapidly expanded its force of undercover agents trained to intervene in hijackings after 9/11. And a source within the agency told me that a number of air marshals had recently been arrested or gotten in trouble for hiring prostitutes on missions overseas.

I knew the FOIA request would take a while — perhaps a few months — but I figured I’d have the records in time for my first ProPublica project.

Instead, I heard nothing but crickets from the Transportation Security Administration.

Finally, last Wednesday, an email popped into my inbox with the data I had been fighting for since my fourth day at ProPublica.

The saga to get the air marshal data reveals a lot about the problems with FOIA, which is supposed to guarantee the public’s access to government records, as well as what happens when an agency decides to drag out the process.

Even though the Federal Air Marshal Service insists it has taken steps to build an agency steeped in professionalism with no tolerance for misconduct, it continues to face the same issues it was battling when I filed my FOIA request in 2008.

While waiting for the data, I found dozens of air marshals who had been arrested for crimes ranging from aiding a human trafficking ring to attempted murder. One air marshal used his badge to smuggle drugs past airport security while another used his to lure a young boy to his hotel room, where he sexually abused him.

Air marshals had hired prostitutes in Barcelona and gotten into a fight with security guards after patronizing a brothel in Frankfurt.

Another marshal’s in-air behavior concerned flight attendants so much that they reported it to the agency, saying “I can’t believe he is able to carry a gun!” (That officer was later convicted of bank fraud for trying to cash a $10.9 million check that he said was a settlement after he was a scratched by a friend’s cat.)

As time passed, the problems continued.

Last year, several other news outlets published troubling reports about air marshals that sound remarkably similar. A few selections: Air marshals accused of hiring prostitutes in Europe and recording the sex on their phones. Air marshals describe a “party-hearty” atmosphere. Air marshal kicked off plane after throwing a fit when he was offered only one dinner choice instead of three.

Oddly, when the TSA finally responded to my seven-year-old request, it included its own analysis of the data along with an unsolicited statement.

“The vast majority of FAMs [federal air marshals] are dedicated law enforcement professionals who conduct themselves in an exemplary manner,” it said. “TSA and FAMS continually strive to maintain a culture of accountability within its workforce.”

The statement also said the agency saw a “significant reduction” in misconduct cases in 2015 as a result of its initiatives. But notably, the agency only provided data through February 2012, even though in my last email exchange with the office last month I requested the entire database.

This has become standard practice for many agencies. By delaying FOIA requests for years, the TSA gets to claim the data it releases is old news. (The agency made the same claim back in 2008, which — because of the data we received recently — we now know wasn’t true.)

So what did the data tell us about misconduct by air marshals?

For starters, air marshals were arrested 148 times from November 2002 through February 2012. There were another 58 instances of “criminal conduct.”

In addition, air marshals engaged in more than 5,000 less serious incidents of misconduct, ranging from 1,200 cases of lost equipment to missing 950 flights they were supposed to protect.

Is that a lot or a little? It’s hard to say because the number of air marshals is classified and the estimates of the size of the force don’t include turnover.

The TSA says the misconduct represents just a “handful of employees.” But concerned air marshals I spoke with said they should all show sound judgment, given that air marshals are allowed to carry guns on planes and must make split-second life-and-death decisions.

Some other highlights found in our analysis of the data:

• 250 air marshals have been terminated for misconduct; another 400 resigned or retired while facing investigation.
• Air marshals have been suspended more than 900 times, resulting in more than 4,600 days lost to misconduct.
• The Washington field office had the most incidents with 530 cases, followed by New York with 471, Chicago and Dallas with 373 each and Los Angeles with 363. There were 85 cases at air marshal headquarters, highlighting that in some cases, misconduct has extended to the top brass.

After our story ran in late 2008, Robert Bray, the director of the air marshal service at the time, vowed to create a “culture of accountability” within the agency and raised the penalty for drunk driving arrests to a 30-day suspension.

We now know the number of misconduct cases remained fairly steady, about 600 a year, in the years before and after our investigation.

It’s unclear if the agency got tougher or weaker. Before the story ran, only 4 percent of air marshals who had been arrested received a suspension of 14 days or longer. After the story ran, that number jumped to 20 percent. But at the same time, a much higher percentage of arrested air marshals got off with minor discipline such as a letter of reprimand, a warning or no action at all.

After the story, I continued to talk to air marshals and pursue the FOIA request. Inspired by the Obama administration’s memo on transparency, and armed with new information that there was a specific misconduct database, I filed a second FOIA request in 2010.

This was perhaps a mistake. Rather than respond to my first request, the TSA merged it with my new request.

In 2012, the agency responded. But the TSA only released two columns — one showing allegations against air marshals, the other listing disciplinary actions taken in response. Notably, there were no dates, which would have allowed us to check if the agency’s “culture of accountability” was working.

I immediately appealed. In addition, I filed another FOIA request for the entire database — “all columns and rows.”

Two more years passed. Meanwhile, air marshal director Bray himself became embroiled in a misconduct investigation. A supervisor was accused of obtaining free and discounted guns from the air marshals’ weapons supplier and providing them to top officials, including Bray, for their personal use. In 2014, Bray retired.

Around that time, I partially won my appeal. But the data was still incomplete.

After nearly six years, I had pretty much given up.

Until late December. That’s when an email arrived from TSA telling me my request from 2012 had been sitting in a backlog and wanting to know if I was still interested.

Indeed I was. (The TSA had asked me this question a few times during my pursuit of these records.)

A month later, I had the information I had been seeking. It only took seven years, seven months and 29 days.