In April 2017, partners from McKinsey & Company sent a confidential final report to the New York City corrections commissioner. They had spent almost three years leading an unusual project for a white-shoe corporate consulting firm like McKinsey: Attempting to stem the tide of inmate brawls, gang slashings and assaults by guards that threatened to overwhelm the jail complex on Rikers Island.

The report recounted that McKinsey had tested its new anti-violence strategy in what the firm called “Restart” housing units at Rikers. The results were striking. Violence had dropped more than 50% in the Restart facilities, the McKinsey partners wrote.

The number was bogus. Jail officials and McKinsey consultants had jointly rigged the Restart program in its earliest phase to all but guarantee there would be few violent episodes, according to documents and interviews. They stacked the units with inmates they believed to be compliant and unlikely to get into fights or to attack staff.

Publicly, McKinsey and top corrections officials touted the drop in violence in these units as an early sign of their project’s success — without disclosing that they had tilted the scale in favor of that result. After McKinsey handed off the inmate selection process, about a year into the firm’s work at Rikers, jail officials continued to manipulate the population of the Restart units to keep their violence numbers low.

In October of this year, the New York City Council voted to approve Mayor Bill de Blasio’s proposal to close Rikers. The vote occurred during the same month that a federal monitor, appointed by a court to oversee reform at Rikers, revealed that violence by jail guards there continues to worsen. Overall, using the metrics employed by McKinsey, jailhouse violence has risen nearly 50% since the firm began its assignment.

The full story of how New York City came to pay McKinsey $27.5 million only to abandon many of the firm’s recommendations and decide to shut Rikers has never been told. A ProPublica investigation, based on interviews with 36 people, half of whom worked directly on the project, as well as more than 10,000 pages of project documents, internal emails and other records, reveals that problems dogged the project at every stage.

Among the issues that plagued the project: McKinsey, which had never before advised a jail or prison system, made data errors that further undercut the results it reported from Restart units. The firm also persuaded the Department of Correction to spend millions on the sorts of advanced data analytics favored by McKinsey’s corporate clients. The department never ended up using many of the those data products, some of which simply did not work very well.

What happened at Rikers is a cautionary tale of a public-sector consulting boom that has emerged over the past decade. In recent years, government agencies across the United States have entrusted management consultants with more and more facets of public administration, from designing school systems to shaping Medicaid policy. Public-sector consulting in North America is a more than $9 billion industry, with an average yearly growth rate of about half a billion dollars, according to ALM Intelligence, which monitors the consulting business. McKinsey was anxious to expand into a potentially lucrative branch of public-sector work, corrections consulting, according to a former McKinsey consultant who worked on the project.

A McKinsey spokesman defended the firm’s work. “The Restart program had a significantly positive impact in the housing areas of the jails in which the program was piloted,” he said. “These units experienced substantial reductions in violence as measured against comparably classified inmate populations.” He added that morale among corrections staff assigned to the Restart units improved.

The spokesman denied that McKinsey’s consultants were involved in or aware of efforts to “improperly skew or reduce the reported levels of violence in the Restarts.” He added, “We stand behind our team’s professionalism and integrity, and the results of their work.”

The Department of Correction referred a request for comment to the mayor’s office. A spokeswoman there, Avery Cohen, maintained that the decision to hire McKinsey was sound and denied that city officials were involved in manipulating Restart units. “We sought the help of an outside firm for a singular purpose: to reduce the level of violence against staff and inmates and improve the safety of our overall facilities,” Cohen wrote in a statement. “We stand by the reduced rates of violence we saw in our Restart Units.”

But the gaming of the Restart units was confirmed by Joseph Ponte, who was the corrections commissioner from 2014 to 2017 and trumpeted the Restart results in public hearings. In a deposition last year, Ponte was asked whether he had been told that the teams running the Restart units “were cherry-picking docile inmates” to ensure that rates of violence would fall.

“We did that, so that wasn’t news,” Ponte testified. “I was well aware of how we were selecting inmates in the Restarts — very carefully. It was a new process for us, and we wanted it to be successful.”

---

McKinsey was hired in the wake of media reports in early 2014 that revealed an alarming rise in violence at Rikers. In just two years, the most serious inmate violence and use of force by guards had both jumped more than 50%. The United States attorney’s office in Manhattan began threatening to take legal action to force reforms.

De Blasio, then just a few months into his first term as mayor, faced a groundswell of outrage. His top deputy, Anthony Shorris, and other aides decided to hire a consulting firm. They solicited proposals from firms on a pre-approved list from the previous administration. Despite its lack of corrections experience, McKinsey won the contract.

The mayor’s aides saw hiring a firm like McKinsey as a way to stave off the need for a federal jail monitor, three current and former corrections officials recall from meetings with them. Hiring McKinsey also offered political cover, those sources said, and the imprimatur of having a respected firm tackle the problem.

McKinsey was selected because of its reputation for “dealing with complex organizations” and its history of working with New York City government, according to Shorris, who left the de Blasio administration in late 2017. McKinsey, for example, had previously prepared a report on 9/11 emergency preparedness and, during the administration of former Mayor Michael Bloomberg, had consulted for the New York Police Department on a reorganization and modernization.

The mayor’s office denied that McKinsey was hired partly to provide political or legal cover, as did Shorris, who is now a senior adviser to McKinsey and teaches at Princeton University.

Early on, the consultants took steps to avoid transparency. At McKinsey’s behest, a few top consultants and corrections officials communicated using Wickr, an encrypted messaging app that automatically deletes messages within hours or days. Ponte and his chief of staff, Jeff Thamkittikasem, downloaded Wickr, a spokeswoman for the mayor said, but neither could recall using it “in any significant way.”

But one person with direct knowledge of the practice said corrections officials and McKinsey consultants exchanged project documents over Wickr. That shielded portions of McKinsey’s work from government oversight bodies and public records requests. “It seems like there’s an intentional act to evade that system of accountability,” said Douglas Cox, an expert on public records law at the City University of New York School of Law. (A spokesman for McKinsey did not dispute that Wickr was used, adding, “our policies require colleagues to adhere to all relevant laws and regulations.”)

McKinsey’s work began in September 2014 with a $1.8 million contract and an ambitious mission: to determine the causes of violence at Rikers and propose solutions.

Almost immediately, signs emerged that McKinsey’s corporate orientation made for an awkward fit. When the consultants pitched Ponte on a proprietary workplace survey, they spotlighted how it helped lift output at a strip mine. “Productivity increased by 50%,” McKinsey’s pitch deck crowed. “~$180MM in annual run rate savings.”

As they formulated a reform plan, the consultants did not solicit the views of inmates, clinic staff or others with direct insights into drivers of violence. The closest the consultants came to interacting with the inmates during this stage was to watch them through Plexiglas from inside guard booths.

McKinsey began adopting the mindset of the correction officers, according to one of its consultants. The firm’s initiatives included facilitating the expanded use of Tasers, shotguns and K9 patrol dogs (“aggressive dogs,” in the words of one McKinsey presentation) at Rikers, a stance that seemed at odds with de Blasio’s claims of wanting to foster a more humane jail system. A McKinsey spokesman said these tactics were intended for use only in extreme situations and were designed to minimize injury while protecting inmate and staff safety.

At a 2015 meeting, a group of McKinsey consultants grew heated reviewing video of a jailhouse assault. There was talk of wanting to beat inmates to get the situation under control, according to a former McKinsey consultant with direct knowledge of the meeting.

Finally, Anish Melwani, a senior McKinsey partner, grew exasperated by the tough-guy posturing. “Guys, you’ve gone native,” the former consultant recalls him saying. “I think you’ve been spending too much time with the correctional officers.” A McKinsey spokesman said its team had no recollection of such a conversation. Melwani, who has since become CEO of luxury retailer LVMH Moët Hennessy Louis Vuitton’s North American operations, said he did not recall the specific meeting or comment but “reviewing footage of incidents was a routine part of the work while I was involved, so such a meeting would not have been unique.”

---

By March 2015, McKinsey and corrections officials had settled on a strategy to curb jail violence, which they called the “14-Point Plan.”

The provisions were straightforward. They ranged from adding educational opportunities for inmates to improving staff training to revamping the internal affairs division.

Some observers thought the recommendations were obvious. “A lot of the points were things the department was saying before they had McKinsey say it,” said Elizabeth Crowley, who at the time chaired the City Council committee that oversaw the Department of Correction.

McKinsey “contributed” to the development of the 14-Point Plan, a spokeswoman for the mayor said, but corrections officials primarily drove its creation and its tenets pre-dated McKinsey.

Top city officials, pleased to have a reform strategy to present to the public, were untroubled. In the spring of 2015, the city extended McKinsey’s engagement, assigning the consultants a long-term task: to help Ponte implement the 14-Point Plan. The firm’s original $1.8 million contract had now grown to more than $6 million.

---

The Restart units were conceived in early 2015. Outside pressure to fix the city’s jails had increased. The United States attorney’s office had joined a lawsuit over civil rights abuses at Rikers, and city officials were embroiled in often contentious settlement negotiations. The mayor’s top deputy, Shorris, was impatient for signs of improvement.

A series of tense meetings with Shorris in February and March created a sense of urgency for Ponte and his staff. After one particularly stressful meeting, a handful of Ponte’s aides met Benjamin Cheatham, a senior McKinsey partner who was heading up the project, at a bar near City Hall. Known by colleagues as the “little gray tornado” — a reference to his diminutive stature, muted color palette and overcaffeinated style — Cheatham had previously worked on McKinsey’s assignment for the New York Police Department.

Over margaritas, a person familiar with the meeting said, the group spent several hours sketching out what reform initiatives to test in the Restart units. A McKinsey spokesperson said it was one among many discussions about how to design the new program.

Shorris said his interactions with top corrections officials were generally “cordial and supportive.” His impatience, he said, was impelled by his desire to reduce jail violence, not by any pressure to resolve the civil rights lawsuit concerning Rikers.

Ponte and his staff would come to embrace the Restart program as their best bet for demonstrating impact quickly. The Restart units would also serve as a proving ground for the centerpiece of McKinsey’s work, an algorithm called the Housing Unit Balancer, or HUB. The HUB was designed to predict each inmate’s propensity for violence and then calculate how to distribute inmates across housing units to minimize the risk of conflict.

By July 2015, eight Restart units, housing a total of about 250 inmates, were in place at the George Motchan Detention Center, one of the 10 jails on Rikers Island at the time and known as GMDC.

Within weeks, the data showed violence far below the norm. Restart units for higher-security inmates, for example, experienced only a single incident in their first few months. Unreformed housing units, by contrast, saw 32 violent episodes over a similar period.

But for each Restart unit, jail officials had replaced inmates selected by the HUB but considered troublesome with alternates, believed to be more accommodating, chosen from a list provided by McKinsey. One email, forwarded to several consultants and obtained by ProPublica, shows a jail guard carefully curating the gang members on McKinsey’s initial list. One inmate was a Blood “but acceptable for program,” the guard wrote. The two other gang members on the list — a Latin King and a Crip — were “not acceptable.”

Today, McKinsey acknowledges that potentially unruly inmates were intentionally excluded from the earliest Restart units, but says it was done only to give the consultants time to fine-tune their approach. Yet neither they nor their clients disclosed that caveat when Ponte used a McKinsey PowerPoint to plug the violence levels at initial Restart units as “lower than GMDC and DOC overall” during a November 2015 hearing before the Correction Department’s oversight board.

Ponte began expanding the Restart program to other Rikers facilities. In time, according to Ponte’s deposition and McKinsey, the filtering of problematic inmates stopped. But seven current and former senior corrections officials interviewed by ProPublica said the practice persisted. (These sources did not know whether the McKinsey consultants were aware that the practice continued.)

A 2017 department analysis supports the contention that the manipulation continued. The analysis showed that, in one facility, inmates selected for Restart units had a significantly lower rate of violent run-ins with guards than inmates placed in other units.

On top of simply swapping out difficult inmates assigned to a Restart unit, jail officials began manually overriding some inmates’ HUB-generated risk scores to steer them toward or away from an available spot in a Restart unit.

A McKinsey spokesman said the firm’s consultants were not aware of these practices. In depositions last year, Ponte and Thamkittikasem also denied knowledge of them. Ponte did not respond to multiple requests for comment for this article. A City Hall spokeswoman said that, while McKinsey continued to monitor the HUB and its underlying algorithm, its consultants were not involved in final inmate housing decisions.

Top corrections officials, relying on data provided by McKinsey, continued to sell the Restart program’s success to the public by heralding dramatic reductions in violence within those units.

When there were flare-ups in the Restart units, the consultants and jail officials took action to stamp them out. After one jump in violence, jail managers barred a demographic prone to violence, inmates aged 18 to 21, from Restart units. “Effective immediately, no Young Adult shall be placed in any Restart Program Housing Units within the facility,” Chantelle Johnson, a deputy warden at GMDC, wrote in a memorandum in November 2016. “Failure to comply with this memorandum will result in disciplinary action.” A McKinsey consultant had a hand in producing the memorandum, metadata shows. (A City Hall spokesperson said the directive on young adults was part of a broader shift in how to handle young inmates and the Restart process was useful in demonstrating the need for a different approach.)

“They were trying to say that this model will work,” said Greg Kuczinski, a deputy corrections commissioner at the time who later sued the city for wrongful termination in an ongoing case that is unrelated to the McKinsey project. “But if you started with ideal inmates, if you will, how is that going to translate into real change when you throw it into the general population?”

---

For the mayor’s office, showing progress was crucial. The Restart units seemed successful, which led the mayor’s office to approve another contract extension in November 2015. The city committed to pay McKinsey $7.5 million on top of the $6 million already billed.

But the stakes for McKinsey extended far beyond Rikers. Data-analytics products, such as the HUB, offered a way for McKinsey to distinguish itself from the competition as consultants vied for fees in the corrections-consulting field. “The HUB was definitely driving a lot,” Kuczinski said.

McKinsey pushed the city to pay it to develop data-analytics software aimed at everything from “workforce optimization” and “workforce absenteeism” to “phone monitoring applications” and “decryption systems,” project documents show. Ultimately, the city’s bill from McKinsey would include $5.5 million for data analytics.

These tools, however, proved ill-suited to the Department of Correction. Many cellblocks lacked computer access, and the department’s antiquated IT system froze constantly. Jail guards and supervisors were unused to recording data electronically, let alone making sense of it.

Some of the McKinsey analytics products simply didn’t work very well. “IntelWatch,” for example, was an algorithm designed to predict gang affiliation and future violence. It was part of a broader McKinsey effort to import surveillance and analysis methods “widely used in the intelligence community,” as one PowerPoint slide put it.

IntelWatch, according to McKinsey slide decks, was designed to map each inmate’s social network by analyzing data from inmates’ phone calls, visitors and commissary deposits. The slides seemed detached from reality: As stand-ins for real inmates, they used the names of characters from “Gangs of New York,” Martin Scorsese’s movie about a battle for dominance among the Irish-immigrant underclass in Civil War-era Manhattan slums.

The surveillance algorithm’s effect on real-life inmates was never tested. IntelWatch, finalized in a rush at the end of McKinsey’s engagement, was riddled with bugs and errors, current and former corrections officials say. Today, McKinsey defends the program as version 1.0, a tool the Correction Department was meant to adjust and improve. But as one former consultant concedes, the department lacked the capacity to do so. The Correction Department paid for IntelWatch but never used it.

---

By the fall of 2016, McKinsey’s days at Rikers were starting to look numbered. Oversight bodies and the city comptroller were starting to question, in increasingly strident tones, what the city was getting for its money, even as the city was poised to extend McKinsey’s contract again, bringing the total cost to city taxpayers to $27.5 million.

By then, McKinsey had given officials a firm violence-reduction figure to quote, project documents show. In Restart units, “violence is down by over 70 percent when compared to other similar units,” the Correction Department’s chief of staff, Thamkittikasem, told the oversight board in October 2016. Over the next several months, he and Ponte would cite that figure repeatedly at City Council hearings.

Yet systemwide, inmates and jail staff were at significantly greater risk than when McKinsey’s engagement began. Inmate-on-inmate attacks had jumped by 25% compared with 2014 and assaults on jail staff and use of force by guards were up by, respectively, 15% and 25%, according to city data.

“You’re trying to paint a rosy picture, and so is the mayor,” Crowley, the councilwoman, told top corrections officials at a City Council oversight hearing that November. But “if I’m an inmate, I’m more likely to be hurt seriously.”

By the start of 2017, the city had decided to give McKinsey an end date: April 30. McKinsey’s mounting cost and a sense that the partnership had run its course prompted the decision, Ponte recalled in his deposition last year. “How much money you’re spending — eventually that’s really your job, so to speak,” he testified. “So you’ve got to wean yourself off of that.”

As the firm handed off its work, corrections officials at department headquarters were disturbed to receive documents that cast doubt on the integrity of the data provided by McKinsey. Officials discovered coding errors in spreadsheets used to collect violence data from Restart units. The flaws were significant. They inflated the baseline rate of violence, against which Restart units were measured, by 10% to 15%, according to City Hall. That had the effect of increasing the seeming drop in violence in the units. The officials, emails and interviews show, lost confidence in the violence-reduction figures McKinsey had provided them, figures that department leaders for months had touted publicly as evidence the Restart process was working.

“Validity of currently reported data is questionable, and is not liable to stand up to audit,” Kyle McDonnell, a Correction Department project specialist, wrote to his supervisors in a June 2 email, just over a month after McKinsey’s last day. (The New York Daily News first reported aspects of these concerns.)

Cohen, the City Hall spokeswoman, said McKinsey consultants helped corrections officials fix the problem. But internal Correction Department emails show that the fix was only partial, with some flaws remaining unresolved nearly six months after the consulting firm left Rikers.

In March 2017, de Blasio announced that he intended to close Rikers, and a little over a year later, GMDC, where McKinsey had pioneered the Restart units, became the first jail on Rikers to be shut. The Correction Department abandoned the Restart process altogether. Cohen said the city plans to export a limited subset of reforms McKinsey worked on, among them the HUB, to the jails that will replace Rikers.

McKinsey never stopped soliciting projects at Rikers, even after de Blasio announced his plan to close the jail complex. In its final report, the firm recommended redesigning Rikers to streamline jail operations and save the city money. It was an area, McKinsey asserted, where it could help.