Calls for police accountability have included demands to examine the public money spent on law enforcement and the ways officers are disciplined. The Asbury Park Press and ProPublica examined the collective bargaining agreements that govern the relationship between unionized police forces in New Jersey and the towns where they operate.

There are 565 municipalities in the state, and some that contract with police unions are as small as 1 square mile. Police unions consolidate their legal and collective bargaining expertise across the state.

Our analysis found many examples of contract terms that experts say are costly to the cash-strapped towns that fund the agreements. Our story, How the Police Bank Millions Through Their Union Contracts, lays out benefits given to officers that may be more generous than towns can afford, but that are difficult to cancel due to ironclad contract language and the power of police unions. Other provisions could make it more difficult to hold officers accountable for misconduct.

Reporters at the Press and ProPublica wanted to determine how prevalent these terms are across the state. We collected thousands of current and past New Jersey police union contracts and developed computer programs and machine learning models to find sample clauses that experts say could waste taxpayer money or impede discipline. We then used those samples to identify similar clauses in police union contracts.

We quantified the provisions in 245 recent and current contracts, which can be seen here. Our analysis found that in at least 22 towns, the contracts guarantee senior retiring cops months of paid time off, known as “terminal leave,” as a parting benefit. We found that 26 towns offer “perfect attendance” incentive bonuses each year for using few or no sick days. In 54 towns, officers are allowed to sell back their unused sick days each year in a way that subverts the spirit of a $15,000 limit the state put on lump-sum sell-backs at retirement for public employees who have been hired since 2010.

We found at least 66 “past practice” clauses that require departments to continue handing out previously established benefits. This clause makes it difficult to cancel past perks — even those that were never established in writing — outside of official contract negotiations. We also found that at least 84 towns guarantee officers access to “extra duty” jobs that are a source of additional income for many officers. In 19 cases, the contract guarantees a minimum number of hours of pay regardless of how long the extra duty work takes. Finally, we found that contracts in 20 towns say police officers facing discipline are entitled to know the name of the person who complained about them.

We also estimated the aggregate liability imposed on New Jersey towns by officers owed money for their unused sick and vacation time to be at least $492.9 million in 2019.

Below, we describe how we gathered our data and arrived at conclusions, as well as what assumptions we made and what the limitations or our analysis are.

Data Collection

We collected the contracts for our analysis from the website of the New Jersey Public Employment Relations Commission, also known as PERC. Public employers are required to file with PERC the contracts they negotiate with public employee representatives. The site contains an extensive database of public contracts, including police union contracts, for nearly all the state’s municipalities, going back decades.

We downloaded all 6,366 of the contracts on the site, along with associated metadata, such as the contract’s start and end date, county and municipality, and the parties involved. We then removed any public sector contracts which were not for rank-and-file police officers. (Contracts negotiated by the unions representing firefighters, teachers and other types of police are also on the site.) The remaining contracts formed our starting body of 2,957 New Jersey police union contracts.

We used language from all the contracts, including historical ones, for our computational analysis. But our investigation focused on provisions in current contracts. Thus, we limited our final manual evaluation of provisions to the 245 contracts that are likely to still be in effect, meaning those that were effective on or after Jan. 1, 2019. This left us with a pool of contracts for under half the municipal agencies in the state. See the below section “Incomplete Collection of Contracts” for reasons that a department’s agreement may not be included.

Text Digitization and Cleaning

Using Google Cloud Vision, we converted the digital image PDF contracts into machine readable text. We then trained a type of machine learning model called a conditional random field to weed out extraneous text, such as cover and signature pages or the table of contents, leaving only the contract text we were interested in.

Sentence Segmentation and Encoding

We used a sentence as the basic unit of analysis. We segmented the body text for each contract into individual sentences using the language processing library spaCy. Then we fed each sentence of more than three words into Google’s Universal Sentence Encoder, which converted them from a textual to a numerical representation. These numerical representations allow us to computationally compare the similarity in meaning between sentences without having to compare their text. This is especially useful where the words used in two sentences are completely different, but the meaning remains similar.

Original sentence:
Terminal leave may be taken at the Employee's sole option as a six month leave of absence with full pay and benefits or as a lump sum payment of six months salary at retirement.[Carlstadt Boro and PBA Loc 312 2020.pdf]

Similarity: 0.7607
an employee shall have the option to be paid said terminal leave pay in a lump sum check on his last day of active service , or be permitted to use such accumulated terminal leave as a paid leave of absence prior to retirement .[Middletown Tp and PBA Loc 124 2011.pdf]

Original sentence:
if an employee is under arrest or is likely to be , that is if he is a suspect or the target of a criminal investigation , he shall be given his rights pursuant to the miranda decision .[Passaic and PBA Loc 14 2007.pdf]

Similarity: 0.8442
in cases other that departmental investigations , if an officer is under arrest or if he is a suspect or the target of a criminal investigation , he shall be given his rights pursuant to the current decision of the united states supreme court . [Buena Boro and PBA Loc 266 2013.pdf]

Similarity: 0.8099
5.3.8 if an employee is under arrest or likely to be , or if he is a suspect or the target of a criminal investigation , he shall be afforded and advised of all his rights pursuant to the united states constitution . [Paterson and PBA Loc 1 2008.pdf]

Collecting Representative Contract Provisions

The process described above allowed us to convert our collection of contracts into a database of sentences. We then identified representative provisions from across our database in each category of clauses. The clauses we examined were:

• Past practice clauses, which require employers to continue previously established benefits to officers, even if the benefits weren’t written into the contracts, unless the benefits are explicitly ended by a new contract.

• Sick day sell-back provisions, which allow officers to convert unused sick days to cash or paid time off. In some cases the provisions could subvert a 2010 state law that limited sick day payouts at retirement to $15,000 for employees hired after that date.

• End-of-career payments, which call for a town to pay an officer months of salary upon retirement after 20 or 25 years of service, as a parting benefit.

• Extra duty provisions, which govern side work that towns direct to their officers, and have become a routine source of cash.

• Rules about sharing the identity of complainants, which require that the identity of a person who complains about an officer to be provided to the officer under investigation.

We used two strategies to find various versions of these clauses. We first developed a lookup tool for the reporting team that could take any sentence as input and return the most-similar sentences from the database. Starting with provisions identified in our reporting, we used this lookup tool to find alternative wording or different terminology in otherwise semantically similar provisions in our data. We also used clustering algorithms to automatically group sentences in the database with similar meaning, to see if known provisions were connected with other sentences. For each clause, we repeated this exploration until we could add no alternative wordings to our collection of representative provisions. We ultimately identified dozens of representative provisions for the categories.

Evaluating Contracts

We evaluated the 245 recent and current contracts in our collection against these representative provisions. Any contract with at least one sentence with a high similarity to a representative provision was considered likely to fall in the corresponding category and became a candidate for further review.

Next, to eliminate false positives, we manually reviewed and verified the language and context in which the candidate sentence appears. To ensure consistency, we defined a standard for each category. Past-practice clauses were flagged if they required continuation of past benefits for officers that went unaddressed by the current contract. Extra duty provisions were tagged whenever the contract acknowledged the existence of such an arrangement. Provisions that entitled officers to know the names of citizens who had filed complaints against them were counted only if the text explicitly required it. Most contracts simply provided that the officer will be “informed of the nature of the investigation,” and few contracts explicitly prohibited disclosure of the complainant’s name to the officer who had been the subject of the complaint.

The sick day and retirement payout category was the most complicated. Towns use a variety of formulas and tiered calculations to determine the size of an individual officer’s benefit. To be consistent, we spotlighted such provisions if they violated the spirit of a 2010 state law (N.J.S.A. 40A:9-10.4 and N.J.S.A. 11A:6-19.2) that limited sick day cash-outs at the end of a career to no more than $15,000 for employees hired after that date. If a contract negotiated after 2010 allowed for sick day sell-back benefits that would amount to more than $15,000, whether in the form of cash or paid time off, over the career of an officer newly hired under that contract, we included it in the sick day cash-out category. Since the law supersedes contracts that allow an end-of-career sellback that exceeds the $15,000 cap, we set those contracts aside.

Calculating Future Payments for Unused Time Off

Getting paid for unused sick and vacation time is a significant perk in many union contracts, and it’s often available to police and other municipal employees. Many employees hired before 2010 law still receive the benefit. These checks, cut at retirement, have made headlines because they sometimes reach six-figure sums. Using more than 500 budget documents obtained primarily from the New Jersey Department of Community Affairs website, our analysis found that New Jersey towns were liable for $938.6 million of unused time off in 2019, the total owed to all municipal employees.

We then manually reviewed the organizations named in these documents to identify how much of this liability was attributed specifically to law enforcement employees. Our review found $492.9 million was due to law enforcement officers in towns throughout the state. However, this understates the true burden attributed to police because some towns didn’t specify how the total was divided among their employees.

The Department of Community Affairs clearinghouse for budget reports was missing 2019 data for about 40 towns. We compiled information for about 30 of those from town websites and public records requests.

Assumptions and Limitations

We examined a large number of police union contracts, and there was a wide range in contract language. Below are the assumptions we made in our analysis and the limitations of our conclusions.

Incomplete Collection of Contracts

First, the universe of contracts we examined is not complete. While public employers across the state are required to file these agreements with PERC, many municipal police departments did not have current contracts on the site. In many cases, municipalities have not yet provided these agreements to PERC. In others, these contracts may have automatically continued upon expiration if no replacement agreement was negotiated. It was impractical to manually examine whether each municipality’s contract was current, so we only looked at contracts that expired after our cutoff date of Jan. 1, 2019. We also scraped our universe of contracts in July 2020, so there may have been updates since then.

Risk of Compounding Errors

Machine learning models save time on manual review, but their results are bound to be imperfect. Image-to-text conversion is not perfect. Like any model, the one we trained to strip extraneous language can make errors. We caught some of these during manual checks, but reading through all the documents would be practically impossible and would negate the benefit of building a model in the first place.

In any similar data pipeline, there is potential for these errors to compound, leading to blind spots — potentially large ones — in the final outcome. We accounted for this by reviewing results by hand to rule out false positives and by estimating our false negative rate using random sampling. Our false negative rate for each category was low: 15% for past practice clauses, 15% for extra duty provisions, and 3% for clauses that required sharing the identity of a complainant. We also decided to manually review all contracts for sick day sell-backs because of the provision’s complexity. These limitations are inherent in data analyses of this kind and were not unique to this project.

In addition, since the contracts we analyzed and reviewed consisted only of those we could obtain from the PERC website, they may not be representative of all contracts in New Jersey. We reported accurate numbers for towns that use a given contract provision, but a reader should not use the data or our analysis to draw inferences about the state as a whole, or about towns that are not in our analysis.

Evaluating Sick Day Cash-Out Provisions

We also made a number of assumptions, guided by experts, when determining whether a particular provision violated the spirit of the law. This was especially true for the complex category of sick day sell-back provisions, which appear in a variety of forms. As we mentioned, a 2010 state law limits sick day sell-backs to $15,000 at retirement for employees hired after that date. We counted a sell-back benefit in our tallies if it was structured in a way that could violate the spirit of the law through a loophole: allowing annual sell-backs that could, over many years, tally up to more than $15,000.

We searched for contracts that allowed sick days to be turned into a benefit as a recurring annual option. Accumulated unused sick days could be converted to cash or paid time off, or a combination thereof. The state law does not limit these benefits for officers hired before the law. So we only counted contracts that would allow the annual sick time benefits for officers hired after the state law to exceed $15,000.

We estimated whether a typical officer could get around the state cap by selling back unused time off annually. To calculate the typical officer’s payout, we assumed an officer would work a 20-year career with 8-hour work days, adding up to 2,080 hours in a year, which is the method used to calculate the hourly rate in most of the contracts. We estimated a median police salary for a typical officer by filtering pension data to find the base salaries of officers with 10 years on the job, that is, those who were halfway through a 20-year career.

We calculated the maximum possible benefit assuming an officer takes 10% of his or her sick days over the course of a career, which we understand to be a reasonable assumption according to our conversations with experts.

To arrive at the final dollar value of a benefit, we used the formulas specified in each contract. Those typically consisted of the number of sick leave hours per year, the rate of conversion for sell-back (e.g. one day’s pay or paid leave for two days of sick days sold), and sometimes a limit on those sellable hours. In some contracts, unused sick days were instead awarded as paid time off. We calculated conversions of unused sick days to paid leave at the wages paid for that time off, with the same salary assumptions as above.

In some cases, we checked town payroll records to see whether officers hired after 2010 were selling back unused sick time annually. In one town, Norwood, we found that four officers had already exceeded the $15,000 limit through annual sick time sell-backs. In other towns, we found sick day sell-backs taking place in a way that, if it continued, would put some officers on track to exceed the $15,000 limit.