Massachusetts prosecutors run their offices on software from the dial-up era

Massachusetts has decided to lead the country on artificial intelligence. Over the past two years, the state has put a hundred million dollars into a new AI Hub, placed a ChatGPT assistant on the desks of forty thousand state employees, and directed every agency to work out what the technology can do for the public's business. The public is warier than its government: two thirds of Massachusetts voters tell pollsters they are more concerned than excited about artificial intelligence.^[1]

Massachusetts has not extended its technology modernization to the offices with the most direct power over ordinary life in the Commonwealth: the eleven District Attorneys. The state elects one for each county or small group of counties, and each runs an independent office that answers, between elections, to no governor, no mayor, and no oversight board.^[2] Their Assistant District Attorneys (ADAs) decide every day who gets charged with a crime, who gets offered a way out of one, and who gets left alone.

All eleven District Attorneys' offices run on a shared database called DAMION, which holds the record of essentially every criminal case in the state and is roughly twenty-five years old.^[3] Even the offices that use it every day do not defend its age. The Massachusetts District Attorneys Association (MDAA), the nonprofit through which the eleven offices share funding and technology, describes DAMION in its request for proposals for a replacement as "implemented approximately 25 years ago" and "nearing its end of life." The State Auditor went further this past November, reporting that the case management system used by all eleven offices "is obsolete and… may soon no longer be supported by the software manufacturer."^[4] When DAMION came online in the early 2000s, most Americans with internet at home still connected through a screeching dial-up modem, and the iPhone was still years away.^[5] AOL shut down its dial-up service last fall, by which point about one American household in a thousand still used it. DAMION remains in daily use by every District Attorney in Massachusetts.

What a prosecutor decides, and who can check it

Once someone is arrested in Massachusetts, no one has more power over what happens to them than the prosecutor. If you are ever arrested in Massachusetts, for shoplifting or for driving on a suspended license, the officer who arrests you can recommend charges but does not decide them; an ADA does.^[1] The ADA decides whether you are charged and with what, whether to ask a judge to jail you while the case is pending, what plea deal to offer, and whether to drop the matter entirely; a judge can rule only on what the prosecutor chooses to bring. DAMION is where every one of those decisions is recorded, when it is recorded at all. After years of working inside and alongside these offices, I have come to think of that bundle of decisions, the charging power, as the largest unreviewed power in Massachusetts government.^[2]

As a general rule, no court will second-guess a prosecutor's decision about whom to charge. The United States Supreme Court has held that, so long as a prosecutor has probable cause, the decision whether to prosecute and what to charge "rests entirely in his discretion."^[3] The Court left one narrow opening: a person can try to prove that prosecutors singled them out unfairly. But proving that claim requires records comparing their case against everyone else's, and no such records exist, because no office is required to keep them.

In Massachusetts, the one reliable way to see how prosecutors use the charging power is the state's public records law. The District Attorneys' files, policies, and data fall under the law, and the law reaches all the way into DAMION itself: offices have answered records requests by running queries against the database and producing the results.^[4] The law's reach has limits: the courts are exempt, and so, for now, are the Legislature and the Governor's office.^[5] Among the institutions that decide who gets prosecuted in Massachusetts, the District Attorneys are the ones whose data the public can demand most directly, and even that right lags other states; Florida and Connecticut now require their prosecutors to publish case data without anyone having to ask.^[6]

You do not need to share my politics to care about whether prosecution records are public. Right now the offices get to describe their own work on their own terms, because no one outside the offices can check the description against the record. Every other participant in the debate over prosecution, reformer and traditionalist alike, is asked for evidence; the offices alone are taken at their word.^[7] Requesting the records is how that changes.

Introducing the Massachusetts Prosecutor Data Project

This year I requested the records from every office at once. Beginning in February, I sent a public records request to each of the eleven District Attorneys, asking for data on the cases the office filed and resolved from the shared DAMION system. Ten of the eleven offices produced records. The eleventh, Bristol County, did not respond fully, and that failure is now on appeal before the state's records authority.

What the offices sent back has become the Massachusetts Prosecutor Data Project: more than six million criminal charges, reaching back decades in some counties, cleaned and matched into a single record that lets prosecution in Massachusetts be compared county to county and year to year for the first time. The county-by-county research begins publishing soon; if you want it as it lands, sign up at prosecutordata.org.

The transparency Massachusetts ordered in 2018

Massachusetts already promised itself a clear view of its criminal justice system once, in its 2018 criminal justice reform law. That law, the largest overhaul of the system in the state's modern history, ordered the construction of a single "cross-tracking" system: one database fed by every agency, from the police who make an arrest to the parole board that ends a sentence, with the District Attorneys named on the list, so the public could follow how people move through the system instead of guessing from each agency's separate reports. MassINC's research director called the law's data provisions "among the best in the country."^[1]

The surprising part is that the cross-tracking system was actually built. It is a state website: you choose a question from a menu, about who is in custody, who is being admitted or released, or who returns, and it counts people for you. This summer I downloaded all eleven of the data extracts its public tools will produce and compared what the system publishes against what the 2018 law ordered.

My test of the cross-tracking system surfaced three problems.

• The state undercounts how many people return to prison. The official measure does not start its clock when a person walks out of prison; it starts months later, when the person finishes parole supervision, and anyone who comes back during those months is never counted. Counted from the day of actual release, the one-year return rate for people released in 2022 is 20.8 percent, not the official 14.7.^[1]
• The closer you look, the less you are allowed to see. Broad questions get answers; the website will tell you how many people are in custody statewide. But ask a question with a face on it, like how many Black women in their twenties were held pretrial in Suffolk County, and the answers vanish: in the detailed views, roughly seven of every ten rows are hidden by privacy rules, and when I requested the most detailed table the tools would build, crossing eight characteristics at once, only two of its 6,573 rows came back with nothing hidden.^[2]
• The District Attorneys are not in the system at all. Eight years after the Legislature ordered every agency linked, the offices that decide who enters the system have never been connected, and the state board that oversees the law reports "no plan or schedule" to add them, because connecting the prosecutors depends on upgrades to DAMION.^[3]

I am not the only one who considers the 2018 law's data promise unkept. When Boston Indicators and MassINC assessed the reform five years after passage, their first recommendation was to "fully implement the data and transparency provisions of the 2018 reform law," and their judgment was blunt: the law had arguably positioned the Commonwealth to have "the finest criminal justice data infrastructure in the United States," and yet "the information available to researchers and the public has not appreciably improved since 2018."^[1]

What happens when the data gets out

The rare times prosecution data has reached the public, it has changed what the country knows. The clearest case is Massachusetts's own: economists at Rutgers, Texas A&M, and New York University used Suffolk County's own DAMION records to measure what happened when prosecutors declined to press nonviolent misdemeanor charges. For borderline cases, the kind one prosecutor might charge and another might not, being left unprosecuted cut the likelihood of a new criminal complaint over the next two years by 53 percent, and the benefit was largest for people with no prior record.^[2] I can vouch for the data because producing it was my job: I worked inside the Suffolk District Attorney's office as its technologist, and the extracts the researchers relied on came out of DAMION's underlying Oracle database. Suffolk's is not the only county's data to carry published research. Duke University researchers tracked plea bargaining with the Berkshire County DA's office across two 2022 reports, and the Urban Institute published a case study of the Northwestern District's drug diversion and treatment program.

Measurement changes prosecutors' offices from the inside as well as from the outside. In one office whose staff I interviewed last year, every attorney who asked a judge to jail someone before trial had to record the reason at that moment, choosing from a dropdown menu the office designed into its case management system. When the office's leaders finally read those reasons across their caseload, they learned that, by the office's own count, its attorneys were detaining forty-three percent of the nonviolent misdemeanor defendants its own written policy said should normally go home.^[3] Nobody had known until the menu existed. The office retrained its attorneys and kept watching the numbers until the numbers moved.^[4]

Most prosecutors' offices record no reasons behind their decisions and run no checks on them, so what the dropdown-menu office learned about itself could be learned almost nowhere else.^[5] At another office whose staff I interviewed, the information technology director discovered, entirely by accident, that one of the office's prosecutors appeared to have been closing cases in ways office policy forbids: no supervisor noticed, no audit existed to notice, and the discovery happened only because the director built himself a report nobody asked for. His explanation is the whole problem in one sentence: "Nobody—I don't know if nobody cared or nobody caught it." That sentence describes the ordinary American prosecutor's office, and, for the years I was inside one, it described mine.

The offices' objection, answered

Ask a District Attorney's office why the public cannot understand its data records, and the standard answer is that the data is too messy to mean anything. After months inside the data, I can say exactly how far that answer goes. For straightforward questions, like what was charged and what happened to the charge, the data is good, and more than six million charges resolve into consistent, comparable outcomes across ten counties. Past those basics, the quality of DAMION data depends on each county's habits, because every office uses the shared system its own way. One county delivered no bail data because it did not use DAMION to track the bail requests it made before District Court. Another office records nearly every non-conviction under one catch-all code, regardless of the true outcome. A third recorded thousands of successful diversions, cases resolved through programs instead of conviction, as "plea," which made its conviction rate look higher than it was until this project separated them.^[6] That variation should not exist. All eleven offices prosecute under the same criminal code and enter cases into the same statewide system; discretion in charging is the design, but discretion in record-keeping is not.

Messy is not the same as meaningless, and the offices treat their own records as meaningful every day. Suffolk County's office told the state's highest court that staff spend an estimated nine to eighteen thousand hours a year manually keying case data into DAMION.^[7] The District Attorneys' association compiles office data into a child-welfare report that state law requires it to file with the Legislature, a report the State Auditor just found inaccurate and incomplete.^[8] Middlesex County publishes its case data on its own website. The offices use these records to run their business, justify their budgets, and answer lawmakers.

The offices already act as if both claims are true: that the records are too flawed to publish, and that the records are good enough to run their business on. Suffolk described its own records to the state's highest court as the product of manual data entry "resulting in high rates of data error, delay,"^[7:1] and the report the law requires was found wrong by the State Auditor;^[8:1] the same offices rely on the same records every day to prosecute, to budget, and to report. If the records are reliable enough to run a prosecutor's office, they are reliable enough to answer for in public. If they are not, the Commonwealth has a far bigger problem than transparency. Either way the answer is to look.

The replacement is being decided now

The District Attorneys are now purchasing DAMION's replacement, which makes this the moment for the public to pay attention. In fairness to the offices, they did not choose this situation. The MDAA has been asking the state for money to replace DAMION for years, and the auditor's central recommendation was not that the offices try harder but that the funding finally come through; eleven offices have been running the Commonwealth's prosecutions on a system everyone, including them, agrees is at the end of its life.^[9]

The sympathy runs out at the association's silence about its own procurement. The MDAA announced the purchase in February and released its full request for proposals in March. The document never mentions the 2018 law that ordered the cross-tracking system, and none of its 118 reporting requirements or 44 named system interfaces connects the new system to it. When prospective bidders asked what the project would cost and who would pay for it, the association declined, twice and in writing, to answer, a silence worth weighing against the hundred million dollars the state found, quickly, for artificial intelligence.^[10]

Whatever the offices purchase will determine what the public can know about prosecution in Massachusetts into the 2050s. A database that never records a fact can never disclose it. The last system lasted twenty-five years, and unless something changes, the offices will choose its successor alone, the way they chose the last one.

The Massachusetts Prosecutor Data Project is my attempt to give the public a voice in that choice. The county research begins publishing soon, the audit of the cross-tracking system follows, and all of it arrives first at prosecutordata.org, where you can sign up.

And to the eleven offices, the invitation stands: talk to me about your data and your practices, because your records are imperfect and you are best placed to improve them. The state is about to spend years and millions replacing the software you work in every day, and the public deserves a say in what the replacement records and what it publishes. DAMION remains in daily use in every office in the Commonwealth, and its successor is being decided right now.