The Machine Learned His Voice. Then He Had to Censor Himself to Teach It.
Tobias Weinberg spent seven months logging every message he typed into his speech-generating device — and found that the act of recording his own speech changed what he was willing to say.
The finding, published in April at the CHI conference in Barcelona, comes from an arXiv preprint by Weinberg, a doctoral student at Cornell Tech who uses augmentative and alternative communication (AAC) devices after a neurological condition took his ability to speak at age 15. His research team collected seven months of his AAC communication data, fine-tuned a language model on that record, then deployed it as his primary communication device for three months. The goal was to find out whether a machine could learn to speak the way a specific person actually speaks — and what that process would cost.
The answer is not clean.
"I didn't expect it," Weinberg told the Cornell Chronicle. "Just knowing I was logging changed how I spoke. I was no longer just speaking in the moment. I was curating a future dataset, and that changed my sense of freedom in conversation."
What he edited out first were the parts that did not fit a clean record: swear words, dark humor, venting about people he did not like. The content that, if it surfaced at the wrong moment in the wrong conversation, could cause problems. Weinberg was not a different person during those seven months. He was a person performing a version of himself for an audience of a model that did not yet exist. When the model was trained and deployed, the self-censorship was already baked in.
"The first things to disappear were informal, emotionally charged expressions," the paper notes — dark humor, gossip, venting. The AI had been trained on a person who had already been edited. Weinberg called this the "socially well-behaved" model: the version of him that would pass a job interview, not the version that tells the joke at the dinner table that makes everyone groan and then laugh anyway.
There was a second failure on the other end. The system occasionally surfaced things that should have stayed buried: private biographical details and cultural markers that made sense in one context and made no sense in another. Weinberg described this as a privacy breach, not a technical glitch. The model had learned him too well and too literally — it did not understand that the same fact about a person's life belongs in some rooms and destroys them in others.
"Can you build a truly personal AAC without also building a surveillance system for your own speech?" he asked in the Cornell Chronicle. The paper does not answer the question. The paper is the question.
The work comes from Cornell Tech's Matter of Tech Lab, led by assistant professor Thijs Roumen, who received a seed grant in December to explore setting up a center for assistive technology and launched a Cornell Initiative for Assistive Technology and AI in February. Weinberg is also an Apple Scholar in machine learning for 2026. The study has documented limits: one person, literate, typing on a full keyboard, using AAC for more than a decade. The findings do not transfer automatically to every person with a speech impairment. The technical implementation also relied on a cloud-hosted model — a version running on the device itself would leave the user's data off remote servers, which would change the privacy math even if it did not eliminate the self-censorship problem.
The uncomfortable frame that survives all the caveats: the people assistive technology is supposed to protect are, in the current architecture, its training data. That is not an accident of implementation. It is the design. If you want a model that speaks like a specific person, you need that person's language history. If you have that history, you have built a surveillance system for their speech. When the person doing the surveillance is also the person speaking, the paradox is not solvable by being more careful. It is structural.
Commercial AAC systems — Tobii Dynavox, TouchChat, Proloquo2Go — are not doing what Weinberg did. According to Cornell, they use generic language models, not ones trained on individual users' communication patterns. That makes them less personalized for any given person and less invasive. The current market choice is between a system that does not really know you and one that learns too much. Weinberg tried the second option and found it carried its own costs.
More than two million Americans use AAC devices for speech disabilities. The question his paper poses — without resolution — is what it means that the most personalized version of those devices requires building, and then living inside, a record of yourself that you cannot fully control.