AI Workplace Writing Study Raises Questions for Property Firms and Employers
For property firms and other employers using AI to draft workplace messages, a small change in prompt wording may produce writing that sounds less formal and less complex. A Johns Hopkins University study found that four chatbots returned different results when requests included language more commonly associated with women, raising questions about how AI-written communications may affect perceptions of professionalism.
The research, from Johns Hopkins in Baltimore, tested GPT-4, Llama, Gemma and Mistral on workplace emails, job applications and resignation letters. Researchers varied the wording of prompts, adding features such as “maybe” and “I think,” collective terms such as “we” and “our team,” and expressive adjectives including “lovely” and “wonderful.” The resulting text was less complex, written at a lower grade level and less formal than responses to prompts using language associated with men.
That distinction matters in workplaces where employees may use generative AI to prepare messages that colleagues, clients or candidates will read as their own. The findings don’t establish how every reader would judge a particular email. They do suggest that phrasing could affect the polish and perceived authority of text produced by the same tool.
Prompt wording changed more than style
Katherine Van Koevering, an inaugural postdoctoral fellow at Johns Hopkins’ Data Science and AI Institute and the study’s lead author, said the scale of the difference caught the research team off guard. The outputs shifted in complexity, reading level and formality when the researchers changed the linguistic features in the prompts.
“The headline finding, of course, is that when we rewrote prompts asking for workplace communication using more women-associated linguistic features, the model output was of a lower complexity level, lower grade level, I believe less formal,” Van Koevering said. “It looked surprisingly different.”
The study focuses on how people interact with AI systems, a research angle that Anjalie Field, its senior author and a Johns Hopkins computer scientist, said has received less attention than the content models generate. Field studies ethics and discrimination in AI. She said researchers have examined bias in model outputs, but have done less work on how the systems perform for different users or in response to different ways of asking.
For businesses that rely on written communication, the practical question is not limited to whether a chatbot can produce a grammatical email. Teams also need to consider whether the final text fits the purpose and the sender’s intended voice. In property businesses, where staff regularly write to colleagues, applicants and customers, that distinction can shape how a message lands. The study did not test a real estate workplace specifically, so its findings should be read as evidence about workplace writing tools more broadly.
Researchers tested for simple style imitation
A possible explanation was that the models merely copied the prompt’s style. The team measured the complexity of each prompt and compared it with the complexity of the answer to estimate how much of the gap reflected that kind of mirroring.
“We would measure the complexity of the prompt, and then we compare that to the complexity of the output,” Van Koevering said. “That would tell us how much of this is mirroring versus the AI picking up on something else.”
Mirroring accounted for no more than about a third of the complexity difference, even at its highest measured level, she said. In most cases, the share was lower. The researchers said that amount generally wasn’t enough to undermine the statistical significance of their results.
The effect appeared in all four chatbots, though its strength differed by model. Van Koevering said the results were significant across every system tested, even though the responses did not change in precisely the same way. That consistency across tools makes the finding relevant to organizations whose staff may use more than one AI product.
Perceptions of authority and competence
The team also asked an AI system to compare responses generated from prompts with women-associated and men-associated linguistic features, then judge which sounded more authoritative and competent. It selected the response based on the men-associated prompts most of the time, according to Van Koevering.
That result points to a possible effect beyond writing complexity. Field said people using hedges or expressive adjectives could receive text that reads as less sophisticated or formal, and that a recipient may then connect the message’s style to the person who sent it.
Van Koevering cautioned that the pattern may apply to linguistic differences beyond gender. British spelling or slang, for example, could also lead to different model responses. The researchers tested whether adding a traditionally male or female name to a prompt affected the result; according to the Johns Hopkins release, it had virtually no effect.
Other research has examined gender and AI in employment contexts. A University of Washington study of AI resume screening, summarized in an April 2025 Brookings Institution article, found that men’s and women’s names were selected at equal rates in only 37% of 27 screening tests spanning three language models and nine occupations.
Separate reporting has covered University of Chicago research indicating that women were less likely than men in the same occupation to use ChatGPT, as well as an International Labour Organization warning that generative AI could affect women’s jobs more than men’s. Those findings address different questions, but together they put the new study within a broader discussion of gender and workplace AI.
What employers and workers can do now
Field suggested employers make staff aware that prompt wording can influence the tone and complexity of AI-generated text. That kind of guidance may fill a gap: in a May 2026 survey of 1,020 employees by career platform Resume Now, 41% said their employer had provided no AI tools, training or guidance, according to HRD America.
For individual users, Field advised checking whether a response’s tone matches the message they intend to send. A prompt can also specify the desired tone, although the researchers did not test that approach directly. Van Koevering suggested comparing drafts by rewriting the same request in different ways and looking at how the outputs change.
Those checks take time, especially when a worker has already invested effort in drafting a request. Field said she doesn’t see repeated user adjustments as a lasting answer because they place responsibility on the person using the tool. She argued that developers should train models to respond appropriately to the request’s purpose across different styles of phrasing.
For employers, the immediate takeaway is practical: staff may need time and guidance to review AI-written messages before sending them. For model developers, the study raises a design question about whether the same workplace task should produce comparable levels of formality and complexity regardless of who is asking or how they naturally phrase the request. Until that issue is addressed, prompt wording remains a variable worth checking in everyday workplace communication.