AI risks turning organizations into self-serving organisms if humans removed

With human bias removed, organizations looking to improve performance by harnessing digital technology can expect changes to how information is scrutinized.

The proliferation of digital technologies like Artificial Intelligence (AI) within organizations risks removing human oversight and could lead institutions to autonomously enact information to create the environment of their choosing, a new study has found.

New research from the University of Ottawa’s Telfer School of Management delves into the consequences of removing human scrutiny and measured bias from core organizational processes, identifying concerns that digital technologies could significantly transform organizations if humans are removed.

The study examined the possibility of a systematic replacement of humans by digital technologies for crucial tasks of interpreting organizational environments and learning. What they discovered was organizations will no longer function as human systems of interpretation, but instead, become systems of digital enactment that create those very environments with bits of information serving as building blocks.

“This is highly significant because it may limit or entirely prevent organizational members from recognizing automation biases, noticing environmental shifts, and taking appropriate action,” says study co-author Mayur Joshi, an Assistant Professor at Telfer.

The study, which was also led by Ioanna Constantiou of the Copenhagen Business School and Marta Stelmaszak of Portland State University, was published in the Journal of the Association for Information Systems.

The authors found replacing humans with digital technologies could:

• Increase the efficiency and precision in scanning, interpreting, and learning, but constrain the organization’s ability to function effectively.

• Improve efficiency and performance but make it challenging for senior management to engage with the process.
• Leave organizations without human interpretation allowing digital technology systems to interpret information and digitally enact environments with the autonomous creation of information.

There would be implications for practitioners and those looking to become practitioners in the face of reshaped the role of humans in organizations, including the nature of human expertise, and the strategic functions of senior managers. Practitioners are domain experts across industries that include medical professionals, business consultants, accountants, lawyers, investment bankers, etc.

“Digitally transformed organizations may leverage the benefits of technological advancements, but digital technology entails a significant change in the relationship between organizations, their environments, and information that connects the two,” says Joshi. “Organizations no longer function as human systems of interpretation, but instead, become systems of digital enactment that create those very environments with bits of information serving as building blocks.”