Humans throttled AI in tackling a series of business-related tasks in a workplace setting.
The artificial intelligence programs performed poorly compared to their human counterparts, a 47% score compared to the humans’ 81% score. According to the team that conducted the study, the AI performed especially badly when it came to finding references to the AISC in the documents it was supposed to use to create its summaries.
“Finding references in larger documents is a notoriously hard task for LLMs due to context window limitations and embedding strategies,” the team wrote. “Page references are not traditionally stored in the embedding models as the contents of PDF documents are ingested as plain text. To achieve better accuracy with this issue, substantial progress was made by splitting documents into pages and treating pages as chunks with associated metadata.”
According to the study, the reviewers often had “to refer back to the source material to confirm AI summary details,” and they “generally agreed that the AI outputs could potentially create more work if used (in current state), due to the need to fact check outputs, or because the source material presented information better.”
Although this study is limited in its application of artificial intelligence, it does imply that the expected benefit of AI could be tempered by humans having to do more work to fit AI’s specific needs in the workplace.
According to the researchers, “While AI holds the potential to deepen racial inequalities, it can also benefit Black communities. If deployed carefully, AI has the power to improve access to healthcare and education, as well as create new economic opportunities. For example, AI can help doctors make more accurate diagnoses and provide personalized treatment plans, particularly in underserved communities where access to healthcare is limited.”
RELATED CONTENT: Job Concerns Rise Amid AI’s Impact On The Gaming Industry
Source: Black Enterprise
