Pentagon pilot program has discovered over 800 potential vulnerabilities and bias in the use of large language models (LLM) for military medicine, The US Department of Defense.
The study was carried out by the Digital Technology and Artificial Intelligence Management (CDAO) as part of the Crowdsourced AI Red-Teaming Assuraance with the participation of the executive programs of medical systems of the Ministry of Defense and the Military Health Agency. The technological non -profit organization Humane Intelligence also participated in the work.
During the testing, the risks of using LLM were analyzed for a brief presentation of clinical records and the work of a medical consulting chatboard. More than 200 people took part in the tests, including medical specialists and analysts of the defense department. Three popular artificial intelligence models were tested.
According to the results of the work, the Pentagon said that the program was “successfully completed”. The collected data will help create test sets to evaluate future suppliers and tools for compliance with productivity expectations. Also, the results will become the basis for the formation of the Pentagon policy in the field of responsible use of generative AI in military medicine.
The head of the CDAO responsible AI Matthew Johnson noted that the program will identify problem areas and check the mechanisms of their elimination, which will contribute to the development and improvement of generative models.
CDAO has been operating since June 2022, engaged in testing, expansion and integration of AI into defense structures. In August 2023, the department launched the initiative of Task Force Lima to study promising technologies, and in December 2024 created the Center for the quick deployment of AI, which will engage in the scaling of AI tools in partnerships with the management of defense innovations.
The Ministry of Defense noted that the work within the framework of Crowdsourced Ai Red-Teaming Assaurance will help accelerate the implementation of II solutions into military medicine.