Optimizing Multimodal Large Language Models for Detection of Alcohol Advertisements via Adaptive Prompting

Daniel Cabrera Lozoya, Jiahe Liu, Simon D’Alfonso, Mike Conway

Abstract
Adolescents exposed to advertisements promoting addictive substances exhibit a higher likelihood of subsequent substance use. The predominant source for youth exposure to such advertisements is through online content accessed via smartphones. Detecting these advertisements is crucial for establishing and maintaining a safer online environment for young people. In our study, we utilized Multimodal Large Language Models (MLLMs) to identify addictive substance advertisements in digital media. The performance of MLLMs depends on the quality of the prompt used to instruct the model. To optimize our prompts, an adaptive prompt engineering approach was implemented, leveraging a genetic algorithm to refine and enhance the prompts. To evaluate the model’s performance, we augmented the RICO dataset, consisting of Android user interface screenshots, by superimposing alcohol ads onto them. Our results indicate that the MLLM can detect advertisements promoting alcohol with a 0.94 accuracy and a 0.94 F1 score.
Anthology ID:
2024.bionlp-1.42
Volume:
Proceedings of the 23rd Workshop on Biomedical Natural Language Processing
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Dina Demner-Fushman, Sophia Ananiadou, Makoto Miwa, Kirk Roberts, Junichi Tsujii
Venues:
BioNLP | WS
SIG:
SIGBIOMED
Publisher:
Association for Computational Linguistics
Note:
Pages:
514–525
Language:
URL:
https://aclanthology.org/2024.bionlp-1.42
DOI:
10.18653/v1/2024.bionlp-1.42
Bibkey:
Cite (ACL):
Daniel Cabrera Lozoya, Jiahe Liu, Simon D’Alfonso, and Mike Conway. 2024. Optimizing Multimodal Large Language Models for Detection of Alcohol Advertisements via Adaptive Prompting. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, pages 514–525, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Optimizing Multimodal Large Language Models for Detection of Alcohol Advertisements via Adaptive Prompting (Cabrera Lozoya et al., BioNLP-WS 2024)
Copy Citation:
PDF:
https://preview.aclanthology.org/nschneid-patch-5/2024.bionlp-1.42.pdf
PDF Search