\section{Introduction}
\label{sec:intro}
The recent advancements in LLMs have had a significant impact across various fields, from general domains to those requiring specialized knowledge~\cite{chen2024internvl}. Especially, Multi-modal Large Language Models (MLLMs) are capable of answering both specific and general user queries based on detailed reasoning and understanding of visual inputs~\cite{chen2024lion,wu2024controlmllm,chen2023large,Lee_2025_CVPR}.

These advancements have also significantly increased the potential for malicious users to exploit unethical and harmful prompts for malicious attacks~\cite{han2024wildguard, wei2023jailbroken, zou2023universal}. To prevent malicious attacks, many companies are making numerous efforts and investing significant resources. These studies~\cite{inan2312llama, han2024wildguard,xie2024gradsafe} have made significant progress and provide models as open-source. 

However, such approaches~\cite{inan2312llama,han2024wildguard,xie2024gradsafe,gu2025mllmguard,HarmfulPromptClassification} typically require fine-tuning for each dataset and demand additional resources to prevent recent harmful prompt attacks. Moreover, their models may struggle to determine why a user's input has been classified as harmful, making this one of the important and challenging tasks in the real world~\cite{efficientdetectiontoxicprompts}.

In the real-world, small companies may rely on fine-tuned open-source models~\cite{gu2025mllmguard, han2024wildguard,UniGuard}, yet guard mechanisms may still be imperfect, and multi-modal harmful inputs remain a significant vulnerability. Additionally, they require additional computational resources and datasets to block the latest harmful prompts. It remains a challenging task.

Therefore, we propose \textbf{QGuard,}, a simple yet effective method, to guard harmful prompts without any fine-tuning, making it suitable for real-world applications in a zero-shot manner. 

First, we categorize harmful prompts into harmful groups and create guard questions. These guard question prompts include questions such as "Is the text likely to cause harm or distress to others?" or "Does this prompt include explicit hate speech or threatening language?", allowing the model to assess whether a user's input is a harmful prompt. 

Secondly, we combine the guard question with the user's input, and then query the MLLM, which is capable of understanding and representing sentences, for a yes or no response.

We define this process as \textit{question prompting} as shown in stage (1) of Figure~\ref{figure:overview}. Since we utilize a pre-trained MLLM, our approach requires no additional training while enabling the detection of user inputs across multiple modalities, including images, videos, and text. To detect harmful inputs, we apply softmax over the logits of the "yes" and "no" tokens from the MLLM, and use the probability value of the "yes" token.

Finally, as shown in stage 2 of Figure~\ref{figure:overview}, we use the PageRank algorithm as a filtering method and apply a threshold to the "yes" probabilities of guard questions to distinguish between harmful and unharmful inputs.

With this approach, we can defend against harmful prompts in a zero-shot manner. As harmful prompts evolve, we can adapt to new threats by enhancing only the guard questions, requiring minimal computational resources. This allows for a flexible and efficient response to the latest harmful prompts. Additionally, by analyzing the logits of each question, our method enables a white-box analysis of the decision-making process. 

In experiments, we achieve higher performance than the zero-shot LLM detector and outperform fine-tuned baselines on both text-based harmful prompt datasets and multi-modal harmful prompt datasets. These results demonstrate that our method is simple yet effective. Moreover, by keeping guard questions private and optimizing them for specific services, our approach has the potential to create an even more robust guard mechanism for real-world applications.

In summary, our contributions are as follows: 
\begin{itemize}
  \setlength{\itemsep}{3pt}
  \setlength{\parskip}{3pt}
 \item We propose a simple yet effective method for detecting harmful prompts using question prompting in a zero-shot manner.
 \item By refining the guard questions, our method can provide a more robust defense against the latest harmful prompts with minimal computational resources, without requiring any fine-tuning or additional datasets. 
  \item Since we utilize the logits of the MLLM, we can perform white-box analysis to understand why an input is harmful, and we provide such analysis.
 \item Experimental results show that our model performs competitively on both text-only and multi-modal harmful datasets.

\end{itemize}
