\section{Conclusion}
We propose a simple yet effective method using question prompting for detecting harmful prompts in a zero-shot manner. Our approach leverages pre-trained MLLM without fine-tuning and classifies harmful prompts through guard questions, question prompting, and a filtering algorithm. Experimental results show that our model outperforms fine-tuned baselines. The method also enables white-box analysis, providing transparency in classification. By refining guard questions, our approach can flexibly adapt to new harmful prompts with minimal computational overhead, making it a practical solution for real-world LLM safety applications. We believe that our approach presents a practical and effective solution for real-world LLM safety applications.

\noindent\textbf{Limitation.} Although our method does not require fine-tuning, it relies on a pre-trained MLLM for inference. Additionally, extracting logits from the MLLM may take some extra time, and the use of dataset-specific thresholds can pose challenges to generalization. In the future, we aim to enhance the model's generalization capabilities and optimize the filtering algorithm to improve efficiency.

\subsubsection*{Acknowledgments}
This research was supported by Brian Impact Foundation, a non-profit organization dedicated to the advancement of science and technology for all.