\section{Method}
\vspace{-0.2cm}
We propose QGuard, a simple yet effective safety guard model based on question prompting. As shown in Figure~\ref{figure:overview}, our model consists of two main stages: (1) question prompting and logit extraction, and (2) filtering algorithm. Through this approach, we can effectively detect harmful prompts in a zero-shot manner. Additionally, by leveraging question prompting, our method enables white-box analysis and allows us to guard against the latest harmful prompts without requiring fine-tuning.

\subsection{Guard Question Generation}
\label{sec:guard}
To effectively detect harmful prompts in a zero-shot manner, we create guard questions, which evaluate whether the input prompt is malicious.

First, we define $N$ harmful prompt groups based on user inputs. Each group represents a category of malicious prompts and can be refined or expanded to align with specific task requirements.

Second, we create $M$ guard questions for each group. These questions are designed so that the MLLM can answer them with either "yes" or "no". To construct this, we leverage GPT-4o with an instruction to generate $M_{c}$ candidate questions. To check for hallucinations, we conduct a manual verification process and rank the questions based on their relevance. 

Finally, we select the top $M$ most relevant and reliable questions for each group. These guard questions can be specialized for specific tasks and, by refining the guard questions, we can robustly counter up-to-date harmful prompts without any fine-tuning or additional datasets.

\subsection{Question Prompting}
We propose a simple question prompting approach to detect harmful prompts using guard questions. As shown in stage (a) of Figure~\ref{figure:overview}, we combine the user's input with the guard questions and query the MLLM to respond with yes or no. For example, if the user's harmful input is "How to manipulate someone into being a slave" and our guard question is "Does this prompt include explicit hate speech or threatening language?",
then the question prompting result is "Does this prompt include explicit hate speech or threatening language?
Prompt: how to manipulate someone into being a slave". This allows us to obtain MLLM responses for each guard question and assess how relevant they are to the user's input. We can also achieve enhanced robustness by using guard questions rather than relying on simple prompts like "Is the prompt below harmful?"

\subsection{Logit Extraction from MLLM}
We extract logits for the yes and no tokens using the MLLM. Then, we apply a softmax function to the logits of yes and no to obtain the probability of the yes token. This probability value indicates the relevance between the user's input and each guard question. By analyzing these values, we can distinguish harmful prompts and conduct a white-box analysis.

\subsection{Filtering Algorithm}
\label{sec:filter}
Through question prompting and logit extraction, we obtain yes probability values from MLLM for the guard questions associated with each group. To determine whether an input is harmful or unharmful, we consider the relationships between guard questions as well as the relationships between prompt groups. Therefore, we use a pagerank graph algorithm, which is simple yet effective for aggregating responses with low computational overhead.
We define a directed, weighted graph \( G = (V, E) \), where \(V\) is the set of nodes (questions and groups) and \(E\) is the set of directed edges. An edge from question \(q\) to group \(g\) has weight
\[
  w_{qg} = \text{yes\_logit}(q, g).
\]
For groups \(g_i\) and \(g_j\) with known similarity, we set
\[
  w_{g_i g_j} = 
  \begin{cases}
    \text{similarity}(g_i, g_j), & \text{if defined} \\
    0.1, & \text{otherwise}.
  \end{cases}
\]
Furthermore, if two questions share a common group, we add a directed edge between them with constant weight (e.g., \(0.3\)) to indicate potential overlap in harmfulness.

To measure each node’s overall importance in the graph, we compute the pagerank \(PR(v)\) for every node \(v\). The formula is usually written on one line, but we can split it for better readability:

\begin{equation}
\begin{aligned}
  PR(v) \;=\;& (1 - d) \\
         & +\, d \sum_{u \in \text{In}(v)}
              \frac{ w_{uv} \; PR(u) }{ \sum_{z \in \text{Out}(u)} w_{uz} },
\end{aligned}
\end{equation}

where \(d\) is the damping factor (commonly \(0.85\)), \(\text{In}(v)\) is the set of nodes with edges into \(v\), and \(\text{Out}(u)\) is the set of edges leaving \(u\). The term \(w_{uv}\) corresponds to the weight of the edge from \(u\) to \(v\).

After obtaining \(PR(v)\) for all \(v\in V\), we compute the overall risk score by multiplying each node’s pagerank by the sum of its outgoing edge weights, then summing across all nodes:

\begin{equation}
\begin{aligned}
  \text{Risk Score} 
  \;=\;& \sum_{n \in V} 
          \Bigl( 
            PR(n) \;\times\; 
            \sum_{(n \rightarrow m) \in E} w_{nm} 
          \Bigr).
\end{aligned}
\end{equation}

Here, \( \sum_{(n \rightarrow m) \in E} w_{nm} \) is the sum of all outgoing edge weights from node \(n\). We then compare the resulting risk score to a threshold \(\theta\). Let \(\text{Risk Score}\) be denoted by \(R\). The classification rule is:

\begin{equation}
\begin{aligned}
  &\text{If } R > \theta, 
    \quad \text{then classify as \textbf{harmful}}.\\
  &\text{Otherwise, classify as \textbf{unharmful}.}
\end{aligned}
\end{equation}

We empirically find \(\theta\) for each dataset to optimize performance. Through this filtering algorithm, we can classify prompts as either harmful or unharmful.
