\appendix


\section{Hyperparameter Search Space}
\label{sec:hparams-space}
We performed a hyperparameter search for the attentive-LSTM and pointer-generator models using the \textbf{sentence copying} data augmentation strategy. We used random search with the goal of maximizing validation accuracy. We report the search space we considered in \autoref{fig:opt}.
\begin{table}[h]
\small
    \centering
    
    \begin{tabular}{l|l|c}
        \toprule        \textbf{Hyperparameter} & \textbf{Distribution} & \textbf{Values} \\
        \midrule
        Batch Size & categorical & 16, 32, 64 \\
        Embedding Size& q\_uniform& 128 to 1024; q=64\\
        Hidden Size& q\_uniform& 128 to 2048; q=64\\
        Attention Heads& values& 1, 2\\
        Encoder Layers& values & 1, 2, 3\\
        Decoder Layers& values & 1, 2, 3  \\
        \bottomrule
    \end{tabular}
    \caption{Hyperparameter Search Space}
    \label{fig:opt}
\end{table}

\input{tables/appLargeArch}

\section{Larger Architectures}
\label{sec:larger-arch}
For thoroughness, we also compare architectures using the architecture size described in \citet{yang-etal-2022-generalizing}. We report these results in \autoref{tab:larger-arch-results}.

Except for the transformer models, these larger models well underperform their smaller counterparts, in many cases overfitting the training data and completely failing to generalize. The transformer models perform more robustly, and seem to benefit from deeper and larger architectures.
%Meanwhile, the transformer models are not only more robust to deeper and larger architectures, but seem to benefit.

\section{LLM Prompting}
\label{sec:chatgpt-app}
% We describe our prompt for few-shot learning with the ChatGPT API. \cyan{TODO}

We attempted two different prompting strategies for our Chat-GPT implementation. 

In the first strategy, we used a full-context approach, using the entire language’s training split as the context. We tried these two different chunk size settings, calling the API with chunks of 20 or 80 test sentences at a time. 

In the second strategy, we tried a smart-retrieval approach with a chunk size of one to only provide relevant examples as context. Relevant examples were those with the same changes as the test sentences within the language’s training split.

In \autoref{tab:gpt-prompt}, an example of the prompt we provided using the smart-retrieval approach for a sentence in Bribri is shown. Note that this prompt provides just one training instance; in our experiments we provided multiple instances per prompt.

% The Table %


\begin{table*}[h!b]
\begin{tabular}{ll}
\multicolumn{2}{l}{\begin{tabular}[c]{@{}l@{}}\textbf{**Prompt**}\\ *\texttt{Below is an example of a sentence in Bribri,  the linguistic change, and the target} \\ \texttt{sentence after applying the change.}\end{tabular}}                                                                                \\
                                                                                                                             &                                                                                                                                                       \\
%\multicolumn{2}{l}{\textit{One example to give to model:}}                                                                                                                                                                                                                                    \\ 
\hline
\multicolumn{1}{|l|}{ID:}                                                                                                    & \multicolumn{1}{l|}{Bribri0303}                                                                                                                       \\ \hline
\multicolumn{1}{|l|}{Source:}                                                                                                & \multicolumn{1}{l|}{Ye' shka'}                                                                                                                        \\ \hline
\multicolumn{1}{|l|}{Change:}                                                                                                & \multicolumn{1}{l|}{TYPE:NEG, TENSE:PRF\_PROG}                                                                                                        \\ \hline
\multicolumn{1}{|l|}{Target:}                                                                                                & \multicolumn{1}{l|}{Ye' k{\`{\"e}} k{\b{u}}'bak shk{\'{\"o}}k}                                                                                                           \\ \hline
                                                                                                                             &                                                                                                                                                       \\
\multicolumn{2}{l}{\begin{tabular}[c]{@{}l@{}} \texttt{Below is a similar example, where the source sentence and linguistic change are} \\ \texttt{given, and the output sentence is not known. For this example, please output only} \\ \texttt{the id and target sentence values, as in:}\end{tabular}} \\
                                                                                                                             &                                                                                                                                                       \\ \hline
\multicolumn{1}{|l|}{ID:}                                                                                                    & \multicolumn{1}{l|}{Some ID}                                                                                                                          \\ \hline
\multicolumn{1}{|l|}{Target:}                                                                                                & \multicolumn{1}{l|}{Sentence after applying the change}                                                                                               \\ \hline
                                                                                                                             &                                                                                                                                                       \\
\multicolumn{2}{l}{\begin{tabular}[c]{@{}l@{}} \texttt{Do not output any additional text, and do not output the Source or Change fields.} \\ \texttt{This is very important, take your time and do not mess up  or I will lose my job.}\end{tabular}}                                             \\
                                                                                                                             &                                                                                                                                                       \\
\multicolumn{2}{l}{\textit{Example Input:}}                                                                                                                                                                                                                                                   \\ \hline
\multicolumn{1}{|l|}{ID:}                                                                                                    & \multicolumn{1}{l|}{Bribri0367}                                                                                                                       \\ \hline
\multicolumn{1}{|l|}{Source:}                                                                                                & \multicolumn{1}{l|}{P{\^u}s kap{\"e}'w\b{a}}                                                                                                                     \\ \hline
\multicolumn{1}{|l|}{Change:}                                                                                                & \multicolumn{1}{l|}{TYPE:NEG, TENSE:PRF\_PROG}                                                                                                        \\ \hline
\multicolumn{1}{|l|}{Target:}                                                                                                & \multicolumn{1}{l|}{*}                                                                                                                                \\ \hline
                                                                                                                             &                                                                                                                                                       \\
\multicolumn{2}{l}{\textit{Model Response:}}                                                                                                                                                                                                                                                  \\ \hline
\multicolumn{1}{|l|}{ID:}                                                                                                    & \multicolumn{1}{l|}{Bribri0367}                                                                                                                       \\ \hline
\multicolumn{1}{|l|}{Target:}                                                                                                & \multicolumn{1}{l|}{P{\^u}s k{\`{\"e}} k{\b{u}}'bakap{\"e}'w{\b{a}}}                                                                                                           \\ \hline
\end{tabular}
\caption{Example prompt given while LLM prompting.}
    \label{tab:gpt-prompt}
\end{table*}




% fixes to the diacritics %

% Bribri 303 source: no change
% Bribri 303 target: {Ye' k{\`{\"e}} k{\b{u}}'bak shk{\'{\"o}}k
% Bribri0367 source: P{\^u}s kap{\"e}'w\b{a}
%Bribi0367 response: P{\^u}s k{\`{\"e}} k{\b{u}}'bakap{\"e}'w{\b{a}}

           

% **Prompt**

% *Below are examples of a sentence in Bribri, the linguistic change, and the target sentence after applying the change.
    
% Id: Bribri0303
% Source: Ye' shka'
% Change: TYPE:NEG, TENSE:PRF_PROG
% Target: Ye' kë̀ ku̠'bak shkö́k*


% *Below is a similar example, where the source sentence and linguistic change are given, and the output sentence is not known. For this example, please output only the id and target sentence values, as in

% ID: some id
% Target: sentence after applying the change

% Do not output any additional text, and do not output the Source or Change fields. This is very important, take your time and do not mess up or I will lose my job.*

% *Id: Bribri0367
% Source: Pûs kapë'wa̠
% Change: TYPE:NEG, TENSE:PRF_PROG
% Target:*

% **RESPONSE:**
% ID: Bribri0367
% Target: Pûs kë̀ ku̠'bakapë'wa̠



% different section







\section{Augmentation Examples}
\label{sec:aug-appendix}
We provide examples of the rows created by each augmentation strategy in \autoref{tab:aug_examples}

\input{tables/appAugExamples}

%\begin{table*}[tb]
%\small
%\renewcommand{\arraystretch}{2}
%    \centering
%    \begin{tabular}{| l | c c c | c c c |}
%    \hline
%        & \multicolumn{3}{c|}{Original example} & \multicolumn{3}{c|}{Augmented example} \\
%        Strategy & Source & Change & Target & Source & Change & Target  \\
%        \hline 
%        \textsc{Copy} & Ko po ojupi & TENSE:FUT\_SIM &  Ko po ojupíta & Ko po ojupi & NOCHANGE & Ko po ojupi \\
%        $\textsc{Copy}_{ext}$ & - & - & - & Nde ruvichápe & NOCHANGE & Nde ruvichápe \\
%        \textsc{Trans} & \makecell[tc]{Che rasy} & \makecell[tc]{PERSON:2\_SI,\\PERSON:1\_PL\_EXC}  & \makecell[tc]{Nde nderasy\\ Ore rorasy} & Nde nderasy & PERSON:1\_PL\_EXC & Ore rorasy \\
 %       \textsc{Per} & Ha’e oguapy & PERSON:3\_PL & Hikuái oguapy  & Ha’e ocguapy & PERSON:3\_PL & Hikuái ocguapy \\
  %      \textsc{Con} & \makecell[tc]{Nde nderejapói\\Apurahéi kuri} & PERSON:3\_PL & \makecell[tc]{Ha’ekuéra ndojapói\\Ha’ekuéra opurahéikuri} & Nde nderejapói apurahéi kuri & PERSON:3\_PL & Ha’ekuéra ndojapói ha’ekuéra opurahéikuri \\
   %     Embedding & \\
%    \hline
%    \end{tabular}
%    \caption{Example rows from each of our augmentation strategies. All examples are given in Guarani.}
%    \label{tab:aug_examples}
%\end{table*}