%
% File acl2019.tex
%
%% Based on the style files for ACL 2018, NAACL 2018/19, which were
%% Based on the style files for ACL-2015, with some improvements
%%  taken from the NAACL-2016 style
%% Based on the style files for ACL-2014, which were, in turn,
%% based on ACL-2013, ACL-2012, ACL-2011, ACL-2010, ACL-IJCNLP-2009,
%% EACL-2009, IJCNLP-2008...
%% Based on the style files for EACL 2006 by 
%%e.agirre@ehu.es or Sergi.Balari@uab.es
%% and that of ACL 08 by Joakim Nivre and Noah Smith

\documentclass[11pt,a4paper]{article}
\usepackage[hyperref]{acl2019}
\usepackage{times}
\usepackage{latexsym}
\usepackage{graphicx}
\usepackage{url}
\usepackage{multirow}
\aclfinalcopy % Uncomment this line for the final submission
\def\aclpaperid{027} %  Enter the acl Paper ID here

%\setlength\titlebox{15cm}
% You can expand the titlebox if you need extra space
% to show all the authors. Please do not make the titlebox
% smaller than 5cm (the original size); we will check this
% in the camera-ready version and ask you to change it back.

\newcommand\BibTeX{B\textsc{ib}\TeX}

\title{Non-native Accent Partitioning for \\Speakers of Indian Regional Languages}

\author{G Radha Krishna \\
  Dept. of E.C.E,VNRVJIET \\
   Hyderabad \\
   India \\
  \texttt{\small gunturrkr@gmail.com} \\\And
  R Krishnan \\
   Amritha University \\
   Coimbattore \\
   India \\
  \texttt{\small drrkdrrk@gmail.com} \\\And
 V K Mittal \\
   K L University \\
   Vijayawada \\
    India \\
  \texttt{{\small drvinaykrmittal@gmail.com}} }

\date{}

\begin{document}
\maketitle
\begin{abstract}
  Acoustic information from the speech signal permits the identification of speaker related information, chief among them are the \textit{geographical origin} and \textit{ethnicity}. This paper validates the use of acoustic features for the classification of the native language of speakers of South Indian languages  based on their English speech. The native speakers of \textit{Kannada}, \textit{Malayalam}, \textit{Tamil}, and \textit{Telugu} are chosen for this study. A database of native language speech and non-native English speech from a non- overlapping set of speakers of the above language groups were collected. Segmental acoustic features including the Mel-frequency cepstral coefficients (MFCCs) are used. Accent partitioning on non-native English speech data has been investigated. Classification accuracies up to $89\%$ are observed in the four class problem. This rate is more than  $90\%$ when number of classes is less. Statistics of $F_0$ contour related to $L_2$ intonation showed that native speakers of \textit{Kannada} language are clearly distinct compared to those of \textit{Tamil} and \textit{Telugu}. It is found that identification of \textit{Malayalam} and \textit{Kannada}  speakers from their English speech is relatively easier than \textit{Telugu} or \textit{Tamil} speakers.
  
\end{abstract}

\section{Introduction}

  Identification of speakers, classification of their dialectal zones is important in a multilingual country like India \cite{Bhattacharjee2012}. Speaker uniqueness is manifested in both anatomical and learned traits. When the context is constrained, speaker characteristics can be used reliably to identify individuals \cite{Arslan1996}. The \textit{accent} is one of the glaring indications of linguistic and social background of a speaker. Studying the characteristics of dialect on a phonetic or phonemic level  belongs to accent recognition \cite{Mittal2014}. Earlier studies have concluded that native language ($L_1$) affects the speaker's traits of their second language ($L_2$) \cite{Ghorbani2018, Graham2018}. Analysis and classification of utterances that belong to specific groups of learners is the main objective of \textit{Native Language Identification} (NLI) \cite{Nisioi2015}.   However, there is very little research on the question of accuracy with which accent features can be used to identify a speaker's \textit{regional} or \textit{ethnic origin} \cite{Harper2008}. A solution to the problem of regional accent classification across English speaking South Indians is attempted in the present research, using a specifically developed corpus. 
  
   Discriminative classifiers based on characterizing acoustic differences across foreign accents can be employed to direct an accent dependent recognition system \cite{Omar2010, Ikeno2006}. Systems with an automatic evaluation of non-native speech, which includes characteristics of the mother tongue will have better performance over similar algorithms that depend upon target languages \cite{Qian2017}. This is particularly true when the text uttered is unknown. Native listeners are mostly aware of the speaker's regional accent and also the social or geographical subgroup within the region \cite{Hanani2013}. Automatic speaker characterization is vital in real-world applications  and the advantages are widely open \cite{Zampieri2017, Krishna2014}.   
   
   Pattern recognition approach of collecting data, extracting suitable features, and training classification module using machine learning is a powerful tool in applications like Computer-Assisted-Pronunciation-Training (CAPT) programs. Acoustic descriptors are critical in tasks such as sound Classification \cite{Day2007}. State-of-the-art Accent Identification (AID) systems widely rely on spectral acoustic distribution for modeling the pronunciation. In applications like accent recognition, features distinguishing different phonemes of a language will be useful \cite{Neumeyer2000}. Language-specific differences in phonological development might be related to differences in phoneme and phoneme sequence frequency across languages \cite{Ikeno2006}. Such variations are also represented by the intonation patterns of individuals \cite{Mary2008,Li2017}. Apart from cepstral features that capture underlying acoustic characteristics, information from higher-level prosodic traits \cite{Doddington2001, MALMASI2017} were examined in the present study. 

  \begin{table}[t]
  \caption{Summary of data used for training and testing: (a) attributes    (b) values for \textit{training set}  and (c) values for \textit{testing set} }
  \label{tab:example}
  \centering
  \begin{tabular}{p{2.4cm}ccc}
\hline
(a) \textit{Attributes} & (b) \textit{Training set}   & (c) \textit{Test set}           
\\
\hline
Total number of speakers             & 60 & 75
  \\
  \hline
Speakers per language group (KAN, MAL*, TAM, TEL) &20 &25 
\\
\hline
Speech Duration per speaker & 300 sec & 60 sec \\
\hline
\end{tabular}
Note: *MAL-Malayalam data set is used only in tests related to cepstral features.
  \end{table}
  
  
English is the most widely spoken second language in India and elsewhere in the world \cite{Saha2015, Guntur2018}. Indian English has several varieties with their specific accents and phonological features and often a distinct lexicon. Research on spoken English of Indian speakers is urgently needed from a multidisciplinary perspective \cite{Cheng2013, Krishna2019}. Present work is aimed at comparing the acoustic properties that are likely to differ between English accents different groups of South Indian language of speakers. The non-native prosodic traits are a hindrance to proficiency in a second language ($L_2$), and also to the mutual understanding. Present work also examines the local prosodic changes in the non-native English speech, without incorporating any phonology of the specific languages. The ability to compensate against prosodic deviation during English production can be improved by identifying the articulatory gestures that emphasize the non-native speaker accent.  

  
  \begin{table}[h]
  \caption{ Template of file naming for data recording}
   \label{tab:example}
  \centering
  \begin{tabular}{cccc}
\hline
Native \\language  & Name    & Age / Sex  & File Name\\
\hline 
\end{tabular}
\end{table}  

The paper is organized as follows: Section 2 presents the details of the database, including the recording methodology.  Section 3 describes acoustic and prosodic features used in foreign accent recognition. Section 4 describes the classification procedures employed in the NLI experiments. Section 5 gives the details of the experiments and results. Analysis of results of regional accent classification is given in section 6. Section 7 describes the key outcome and contributions. Conclusions drawn are given in Section 8. 

 
   
 
  
 
%\end{table}

%\begin{figure}
%\includegraphics[width=8cm,height=9cm]{fig1BD-new1.pdf}
%\caption{Hierarchy of Language Identification tasks}
%\end{figure}  

\section{Data Sets of 4 Indian Regional Languages}

The main focus of current research work is on differentiating the regional non-native English accents of speakers, and also describing foreign accent in terms of a common set of fundamental speech attributes. A database has been specifically developed \cite{G.RadhaKrishna2018} with native and non-native speech samples containing utterance by the speakers belonging to language groups Kannada (KAN), Malayalam (MAL), Tamil (TAM), and Telugu (TEL). Table 2 shows the template of file naming process.  

\subsection{Selection of Regional languages}
         Among more than six thousand languages in the world, less than $10\%$ of the languages are spoken by more than $90 \%$ of the people. Speakers and learners of the English language constitute a large proportion in countries like  India, South Africa, and much of the developing world. In South Indian cities, many people speak at least two second languages. It would be beneficial if speech based systems can store models of all known languages and carry out the task of NLI automatically. India has distinct linguistic communities, each of which shares a common language and culture. English, Hindi and dominant local languages are spoken non-natively by a large number of Indians.  
         


\begin{table}[t]
  \caption{Summary of speaker traits and related speech features \cite{Day2007}.}
  \label{tab:example}
  \centering
  \begin{tabular}{p{2.1cm}p{2.1cm}p{2.6cm}}
\hline
Speech characteristic & Speaker trait   & Speech feature  
\\
\hline
Lexical, Syntactic & Socio economic & Vocabulary, Word
\\
(Idiolect, Semantics, Pronunciations, dictions, Idiosyncrasies) & Educational status (Language use and sentence construction) & arrangement \& grammatical cues. 
\\
\hline
Prosodic (Rhythm, Intonation, articulation rate etc.) & Personality type, Parental influences & Durational features. Pitch dynamics, Energy (likely to be Text / time dependent).
\\
\hline
Low level acoustic features & Anatomical structure of speaker's vocal apparatus & Short-time spectrum, Predictor coefficients, Intensity, Pitch.
\\
\hline 

\end{tabular}
\end{table}   
   
         
\subsection{Speech Corpus Recording Methodology}
The details of speech corpus developed for each of the languages is shown in Table 1. Native speech utterances of 20 speakers from each of the native language groups KAN, TAM, and TEL, each with a duration of 300 seconds formed the training set. English test samples for a duration of 60 seconds were collected from 25 speakers belonging to each of the four groups KAN, MAL, TAM, and TEL. As the sufficient num ber of native speakers of MAL are not readily available, it is included in the testing set only.  The test utterances were recorded under identical conditions as training speech samples and there is no overlap between training and testing sets with respect to speakers and sentences. Each of the test samples is recorded for a duration of 60 seconds. The non-native English speech samples are collected from a set of speakers with nearly uniform geographical distribution within a region with an educational background of at least graduation, but who do not use English routinely.    
 
  
   Recordings of speakers were made in quiet office room conditions using Logitech h110 microphone and waveforms are sampled at a rate of 16 kHz. The recordings were made in a laboratory environment with written text, with negligible reverberation. The participants were asked to read aloud passages of a text from general topics. For applications like screening of non-native speech, read data can be used for both training and testing \cite{Schuller2013}. It is ensured that Gender weightages are equally distributed in training as well as testing data sets. The speakers in the training set are considered representative of the regional languages KAN, TAM and TEL.  However, for testing set speakers of Malayalam were also included. These speakers are so chosen from language heartlands. The speakers in the test set are considered potential users of future systems augmented with automatic Accent Identification (AID) capability.         
  


\begin{table}[t]
  \caption{ Major Text Independent features used in prosodic analysis.}
  \label{tab:example}
  \centering
  \begin{tabular}{p{1.35cm}p{5.5cm}}
\hline
Prosodic features   & Factors that influences speech \\
\hline
Dynamics of $F_0$ contour  & Identity of sound unit, its position from phrase, word; Speaking style; Intonation rules; Type of sentence (Interrogative, Declarative)\\
\hline
Intonation, Rhythm, Stress   & Attitudinal, Accentual, Discourse, Grammatical\\
\hline  
\end{tabular}
\end{table}

   

\section{Features for Non-native Accent Partitioning}

Understanding similar variations in foreign accents is a crucial factor for the development of an NLI system. The dominant articulatory traits of different languages are different \cite{Koreman2018}. In applications like accent recognition, features distinguishing different phonemes of a language will be useful \cite{Li2013}. The acoustic signature or the \textbf{voice individuality} of the speech signal are available as differences in transformations occurring at semantic, linguistic articulatory, and acoustic levels. Out of all the factors affecting speech, accent is a week factor in the sense that speech variation is not as evident as that due to speaker/gender. 

Language-specific differences in phonological development might be related to differences in phoneme and phoneme sequence frequency across languages \citep{Graham2018}. Speakers of the second language (SL) are expected to import certain patterns from their native language (NL) which are audible in SL. The influence of the surrounding speech prosody on new-born cry melody has been shown \citep{Monnin2010}. The non-native speech detection is thus very challenging . 

Characterization of a foreign accent is mostly based on either auditory analysis or manual transcriptions of deviations. The auditory spectrum is consistent with several phenomena observed in speech perception and is useful in automatic speaker independent speech recognition.  Features used for nonnativeness detection include cepstral vectors, phone strings and a variety of prosodic features, but when used alone, systems based on acoustic features perform better \cite{Shriberg2005}. We can consider acoustic features, which are proxy of phonetic reproduction as acoustic-phonetic features \cite{Li2013}.   




   
   \begin{figure}
\begin{center}
\includegraphics[width=7.6cm,height=4cm]{front.pdf}
\caption{Front end signal processing for feature extraction}
\end{center}
\end{figure} 

\subsection{Acoustic Features}  

Earlier investigations on text-independent non-native speech tied to underlying native language structure are based on (i) Global acoustic distribution of phonemes (which requires no language knowledge) (ii) Different intonations corresponding to uniqueness in the manner in which articulators are manipulated. The shape of the vocal tract is manifested in the envelope of the short-time power spectrum \cite{Reynolds1995}.  The attributes that contain speaker identifiability for machine as well as for humans are of interest \cite{Zheng2007, Franco2000}. 

In this study, acoustic features used for phonetic modeling of the accent differences consists of the cepstral features: Perceptive Linear Prediction Coefficients (PLPs), Linear Predictive Cepstral Coefficients (LPCCs), and MFCCs \cite{Hermansky1990, Luengo2008, Mittal2013}. The steps followed are shown in Figure 1. Given all the alternative spectral features based on LPC - cepstrum and FFT – cepstrum for speaker recognition, MFCCs, give a highly compact representation of the spectral envelope of a sound \cite{Lopez2014}. The LPCCs are known to capture extra information from a speech that discriminates different languages. The PLPs which take advantage of psychoacoustic principles are robust against noise. A hierarchy of speech characteristics, related  speaker traits, and possible speech features are listed in Table 3. 


    \begin{figure}
 \begin{center}
 \includegraphics[width=7.6cm,height=4cm]{slengm-EH.pdf}
 \caption{Waveform and Pitch contour of non-native English speech by female Kannada speaker}
 \end{center}
\end{figure} 
\subsection{Prosodic Features}

 The prosodic structure is a critical aspect of language contact and gives important information related to the speaking habit of a person \cite{Kinnunen2010, Farrus2010}. The goal is to capture prosodic idiosyncrasies of speakers belonging to different native languages. Prosodic cues Stress, Rhythm, and Intonation are each complex entities expressed  using (i) Pitch (ii) Energy (iii) Duration. Major text-independent features used in prosodic analysis are given in Table 4.
 
  In this study Prosodic statistics were obtained by performing different measurements of \textit{pitch}, which are derived supra segmentally. The power of accent in voice identification is investigated as explained below. A \textit{Generative model} of pronunciation describes what is acceptable, and \textit{Discriminative model} both acceptable and unacceptable pronunciation, and the pronunciation score is the direct output of the classification module. 

 \begin{figure}
 \begin{center}
 \includegraphics[width=7.6cm,height=4cm]{ENGLISH.jpg}
 \caption{Distribution of MFCC Coefficients as a Scatter plot of $C_0$ versus $C_1$ for native English speakers }
 \end{center}
\end{figure} 

 \begin{figure}
 \begin{center}
 \includegraphics[width=7.6cm,height=4cm]{KANNADA.jpg}
 \caption{Distribution of MFCC Coefficients of $C_0$ versus $C_1$ for  English speech by KANNADA speakers}
 \end{center}
\end{figure} 
 
 

 Non-native prosodic traits limit proficiency in a second language (L2). Prosodic phenomena located on word level and above, help listeners to structure the speech signal and to process the linguistic content successfully. Table 4 shows some of the features useful for detecting non-native speech without annotation of prosodic events. The experiment by Rosenberg to foil a Speaker Verification system says that even an identical twin was unable to imitate the enrolled sibling well enough to get accepted by the system, tells the need to look at learned speaking behaviour. 

\begin{figure}
 \begin{center}
 \includegraphics[width=7.6cm,height=4cm]{MALAYALAM.jpg}
 \caption{Distribution of MFCC Coefficients of $C_0$ versus $C_1$ for  English speech by MALAYALAM speakers}
 \end{center}
\end{figure} 
   
 


\section{Classification for Non-native Accent Partitioning}
   \textit{Speaker Classification} can be conveniently defined as a grouping of speakers speaking in a similar manner, on the basis of acoustic characteristics \cite{Chen2014}. Classification of foreign accents directly from the acoustic features is attempted by using a test data set described in Table 1. The role of accent in voice identification is investigated as explained below. There exists a significant overlap between NLI approaches and computational methods for dialect and language identification (LID), and Support Vector Machine (SVM) classifiers are a very good fit for NLI \cite{Zampieri2017}. 
   
   \begin{figure}
 \begin{center} 
 
 \includegraphics[width=7.6cm,height=4cm]{TAMIL.jpg}
 \caption{Distribution of MFCC Coefficients of $C_0$ versus $C_1$ for  English speech by TAMIL speakers}
 \end{center}
\end{figure}

\begin{figure}
 \begin{center}
 \includegraphics[width=7.6cm,height=4cm]{telugu.jpg}
 \caption{Distribution of MFCC Coefficients $C_0$ versus $C_1$ for English speech by TELUGU speakers}
 \end{center}
\end{figure}
   
 \subsection{Accent partitioning using SVM Classifier} SVM is one of the most popular supervised classifiers on a wide range of data sets, which looks for a \textit{maximum-margin hyper plane} for data separation \cite{Wu2010,Bahari2013, Campbell2006}. Accuracies of non-native accent classification were studied for the present problem by using the SVM classifier. The speech signal is first processed to extract attributes relevant to the \textit{foreign accent} \cite{Moustroufas2007}.  The most representative acoustic features, the  (LPCC), the  (PLP) \cite{Li2013} have been tested but were found to be less efficient. The input to the system is a 13 dimensional MFCC vector  consisting of 12 cepstral coefficients and one energy coefficient. Thus the front end for the proposed classification system consisted of only 13 dimensional MFCC vector including $C_0$. 
   
\begin{figure}
 \begin{center}
 \includegraphics[width=8cm,height=6cm]{GRK-SP-rajesh-AUG27.pdf}
 \caption{Distribution of MFCC Coefficients $C_0$ versus $C_1$ for non-native English speech by four South Indian language speakers against native English speech.}
 \end{center}
\end{figure} 

  

   \subsection{Intonation analysis} Native traits located at a word and sentence levels help listeners structure the speech signal. In many approaches that apply prosody to either \textit{Language Identification} (LID) or \textit{Speaker Recognition}, extracted features are based on statistics of pitch / energy contour segments or piecewise linear stylization of pitch / energy contours. Intonation is a key expressive factor which can covey the intent of a speaker, contains a lot more information than words and utterance \cite{Ward2017}. \textit{Intonation} is more used than \textit{energy} and \textit{duration} features in the context of prosody. Listeners can discern a speaker's regional accent from intonation alone \cite{Eady1986, Tepperman2008}.  
   
    
  
   
    Dynamics of $F_0$ contour corresponding to a sound is influenced by several factors such as the identity of the sound unit, its context, the speaking style of the speaker, intonation rules of the language, type of the sentence, etc. \cite{Arias2010}. The focus was mainly on the pitch since it is one of the most important characteristics of prosody and helps in predicting human intonation rating. These suprasegmental parameters can be used to model non-native English prosody \cite{Honig2012}. In the present study, the main aim is to ascertain the influence of  linguistic background on $F_0$ across regional varieties of English, future studies are planned to include the aperiodic components of excitation of expressive voices like Noh voice \cite{mittal2015}  
    
     
  
  
   
 
  \section{Experiments and Results}
  
  
  	To validate the hypothesis that the accent of the mother tongue is separable, experiments were performed to understand and to calibrate idiolectal differences in the non-native speech samples of the language groups KAN, MAL, TAM and TEL. The corpus is sampled at 16000 samples per second and the bit rate was 32 bits per sample. Silence removal has been implemented using a VAD algorithm \cite{Kinnunen2010}. The feature vectors are computed over 20 msec windowed frames every 10 msec. Fourier spectra were computed for sequential frames 160 points apart by using a 320 point Hamming window. Finally Cepstral Mean Normalization (CMN) is applied by subtracting the mean value of each feature over the entire utterance. MFCCs are generated by windowing the signal, application of DFT, taking the log of the magnitude and warping the frequencies on Mel scale and finally application of DCT.   
  	
  	

\begin{figure}
 \begin{center}
 \includegraphics[width=7.6cm,height=6cm]{CM-SI-EH.pdf}
\caption{ Confusion Matrix for SVM classification of South-Indian English including \textit{Malayalam}. Note: TPR is True Positive Rate, FNR is False Negative Rate.}
 \end{center}
\end{figure}       
    
    
    \begin{table}[t]
  \caption{\textit{Non-native Regional English Accent Classification accuracies using(a)k-nearest neighbor (KNN), (b)Linear Discriminant (LDA), and (c) SVM}}
  \label{tab:example}
  \centering
  %\scriptsize
  \begin{tabular}{cccc}
\hline
\textit{Classifier}	&(a) \textit{KNN}	& (b) \textit{LDA}  &(c) \textit{SVM}     
\\
\hline
Accuracy & $\textbf{86.6\%}$ & $\textbf{82.5\%}$ & $\textbf{89.2}$ \\
%EER & $\textbf{36.36\%}$ & $\textbf{33.33\%}$ &$\textbf{6.1\%}$ \\
\hline
\end{tabular}
  \end{table}            
       
   \subsection{Non-native accent Classification based upon Acoustic Features}
         Experiments were performed to establish the differences in the distribution of acoustic features in the non-native speech samples of four language groups KAN, MAL,TAM, and TEL. Graphical illustration of accent partitioning on test data is shown in Figures 3,4,5,6,7, and 8. It indicates that the high classification  accuracies are possible in the present task.  Classification of foreign accents directly from the acoustic features is attempted, by using data set described in Table 1.  Figure 9 shows the confusion matrix for best performing SVM classifier for the five class classification. Figure 11 shows the confusion matrix for the three class classification. 
         
         The confusion matrix indicates that the identification rates for Kannada and Tamil language speakers from their non-native English speech can be high compared to that of Telugu native speakers . The Receiver Operating Point Curve (ROC) shown in Figure 10 is a plot of true positive rate as a function of false positive rate, which is very close to the upper left hand corner, indicates that the classifiers can achieve good overall accuracies.                  
       
    Verification of accent partitioning of non-native speech using a series of classification techniques: k-nearest neighbour, and Linear Discriminant Analysis was also implemented. English speech samples of the native speakers of KAN, MAL, TAM, and TEL are tested against standard English speech corpus using TIMIT corpus. The resulting accuracies are $86.6 \%$ when a KNN clssifier is used, $82.5 \%$ when Discrimination classifier is used, and $89.2 \%$ using SVM classifier is used. These results are consolidated in Table 5. Figure 4, and 6 shows the corresponding confusion matrices, obtained during SVM classification. 
    
 \begin{figure}
 \begin{center}
 \includegraphics[width=8cm,height=6cm]{image-ROC-oct22.jpg}
\caption{: ROC curve for SVM classification of Non-native speech.}
 \end{center}
\end{figure}           

%            
 
\subsection{Foreign Accent discrimination based upon Prosodic Features }

       Experiments were conducted on native and non-native speech samples of bilingual and multilingual speakers. The pitch frequency was extracted using the "pitch contour" function of the \textit{Wave Surfer} software, and $F_0$ data was extracted. Typical waveform showing the non-native speech by a female Kannada speaker and the pitch contour were shown in Figure 2. The speakers in this study were asked to speak in their mother tongue or in English, and 20 exemplars were analysed from each group KAN, TAM, and TEL. In few cases the same speakers have spoken in other Indian language of the neighbouring state. 
       
       The difference in $F_0$ contour between native and non-native speech for speakers from each group has been tested. These results shown in Table 6 clearly indicate that the mean value of non-native pitch is markedly high in the case non-native speakers in all the three groups. The percentage deviation from native language to English speech for a group of 20 speakers in each of the three languages has been estimated and is presented in Table 7. It is evident from the scores presented in Table 7 that the dynamic variation of pitch is the least at $3.7 \%$  for the regional variant of KAN speakers, which is significantly less when compared to $9.5 \%$,and  $27\%$ corresponding to native TAM and TEL speakers respectively. 
       
           
    
     
  \begin{figure}
 \begin{center}
 \includegraphics[width=7.6cm,height=6cm]{CM3-v2-Enhanced.jpg}
\caption{ Confusion Matrix for SVM classification of English by speakers of KAN, TAM, and TEL. Note: TPR is True Positive Rate, FNR is False Negative Rate. }
 \end{center}
\end{figure}      

     

 






 \begin{table}[h]
  % \scriptsize
  \centering
  \caption{Mean($\mu$) and SD($\sigma$) of Pitch variation of single speaker from three groups of native speakers when speaking (a) Native Language (NL) (b) English (c) Other South Indian language (OSIL)}
    \begin{tabular}{|l|rr|rr|rr|}
    \hline
    \multicolumn{1}{|c|}{\multirow{3}[6]{*}{L1}} & \multicolumn{6}{c|}{LANGUAGE SPOKEN} \\
%\hline         
 & \multicolumn{2}{c|}{(a) NL} & \multicolumn{2}{c|}{(b) English} & \multicolumn{2}{c|}{(c) OSIL } \\
%\hline{2-7}       
   & \multicolumn{1}{l|}{$\mu$} & \multicolumn{1}{l|}{$\sigma$} & \multicolumn{1}{l|}{$\mu$ } & \multicolumn{1}{l|}{$\sigma$} & \multicolumn{1}{l|}{$\mu$} & \multicolumn{1}{l|}{$\sigma$} \\  
    \hline
    Kan   & 214      &  32.2     &  254    &  32.3     &  235    & 32.4 \\    
    \hline
    Tam   & 227      &  21.7     & 248      &  28.9     &  230     & 30.6 \\    
    \hline
    Tel  &  133     &  21.5     &  157     &   22.9    &   150    & 26.3 \\    
    \hline
    \end{tabular}%
  \label{tab:addlabel}%
\end{table}%

\section{Analysis of Results}
  
  \begin{itemize}

\item  Figures 3,4,5,8 reveals that English by native Kannada and Malayalam speakers is distinct than native Tamil or Telugu speakers, when compared to standard English.

\item  Accent partitioning experiments from a short utterance of 60 seconds of test data, indicates the suitability of the SVM classifier, as can be seen from accuracies shown in Table 5.
 
\item  Figures 1 reveals that the English spoken by  Telugu native speakers are marginally closer to standard English, compared to that of Kannada and Malayalam language speakers. 

\item   Higher mean values of the non-native pitch   shown in Table 6 indicates the  accommodation of speakers of all native languages to suit different social groups.
 
 
\item   Table 7 shows that English speakers of Tamil and Telugu would produce statistically significant higher pitch contour deviations than KAN speakers.  


\end{itemize}


 
   
   
   \section{Key Outcome and Contributions}
 
\begin{itemize}    
%%%   
  

\item 	 A framework to handle the deviations of $L_2$ influenced by closely related $L_1$s and to achieve better performance for a given NLI task, even with fewer features is proposed

\item 	Current study is significant when the target languages are linguistically close, and large resources of spoken English are not available 



 \begin{table}[h]
  \caption{ Percentage increase in Standard Deviation of pitch contour from native language speech to English  speech (using two non-overlapping sets of 20 speakers from each native language group Kannada, Tamil, and Telugu).}
   \label{tab:example}
  \centering
  \begin{tabular}{cccc}
\hline
Language group  & Male   & Female  & Average\\
\hline
 Kannada  & 0.9    & 6.5  & 3.7 \\
\hline
Tamil  &  9    &  10 &  9.5 \\
\hline
Telugu  &  33    & 21  &  27\\
\hline  
\end{tabular}
\end{table}  

\item Prosodic differences across the South Indian English accents has been experimentally illustrated, which is useful in automatic intonation classification for $L_2$ speech acquisition. 

\item Present work helps in accurate recognition of regional accent, that can improve the speech and speaker recognition system performance. 

\item Distinct pitch pattern variations in non-native English speech by Malayalam, and Kannada speakers compared to that of Tamil and Telugu varieties can help in distinguishing them. 

%\item Present research on English spoken as a second language in the Indian setting helps in overcoming language barrier, and also useful for speakers of Indian English to correct their errors.  
\end{itemize}

   
 

\section{Conclusion}


 It can be concluded that the regional native language classification has been achieved with an accuracy of nearly $90 \%$, by using the acoustic distribution of cepstral features on the four types of non-native South Indian English speech.  It is known that systems make more mistakes among regionally close languages.  
Accent differences among the non-native speakers are reflected as the deviation of $L_2$ influenced by $L_1$ on prosodic level. Studies carried out based on intonation distribution indicates that English speaking South Indian groups corresponding to Kannada, Malayalam, Tamil, and Telugu are clearly divided as per their native languages. Prosodic differences in the native and English speech by South Indian speakers were detected without annotation. Present method can potentially be applied to other languages like Hindi, and in addressing the important question of finding a universal feature set for identifying the non-native speech. 
   
   Present research is useful in applications such as voice based wireless services like mobile health care, agriculture. Automatic accent characterization can also be applied to fields such as sociolinguistics and speech pathology. Future work can employ different speech styles, and characteristics of speaker population to be carefully scrutinized, and also by including multi-disciplinary information. Further, the results can be extended to separating language families and also for rating $L_2$ proficiency.  

\newpage

\bibliographystyle{acl_natbib}
\bibliography{MyRef}
%
% \begin{thebibliography}{9}
%  
% 
% 
%
%\end{thebibliography}




\end{document}

%
%
%\bibitem[1]{p1}
%     D.Crystal,
%   ``A Dictionary of Linguistics and Phonetics'',
%   in \textit{2nd edition, Oxford: Basil Blackwell, 1985.} 
%
% \bibitem[2]{p2} Hamid Behravan,\ Ville Hautamaki, and \ Tomi Kinnuneu,
%   ``Foreign Accent Detection from Spoken Finnish Using i-Vectors'', \textit{Interspeech, 2013}, pp.~79-83.
%   
%   \bibitem[3]{p3}
%  	 Mary P. Harper, and \ Michael Maxwell, Spoken Language Characterization,
%  in {Springer Handbook on Speech Processing and Speech Communication},  2006, pp.1-15.
%  
%  \bibitem[4]{p4}
%  Sergiu Nisioi, 
%   ``Feature Analysis for Native Language Identification'',
%   in \textit{16th International conference on Intelligent Text processing and computational linguistics}, 2015, pp.~1-15.
%   
%   \bibitem[5]{p5}
%  Shambhu Nath Saha , and \ Shyamal Kr Das Mandal,   ``Study of Acoustic Correlates of English Speakers Compared to Native (L1) English Speakers " Listener Accent and Perceived Accent, and Comprehension",
%   in \textit{INTERSPEECH 2015}, pp. ~815-819.
% 
% \bibitem[6]{p6}
% Calbert Graham and \ Brechtje Post,
%   ``Second Language acquisition of Intonation: peak alignment in American English'',
%   in \textit{Journal of phonetics 66}, 2018, pp.~1-14.
%   
%   \bibitem[7]{p7}
%    L.Arslan, \ J.H.L.Hansen, \ Lastname3,
%   ``Language Accent Classification in American English'',
%   in \textit{Speech Communications}, July 1996, vol.~ 18(4), pp.~353-367.
%   
%   \bibitem[8]{p8}
%\ Marcos Zampieri, \ Alina Maria Ciobanu, and \  Liviu P.Dinu,  
%   ``Native Language Identification on Text and Speech'',
%   in \textit{Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, ACL}, 2017, pp.~398-404.
%   
%   \bibitem[9]{p9}  
%  Masahiko Komatsu, ``Reviewing human language Identification'',  \textit{Speaker Classification II,Springer Verlag: Berlin/ Heidelberg, Germany,}, pp.~206-228, 2007.
%
%\bibitem[10]{p10} 
%
%   	 A. Ikeno , and \ J. H. L. Hansen,   ``Perceptual recognition cues in Native English Accent Variation: " Listener Accent and Perceived Accent, and Comprehension",
%   in \textit{ IEEE ICASSP, }, 2006, pp.~401-404.
%
%\bibitem[11]{p11}
%
% L.Mary, and \ B.Yegnanarayana, 
%  ``Extraction and representation of prosodic features for language and speaker recognition'',
%   \textit{Speech Communication}, vol.50, pp. ~782-796, 2008.   
%
%\bibitem[12]{p12}
%     Julia Monnin, and \ Hélène Loevenbruck, 
%  ``Language-Specific Influence on Phoneme Development: French and Drehu Data'',
%   \textit{INTERSPEECH 2010}, pp. ~1882-1885.
%
%\bibitem[13]{p13}
%   	Markham D.,  
%   ``Listeners and disguised voices: the imitation and perception of dialectal accent. '',
%   in \textit{Forensic Linguistics}, vol 6(2), 1999, pp 289-299. 
%
%
%\bibitem[14]{p14}
%   Hynek Hermansky,
%   ``Perceptual linear predictive (PLP) analysis of speech,''
%   in \textit{Journal of Acoustic Society of America}, 2010,vol.~87, no.~4, pp.~1738-1752.
%
%
% \bibitem[15]{p15}
% 
%  Helen Fraser,
%   ``The role of 'educated native speakers’ in providing language analysis for the determination of the origin of asylum seekers'',
%   in \textit{International Journal of Speech Language and Law}, 2009, pp.~113-138.
%
%\bibitem[16]{p16}
% L. Neumeyer,\ H. Franco,\ V. Digalakis,  and \ M. Weintraub,
%   ``Automatic Scoring of Pronunciation Quality'',
%   in \textit{Speech Communication}, 2000, vol.~30, pp.~83-93.
%
%\bibitem[17]{p17}
%     Marcel Kockmann, \ Lukas Burget, \ and \  Jan Honza Cermocky,  
%   ``Investigations into prosodic syllable contour features for speaker recogntion'',
%   in \textit{”, in Acoustics, Speech, and Signal Processing ICASSP,}, 2010, %pp.~4418-4421.
%
%\bibitem[18]{p18}
%   Sadaoki Furui,
%   ``Language identification using pitch    contour information,''
%   in \textit{in Proc. IEEE ICASSP, Philadelphia, PA}, Mar 2005,  pp.~601604.
%
%\bibitem[19]{p19}
%   	 \ Vinay Kumar Mittal,
%   ``Analysis of nonverbal speech sounds,''
%   Ph.D. Thesis, \textit{International Institute of Information Technology Hyderabad}, November 2014.
%   
%\bibitem[20]{p20} 
%
%   	 Sirsa H , and \ Redford,   ``Structure of Indian English sounds and timing patterns”, in Journal of Phonetics 2013",
%   in \textit{ Journal of Phonetics, }, 2013, pp.~3931-406.
%
%  \bibitem[21]{p21}
%  	 J. Cheng,\ N. Bojja, and \ X. Chen ,
%   ``Automatic Accent Quantification of Indian Speakers of English,'',
%   in \textit{Interspeech}, 2013, pp.~2574-2578.
%
%\bibitem[22]{p22}
%   H.Li, \ K.A.Lee, \  B.Ma,
%   ``Spoken language recognition, From fundamentals to practice'',
%   in \textit{Proceedings of the IEEE}, May 2013, vol.~101, no.~5,  PP.~1136-1159.
%
%\bibitem[23]{p23}	
%Doddington G. ,
%   ``Speaker recognition based on idiolectical differences between speakers,''
%   in \textit{In: Proc. EUROSPEECH, Aalborg, Denmark}, 2001, pp.~2521-2524.	
%
% 
% \bibitem[24]{p24}
%Shervin Malamasi, and \  Mark Dras, ``Multilingual Native language Identification,'' %\in{Journal of Natural Language Engineering}, vol.\ 23, no.\ 2, pp.~163--215, 2017.
%
%\bibitem[25]{p25}
%  K. Sreenivasa Rao, and \ B. Yegnanarayana ,
%   ``Intonation modeling for Indian languages'',
%   in \textit{Science Direct , Coputer Speech and Language}, 2009, pp.~240-256.
%
%\bibitem[26]{p26}
%G. Radha Krishna, and \ R.Krishnan,
%   ``Native Language Identification Based on English Accent'',
%   in \textit{International Conference on Natural Language Processing (ICON), 2014}, pp.~69-74.
%
%\bibitem[27]{p27}
%   	 Iskra, D., \ Grosskopf, B, \ Marasek,K \ Heuvel, \ H. van den, \ Diehl, F, and  \ Kiessling, A, ``SPEECON - Speech Databases for Consumer Devices: Database Specification and Validation” ,
%   in \textit{Proceedings LREC 2002. Third International Conference on Language Resources and Evaluation}, 2002, pp.~329-333.
%
% \bibitem[28]{p28}D. A. Reynolds and \ Richar C. Rose,
%   ``Robust Text-Independent Speaker Identification Using  Gaussian Mixture Speaker Models'',
%   in \textit{IEEE Transactions on Speech and Audio Processing}, 1995, Vol.~3, No.~1,  pp.~72-83. 
%     
%   
%\bibitem[29]{p29}
%   	Nengheng Zheng, and \ P.C.Ching ,
%   ``Integration of Complementary Acoustic Features for Speaker Recognition'',
%   in \textit{IEEE Signal Processing Letters}, March 2007,vol.~14, no.~3, pp.~181-184.
%  
%   \bibitem[30]{p30}
%   	Joseph Tepperman, and \ Shrikanth Narayanan,
%   ``Better Nonnative Intonation scores through Prosodic theory,''
%   in \textit{INTERSPEECH 2008}, pp.~1813-1816.
%
%  \bibitem[31]{p31}
%   	Eady, S. J., and \ Cooper, W. E.,
%   ``Speech intonation and focus location in matched statements and questions'',
%   in \textit{Journal of the Acoustical Society of America }, 1980, pp.~402-416.
%   
%   
%
% \bibitem[32]{p32}
%    Juan Pablo Arias, \ Nestor Becerra Yoma, \ Hiram Vivanko, ,
%   ``Automatic intonation assessment for computer aided language learning'',
%   in \textit{Speech Communication}, 2010, vol.~52, pp.~254-267.
%   
%    \bibitem[33]{p33}
%  N. Moustroufas, and \ V. Digalakis,
%   ``Automatic pronunciation evaluation of foreign speakers using unknown text,''
%   in \textit{Computer Speech and Language}, 2007,  vol.~ 21, pp.~219-230.
%
%\bibitem[34]{p34}
%   Xinhao Wang, \ Keelan Evanini, \ Su-Youn Yoon,
%   ``Word –Level F0 modeling in Automated Assessment of Non-native read speech'',
%   in \textit{Word –Level F0 modeling in Automated Assessment of Non-native read speech” SLaTE (2015), ISCA workshop on Speech and Language Technology in Educatio},  pp.~23-28, 2015.  
%   
% \bibitem[35]{p35}
%Yao Quian,\ Keelan Evanini,\ Xinhao Wang ,\ David Sundermann Oeft,\ Rober A Pugh, \ Patrick L Lange, \ Hillary R Molloy, and \ Frank K Soong, ``Improving Sub-phone Modeling for Better Native Language Identification With Non-native Englsih Speech,'' \textit{Interspeech, 2017}, pp.~2586-2590.
%
%\bibitem[36]{p36}
%  Wave Surfer ,
%    
%    \textit{http://www.speech.kth.se/wavesurfer/}  
% 