2022
pdf
Stability of Forensic Text Comparison System
Susan Brown

Shunichi Ishihara
Proceedings of the The 20th Annual Workshop of the Australasian Language Technology Association
pdf
abs
Estimating the Strength of Authorship Evidence with a DeepLearningBased Approach
Shunichi Ishihara

Satoru Tsuge

Mitsuyuki Inaba

Wataru Zaitsu
Proceedings of the The 20th Annual Workshop of the Australasian Language Technology Association
This study is the first likelihood ratio (LR)based forensic text comparison study in which each text is mapped onto an embedding vector using RoBERTa as the pretrained model. The scores obtained with Cosine distance and probabilistic linear discriminant analysis (PLDA) were calibrated to LRs with logistic regression; the quality of the LRs was assessed by log LR cost (Cllr). Although the documents in the experiments were very short (maximum 100 words), the systems reached the Cllr values of 0.55595 and 0.71591 for the Cosine and PLDA systems, respectively. The effectiveness of deeplearningbased text representation is discussed by comparing the results of the current study to those of the previous studies of systems based on conventional feature engineering tested with longer documents.
2020
pdf
abs
The Influence of Background Data Size on the Performance of a Scorebased Likelihood Ratio System: A Case of Forensic Text Comparison
Shunichi Ishihara
Proceedings of the The 18th Annual Workshop of the Australasian Language Technology Association
This study investigates the robustness and stability of a likelihood ratio–based (LRbased) forensic text comparison (FTC) system against the size of background population data. Focus is centred on a scorebased approach for estimating authorship LRs. Each document is represented with a bagofwords model, and the Cosine distance is used as the scoregenerating function. A set of population data that differed in the number of scores was synthesised 20 times using the MonteCarol simulation technique. The FTC system’s performance with different population sizes was evaluated by a gradient metric of the log–LR cost (Cllr). The experimental results revealed two outcomes: 1) that the scorebased approach is rather robust against a small population size—in that, with the scores obtained from the 40~60 authors in the database, the stability and the performance of the system become fairly comparable to the system with a maximum number of authors (720); and 2) that poor performance in terms of Cllr, which occurred because of limited background population data, is largely due to poor calibration. The results also indicated that the scorebased approach is more robust against data scarcity than the featurebased approach; however, this finding obliges further study.
pdf
abs
FeatureBased Forensic Text Comparison Using a Poisson Model for Likelihood Ratio Estimation
Michael Carne

Shunichi Ishihara
Proceedings of the The 18th Annual Workshop of the Australasian Language Technology Association
Score and featurebased methods are the two main ones for estimating a forensic likelihood ratio (LR) quantifying the strength of evidence. In this forensic text comparison (FTC) study, a scorebased method using the Cosine distance is compared with a featurebased method built on a Poisson model with texts collected from 2,157 authors. Distance measures (e.g. Burrows’s Delta, Cosine distance) are a standard tool in authorship attribution studies. Thus, the implementation of a scorebased method using a distance measure is naturally the first step for estimating LRs for textual evidence. However, textual data often violates the statistical assumptions underlying distancebased models. Furthermore, such models only assess the similarity, not the typicality, of the objects (i.e. documents) under comparison. A Poisson model is theoretically more appropriate than distancebased measures for authorship attribution, but it has never been tested with linguistic text evidence within the LR framework. The logLR cost (Cllr) was used to assess the performance of the two methods. This study demonstrates that: (1) the featurebased method outperforms the scorebased method by a Cllr value of ca. 0.09 under the bestperforming settings and; (2) the performance of the featurebased method can be further improved by feature selection.
2018
pdf
bib
abs
Textdependent Forensic Voice Comparison: Likelihood Ratio Estimation with the Hidden Markov Model (HMM) and Gaussian Mixture Model
Satoru Tsuge

Shunichi Ishihara
Proceedings of the Australasian Language Technology Association Workshop 2018
Among the more typical forensic voice comparison (FVC) approaches, the acousticphonetic statistical approach is suitable for textdependent FVC, but it does not fully exploit available timevarying information of speech in its modelling. The automatic approach, on the other hand, essentially deals with textindependent cases, which means temporal information is not explicitly incorporated in the modelling. Textdependent likelihood ratio (LR)based FVC studies, in particular those that adopt the automatic approach, are few. This preliminary LRbased FVC study compares two statistical models, the Hidden Markov Model (HMM) and the Gaussian Mixture Model (GMM), for the calculation of forensic LRs using the same speech data. FVC experiments were carried out using different lengths of Japanese short words under a forensically realistic, but challenging condition: only two speech tokens for model training and LR estimation. Loglikelihoodratio cost (Cllr) was used as the assessment metric. The study demonstrates that the HMM system constantly outperforms the GMM system in terms of average Cllr values. However, words longer than three mora are needed if the advantage of the HMM is to become evident. With a sevenmora word, for example, the HMM outperformed the GMM by a Cllr value of 0.073.
2017
pdf
A Comparative Study of Two Statistical Modelling Approaches for Estimating Multivariate Likelihood Ratios in Forensic Voice Comparison
Shunichi Ishihara
Proceedings of the Australasian Language Technology Association Workshop 2017
2016
pdf
An Effect of Background Population Sample Size on the Performance of a Likelihood Ratiobased Forensic Text Comparison System: A Monte Carlo Simulation with Gaussian Mixture Model
Shunichi Ishihara
Proceedings of the Australasian Language Technology Association Workshop 2016
2015
pdf
Likelihood Ratiobased Forensic Voice Comparison on L2 speakers: A Case of Hong Kong native male production of English vowels
Daniel Frost

Shunichi Ishihara
Proceedings of the Australasian Language Technology Association Workshop 2015
2013
pdf
The Effect of the Withinspeaker Sample Size on the Performance of Likelihood Ratio Based Forensic Voice Comparison: Monte Carlo Simulations
Shunichi Ishihara
Proceedings of the Australasian Language Technology Association Workshop 2013 (ALTA 2013)
pdf
A Comparative Study of Likelihood Ratio Based Forensic Text Comparison in Procedures: Multivariate Kernel Density vs. Gaussian Mixture ModelUniversal Background Model
Shunichi Ishihara
Proceedings of the Australasian Language Technology Association Workshop 2013 (ALTA 2013)
pdf
Differences in Speaker Individualising Information between Case Particles and Fillers in Spoken Japanese
Shunichi Ishihara
Proceedings of the Australasian Language Technology Association Workshop 2013 (ALTA 2013)
2011
pdf
A Forensic Authorship Classification in SMS Messages: A Likelihood Ratio Based Approach Using Ngram
Shunichi Ishihara
Proceedings of the Australasian Language Technology Association Workshop 2011
2010
pdf
Variability and Consistency in the Idiosyncratic Selection of Fillers in Japanese Monologues: Gender Differences
Shunichi Ishihara
Proceedings of the Australasian Language Technology Association Workshop 2010