Jianbo Zhao
2024
Vision Language Models for Spreadsheet Understanding: Challenges and Opportunities
Shiyu Xia
|
Junyu Xiong
|
Haoyu Dong
|
Jianbo Zhao
|
Yuzhang Tian
|
Mengyu Zhou
|
Yeye He
|
Shi Han
|
Dongmei Zhang
Proceedings of the 3rd Workshop on Advances in Language and Vision Research (ALVR)
This paper explores capabilities of Vision Language Models on spreadsheet comprehension. We propose three self-supervised challenges with corresponding evaluation metrics to comprehensively evaluate VLMs on Optical Character Recognition (OCR), spatial perception, and visual format recognition. Additionally, we utilize the spreadsheet table detection task to assess the overall performance of VLMs by integrating these challenges. To probe VLMs more finely, we propose three spreadsheet-to-image settings: column width adjustment, style change, and address augmentation. We propose variants of prompts to address the above tasks in different settings. Notably, to leverage the strengths of VLMs in understanding text rather than two-dimensional positioning, we propose to decode cell values on the four boundaries of the table in spreadsheet boundary detection. Our findings reveal that VLMs demonstrate promising OCR capabilities but produce unsatisfactory results due to cell omission and misalignment, and they notably exhibit insufficient spatial and format recognition skills, motivating future work to enhance VLMs’ spreadsheet data comprehension capabilities using our methods to generate extensive spreadsheet-image pairs in various settings.
2018
Contextualized Character Representation for Chinese Grammatical Error Diagnosis
Jianbo Zhao
|
Si Li
|
Zhiqing Lin
Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications
Nowadays, more and more people are learning Chinese as their second language. Establishing an automatic diagnosis system for Chinese grammatical error has become an important challenge. In this paper, we propose a Chinese grammatical error diagnosis (CGED) model with contextualized character representation. Compared to the traditional model using LSTM (Long-Short Term Memory), our model have better performance and there is no need to add too many artificial features.
2017
N-gram Model for Chinese Grammatical Error Diagnosis
Jianbo Zhao
|
Hao Liu
|
Zuyi Bao
|
Xiaopeng Bai
|
Si Li
|
Zhiqing Lin
Proceedings of the 4th Workshop on Natural Language Processing Techniques for Educational Applications (NLPTEA 2017)
Detection and correction of Chinese grammatical errors have been two of major challenges for Chinese automatic grammatical error diagnosis. This paper presents an N-gram model for automatic detection and correction of Chinese grammatical errors in NLPTEA 2017 task. The experiment results show that the proposed method is good at correction of Chinese grammatical errors.
Search
Co-authors
- Si Li 2
- Zhiqing Lin 2
- Hao Liu 1
- Zuyi Bao 1
- Xiaopeng Bai 1
- show all...