Abstract
This study explores the use of large language models (LLMs), such as ChatGPT and GPT-4, in creating high-quality text-based social media content for businesses on LinkedIn. We introduce a novel architecture incorporating external knowledge bases and a multi-step writing approach, which extracts facts from company websites to form a knowledge graph. Our method’s efficacy is assessed using the “Long-LinkedIn” evaluation dataset designed for long-form post generation. Results indicate that our iterative refinement significantly improves content quality. However, knowledge-enhanced prompts occasionally reduced quality due to potential formulation issues. LLM-based evaluations, particularly using ChatGPT, showcased potential as a less resource-intensive alternative to human assessments, with a notable alignment between the two evaluation techniques.- Anthology ID:
- 2023.gem-1.3
- Volume:
- Proceedings of the Third Workshop on Natural Language Generation, Evaluation, and Metrics (GEM)
- Month:
- December
- Year:
- 2023
- Address:
- Singapore
- Editors:
- Sebastian Gehrmann, Alex Wang, João Sedoc, Elizabeth Clark, Kaustubh Dhole, Khyathi Raghavi Chandu, Enrico Santus, Hooman Sedghamiz
- Venues:
- GEM | WS
- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 22–41
- Language:
- URL:
- https://aclanthology.org/2023.gem-1.3
- DOI:
- Cite (ACL):
- Christian Jensen and Axel Højmark. 2023. Formalizing content creation and evaluation methods for AI-generated social media content. In Proceedings of the Third Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), pages 22–41, Singapore. Association for Computational Linguistics.
- Cite (Informal):
- Formalizing content creation and evaluation methods for AI-generated social media content (Jensen & Højmark, GEM-WS 2023)
- PDF:
- https://preview.aclanthology.org/nschneid-patch-4/2023.gem-1.3.pdf