EXTRA: Integrating External Knowledge into Multimodal Hashtag Recommendation System

Hyun Sik Won; Su Min Roh; Dohyun Kim; Min Ji Kim; Huiju Kim; Kang Min Kim

doi:10.1109/ICWS60048.2023.00096

EXTRA: Integrating External Knowledge into Multimodal Hashtag Recommendation System

Hyun Sik Won, Su Min Roh, Dohyun Kim, Min Ji Kim, Huiju Kim, Kang Min Kim

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

As microblogs have become commonplace, recommending relevant hashtags for microblog posts has become increasingly important. However, recommending appropriate hashtags for a post is challenging because it requires a high-level understanding of the context and relationships of the information in the post. In this paper, we propose a novel hashtag recommendation framework that incorporates external knowledge to enrich the context of posts. Using an image of the post, we obtain the hierarchical external knowledge extracted by the Open Directory Project (ODP)-based classifier. Experimental results show that our framework performs better than the baselines on a multimodal hashtag recommendation benchmark dataset. It outperformed the existing state-of-the-art model by providing a 39.86% increase in the average F1-score.

Original language	English
Title of host publication	Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023
Editors	Claudio Ardagna, Boualem Benatallah, Hongyi Bian, Carl K. Chang, Rong N. Chang, Jing Fan, Geoffrey C. Fox, Zhi Jin, Xuanzhe Liu, Heiko Ludwig, Michael Sheng, Jian Yang
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	719-721
Number of pages	3
ISBN (Electronic)	9798350304855
DOIs	https://doi.org/10.1109/ICWS60048.2023.00096
State	Published - 2023
Event	2023 IEEE International Conference on Web Services, ICWS 2023 - Hybrid, Chicago, United States Duration: 2 Jul 2023 → 8 Jul 2023

Publication series

Name	Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023

Conference

Conference	2023 IEEE International Conference on Web Services, ICWS 2023
Country/Territory	United States
City	Hybrid, Chicago
Period	2/07/23 → 8/07/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

Hashtag Recommendation
Knowledge Base
Multimodal Learning
Natural Language Processing
Vision and Language Pretrained Models

Access to Document

10.1109/ICWS60048.2023.00096

Cite this

Won, H. S., Roh, S. M., Kim, D., Kim, M. J., Kim, H., & Kim, K. M. (2023). EXTRA: Integrating External Knowledge into Multimodal Hashtag Recommendation System. In C. Ardagna, B. Benatallah, H. Bian, C. K. Chang, R. N. Chang, J. Fan, G. C. Fox, Z. Jin, X. Liu, H. Ludwig, M. Sheng, & J. Yang (Eds.), Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023 (pp. 719-721). (Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICWS60048.2023.00096

Won, Hyun Sik ; Roh, Su Min ; Kim, Dohyun et al. / EXTRA : Integrating External Knowledge into Multimodal Hashtag Recommendation System. Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023. editor / Claudio Ardagna ; Boualem Benatallah ; Hongyi Bian ; Carl K. Chang ; Rong N. Chang ; Jing Fan ; Geoffrey C. Fox ; Zhi Jin ; Xuanzhe Liu ; Heiko Ludwig ; Michael Sheng ; Jian Yang. Institute of Electrical and Electronics Engineers Inc., 2023. pp. 719-721 (Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023).

@inproceedings{49810f5ae5584be5a0cffc9097d9fa7c,

title = "EXTRA: Integrating External Knowledge into Multimodal Hashtag Recommendation System",

abstract = "As microblogs have become commonplace, recommending relevant hashtags for microblog posts has become increasingly important. However, recommending appropriate hashtags for a post is challenging because it requires a high-level understanding of the context and relationships of the information in the post. In this paper, we propose a novel hashtag recommendation framework that incorporates external knowledge to enrich the context of posts. Using an image of the post, we obtain the hierarchical external knowledge extracted by the Open Directory Project (ODP)-based classifier. Experimental results show that our framework performs better than the baselines on a multimodal hashtag recommendation benchmark dataset. It outperformed the existing state-of-the-art model by providing a 39.86% increase in the average F1-score.",

keywords = "Hashtag Recommendation, Knowledge Base, Multimodal Learning, Natural Language Processing, Vision and Language Pretrained Models",

author = "Won, {Hyun Sik} and Roh, {Su Min} and Dohyun Kim and Kim, {Min Ji} and Huiju Kim and Kim, {Kang Min}",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 2023 IEEE International Conference on Web Services, ICWS 2023 ; Conference date: 02-07-2023 Through 08-07-2023",

year = "2023",

doi = "10.1109/ICWS60048.2023.00096",

language = "English",

series = "Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "719--721",

editor = "Claudio Ardagna and Boualem Benatallah and Hongyi Bian and Chang, {Carl K.} and Chang, {Rong N.} and Jing Fan and Fox, {Geoffrey C.} and Zhi Jin and Xuanzhe Liu and Heiko Ludwig and Michael Sheng and Jian Yang",

booktitle = "Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023",

address = "United States",

}

Won, HS, Roh, SM, Kim, D, Kim, MJ, Kim, H & Kim, KM 2023, EXTRA: Integrating External Knowledge into Multimodal Hashtag Recommendation System. in C Ardagna, B Benatallah, H Bian, CK Chang, RN Chang, J Fan, GC Fox, Z Jin, X Liu, H Ludwig, M Sheng & J Yang (eds), Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023. Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023, Institute of Electrical and Electronics Engineers Inc., pp. 719-721, 2023 IEEE International Conference on Web Services, ICWS 2023, Hybrid, Chicago, United States, 2/07/23. https://doi.org/10.1109/ICWS60048.2023.00096

EXTRA: Integrating External Knowledge into Multimodal Hashtag Recommendation System. / Won, Hyun Sik; Roh, Su Min; Kim, Dohyun et al.
Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023. ed. / Claudio Ardagna; Boualem Benatallah; Hongyi Bian; Carl K. Chang; Rong N. Chang; Jing Fan; Geoffrey C. Fox; Zhi Jin; Xuanzhe Liu; Heiko Ludwig; Michael Sheng; Jian Yang. Institute of Electrical and Electronics Engineers Inc., 2023. p. 719-721 (Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - EXTRA

T2 - 2023 IEEE International Conference on Web Services, ICWS 2023

AU - Won, Hyun Sik

AU - Roh, Su Min

AU - Kim, Dohyun

AU - Kim, Min Ji

AU - Kim, Huiju

AU - Kim, Kang Min

PY - 2023

Y1 - 2023

N2 - As microblogs have become commonplace, recommending relevant hashtags for microblog posts has become increasingly important. However, recommending appropriate hashtags for a post is challenging because it requires a high-level understanding of the context and relationships of the information in the post. In this paper, we propose a novel hashtag recommendation framework that incorporates external knowledge to enrich the context of posts. Using an image of the post, we obtain the hierarchical external knowledge extracted by the Open Directory Project (ODP)-based classifier. Experimental results show that our framework performs better than the baselines on a multimodal hashtag recommendation benchmark dataset. It outperformed the existing state-of-the-art model by providing a 39.86% increase in the average F1-score.

AB - As microblogs have become commonplace, recommending relevant hashtags for microblog posts has become increasingly important. However, recommending appropriate hashtags for a post is challenging because it requires a high-level understanding of the context and relationships of the information in the post. In this paper, we propose a novel hashtag recommendation framework that incorporates external knowledge to enrich the context of posts. Using an image of the post, we obtain the hierarchical external knowledge extracted by the Open Directory Project (ODP)-based classifier. Experimental results show that our framework performs better than the baselines on a multimodal hashtag recommendation benchmark dataset. It outperformed the existing state-of-the-art model by providing a 39.86% increase in the average F1-score.

KW - Hashtag Recommendation

KW - Knowledge Base

KW - Multimodal Learning

KW - Natural Language Processing

KW - Vision and Language Pretrained Models

UR - http://www.scopus.com/inward/record.url?scp=85173803511&partnerID=8YFLogxK

U2 - 10.1109/ICWS60048.2023.00096

DO - 10.1109/ICWS60048.2023.00096

M3 - Conference contribution

AN - SCOPUS:85173803511

T3 - Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023

SP - 719

EP - 721

BT - Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023

A2 - Ardagna, Claudio

A2 - Benatallah, Boualem

A2 - Bian, Hongyi

A2 - Chang, Carl K.

A2 - Chang, Rong N.

A2 - Fan, Jing

A2 - Fox, Geoffrey C.

A2 - Jin, Zhi

A2 - Liu, Xuanzhe

A2 - Ludwig, Heiko

A2 - Sheng, Michael

A2 - Yang, Jian

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 2 July 2023 through 8 July 2023

ER -

Won HS, Roh SM, Kim D, Kim MJ, Kim H, Kim KM. EXTRA: Integrating External Knowledge into Multimodal Hashtag Recommendation System. In Ardagna C, Benatallah B, Bian H, Chang CK, Chang RN, Fan J, Fox GC, Jin Z, Liu X, Ludwig H, Sheng M, Yang J, editors, Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023. Institute of Electrical and Electronics Engineers Inc. 2023. p. 719-721. (Proceedings - 2023 IEEE International Conference on Web Services, ICWS 2023). doi: 10.1109/ICWS60048.2023.00096

EXTRA: Integrating External Knowledge into Multimodal Hashtag Recommendation System

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this