Analysis of Transformer Model Applications

Date

2023-08-29

Advisors

Journal Title

Journal ISSN

ISSN

Volume Title

Publisher

Springer

Type

Conference

Peer reviewed

Yes

Abstract

Since the emergence of the Transformer, many variations of the original architecture have been created. Revisions and taxonomies have appeared that group these models from different points of view. However, no review studies the tasks faced according to the type of data used. In this paper, the modifications applied to Transformers to work with different input data (text, image, video, etc.) and to solve disparate problems are analysed. Building on the foundations of existing taxonomies, this work proposes a new one that relates input data types to applications. The study shows open challenges and can serve as a guideline for the development of Transformer networks for specific applications with different types of data by observing development trends.

Description

Keywords

Citation

Cabrera-Bermejo, M.I., Del Jesus, M.J., Rivera, A.J., Elizondo, D., Charte, F., Pérez-Godoy, M.D. (2023) Analysis of Transformer Model Applications. In: García Bringas, P., et al. Hybrid Artificial Intelligent Systems. HAIS 2023. Lecture Notes in Computer Science, 14001. Springer, https://doi.org/10.1007/978-3-031-40725-3_20

Rights

Research Institute

Institute of Artificial Intelligence (IAI)