Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
This research report explores the effectiveness of emojis in user experience questionnaires (ueq) by examining a case study at fpt university in vietnam. The study investigates the impact of emojis on user experience, focusing on six key variables: appeal, clarity, efficiency, reliability, stimulation, and novelty. The report analyzes data collected from a survey of 250 participants, providing insights into the potential of emojis to enhance user engagement and communication in digital environments.
Typology: Study Guides, Projects, Research
1 / 16
This page cannot be seen from the preview
Don't miss anything!
The Evaluation of User Experience of Emoji-based Form Using UEQ: A Case Study of University in Vietnam RESEARCHERS Trần Mai Anh - SS Nguyễn Trung Tín - SS Nguyễn Vân Hương - SS Phạm Anh Thy - SS Hà Hiển Long - SS Group: EMZOJI Lecturer: Lê Thị Mỹ Danh Subject: RMC FPT UNIVERSITY HCMC, 2023.
Abstract................................................................................................................................................................
**1. Introduction.....................................................................................................................................................
The digital revolution has led to the widespread adoption of emojis across various digital platforms due to their universal acceptance by users. This contemporary mode of communication is valued for its ability to convey multiple meanings, sometimes going beyond their initial definitions, as long as both senders and receivers share a common understanding of the intended message. This advancement speeds up communication while also time-saving. As a result, people are increasingly using emojis as a language-independent substitute for conveying subjective assessments and expressing opinions, and emotions in text-based communication, where nonverbal cues are absent (Walther and D'Addario, 2001). Due to being a relatively new phenomenon, limited research has been conducted on their application, particularly in user experience (UX). In the digital age, there may be potential for using emojis in the realm of UX such as applications in the field of healthcare and educational institutions. For instance, in healthcare, pictorial pain scales (resembling emojis) are employed as tools to assist in pain management during clinical settings (Donovan, 2016). This application can use muliple tool, and the User Experience Questionnaire (UEQ) is one of the most representative instrument to measure individuals' perceptions when anticipating or using a product through evaluating UX based on six variables including: attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty; and a total of twenty-six items. (Schrepp, 2015). The aim of this paper seeks to discover the prospect of emoji-based application along with examine and assess the effectiveness of the six variables in predicting user experiences. This paper shall answer the following research question: How emojis influence user' experiences while completing a form based on the UEQs? To address this research question, an online survey featuring an emoji-based form was conducted and distributed through various social media groups, with data collected from a sample of 250 participants attending Welcoming Students Day at FPT University HCM city campus, Vietnam. The result will provide insights into emojis and their implications for electronic UEQs, opening up a fascinating area for further research.
2.1. Emojis Over the years, we've observed a profound transformation into the digital communication era, now the prevalent means of interaction in today's society. This evolution began with the utilization of email and text messages and has further evolved into the age of instant messaging via specialized applications like WeChat and similar platforms. Modern digital messaging tools, including iMessage and WhatsApp, have incredibly helped and enhanced interpersonal communication (Annamalai & Sobihatun Nur Abdul Salam, 2017). Nonetheless, these implematation frequently struggle to capture the subtleties of spoken communication. To address this limitation, there have been dedicated efforts to improve and explore methods for accurately conveying emotions and non-verbal cues beyond plain text. Fortunately, continuous technological advancements have given rise to an alternative form of "visual language" known as emojis, intending to bridge this communication gap. Emojis, in general, are text-based pictorial symbols used to represent facial expressions and conceptual ideas, especially emotions (Rodrigues et al., 2017). Although emojis come in various categories, and those that simulate human facial pressence are significantly common in everyday communication (Barbieri, Ronzano, & Saggion, 2016). 2.2. Emojis and UEQs Application Emojis representing facial expressions are now more commonly employed to assess user satisfaction due to the subjective and intricate nature of people's emotional states, which can be challenging to perceive and accurately capture. For instance, emoji-based evaluation was applied in rating the new products or brands’s designs in the F&B industry (Desmet, Overbeeke, & Tax, 2001; Jaeger et al., 2017). While the satisfaction scale often displays about 3 to 5 emojis expressing between happy to unhappy states in such places such as airports or stores, this usage may not be adequate in terms of capturing personal and subjective museum visitors’ experiences. Upgrading previous application, De Angeli, Daniela, Ryan M. Kelly, and Eamonn O’Neill (2020) have designed a set of 9 emojis portraying a more diverse range of emotional expressions to assess an interactive museum experience, supporting the idea that emoji enables researchers to capture participants’ feelings, and consequently, gain a greater understanding of its usage. Surprisingly, emoji research is scarce on user experience questionnaires (UEQs) according to Alismail, Sarah, and Hengwei Zhang (2018). Exploring this specific context has the potential to
3.2. Likert scale questionnaires Likert-scale questionnaires are the most commonly used type of instrument for measuring affective variables such as motivation and self-efficacy, given that they allow researchers to gather large amounts of data with relative ease (Nemoto, T., & Beglar, D. 2014), quantify opinions and beliefs and thus obtain more precise indications than with many other methodologies. This method aims to take a long list of possible scaling items for a concept and find the subset that measures the various dimensions. In this study, questions are offered and people are asked to choose from among five positions ranging from strongly agree to strongly disagree. 3.3. User experience questionnaires (UEQ) The emoji-based form employed in this study can be regarded as a commercially oriented product due to its practical utility. In the contemporary competitive market landscape, the attainment of an outstanding user experience is imperative for the commercial success of any product. Therefore, this research study utilized user experience questionnaires to evaluate the user experience facilitated by the emoji-based form. Our survey instrument presented questions and responses structured according to the UEQ scale framework. The UEQ template consists of a five-point scale comprising 26 items designed to assess various aspects of user experience, including perspicuity, efficiency, dependability, novelty, stimulation, and attractiveness.
The following section presents the findings obtained from assessing the effectiveness of integrating emojis into surveys conducted during offline Open Day. 4.1. The Cronbach’s alpha consistency (CAC) According to Bonnet and Wright (2014), ‘Cronbach’s alpha’ is a metric used to assess the reliability of a scale. It is employed to gauge the consistency of a set of items in measuring the same attribute. This statistical measure, ranging from 0 to 1, quantifies the degree of agreement. Higher values signify a stronger consensus among the items (Frost, 2023). In Table 1, all values exceeding 0.7 indicate a notable level of consistency across all six UEQ measurement items and their respective responses.
In Figure 3, there are some data details provided in Table 2. The Emoji survey form assessment does not contain any misinterpretation of items, as all of the measured items are greater than zero. Furthermore, the items on this scale have an uplifting positive mean and none of them have a negative mean. Therefore, the items in this context are not problematic. Table 2 has a detailed illustration of the depiction found in Figure 3, clarified by the adjacent legend. Any confusion about the Emoji survey assessment was avoided since every measured item has a value greater than 0. The high positive mean values of all items on the scale make it apparent that there was no negative interpretation of any item. This shows that the item performed well in the given context.
4.3. Emoji survey form UX compare to benchmark As per the study by Schrepp and Thomaschewski (2017), the User Experience (UX) data gathered from the Emoji survey form incorporated the responses of 250 participants who attended the Open Day events. This data was then compared against the benchmark intervals for the UEQ scales, which were established in the study written by Schrepp, Hinderks, and Thomaschewski (2017). The findings from this comparison are detailed in Table 3, providing a thorough measurement of the scale. Table 3 displays the benchmarked results for the UX quality of the Emoji survey form across six distinct variables. Novelty exhibited an excellent score, while Perspicuity and Stimulation both received high scores. However, the use of the emoji-based survey form showed areas that could benefit from improvements, such as Attractiveness, Efficiency, and Dependability.
continues to progress, the thoughtful integration of emojis holds the potential to yield substantial benefits, elevating communication and offering an appealing avenue for further exploration and innovation.
7.1. Limitations The survey conducted in this study had some limitations in terms of sample size, as the number of participants is still relatively small. Because Open Day was a one-day affair, and the participant pool was relatively limited, thus affecting the precision of the data collected. Besides, the translation of the 26 terms from the UEQ questionnaire from English to Vietnamese posed certain challenges for the participants in our research study, as we encountered difficulties in selecting the most appropriate wording for the translation. Additionally, despite our efforts to formulate concise questions and answers to streamline the survey process, the substantial number of questions could still lead participants to perceive the survey as time-consuming, potentially impacting the quality of responses obtained. 7.2. Future Study Subsequent research endeavors may consider employing supplementary methodologies, such as interviews, to gain a more profound understanding and in-depth analysis of user experiences associated with emoji-based forms. Additionally, conducting large-scale tests involving a significant number of participants can be instrumental in mitigating potential errors and enhancing the reliability of the findings.
[1] Walther, J. B. (2001). The Impacts of Emoticons on Message Interpretation in Computer-Mediated Communication. Social Science Computer Review. https://doi.org/10.1177/ [2] Donovan, J. L., Hamdy, F. C., Lane, J. A., Mason, M., Metcalfe, C., Walsh, E., Blazeby, J. M., Peters, T. J., Holding, P., Bonnington, S., Lennon, T., Bradshaw, L., Cooper, D., Herbert, P., Howson, J., Jones, A., Lyons, N., Salter, E., Thompson, P., … Neal, D. E. (2016). Patient-Reported Outcomes after Monitoring, Surgery, or Radiotherapy for Prostate Cancer. New England Journal of Medicine, 375(15), 1425–
[3] Annamalai, S., & Abdul Salam, S. N. (2017). Undergraduates’ Interpretation on WhatsApp Smiley Emoji. Jurnal Komunikasi, Malaysian Journal of Communication, 33(4), 89–103. https://doi.org/10.17576/jkmjc-2017-3304- [4] Rodrigues, I. B., Adachi, J. D., Beattie, K. A., & MacDermid, J. C. (2017). Development and validation of a new tool to measure the facilitators, barriers and preferences to exercise in people with osteoporosis. BMC Musculoskeletal Disorders, 18(1). https://doi.org/10.1186/s12891-017-1914- [5] Barbieri, F., Kruszewski, G., Ronzano, F., & Saggion, H. (2016). How Cosmopolitan Are Emojis? Proceedings of the 24th ACM International Conference on Multimedia. https://doi.org/10.1145/2964284. [6] Desmet, P., Overbeeke, K., & Tax, S. (2001). Designing Products with Added Emotional Value: Development and Appllcation of an Approach for Research through Design. The Design Journal, 4(1), 32–47. https://doi.org/10.2752/ [7] Jaeger, S. R., Lee, S. M., Kim, K.-O., Chheang, S. L., Jin, D., & Ares, G. (2017). Measurement of product emotions using emoji surveys: Case studies with tasted foods and beverages. Food Quality and Preference, 62, 46–59. https://doi.org/10.1016/j.foodqual.2017.05. [8] De Angeli, D., Kelly, R. M., & O’Neill, E. (2020). Beyond Happy‐or‐Not: Using Emoji to Capture Visitors’ Emotional Experience. Curator: The Museum Journal, 63(2), 167–191. Portico. https://doi.org/10.1111/cura. [9] Alismail, S., & Zhang, H. (2018). The Use of Emoji in Electronic User Experience Questionnaire: An Exploratory Case Study. Proceedings of the Annual Hawaii International Conference on System Sciences. https://doi.org/10.24251/hicss.2018. [10] Frost, J. (2022). Cronbach’s Alpha: A Measure of Internal Consistency and Reliability 21. Retrieved from https://statisticsbyjim.com/basics/cronbachs-alpha/ [11] Paredes, R. K., & Hernandez, A. A. (2017). Measuring the quality of user experience on web services: A case of university in the Philippines. 2017IEEE 9th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management (HNICEM). https://doi.org/10.1109/hnicem.2017. [12] Schrepp, M., Hinderks, A., & Thomaschewski, J. (2017). Construction of a Benchmark for the User Experience Questionnaire (UEQ). International Journal of
