1

Objective: this study aimed evaluate the usefulness and satisfaction of a digital learning tool (DLT) using ICDAS (International caries detection and assessment system) created for learning and training visual detection of carious lesions in a non-controlled setting. Material and Methods: the DLT containing 60 pictures of sound and carious surfaces was distributed through social media and professional associations, aiming at both dentists and dental students. Analysis was based on data collected through a personal, professional and satisfaction questionnaire and Google Analytics. Results: a total of 2300 users accessed the DLT and 1517 completed it, filling out the questionnaires. Users required 2.29 sessions to accomplish the DLT in an average time of 15 min and 14 seconds each. The satisfaction questionnaire showed that 84.8% of users found in the DLT exactly what they expected and 91.8% found it useful. Around 88.4% of users found that the DLT helped in discussion of ICDAS with colleagues and 88.9% with superiors; 88.3% found that the DLT provided immediate learning results. The DLT was most useful to dental students (83.7% of users), and overall satisfaction was 86%. Conclusion: users found the DLT useful for learning and training visual detection of carious lesions using ICDAS, and general satisfaction was high.


INTRODUCTION
Detecting caries lesions is a key element in preventing and treating the disease and is a challenge in dentistry [1,2].The disease's slow progression allows lesions to be detected early and controlled, allowing patient management though preventive measures, avoiding restorative intervention [3,4] The International Caries Detection and Assessment System (ICDAS) was developed by an international research group [5] aiming to standardize the detection system of carious lesions and to increase its sensitivity and decrease its subjectivity, improving the method's reproducibility [6].Its principle is that visual examination should be conducted on clean plaque-free surfaces and with drying of lesions and surfaces to identify initial lesions [5,7].The system consists of seven scores that allow the condition of a tooth surface to be numerically classified and reflect the severity of the disease.Ünal et al. [8] indicated that the low kappa value for examination using ICDAS may be explained by the observers' different clinical experiences and inadequate calibration methods, which highlights the importance of developing better ICDAS training strategies.
The distance teaching modality (e-learning) allows self-learning, which is mediated by systematized and organized didactic resources available in different forms of technological support [9].E-learning platforms may be synchronous or asynchronous.Synchronous content delivery occurs through real-time simultaneous communication.In asynchronous content delivery, the transmission and receipt of information do not occur simultaneously; learners are responsible for pacing their own selfinstruction and learning [10].Blended learning combines e-learning technology with traditional instructor-led training [11].
The ICDAS group has developed an e-learning tool [12]; it spans 90 minutes and is available online in four different languages.However, this e-learning tool does not include an extensive training session.In the context of blended learning, a digital learning tool (DLT) with 60 photo-based questions was created and has been evaluated through an in vitro [13] and a clinical study [14].These studies concluded that the use of this digital resource could accelerate pre-clinical training and increase the sensitivity of dental students for detecting caries lesions even in a low-prevalence child population, helping in learning ICDAS scores.However, no study has tested its usefulness and satisfaction with regards to its use by dental students and dentists.Therefore, the present study aimed to evaluate the usefulness and satisfaction of a DLT using ICDAS [13,14] (www.ufrgs.br/icdas)created for learning and training visual detection of carious lesions in a non-controlled setting.

Development of the experimental DLT
The development and evaluation of the DLT was approved by the local ethics committee (#502.250).
The DLT has been reported in detail previously [13,14] and consists of 60 questions based on pictures of both permanent and primary teeth.A question is assigned to each picture, being questions in which the user has to provide the score of a lesion indicated by an arrow, questions on describing the presented lesion, simple or multiple-choice questions, and true or false questions.During testing, users had two chances to answer a question correctly; after using their chances, the user receives the question's feedback.The ICDAS scores and the progress report could be accessed at any given moment.
The DLT was programmed with a Yii 2.0 framework, converted to a HTML site, and hosted at the following address: www.ufrgs.br/icdas and registered uned the license CREATIVE COMMONS 3.0.
The initial pages of the DLT describes the technical instructions and its resources, followed by the ICDAS scores table.While accessing the DLT, the user could have access to the ICDAS scores table at any moment by clicking on an icon.Users had two chances to correctly respond to each question; after that, they were required to move on to the next question; feedback with an explanation of the answer became available after the user responds correctly or wrongly twice to a question.At any moment the users could view a list with their progress, where the questions that were answered correctly are highlighted in green and the wrong answers are highlighted in red.Users could also leave the DLT at any moment by clicking "log out," so they could return later to the same question at which they had stopped.

Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS After the 60 questions had been answered, the users were invited to answer a questionnaire [15] about their satisfaction with the DLT.Only after the user had completed the questionnaire, a certificate with the final percentage of correct answers became available (Figure 1).

Evaluation of the DLT
For this study the DLT was marketed through social media and professional associations aiming at both dentists and dental students.At this first moment, this DLT is available in Portuguese, and mostly destined to users in Brazil.
Data on age, sex, degree (student, dentist, specialist, master, PhD), university, service (public, private), was collected, which were available on the website administrative page.After logging in, users completed a form with personal and professional questions to characterize the sample; Google Analytics data on the number of users, number of sessions, and the average time were also used to address usefulness.
Users were invited to complete a satisfaction questionnaire available at Google forms after they had completed the DLT, which unlocked their certificate.A modified Wang [15] questionnaire related to content, learner interface, personalization, and learner community was applied.The original Wang [15] questionnaire is composed of 26 questions; there were 17 questions in its modified final version, as we used the questions that were most relevant to the present study.Users rated the questions in a Likert scale, with 1 and 5 indicating the lowest and highest level of satisfaction with the item described, respectively.

RESULTS
Data was collected from June 2019 through July 2022.A total of 2300 users accessed the DLT and 1517 completed it, filling out the questionnaires.Most users identified themselves as female (1738; 75.5%) and 557 (24.2%) as male.Out of the 2300 users that began the DLT, 83.7% were undergraduate students and 16.2% were dentists (8.7% were clinicians and specialists, 2.8% were master's degree holders, and 4.6% hold PhD degrees).In Figure 2, specialists were included in the "dentists" group.The total evasion rate was 34%, and we considered it as users that did not complete all 60 questions.The evasion rate per group was: students, 29.1%; dentists (including specialists), 47.7%; dentists with a master degree, 65.1% and dentists with a PhD, 76.6%.Comparing Google Analytics data and data collected by our site, 1574 users accessed the website but did not feel compelled to engage in the DLT.

Students showed the highest number of finishing
The average session time per user was 15 minutes and 14 seconds, and users used and average of 2.29 sessions to finish the DLT.This shows that, once a learner had logged on to the site, they would stay active for this time and then proceed to do something else on their computer before going back to more 15 minutes and 14 seconds in the DLT.
In the satisfaction questionnaire [15] in Table 1, users were asked to rate questions on a scale from 1 to 5, with 5 indicating the highest level of satisfaction regarding the matter in question.Each question in Wang's questionnaire represents one of four factors responsible for e-learner satisfaction.

DISCUSSION
The most important outcome of this study was developing a modernized and accessible DLT to complement ICDAS e-learning.Given the number of people we reached with our DLT (n = 3874), we hope that e-learning continues to have growing impact.This number represents the total of users who accessed our website, searching for online training in ICDAS.This highlights a great necessity for a tool to complement formal university ICDAS training.Since we created an asynchronous platform fit to be easily modernized, and based on the results of the present investigation, there would not be operational difficulty in translating and validating it to other languages.
Undergraduate dental students seemed to be the group most interested in accessing and using the DLT.Other than that, higher-level professionals appeared to think that the DLT is less interesting, as dentists were our smallest audience.In the profile area of the tool, students may indicate their current period of dentistry school; when a large number of students in the same course period responded to the DLT consecutively, it led us to think that a professor Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS was using the DLT as blended learning, so it has the function of an extra-class activity for undergraduate dental students to train their diagnostic abilities.Frehywot et al. [16] evaluated several studies comparing blended learning with traditional approaches and found that blended learning yielded either promising results or no statistically significant difference in outcomes between the different modalities.This evaluation also indicated that blended learning is effective for conveying medical knowledge and developing practical competences, for example, detecting caries lesions and scoring them.We opted for releasing certificates only after the user responded to the satisfaction questionnaire as a way of stimulating them to respond to it.As reported by others [17], it is easier to motivate learners to gain knowledge than to contribute to a research study.It could be speculated that many undergraduate dental students who used the DLT were encouraged by their professors to use it as an extra-class activity, and therefore, their professors may have asked them to submit the final certificate as confirmation of accomplishment.

Rocha et al. (2021)
[18] compared different teaching methodologies for the training of radiographic detection of proximal carious lesions.The authors observed that students preferred active or a hybrid (blended) methodology, reporting to be easy to learn, making themselves feel more confident in performing the trained diagnosis.A study conducted by Maatuk et al. (2021) [19] stated that students agreed that e-learning is useful and helped them to be confident and improved their academic standards.However, they reported to be worried that this fact could decrease the workload for teaching staff and could raise the pressure on students.These findings are similar to ours, since students found that our DLT helped them on learning, gave them immediate results and fit their needs.
We consider the 34% of evasion rate moderate.This may be due to the length of the tool, which was comprised of 60 questions that were previously tested in the controlled environments [13,14].Also, the user interface factor scored the lowest in our satisfaction questionnaire (79.2%).It is important to state that both previously published studies using the DLT [13,14] were conducted in a controlled setting, so all users who participated in the study had to finish their training.In the present investigation conducted in a non-controlled setting, since 1574 people who visited the site did not feel motivated to start the DLT, it could be suggested the need to invest a greater effort in better design and in a more attractive layout.
The users of the present DLT averaged 15 minutes and 14 seconds to complete the tool; this represents the average time a user was actively interacting with the site, responding to the questions, reading feedback, and checking the ICDAS table.This was expected, as it is known that the average student has an attention span between 10 and 20 minutes [18].Users that finished the tool had great interest in doing so, as they would return even after taking a break.We may also infer that a shorter DLT might have a lower evasion rate.Reducing the number of questions should be considered, however this could be risky, as the learning efficacy of this DLT has been assessed by two studies [13,14], using the same number of pictures and the exact number and format of questions.

E-learning tools have their limitations.
To assess the activity status of a caries lesion, three factors need to be considered when a carious lesion is examined visually and gently probed: plaque accumulation, visual appearance (opacity), and tactile feeling (roughness) [7].In this context, activity assessments would be hardly possible using any DLT.
According to traffic data of the website compiled by Google Analytics, access to the tool peaked in the beginning of quarantine in Brazil due to the Covid-19 pandemic.On March 11, 2020, the World Health Organization (WHO) declared the coronavirus (COVID-19) outbreak a global pandemic.In the period of only a month after the beginning of quarantine in Brazil we had 96 new users, which represents 30% of the number of users of a whole semester in 2019.
The DLT to complement the ICDAS training that is available online had greater reception and interest from the public than expected.We conclude that this DLT, in the present format, is mostly useful for teaching ICDAS in dental schools as a tool for blended learning.This exploratory study has opened our eyes to usability issues we could not perceive in previous controlled studies.In this manner, we can continue to improve this DLT to better fit e-learners' needs and hopes.
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS

CONCLUSION
In a non-controlled setting, the DLT is useful for dentists and dental students for the visual detection of carious lesions with ICDAS, and global satisfaction was high.
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS

Figure 1 -
Figure 1 -Diagram of the DLT for ICDAS.

Figure 2 -
Figure 2 -The number of users who reported a specific area of expertise.