UNIVERSIDADE ESTADUAL PAULISTA
“JÚLIO DE MESQUITA FILHO”
Instituto de Ciência e Tecnologia
Campus de São José dos Campos
ORIGINAL ARTICLE DOI: https://doi.org/10.4322/bds.2023.e3819
1
Braz Dent Sci 2023 July/Sept;26 (3): e3819
Usefulness and satisfaction of a digital learning tool for caries
detection using ICDAS
Utilidade e satisfação de uma ferramenta digital de aprendizado para detecção de cárie utilizando ICDAS
Julia TONIOLO
1
, Rafael da Rosa GRASEL
1
, Jonas Almeida RODRIGUES
1
1 - Universidade Federal do Rio Grande do Sul, Faculdade de Odontologia, Departamento de Cirurgia e Ortopedia, Porto Alegre, RS, Brazil.
How to cite: Toniolo J, Grasel RR, Rodrigues JA. Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS.
Braz Dent Sci. 2023;26(3):e3819. https://doi.org/10.4322/bds.2023.e3819
ABSTRACT
Objective: this study aimed evaluate the usefulness and satisfaction of a digital learning tool (DLT) using ICDAS
(International caries detection and assessment system) created for learning and training visual detection of carious
lesions in a non-controlled setting. Material and Methods: the DLT containing 60 pictures of sound and carious
surfaces was distributed through social media and professional associations, aiming at both dentists and dental
students. Analysis was based on data collected through a personal, professional and satisfaction questionnaire
and Google Analytics. Results: a total of 2300 users accessed the DLT and 1517 completed it, lling out the
questionnaires. Users required 2.29 sessions to accomplish the DLT in an average time of 15 min and 14 seconds
each. The satisfaction questionnaire showed that 84.8% of users found in the DLT exactly what they expected and
91.8% found it useful. Around 88.4% of users found that the DLT helped in discussion of ICDAS with colleagues
and 88.9% with superiors; 88.3% found that the DLT provided immediate learning results. The DLT was most useful
to dental students (83.7% of users), and overall satisfaction was 86%. Conclusion: users found the DLT useful for
learning and training visual detection of carious lesions using ICDAS, and general satisfaction was high.
KEYWORDS
Dental Caries; Distance; Diagnosis; Education; ICDAS.
RESUMO
Objetivo: o objetivo deste estudo foi avaliar a utilidade e a satisfação de uma ferramenta digital de aprendizado (FDA)
utilizando o ICDAS (International caries detection and assessment system), criada para aprender e treinar a detecção
visual desse índice de forma virtual. Material e Métodos: a FDA contendo 60 fotos de superfícies hígidas e cariadas
foi distribuída por meio de mídias sociais, visando prossionais e estudantes de odontologia. A análise baseou-se nos
dados recolhidos através de um questionário de satisfação da experiência e Google Analytics. Resultados: um total de
2.300 usuários acessaram a FDA e 1.517 a completaram até os questionários. Os usuários precisaram de 2,29 sessões
para realizá-la em um tempo médio de 15 min e 14 segundos. O questionário de satisfação mostrou que 84,8% dos
usuários encontraram exatamente o que esperavam e 91,8% a acharam útil. Cerca de 88,4% dos usuários acreditaram
que a FDA ajudou na discussão do ICDAS com colegas e 88,9% com superiores; 88,3% acharam que foi fornecido
resultados de aprendizagem imediatos. A FDA foi mais útil para estudantes de odontologia (83,7% dos usuários) e a
satisfação geral foi de 86%. Conclusão: os usuários acharam a FDA útil para aprender e treinar a detecção visual de
lesões cariosas utilizando o ICDAS. A satisfação geral alcançada foi considerada alta.
PALAVRAS-CHAVE
Cárie Dentária; Distância; Diagnóstico; Educação; ICDAS.
2
Braz Dent Sci 2023 July/Sept;26 (3): e3819
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries
detection using ICDAS
INTRODUCTION
Detecting caries lesions is a key element
in preventing and treating the disease and is a
challenge in dentistry [1,2]. The disease’s slow
progression allows lesions to be detected early
and controlled, allowing patient management
though preventive measures, avoiding restorative
intervention [3,4]
The International Caries Detection and
Assessment System (ICDAS) was developed
by an international research group [5] aiming
to standardize the detection system of carious
lesions and to increase its sensitivity and
decrease its subjectivity, improving the method’s
reproducibility [6]. Its principle is that visual
examination should be conducted on clean
plaque-free surfaces and with drying of lesions
and surfaces to identify initial lesions [5,7]. The
system consists of seven scores that allow the
condition of a tooth surface to be numerically
classied and reect the severity of the disease.
Ünal et al. [8] indicated that the low kappa value
for examination using ICDAS may be explained
by the observers’ different clinical experiences
and inadequate calibration methods, which
highlights the importance of developing better
ICDAS training strategies.
The distance teaching modality (e-learning)
allows self-learning, which is mediated by
systematized and organized didactic resources
available in different forms of technological
support [9]. E-learning platforms may be
synchronous or asynchronous. Synchronous
content delivery occurs through real-time
simultaneous communication. In asynchronous
content delivery, the transmission and receipt
of information do not occur simultaneously;
learners are responsible for pacing their own self-
instruction and learning [10]. Blended learning
combines e-learning technology with traditional
instructor-led training [11].
The ICDAS group has developed an
e-learning tool [12]; it spans 90 minutes and
is available online in four different languages.
However, this e-learning tool does not include
an extensive training session. In the context of
blended learning, a digital learning tool (DLT)
with 60 photo-based questions was created and
has been evaluated through an in vitro [13] and a
clinical study [14]. These studies concluded that
the use of this digital resource could accelerate
pre-clinical training and increase the sensitivity of
dental students for detecting caries lesions even
in a low-prevalence child population, helping
in learning ICDAS scores. However, no study
has tested its usefulness and satisfaction with
regards to its use by dental students and dentists.
Therefore, the present study aimed to evaluate
the usefulness and satisfaction of a DLT using
ICDAS [13,14] (www.ufrgs.br/icdas) created for
learning and training visual detection of carious
lesions in a non-controlled setting.
MATERIAL AND METHODS
Development of the experimental DLT
The development and evaluation of the
DLT was approved by the local ethics committee
(#502.250).
The DLT has been reported in detail
previously [13,14] and consists of 60 questions
based on pictures of both permanent and primary
teeth. A question is assigned to each picture, being
questions in which the user has to provide the
score of a lesion indicated by an arrow, questions
on describing the presented lesion, simple or
multiple-choice questions, and true or false
questions. During testing, users had two chances
to answer a question correctly; after using their
chances, the user receives the question’s feedback.
The ICDAS scores and the progress report could
be accessed at any given moment.
The DLT was programmed with a Yii 2.0
framework, converted to a HTML site, and
hosted at the following address: www.ufrgs.br/
icdas and registered uned the license CREATIVE
COMMONS 3.0.
The initial pages of the DLT describes the
technical instructions and its resources, followed
by the ICDAS scores table. While accessing the
DLT, the user could have access to the ICDAS
scores table at any moment by clicking on an
icon. Users had two chances to correctly respond
to each question; after that, they were required to
move on to the next question; feedback with an
explanation of the answer became available after
the user responds correctly or wrongly twice to a
question. At any moment the users could view a
list with their progress, where the questions that
were answered correctly are highlighted in green
and the wrong answers are highlighted in red.
Users could also leave the DLT at any moment
by clicking “
log out
,” so they could return later
to the same question at which they had stopped.
3
Braz Dent Sci 2023 July/Sept;26 (3): e3819
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries
detection using ICDAS
After the 60 questions had been answered, the
users were invited to answer a questionnaire [15]
about their satisfaction with the DLT. Only after the
user had completed the questionnaire, a certicate
with the nal percentage of correct answers became
available (Figure 1).
Evaluation of the DLT
For this study the DLT was marketed through
social media and professional associations aiming
at both dentists and dental students. At this rst
moment, this DLT is available in Portuguese, and
mostly destined to users in Brazil.
Data on age, sex, degree (student, dentist,
specialist, master, PhD), university, service
(public, private), was collected, which were
available on the website administrative page. After
logging in, users completed a form with personal
and professional questions to characterize the
sample; Google Analytics data on the number of
users, number of sessions, and the average time
were also used to address usefulness.
Users were invited to complete a satisfaction
questionnaire available at Google forms after
they had completed the DLT, which unlocked
their certificate. A modified Wang [15]
questionnaire related to content, learner interface,
personalization, and learner community was
applied. The original Wang [15] questionnaire is
composed of 26 questions; there were 17 questions
in its modified final version, as we used the
questions that were most relevant to the present
study. Users rated the questions in a Likert scale,
with 1 and 5 indicating the lowest and highest
level of satisfaction with the item described,
respectively.
RESULTS
Data was collected from June 2019 through
July 2022. A total of 2300 users accessed the
DLT and 1517 completed it, filling out the
questionnaires. Most users identied themselves
as female (1738; 75.5%) and 557 (24.2%) as
male. Out of the 2300 users that began the
DLT, 83.7% were undergraduate students and
16.2% were dentists (8.7% were clinicians and
specialists, 2.8% were master’s degree holders,
and 4.6% hold PhD degrees). In Figure 2,
specialists were included in the “dentists” group.
Students showed the highest number of nishing
Figure 1 - Diagram of the DLT for ICDAS.
4
Braz Dent Sci 2023 July/Sept;26 (3): e3819
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries
detection using ICDAS
learners (1364; 89.9%) and the highest number
of respondents to the satisfaction questionnaire.
The total evasion rate was 34%, and we
considered it as users that did not complete all
60 questions. The evasion rate per group was:
students, 29.1%; dentists (including specialists),
47.7%; dentists with a master degree, 65.1% and
dentists with a PhD, 76.6%. Comparing Google
Analytics data and data collected by our site,
1574 users accessed the website but did not feel
compelled to engage in the DLT.
The average session time per user was
15 minutes and 14 seconds, and users used
and average of 2.29 sessions to nish the DLT.
This shows that, once a learner had logged on
to the site, they would stay active for this time
and then proceed to do something else on their
computer before going back to more 15 minutes
and 14 seconds in the DLT.
In the satisfaction questionnaire [15] in
Table 1, users were asked to rate questions on a
scale from 1 to 5, with 5 indicating the highest level
of satisfaction regarding the matter in question.
Each question in Wang’s questionnaire represents
one of four factors responsible for e-learner
satisfaction.
DISCUSSION
The most important outcome of this study
was developing a modernized and accessible
DLT to complement ICDAS e-learning. Given
the number of people we reached with our
DLT (n = 3874), we hope that e-learning
continues to have growing impact. This number
represents the total of users who accessed our
website, searching for online training in ICDAS.
This highlights a great necessity for a tool to
complement formal university ICDAS training.
Since we created an asynchronous platform t to
be easily modernized, and based on the results
of the present investigation, there would not be
operational difculty in translating and validating
it to other languages.
Undergraduate dental students seemed to
be the group most interested in accessing and
using the DLT. Other than that, higher-level
professionals appeared to think that the DLT is
less interesting, as dentists were our smallest
audience. In the prole area of the tool, students
may indicate their current period of dentistry
school; when a large number of students in
the same course period responded to the DLT
consecutively, it led us to think that a professor
Figure 2 - The number of users who reported a specific area of expertise.
5
Braz Dent Sci 2023 July/Sept;26 (3): e3819
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries
detection using ICDAS
was using the DLT as blended learning, so
it has the function of an extra-class activity
for undergraduate dental students to train
their diagnostic abilities. Frehywot et al. [16]
evaluated several studies comparing blended
learning with traditional approaches and found
that blended learning yielded either promising
results or no statistically signicant difference
in outcomes between the different modalities.
This evaluation also indicated that blended
learning is effective for conveying medical
knowledge and developing practical competences,
for example, detecting caries lesions and scoring
them. We opted for releasing certicates only
after the user responded to the satisfaction
questionnaire as a way of stimulating them to
respond to it. As reported by others [17], it is
easier to motivate learners to gain knowledge
than to contribute to a research study. It could
be speculated that many undergraduate dental
students who used the DLT were encouraged
by their professors to use it as an extra-class
activity, and therefore, their professors may
have asked them to submit the nal certicate
as conrmation of accomplishment.
Rocha et al. (2021) [18] compared
different teaching methodologies for the
training of radiographic detection of proximal
carious lesions. The authors observed that
students preferred active or a hybrid (blended)
methodology, reporting to be easy to learn,
making themselves feel more confident in
performing the trained diagnosis. A study
conducted by Maatuk et al. (2021) [19] stated
that students agreed that e-learning is useful and
helped them to be condent and improved their
academic standards. However, they reported
to be worried that this fact could decrease the
workload for teaching staff and could raise the
pressure on students. These ndings are similar
to ours, since students found that our DLT
helped them on learning, gave them immediate
results and t their needs.
We consider the 34% of evasion rate
moderate. This may be due to the length of
the tool, which was comprised of 60 questions
that were previously tested in the controlled
environments [13,14]. Also, the user interface
factor scored the lowest in our satisfaction
questionnaire (79.2%). It is important to state
that both previously published studies using the
DLT [13,14] were conducted in a controlled
setting, so all users who participated in the
study had to nish their training. In the present
investigation conducted in a non-controlled
setting, since 1574 people who visited the site
did not feel motivated to start the DLT, it could
be suggested the need to invest a greater effort
in better design and in a more attractive layout.
The users of the present DLT averaged
15 minutes and 14 seconds to complete the
tool; this represents the average time a user was
actively interacting with the site, responding to
the questions, reading feedback, and checking the
ICDAS table. This was expected, as it is known
that the average student has an attention span
between 10 and 20 minutes [18]. Users that
nished the tool had great interest in doing so,
as they would return even after taking a break.
We may also infer that a shorter DLT might have
a lower evasion rate. Reducing the number of
questions should be considered, however this
could be risky, as the learning efcacy of this
DLT has been assessed by two studies [13,14],
using the same number of pictures and the exact
number and format of questions.
E-learning tools have their limitations.
To assess the activity status of a caries lesion,
three factors need to be considered when a
carious lesion is examined visually and gently
probed: plaque accumulation, visual appearance
(opacity), and tactile feeling (roughness) [7].
In this context, activity assessments would be
hardly possible using any DLT.
According to traffic data of the website
compiled by Google Analytics, access to the tool
peaked in the beginning of quarantine in Brazil
due to the Covid-19 pandemic. On March 11,
2020, the World Health Organization (WHO)
declared the coronavirus (COVID-19) outbreak a
global pandemic. In the period of only a month
after the beginning of quarantine in Brazil we
had 96 new users, which represents 30% of the
number of users of a whole semester in 2019.
The DLT to complement the ICDAS training
that is available online had greater reception
and interest from the public than expected.
We conclude that this DLT, in the present
format, is mostly useful for teaching ICDAS in
dental schools as a tool for blended learning.
This exploratory study has opened our eyes
to usability issues we could not perceive in
previous controlled studies. In this manner, we
can continue to improve this DLT to better t
e-learners’ needs and hopes.
6
Braz Dent Sci 2023 July/Sept;26 (3): e3819
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries
detection using ICDAS
CONCLUSION
In a non-controlled setting, the DLT is useful
for dentists and dental students for the visual
detection of carious lesions with ICDAS, and
global satisfaction was high.
Author’s Contributions
JT: Conceptualization, Methodology,
Software, Validation, Formal Analysis, Investigation,
Resources, Writing – Original Draft Preparation,
Funding Acquisition. RRG: Software, Data curation,
Writing – Review & Editing, Formal Analysis.
JAR: Funding Acquisition, Supervision, Project
Administration.
Conict of Interest
No conicts of interest declared concerning
the publication of this article.
Funding
This project was possible thanks to the
financial support of the Office of Distance
Learning (SEAD) and technical support from
the Pedagogical Support for Distance Learning
Nucleus (NAPEAD), Federal University of
Rio Grande do Sul (UFRGS).
Regulatory Statement
This study was conducted in accordance with
all the provisions of the local human subjects
oversight committee guidelines and policies of:
Local Human Research and Ethics Committee.
The approval code for this study is: #502.250.
REFERENCES
1. Bader JD, Shugars DA. A systematic review of the performance
of a laser fluorescence device for detecting caries. J Am Dent
Assoc. 2004;135(10):1413-26. http://dx.doi.org/10.14219/jada.
archive.2004.0051. PMid:15551982.
2.
Diniz MB, Rodrigues JA, Hug I, Cordeiro Rde C, Lussi A. Reproducibility
and accuracy of the ICDAS-II for occlusal caries detection.
Community Dent Oral Epidemiol. 2009;37(5):399-404. http://dx.doi.
org/10.1111/j.1600-0528.2009.00487.x. PMid:19681984.
3. Pretty IA, Maupomé G. A closer look at diagnosis in clinical dental
practice: part 5. Emerging technologies for caries detection and
diagnosis. J Can Dent Assoc. 2004;70(8):540. PMid:15363214.
4. Bader JD, Shugars DA. The evidence supporting alternative
management strategies for early occlusal caries and suspected
occlusal dentinal caries. J Evid Based Dent Pract. 2006;6(1):91-100.
http://dx.doi.org/10.1016/j.jebdp.2005.12.004. PMid:17138407.
5.
Ismail AI, Sohn W, Tellez M, Amaya A, Sen A, Hasson
H, et al. The International Caries Detection and Assessment
System (ICDAS): an integrated system for measuring dental
caries. Community Dent Oral Epidemiol. 2007;35(3):170-8.
http://dx.doi.org/10.1111/j.1600-0528.2007.00347.x. PMid:17518963.
6.
Braga MM, Oliveira LB, Bonini GA, Bönecker M, Mendes FM. Feasibility
of the International Caries Detection and Assessment System
(ICDAS-II) in epidemiological surveys and comparability with standard
World Health Organization criteria. Caries Res. 2009;43(4):245-9.
http://dx.doi.org/10.1159/000217855. PMid:19439944.
Table 1 - Modified Wang questionnaire and percentage of users that rated each question with 5
Question Factor
% of users that rated
the question with 5
Q1. The e-learning system is easy to use. Learner interface 86.8%
Q2. The e-learning system is user-friendly. Learner interface 79.2%
Q3. Content presented through the platform is easy to understand. Learner interface 80.5%
Q4. Operating system is stable. Learner interface 85.9%
Q5. The e-learning system makes it easy to find the content needed. Learner interface 83.1%
Q6. The e-learning system makes it easy to discuss questions with other students. Learner community 88.4%
Q7. The e-learning system makes it easy to discuss questions with teachers. Learner community 88.9%
Q8. The e-learning system makes it easy to share what you learn with other learners. Learner community 86.1%
Q9. The e-learning system provides up-to-date content. Content 90.4%
Q10. The e-learning system provides content that fits your needs exactly. Content 84.8%
Q11. The e-learning system provides useful content. Content 91.8%
Q12. Operating system provides learners a safe place to train. Learner interface 90%
Q13. The e-learning provides immediate results. Content 88.3%
Q14. The e-learning system allows you to learn the content needed. Personalization 84.8%
Q15. The e-learning system enables you to control your learning progress. Personalization 84.2%
Q16. The e-learning system usefully records learning progress and performance. Personalization 84.6%
Q17. Overall, what is your satisfaction rate regarding the e-learning? - 86%
7
Braz Dent Sci 2023 July/Sept;26 (3): e3819
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries detection using ICDAS
Toniolo J et al.
Usefulness and satisfaction of a digital learning tool for caries
detection using ICDAS
7. Ekstrand KR, Martignon S, Ricketts DJ, Qvist V. Detection
and activity assessment of primary coronal caries lesions:
a methodologic study. Oper Dent. 2007;32(3):225-35.
http://dx.doi.org/10.2341/06-63. PMid:17555173.
8. Ünal M, Koçkanat A, Güler S, Gültürk E. Diagnostic performance
of different methods in detecting incipient non-cavitated
occlusal caries lesions in permanent teeth. J Clin Pediatr Dent.
2019;43(3):173-9. http://dx.doi.org/10.17796/1053-4625-43.3.5
PMid:30964722.
9. Romiszowski A. O futuro de e-learning como inovação
educacional: fatores influenciando o sucesso ou fracasso
de projetos. Revista Brasileira de Aprendizagem Aberta e a
Distância. 2003;2:1-15.
10. Ruiz JG, Mintzer MJ, Leipzig RM. The impact of E-learning in
medical education. Acad Med. 2006;81(3):207-12. http://dx.doi.
org/10.1097/00001888-200603000-00002. PMid:16501260.
11. Masie E. Blended learning: the magic is in the mix. In: Rosset A,
editor. The ASTD ELearning Handbook. New York: McGraw-Hill;
2002. p. 58-63.
12.
Topping GVA, Hally J, Bonner B, Pitts NB. International caries detention
and assessment system (ICDAS) e-learning package. Interactive CD
ROOM and web based software. London: Smile on; 2008.
13. Alves LS, de Oliveira RS, Nora ÂD, Cuozzo Lemos LF, Rodrigues
JA, Zenkner JEA. Dental students’ performance in detecting in
vitro occlusal carious lesions using ICDAS with e-learning and
digital learning strategies. J Dent Educ. 2018;82(10):1077-83.
http://dx.doi.org/10.21815/JDE.018.100. PMid:30275142.
14. Luz PB, Stringhini CH, Otto BR, Port AL, Zaleski V, Oliveira
RS,et al. Performance of undergraduate dental students on
ICDAS clinical caries detection after different learning strategies.
Eur J Dent Educ. 2015;19(4):235-41. http://dx.doi.org/10.1111/
eje.12131. PMid:25495379.
15. Wang Y. Assessment of learner satisfaction with asynchronous
electronic learning systems. Inf Manage. 2003;41(1):75-86.
http://dx.doi.org/10.1016/S0378-7206(03)00028-4.
16. Frehywot S, Vovides Y, Talib Z, Mikhail N, Ross H, Wohltjen
H,etal. E-learning in medical education in resource constrained
low- and middle-income countries. Hum Resour Health.
2013;11(1):4. http://dx.doi.org/10.1186/1478-4491-11-4.
PMid:23379467.
17. Oliveira AC, Mattos S, Coimbra M. Development and assessment
of an e-learning course on pediatric cardiology basics. JMIR Med
Educ. 2017;3(1):e10. http://dx.doi.org/10.2196/mededu.5434.
PMid:28490416.
18. Rocha BC, Rosa BSP, Cerqueira TS, de-Azevedo-Vaz SL, Barbosa
GLR, Ferreira LM,etal. Evaluation of different teaching methods
in the radiographic diagnosis of proximal carious lesions.
Dentomaxillofac Radiol. 2021;50(4):20200295. http://dx.doi.
org/10.1259/dmfr.20200295. PMid:33141626.
19. Maatuk AM, Elberkawi EK, Aljawarneh S, Rashaideh H, Alharbi
H. The COVID-19 pandemic and E-learning: challenges
and opportunities from the perspective of students and
instructors. J Comput High Educ. 2022;34(1):21-38http://dx.doi.
org/10.1007/s12528-021-09274-2. PMid:33967563.
Rafael da Rosa Grasel
(Corresponding address)
Universidade Federal do Rio Grande do Sul, Faculdade de Odontologia,
Departamento de Cirurgia e Ortopedia, Porto Alegre, RS, Brazil.
Email: rgrasel0@gmail.com
Date submitted: 2023 March 01
Accepted submission: 2023 July 17
