Introduction

The main aim of this trial was to evaluate whether a teledentistry system would be a valid method of screening new patient orthodontic referrals and whether this system would reduce the inappropriate referral rate. The validity of such a system is important because clinical decision making through electronic means may fall below established standards of care.1 Furthermore, there is a lack of research investigating the validity of telemedicine applications2,3 despite a number of publications outlining principles of its set-up and evaluation.4,5,6,7

This trial proposed to take a step back from full diagnosis and treatment planning via teledentistry and planned only to screen new patients for the appropriateness of their referral. A screening system was used because, on average, the wait for a new patient consultation was 4.6 months (range 0-24 months).8 Additionally, 20-45% of patient referrals are inappropriate and increase waiting times unnecessarily.9,10

Common reasons for inappropriate referrals were reported as mild malocclusion, referral too early, and poor oral hygiene.9 GDPs may often also feel pressurised by parents to refer this type of patient for an opinion, although they themselves realise that the patient is unlikely to receive treatment. The factors outlined above should be easy to assess from clinical photographs and an orthodontic opinion may be obtained quickly thus reassuring patients without the need for a hospital visit. As a result, a 'store and forward' email–based teledentistry system to screen new patient referrals became the focus of this study.

Sample

Background information

An initial study was carried out to determine which dental practices referred more than seven patients per year to one of two hospital departments (Hope Hospital, Salford, and University Dental Hospital of Manchester). Thirty-five practices were identified who referred on average 19 patients (range 8-17). The inappropriate referral rate for all referrals to Hope Hospital over one year was found to be 38%.

Sample size calculation

The sample size was based on the proportion of inappropriate referrals decreasing from 38% to 10% during the trial (0.38-0.10). A sample of 15 referrals (on average) per year from each of 16 practices (eight in the test and eight in the control groups) would have >90% power to detect this reduction, assuming that the intra-class correlation coefficient was 0.05. This resulted in a total sample size for the study of 240 referrals over a one year period.

Method

Practice recruitment

The 35 practices referring more than seven patients per year were approached to take part in the trial. Sixteen practices responded, however, one practice in the control group withdrew before the trial started.

Random allocation method

Dental practices and thus GDPs within practices were randomly allocated, using a random-number table, to teledentistry test group or control. Agreement to participate in the trial was obtained before random allocation to ensure that GDPs did not agree to participate based on the computer equipment donated for the trial.

Interventions and their timing

Written parent and child consent was obtained by the GDPs for all children participating in the trial.

i) Teledentistry test group

The following patient information was sent to the consultant orthodontist via email with the photographs downloaded from a digital camera as JPEG files.

  • Patient name, date of birth and gender

  • Patient complaint, medical and dental history

  • Overjet (mm)

  • Presence of mandibular displacement on closure

  • Extra oral frontal and lateral facial photographs

  • Intra oral anterior, right and left buccal photographs in occlusion

  • Any available radiographs were mounted on a light box and photographed.

Thus the teledentistry method used in this study was 'store and forward'. The orthodontist received an email with attached photographs that could be read outside and away from the clinic. GDPs were trained onsite by our IT consultant and were also given written instructions. IT support was given during the study by telephone and, in two cases, by a practice visit.

ii) Control group

Once this information had been forwarded, GDPs also referred the patient by conventional referral letter to ensure that the patient was also seen on a new patient clinic. The same orthodontist who assessed the email referral saw the patient on clinic a minimum of eight weeks later. Patients in control practices were referred to the orthodontist by letter in the conventional way.

Patient confidentiality

Access to patient records on the teledentistry system was limited to the two consultant orthodontists participating in the trial. In addition to the usual password protection, the data were encrypted with a double password protected encryption key.

Outcomes

The primary outcome was whether the orthodontist would accept or not accept an orthodontic referral based on the teledentistry referral and the clinical examination. These two decisions were compared to assess the validity of the teledentistry system. The following secondary outcomes were compared between the test and control groups:

  • Number of referrals

  • Inappropriate referral rate

  • Failure to attend.

If a patient cancelled or failed to attend an appointment but then subsequently attended, the primary outcomes and the inappropriate referral outcome were recorded and included in the data analysis.

Data collection

Teledentistry data

The orthodontist recorded whether or not they would accept the patient as a new patient referral using the teledentistry system. The reason for not accepting a referral was recorded in both instances. The definition of inappropriate referral used in this study was:

  • Poor oral hygiene

  • Mild malocclusion

  • Referral too early

  • Other – including patient did not want treatment, suitable for a specialist, adult requiring routine orthodontics.

Control group

Clinical data were recorded for all patients in either test or control group. This consisted of attendance and the outcome of the appointment and therefore whether the patient referral would be accepted. In addition, for inappropriate referrals the reason was recorded as previously outlined.

Examiner blinding

The orthodontists were blinded to whether patients were in the test or control group to minimise any bias in recording clinical outcomes. Blinding was done as follows:

  • Test practices were given an unique identification number so subsequent referral letters could not be linked to test or control group.

  • GDPs were asked not to indicate any involvement with the trial on the referral letters.

Memory bias

Taylor5 suggested that the same observers should view the cases in the test group and the effects of learning/remembering cases counteracted by having the clinic a couple of months after the teledentistry system is used. This occurred as a result of natural new patient waiting lists and no patient was seen on a clinic within two months of their electronic referral. In addition, trial patients on clinics were integrated at random with non-trial patients.

Statistics

Simple summary statistics were produced for age, gender and number of referrals in test and control groups. Sensitivity and specificity and kappa values were calculated to assess the validity of the teledentistry system. Failed attendances were compared between test and control using chi square statistic. Inappropriate referral rates were compared using generalised estimating equations (GEE) with a logit link and an exchangeable correlation matrix. These are an extension of logistic regression analysis, which allowed for clustering of patient referrals within practices.

Results

After 15 months, 80 patients had been referred in the teledentistry group and 247 in the control group. This sample had sufficient power to detect statistically significant differences between test and control groups for inappropriate referral rates.

Figure 1 summarises the number of practices and patients participating in the trial. The mean age and gender of the patients is summarised in Table 1 and there were no statistically significant differences between test and control groups for these variables.

Figure 1
figure 1

Number of practices and patients participating in the trial

Table 1 Age and gender of patients in the teledentistry trial

Validity of the teledentistry system

A cross tabulation of the decision to accept a referral for the teledentistry group is shown in Table 2. The sensitivity of the system to screen referrals (true positive or (yes/yes agreement) was 0.80. The specificity of the system (true negative or no/no agreement) was 0.73. Appendix 1. This showed that teledentistry was valid for screening orthodontic referrals. In other words, the decision whether to accept a referral based on teledentistry information was similar to seeing the patient face to face. However, great care still needs to be taken if a patient is refused an appointment because the screening system suggests they should not be seen. Such a patient may not receive full orthodontic assessment when they should have received it. This potentially occurred in 11 cases during this trial.

Table 2 The inappropriate referral rate in the teledentistry and control groups where patients who would have been accepted as a result of the teledentistry system were selected

Thus the positive predictive value of the teledentistry system to screen new referrals was high at 0.92. However the negative predictive value was lower at 0.50. This reflected the finding that half of the 22 children not accepted on the teledentistry system, would have been accepted when seen on clinic (Appendix 1).

This important caveat is reflected in the Kappa score, was calculated as 0.46 (standard error 0.12) and reflects the clinician agreement between the decision to accept a new patient based on teledentistry versus conventional clinical information. A score of 0.46 reflects moderate agreement.10,11

Inappropriate referral rate

Patients who would have been accepted as new patient referrals via the teledentistry system were selected and compared with the control group (Table 2). The inappropriate referral rate was lower in the test group (8.2%) compared with the control group (26.2%) (P = 0.037).

Hospital appointment attendance

The data for failed and cancelled appointment were combined into a non-attendance variable. Comparison of attendance between test and controls is shown in Table 3. There was no difference between attendance for the first appointment between teledentistry and control groups (P = 0.36).

Table 3 Comparison of first appointment attendance between teledentistry and control groups

Discussion

The results of this study suggest that teledentistry is a good way of positively identifying patients who should be referred to a consultant orthodontist. Therefore, a valid, efficient and time saving clinical screening method has been identified. These results support the findings of Davies and Stephens.11,12 They found no discernable differences in intra-examiner consistency for treatment need, extraction decision or type of appliance when using hand-held records or video-conferencing technology.

However, our results did highlight a disadvantage with a teledentistry system in that some cases not accepted for care from electronic data were subsequently accepted as new patient referrals following a full clinical examination (11 out of 56 (19.6%) of patients). An explanation for this may have been that a patient could improve their oral hygiene between the teledentistry referral and being seen on a clinic, however, the results of this study do not support this hypothesis.

Importantly, a significant minority of patients may not be accepted as new patients after electronic screening, but may benefit from a full clinical examination. A way of addressing this problem could be to accept the new patient referrals in cases where there is uncertainty in accepting the patient based on teledentistry information. Thus, it would be beneficial not to be too selective when screening new patient referrals with this method.

The kappa score for intra-clinician agreement between teledentistry and clinical information was moderate but still higher than other reports of clinical decision-making.12,13,14,15 Therefore, it could be suggested that agreement between screening decisions made from teledentistry and clinical information is acceptable in the light of generally poor examiner agreement for orthodontic decision-making.

The number of patients referred and their attendance

At the start of the trial, one practice withdrew from the study and there could have been a number of possible reasons for this. The GDP may have felt he did not have enough time to participate — consent procedures can take 10 minutes per patient. Alternatively, staff training, having enough potential suitable referrals, or workload may have caused them to withdraw. Additionally, it was not as easy to give an incentive to GDPs in the control group, as they did not receive computer and camera equipment given to the test practices.

There were statistically significantly more patients referred in the control group for the district general hospital compared with the teaching hospital (Table 2). This cannot be explained by the number of practices in each group as there were only seven practices in the control group but eight in the teledentistry group. Other possible reasons might include:

  • The teledentistry system reducing the number of referrals because of extra clinical time required to take photographs and send the email referral.

  • A number of patients who did not participate in the trial were referred by practices in the teledentistry group. Unfortunately, as this only accounted for eight patients, it does not explain the large differences between groups.

  • One practice in the control group had five dentists and routinely referred more patients than the other practices.

When attendance is considered, we initially thought that patients agreeing to take part in the teledentistry aspect of the trial might be more motivated towards orthodontic treatment and, therefore, more likely to attend for their appointment. However, this hypothesis was not borne out in this trial.

Inappropriate referrals

The finding that the teledentistry system could potentially reduce the inappropriate referral rate to 8% is clinically important. Previous reports have suggested an inappropriate referral rate of around 45% but interestingly the rate in the control group in this study was only 26%. There is no reason to expect the control group data to be any different compared with previous literature. However, it is possible that because the control dentists knew they were taking part in a study, their referral pattern could have been different.

A statistically significantly lower inappropriate referral rate associated with teledentistry has widespread service implications. Previously up to 45% of clinic appointments may have been taken up with inappropriate referrals resulting in appropriately referred new patients waiting longer to be seen. There is thus a huge potential advantage to shorten new patient waiting times by screening out inappropriate referrals using teledentistry.

Conclusions

Teledentistry is a valid system for positively identifying appropriate new patient orthodontic referrals. However, importantly there is a risk that a patient is not accepted on the teledentistry system but would benefit from a full clinical examination. This highlights a potential flaw in the validity or accuracy of teledentistry to screen orthodontic referrals.

Teledentistry could potentially be a significant factor in reducing the inappropriate referral rate. This would significantly reduce waiting times for orthodontic referrals. Patient participation in a teledentistry system does not appear to mean they are more likely to attend their hospital appointment. Additionally, a reduction in number of referrals attributable to the use of the system cannot be reliably concluded. GDP perception of teledentistry was also evaluated and will be reported in the part II study. See Appendix 2 and 3.