Experimental evaluation of instrumentalcale readings obtained by eye interpolation of fractional interval values

Yutaka Tomita, Satoshi Honda

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)


This paper describes an experimental evaluation of the precision achieved by eye interpolation. It explains the effect on eye interpolation of differences in the position of a pointer. Fiducial lines-a fiducial line is the "0" or leftmost line on a vernier scale and serves as a coarse pointer-were set at different positions between 0 and 1 mm (0.05 mm intervals) using the vernier scales. The vernier scales were then covered with plastic tape. Thi vernier callipers were randomly given to 19 whbjects who were asked to read the position of the fiducial lines on the main scale, and to estimate them to the nearest tenth of a division. As a result of this experiment we found the following: (i) The bias and dispersion of readings just below the scale marks were the smallest. (ii) When the fiducial line were located at less than 0.5, the readings were biased toward 0, and when they were located at greater than 0.5, they were biased toward 1. (iii) Based on an analysis of variance involving the position of the fiducial line, the individual observers, and their interactions, the position of the fiducial line was found to be the most significant. (iv) Thus, personal errors in reading should be categorized together with instrumental errors, and could be reduced with maximum likelihood estimation.

Original languageEnglish
Pages (from-to)147-151
Number of pages5
Issue number2
Publication statusPublished - 1994 Apr


  • Analysis of variance
  • Calibration with maximum likelihood
  • Personal error
  • Space eye interpolation

ASJC Scopus subject areas

  • Instrumentation
  • Electrical and Electronic Engineering


Dive into the research topics of 'Experimental evaluation of instrumentalcale readings obtained by eye interpolation of fractional interval values'. Together they form a unique fingerprint.

Cite this