The extent to which results are consistent over time and an accurate representation of the total population under study is referred to as reliability. In other words, if the results of a study can be reproduced under a similar methodology, then the research instrument is considered to be reliable.
Should you have a question that can be misunderstood, and therefore is answered differently by respondents, you are dealing with low reliability. The consistency with which questionnaire items are answered can be determined through the test-retest method, whereby a respondent would be asked to answer the same question(s) at two different times. This attribute of the instrument is actually referred to as stability. If we are dealing with a stable measure, then the results should be similar. A high degree of stability indicates a high degree of reliability, since the results are repeatable. The problem with the test-retest method is that it may not only sensitize the respondent to the subject matter, and hence influence the responses given, but that we cannot be sure that there were no changes in extraneous influences such as an attitude change that has occurred that could lead to a difference in the responses provided.
Probing for attitudes usually requires a series of questions that are similar, but not the same. This battery of questions should be answered consistently by the respondent. If it is, the instrument shows high consistency. This can be tested using the split-half method, whereby the researcher takes the results obtained from one-half of the scale items and checks them against the results of the other half.
Although the researcher may be able to prove the research instrument’s repeatability and internal consistency, and therefore reliability, the instrument itself may not be valid. In other words, it may consistently provide similar results, but it does not measure what the research proposed to determine.