Approaches Used in Scholarship
Ephraim Segerman

The purpose of scholarship is to generate knowledge. Approaches to doing it need not depend on the field of study, as long as they have the same set of elements. These elements are:
1) questions of desired knowledge that are asked
2) theories or speculations (generalisations that go beyond those that are immediately obvious from the evidence) that are postulated as candidates for answering these questions, and
3) pieces of evidence that are collected and are relevant to the question.

Following are three approaches I have observed in music history scholarship, here called A. B, and C, but can apply to all fields. They are somewhat idealised, and few individuals will consistently follow just one of these stereotypes.

A The Evidence-Up approach (used by scholars for whom 'facts' are what one can safely believe, and 'speculations' are what one cannot):
1) Collect an amount of evidence, compare it with similar evidence previously reported, and present it in a clear and useful way. Any individual piece of evidence may be in error and so is distrusted, with trust given only if it is 'confirmed' by other independent evidence. Trusted evidence, and the obvious generalisations from it, are the basic 'facts' of knowledge, and can be believed.
2) Speculations that go beyond the 'facts' and can explain them are encouraged, and all are welcome. How convincing a speculation is depends on how well it fits into what one already 'knows', how much support it has from the evidence, and how convincing one's respected colleagues find it. There is no public debate on acceptance or rejection since there are no objective criteria, so this is a private subjective process.
3) Whenever there is consensus amongst the respected leaders in the field that a speculation is correct, perhaps because it remains unchallenged for some time, it becomes knowledge that can be believed, becoming 'facts'.
4) Questions to which there are no answers accepted by the consensus, and evidence that is not believed, are deemed 'mysteries'.

B The Theory-Down approach (used by those who believe that their insights are knowledge):
1) A theory that answers a question of desired knowledge appears as a powerful self-evident insight.
2) Collect all of the evidence that can be interpreted to support that theory, and don't believe the evidence that doesn't.
3) Promote the theory in publications (preferably full of impressive scholarly apparatus such as footnotes, quotes, translations, tables, etc) showing how well it is supported by evidence. The pieces of evidence that are contrary to the theory, if mentioned at all, are discredited in any way one can. They are often deemed to be 'mysteries'.

C The Evidence-Control approach (used by those who distrust belief but trust the evidence, and who want to extract the maximum amount of knowledge from the evidence available):
1) Formulate a question, explore competing theories to answer the question, and collect all of the available relevant evidence (there is no particular order in doing these).
2) For each theory, try to provide a reasonable explanation for each piece of evidence.
3) Any theory that cannot offer a reasonable explanation for how every piece of evidence became what it is, is rejected (the theory is 'falsified' by contrary evidence). Amongst the remaining theories, one is considered superior to another if its least reasonably probable explanation for any piece of evidence is more reasonably probable than the least reasonably probable explanation for any piece of evidence by the other theory. The most superior theory is considered to be the knowledge (with respect to the question) of the time. It is accepted that this choice may change with the appearance of further evidence or a better theory.
4) Only questions for which there is no relevant evidence can be considered to be 'mysteries', but speculations that can lead to searching for relevant evidence are encouraged.

All approaches associate knowledge with truth. All accept that the basis for knowledge is the evidence, which is the primary indication of truth, and that knowledge needs generalisations which fill in between pieces of evidence and go somewhat beyond them, and that offer understandings of how the evidence got to be what it is. All also accept that objectivity (i.e. being fair to alternatives) should be basic in scholarship. They all differ in how each of these principles is interpreted.

Basic in approaches A and B is belief that the knowledge produced is true, so there is no distinction made between knowledge and truth. In approach C, it is accepted that there is no way to tell whether knowledge that scholarship has produced is true, since there are many examples where subsequent scholarship has shown that some previous knowledge was not true. So knowledge, properly produced, is to be trusted as the best that scholarship can do at the time, without believing that it is necessarily true. It is an article of faith that scholarship, properly done, is the best known way to get close to truth.

In approaches A and B, individual pieces of evidence are not trusted because it is known that errors in evidence occur. In approach A, 'confirmation' by other evidence is required for trust. In approach B, evidence that cannot be explained by the theory is assumed to be in error. In both of these approaches, there is lack of concern for how the errors could have occurred. In approach C, the evidence must be fully accounted for. Since every theory needs to reasonably explain each piece of evidence, each piece assumed to be in error needs an explanation postulating how the error could have occurred. In this way, an easily made error results in little penalty for the theory, while not being able to find a reasonable way that an assumed error could have occurred results in the ultimate penalty of making the theory invalid. The reasonableness of an error assumption is enhanced if there is evidence of similar errors in the same source or similar ones, or if there is conflict with another piece of evidence. When there is apparent conflict between different pieces of evidence, that conflict must be resolved, with a theory modification preferred to an assumption of error. No evidence is not contrary evidence (needing explanation in a theory) unless there is other evidence that leads us to expect that such evidence should be available.

The role of judgement is very different in the three approaches. In approach A, the only criterion for accepting or rejecting a theory is judgement, which is not trusted unless there is a consensus. This skepticism is justified since such a judgement is largely subjective, often depending more on expectation than evidence. Consensus in acceptance provides the safety in numbers for a theory to be believed. So only a theory that adds to previous knowledge is accepted, and a theory that could replace any of it, no matter how well supported by the evidence, is ignored. No theory is overtly rejected, providing a veneer of tolerance, objectivity and minimum conflict. In approach B, the judgement that the theory is true is not questioned. Maximum apparent objectivity is observed in the style of presentation in publications. In approach C, it is appreciated that judgement cannot be avoided in scholarship. When applied to the truth of a theory, it is largely subjective. But since it is much easier to maintain a degree of objectivity in judging how reasonably probable the explanation of a piece of evidence by a theory is, this is where the judgement is shifted to. The focus is on having a measure of the adequacy of theories in explaining all of the evidence.

Those with approach A expect knowledge to grow only. Controversy must be settled before respectability can be conferred. There are thus a large number of questions that are accepted as mysteries because there is no consensus on acceptance of theories that attempt to answer them. Thus paradigm-changing theories are automatically rejected, and serious consideration of them often needs a new generation of leading scholars. This very conservative approach is mostly followed by scholars who are stronger on respectability than on debate, analysis or creativity. Approaches B or C are more attractive to creative scholars. There is some post-modernism in approach B, in that any distinction between belief in what is true and truth itself is at most vague. The only mysteries are why that truth is not supported by some of the evidence and not accepted by all of the other scholars. Approach C is followed mostly by those in rapidly growing fields where new evidence appears often, such as in some sciences. Then, entreched positions are hard to maintain, and the need for objective choice between theories is urgent to guide further work. The rapid growth allows an added criterion in the theory choice, that of whether it accurately predicts the nature of new evidence that is subsequently collected.

All of these approaches, as described above, apply only to theories concerned with truth, not with what is true statistically. The latter requires theories formulated as statistical theories, for which enough independent evidence to meet measures of statistical significance is required. In a statistical study it is assumed (often with little justification) that the evidence is subject to random error, so it is expected that some individual pieces of evidence will be greatly at variance with the accepted theory. Some knowledge of this has led adherents of approaches A and B to consider that in scholarship it is generally acceptable to ignore some pieces of evidence that contradict a non-statistical theory. Aside from the differences in the relationship between theories and evidence, scholars use any of the above approaches with statistical theories. An alternative approach in statistical studies is Bayesian analysis, which uses judgements based on experience. When collecting evidence, this approach speeds up getting objectively meaningful results if the judgements are relevant and good, but slows it down if these initial judgements are false.

Much of what goes under the name of 'arts scholarship' is refining the judgements that are the current fashions of thinking about their subject matter. This is much more criticism than scholarship. Approaches A and B are usually followed. There could be more objective scholarship in the arts, but few are interested because what the public wants from arts academics is the guidance of convincing explanations of current views of arts fashions old and new. Arts historians deal much with evidence and theories of artist's lives, cultures, methods and historical arts fashions. They can do good scholarship, but get into trouble when evidence of historical arts fashions differs from current fashions of thinking about and responding to the historical works of art. Evidence often doesn't do very well here when in conflict with judgement. Approach C would clarify the distinction between scholarship and criticism and improve scholarship quality greatly.

Return to Guide or PREPUBLICATION DRAFTS.