Part I: Cryptanalysis
You have been using cryptanalysis to help us understand how things hidden can be known. But isn't cryptanalysis deductive?
Cryptanalysis is not deductive. In cryptanalysis the cipher is not deduced from cryptograms generated by the cipher. Cryptograms are deduced from the cipher and the cleartext. This is to say, given a cleartext we can turn it into a cryptogram (also called ciphertext) by using the cipher. This process is called encipherment. Encipherment is deductive.
We can also carry out the reverse process. If we have the cipher, given a cryptogram generated by the cipher we can apply the cipher to the cryptogram to get back the cleartext. This process is called decipherment and is also deductive.
If we have been given only a cryptogram and are required to obtain from it both the cipher and the cleartext, we have to engage in cryptanalysis. Clearly this process cannot be deductive. For, from the cryptogram and the cipher we can deduce the cleartext. If both cryptogram and cipher are required to deduce the cleartext, from the cryptogram alone we obviously cannot deduce both the cleartext and the cipher.
If we do not obtain the cipher and the cleartext from the cryptogram deductively, how then do we obtain them?
The answer I have given in this textbook is that it is done by following the theseological method, that is, by following clues and developing new clues from old.
Is the theseological method the same as inductive reasoning?
No, the theseological method is not the same as inductive reasoning. The theseological method is not a form of reasoning at all, even though in following this method we have to reason. Since the theseological method is not a form of reasoning, it cannot be the same as inductive reasoning——or abductive, or deductive, or reasoning of any other kind. A camera is not a form of reasoning, even though in using a camera we have to reason. A camera takes pictures. The theseological method leads to knowledge of things hidden. A camera needs light to work. The theseological method needs clues.
Is the theseological method the same as abductive reasoning?
Please see last answer.
People who write detective stories would like us to believe that crimes can be solved by following clues. In fact what they write has little resemblance to reality. You would like us to believe that we can crack ciphers. Are we not being naïve to think that we can?
We have answered this question in Chapter 2. Our suggestion there to those who think ciphers cannot be broken is that they create ciphers of their own and challenge others to crack them. They can then make up their own minds afterwards.
We can only crack ciphers when there are clues. But if a cipher leaves behind clues, it means the cipher is not hidden. Cryptanalysis therefore cannot help us understand how we uncover knowledge of the hidden.
We have already explained in some detail in Chapter 3 why a cipher can be properly said to be hidden even when it leaves behind clues. Briefly, the important points are the following:
1)Clues in real life are hidden. In real life clues do not carry labels announcing to the world they are clues.
2)Clues are the characteristics of structures, disguised. A thing in disguise is hidden until the disguise is removed. In cryptanalysis we remove the disguise. The disguise does not come off by itself.
We do not have to use the word 'hidden' to describe ciphers that leave behind clues. We can coin a new word, say, 'midden'. Midden things then cannot be known directly, but they can be known by following the theseological method.
In games in cryptanalysis we can check our results against the originals to see whether they correspond. In real life we cannot do this. In real life, therefore, we have no grounds for thinking that we can uncover knowledge of things hidden even in cryptanalysis.
Checking results against originals is not the only way to determine whether results in cryptanalysis are correct. In fact, it is superfluous as we have explained at some length in the Chapter 2.
Even in games in cryptanalysis we do not have to check our results against originals. This is because we have a method, the theseological method. When we have a method there is no need to check results against originals. This is one of the benefits in having a method. We have a method for adding numbers on paper. This method guarantees that our results will be correct if we follow it correctly. After adding two piles of apples together by following this paper method, no one would go and check the result by counting the apples one by one again.
We have a tendency to think, to ensure that two things correspond, we necessarily have to put them side by side and visually make the comparison. Cryptanalysis teaches us that we do not need to do this when we have a method. The method assures correspondence without requiring comparison.
It is not true either that in real life we can never compare results with originals. Ciphers are sometimes captured after they have been broken, affording cryptanalysts an opportunity to compare their results with originals if they wish.
Compared to other kinds of situations in which we desire knowledge, cryptanalysis is simple. Even if knowledge of the hidden is possible in cryptanalysis, cryptanalysis cannot help us understand how knowledge of things hidden is possible in complex situations——for example, in science and in crime detection. The simple cannot explain the complex.
In trying to understand deductive reasoning we start with simple examples, such as that afforded by a syllogism. No one would introduce students to deductive logic by analysing Godel's Proof or even Pythagoras' Theorem. Reasoning in real life often is more complex than the simple examples we use in the study of logic, but no one would deny the importance of these simple examples in understanding the principles behind deductive reasoning. Similarly, if we want to understand why things hidden can be known, we should not start with complicated examples found in science and in crime detection; we should start with examples a great deal simpler.
Compared to complex examples, simple examples are more revealing where the basic principles and basic structures of an activity are concerned. It may be that there are important elements that simple examples cannot capture, but these can be deferred until we have understood the basic principles and basic structures.
It is doubtful that cryptanalysis can help us understand why the hidden can be known. Given a cryptogram we can rearrange the letters in more than one way to produce messages that make sense. Without knowing the original message, how can we tell which is the right message?
This objection is based on a misunderstanding of cryptanalysis. What is described in the objection——rearranging the same group of letters to produce different messages that make sense——is not cryptanalysis at all. It is anagramming. In anagramming no cipher is used: we can rearrange the letters any way we like so long as we produce a 'message' that makes sense. Such a message is called an anagram. Since many anagrams can be formed from the same set of letters, it is indeed true that without knowing the original message we cannot tell which of the many anagrams corresponds to the original message, if any.
There is a kind of cipher which works by rearranging letters, but in this case the rearrangement is done according to a formula (an algorithm) which is known both to the sender and recipient. Since they both know the formula, they can communicate. A cipher that works this way is called a transposition cipher. Anagramming should not be confused with the cryptanalysis of transposition ciphers. In anagramming the letters are rearranged any way we like, so long as we end up with a ‘message’ that makes sense. In transposition ciphers, the letters are also rearranged but according to a formula. In the cryptanalysis of transposition ciphers, if we are to succeed, we will have to discover this formula. Without it, we will not know how the letters should be rearranged.
Not all ciphers work by rearranging letters. Transposition ciphers do but not substitution ciphers. In substitution ciphers the cleartext alphabet is replaced by a ciphertext alphabet, as in our SBR example. Our SBR example is a substitution cipher, not a transposition.
In cryptanalysis the clues are given. By following them we can recover the cleartext and the cipher. In other areas——for example, in science and crime detection, no clues are given. If we say something is a clue, it is we who say so. How can we then claim that in these areas knowledge of the hidden is possible?
We should not assume that, just because ciphers are sometimes broken, clues in cryptanalysis are always clear, so clear that they can be regarded as given. With simple ciphers clues are relatively easy to detect; and once detected, most people can agree as to what they mean. But not all ciphers are simple. With more difficult ciphers, clues can be hard to detect. In these cases there could be debate as to whether something is a clue and if it is, what it signifies.
Clues can be hard to find in cryptanalysis just as they can be hard to find elsewhere. There is a famous cipher manuscript, called the Voynich manuscript, which no one so far has been able to read. Debates have been going on for centuries as to what the clues are in this manuscript, if any, and what they mean.
We should be careful not to infer from the fact that we sometimes debate about clues to the conclusion that clues are only in the eyes of the beholder. Clues, wherever they occur, are objective: we cannot decree something to be a clue if it is not. As we have been pointing out in the main text, we often mistake something for a clue when it is not
What is a clue? Where do clues come from? These questions we have already answered in Chapter 3. Clues are the characteristics of structures, disguised. They are the characteristics of structures camouflaged. The camouflage makes them hard to discern. But whether they are there does not depend on us but on how successful the camouflage is. If the camouflage is completely successful, the clues will be obliterated, as with the one-time pad. If the camouflage is only partially successful, the clues will be there: we only have to find them.
In science we try to uncover the structure of the world. If the world has a structure; if this structure has not been completely covered up; there will be clues. By following these clues we can find out what this structure is. These clues may be hard to detect; we may debate about them; but if they are there, we can make use of them the same way we can make use of clues in cryptanalysis. Cryptanalysis is not an exception where the knowledge of the hidden is concerned. In cryptanalysis we can uncover knowledge of things hidden because there are clues. But since clues come from structures, clues can occur outside of cryptanalysis as well as within.
Can we ever tell we have found a genuine clue? Are clues always open to debate? Are these debates necessarily endless?
We can tell if we have found a genuine clues. When a clue leads to more and more new clues, it is likely to be genuine (see Chapter 2). False clues peter out.
It is useful to ask why we have this common but erroneous belief that clues in cryptanalysis are given, that is, that they are always so clear that any one with any intelligence will know where they are and what they mean. Why do so many people believe in such a view when in fact it is false?
There is a simple explanation. Most of us know about cryptanalysis second-hand, from stories told from hindsight by people already in the know. Now there is this thing about clues: they are always clear from hindsight.1 This is true in cryptanalysis as it is true elsewhere. Mendel’s experiments gave rise to genetic theory. This is clear from hindsight. At the time, nobody noticed his paper.2 That clues are always clear from hindsight gives rise to the expression 'I could kick myself …'. Why is this expression common? It is common because most of the time we don't notice the clues until only afterwards, so we kick ourselves.
In cracking a cipher we know in advance there is a message to be found. In other areas——science, for example, or even crime detection——we don’t know if there is anything to be found. How then can we approach these other areas in the same way we crack a cipher?
Those who make up exercises in cryptanalysis usually tell those who work on them that there are messages behind the cryptograms in the exercises. In this sense those who work on these exercises know in advance that there are messages to be found. But why should those who make up exercises consistently do this? Can they not sometimes have a little fun on the side and make up a 'cryptogram' without a message behind it?
In the latter case we would say they are not being fair; they are misleading those who work on the exercises. But in real life, is fairness a concern for those who make up ciphers? Would they feel bad if they had misled those who wanted to crack their ciphers? Would they be unhappy if the latter were to waste their time on cryptograms that had no messages behind them? And in real life what do cryptanalysts do? Do they demand that all cryptograms should be accompanied by notarised certificates guaranteeing that there is a message behind every single one of these cryptograms? Would they make sure this demand is met before they start?
If we have to know that there is a message behind a cryptogram before we can decipher it (through cryptanalysis), would any one engage in cryptanalysis except in games? Would the theseological method work if every time we crack a cipher we have to know in advance that there is a message behind the cryptogram? The theseological method asks us to follow clues and develop new clues from old. It does not ask that we obtain assurances from our enemies that there are messages behind the cryptograms.
There are ciphers that have remained unbroken to this very day. For example, there are the Beale Ciphers. In this case there are three letters, all encoded. Story has it that these three letters tell of the burial and the location of a large treasure in silver. The second of these three letters has been deciphered but while it tells the story about how the treasure was obtained, it does not disclose the location. The other two letters have so far defeated all attempts. Are there messages behind the remaining two letters? Do we know? Should we find out first before we try to decipher them? From whom?
In cryptanalysis the cryptogram is given: we do not collect a symbol here and a symbol there and make up our own cryptogram. But at a crime scene we do pick up a bit here and a bit there and use them as evidence on which to base a solution. In the investigation of nature we do the same thing. When there is such a big difference between cryptanalysis and these other areas, how can we transfer what we have learnt from cryptanalysis to these other areas?
We use ciphers because we want to communicate in secret. By enciphering our messages; by sending cryptograms instead of plaintexts; we deter others from knowing the contents of our communication.
But ciphers can be broken. If others get hold of our cryptograms, they will try to break our ciphers. What can we do to prevent this from happening? What can we do to prevent our cryptograms from being intercepted?
To prevent our cryptograms from being intercepted we can hide the cryptograms themselves; we resort to steganography: we use cover letters, microdots, invisible inks and other devices of a similar sort. This way we can send our cryptograms without others knowing that we have sent them.
But those interested in our secrets are not dumb. They know about cover letters, microdots and invisible inks, too.
What can we now do if we still want to send cryptograms? How can we prevent our cryptograms from falling into enemy hands?
Try the following. Send your cryptograms by radio. Now you may think this is a poor attempt at humour. To send cryptograms by radio, you have to turn on the radio. But every time you turn on your radio people will listen in. The airwaves are public.
But you don’t turn your radio on. Rather, you don’t turn it off. You keep your radio on all the time. And all the time that it is on, it will be working: it will be sending out signals. Most of the time, the signals will be random; that is, there is no message behind them. When you have a message to send, you first encipher it, then chopped it up into parts. You sandwich the parts between the random signals, then send them on their way. By doing this, there will be no neat packages.
Have you won? Have you deprived your opponents the opportunity to crack your cipher? Are your cryptograms now unretrievable except by your own people?
Your cryptograms are not unretrievable. Your opponents can still find them among the random signals. They will have a hard time: the task is not easy; but it can be done(for hints on how this can be done see again the sections, Gathering Evidence, in Chapter 4 and Parallel Processes in Chapter 5).
Cryptograms are not always neatly packaged even in cryptanalysis. There is no requirement that they have to be if they are to be broken. Nature is not neatly parcelled up, evidence at a crime scene does not draw attention to itself, but this will not prevent us, and has not prevented us, from finding the knowledge that we wish to have in these areas.
Cryptanalysis cannot tell us why things hidden can be known. For what cryptanalysis reveals is not anything that we do not already know. The cipher and the message——they are already known …to those who made them up.
But the important point here is that the cipher and the message are not known to the cryptanalysts. It is true that they are known to some people. But these same people would not tell. They would not tell before nor afterwards. If they would before, there would have been no need for cryptanalysis. If they would tell afterwards, the cryptanalysis would have been superfluous. Because they would not tell either before or after, cryptanalysts have to find out on their own: they have to rescue themselves, bootstrap themselves, from a state of ignorance to a state of knowledge.
Because cryptanalysts have to bootstrap themselves; because their situation is such that they have to find out on their own; whether there is any one who already knows makes no difference. So far as cryptanalysts are concerned, these people who know may as well not exist. In practice, ciphers that are broken are sometimes authored by people long dead. Clearly, in cracking their ciphers we have not resurrected these authors. Nor can they by dying make their ciphers unbreakable.
When cryptanalysts succeed in cracking a cipher, they will have uncovered knowledge they originally did not have. This knowledge is not given to them by God or passed on to them by those who already know; it is knowledge they uncovered on their own, independently of any one who already know. This is what makes cryptanalysis interesting. The puzzling question surrounding knowledge of the hidden has always been, how can human beings with their limited powers discover on their own this kind of knowledge?3 Now this question may appear puzzling but in fact there is an answer: We uncover what we originally did not know by following the theseological method, that is, by following clues and developing new clues from old.
If cryptanalysts cannot ask those who already know, how can they determine that their results are right? How can they make such a determination on their own, in the dark, while cut off from those who know?
We have answered this question many times. Cryptanalysts have a method: the theseological method. This method enables them to determine whether their results are correct, doing so on their own, in the dark, while cut off from those who know.
Clever children watching a magic show will look for smoke and mirror; they know magic is only illusion. There is no smoke and mirror in cracking a cipher. In cracking a cipher we uncover knowledge on our own. We do this by following the theseological method. The existence of this method shows us knowledge can be obtained other than by direct transmission. To uncover knowledge we do not have to get it from the horse’s mouth.
It is not surprising that we can crack ciphers. In cryptanalysis, there is a criterion of truth: coherence. When at the end you arrive at a message that makes sense, you know you have the right message. Unfortunately there is no such thing as a criterion of truth elsewhere. Thus we can uncover knowledge of the hidden only in cryptanalysis but not anywhere else. Would you not agree?
The belief in the existence of a criterion of truth in cryptanalysis arises out of misunderstanding. This we have explained at some length in Chapter 2, but it is useful to remind readers of the example we used there:
MEET ME AT THE T?P
This message, we have said, can be deciphered in at least two ways. Both will lead to a coherent message, a message that makes sense; but, obviously, they cannot both be right.
In cryptanalysis we do not use a criterion of truth. Just as there is no criterion of truth elsewhere, there is no criterion of truth in cryptanalysis. In cryptanalysis we determine whether results are right by reference to method, the theseological method, the same method that can be applied elsewhere.
Cryptanalysis is possible only when you know the language of the plaintext. If you do not know the language of the plaintext, you cannot solve the cipher. Knowledge of the language precedes cryptanalysis, making the latter possible. But in other kinds of situations in which knowledge is desired, for example, in the investigation of nature, we have no advance knowledge of the language. We have no advance knowledge because in these cases no language has ever been put into place. Language is a human invention, not a part of nature.
When cracking a cipher, although it will make things a whole lot easier if you already know the language of the plaintext, it is, strictly speaking, not necessary. Suppose I am cracking a cipher and the plaintext is written in a dialect of the English language, a dialect that I do not know. If this dialect does not differ too much from the English that I know, I can still crack the cipher. For, since the two languages are close my knowledge of English will enable me to detect some clues and thus decipher parts of the plaintext. If these old clues lead to new ones, in time I will have deciphered not just parts of the cleartext, but large parts. Once this has been achieved, context will tell me how to decipher the rest.
But we can go well beyond this. What we have just established is that if we know one language, we can decipher another close to it. But suppose we have twenty-six languages: A, B, C, ..., Z. In this series, A is close to B, B to C, C to D, and so on. Of these twenty-six languages, I know only A to start with. But since B is close to A, I can decipher B. And since C is close to B, I can decipher C. If I go on in this manner, by the time I reach Z, I will be deciphering a language very different from A.
Of course, going from A to Z will take a long time. But it is in theory possible. It is possible to decipher a language very different from the one we initially know.
There is also this to consider. Suppose we have reason to suspect in a particular case that the language of the plaintext is a simple artificial language which we do not yet know. In this case we can start constructing simple artificial languages on our own. If we find among these one that leaves behind the same kind of clues as the one we are trying to decipher, we can start deciphering the latter. This again shows, we do not have to know the language of the plaintext in advance. We can make looking for the language part of the task of cryptanalysis.
When modern science began, much attention was given to the question what language nature speaks. Galileo thinks the book of nature is written in the language of mathematics. By this he meant geometry. Descartes later extended it to analysis; Newton, to the integral and differential calculus; and Leibniz, binary arithmetic. The process has not stopped.
Language indeed is a human invention. So we keep inventing until we find one that best suits whatever it is we are investigating.
Whether or not we need to know the language of the plaintext to start with in cryptanalysis, still in cryptanalysis we know a lot to start with. If we did not know anything to start with, we would not have been able to crack any cipher. But how do we know anything to start with? So long as this question remains unanswered, cryptanalysis cannot help us understand why things hidden can be known. But nobody knows how we know anything to start with. People have been trying to answer this question for ages and have not come up with an answer.
I do not think we have to know how we know anything to start with in order to understand why things hidden can be known. To understand why motor-cars work, no one will say, we have to know how we know anything to start with. Since we don't have to know in one case, we don't have to know either in the other.
It is true that if we do not know anything to start with we cannot crack ciphers. But we have been cracking ciphers. This means human beings do know (approximately) some things 'to start with'.
It is true also that human beings have been asking for a long time how we know anything to begin with and have not been able to find an answer. But this is because by 'how we know anything to begin with' they mean 'how we know anything absolutely for certain to begin with'. There is no answer to the question how we know anything absolutely for certain to begin with because there is nothing we know absolutely for certain. Since there is nothing we know absolutely for certain, there cannot be an answer to how we know some things absolutely for certain.
Now for cryptanalysis to be possible we have to know some things to begin with. But there is no requirement that we know these things absolutely for certain. Whatever we need to know to start with in cryptanalysis, we do not need to know absolutely for certain. In cryptanalysis we do not start with certainties and end at certainties. In cryptanalysis we start with the uncertain and end with the more certain. Clues, for example, can be highly uncertain. If we use cryptanalysis as a model to help us understand how we know, we can only explain how we can become more and more certain, not how we can become absolutely certain. But to explain how we can become more and more certain, we do not need to explain how we know anything absolutely for certain to start with. As we have said, we arrive at the more certain not from the certain but from the less certain.
To explain how we know anything absolutely for certain to begin with is difficult and even impossible because, as far as we know, there is nothing we know absolutely for certain. But it is not hard to imagine how we might have arrived at that uncertain (approximate) knowledge which eventually made it possible for human beings to crack ciphers. Before we point out this how, however, there is one thing we should be aware of. The question, 'How do we arrive at uncertain (approximate) knowledge initially?' is very different from the question, 'How do we know anything absolutely for certain to begin with?'. And since these two questions are different, we do not have to adhere to the same standards when answering them.
And since we do not, we can give the following answer to the first question: We could have arrived at this uncertain knowledge by chance.4
'Chance' is a dirty word where the explanation for knowledge that is absolutely certain is concerned: knowledge we arrive at by chance cannot be absolutely certain! But 'chance' is not a dirty word where uncertain (approximate) knowledge is concerned. In cryptanalysis, we work with uncertain (approximate) knowledge. It is well known in cryptanalysis that luck——another word for chance——has a place. In cryptanalysis, as a general rule, we should follow clues; we should not depend on wild guesses; wild guesses have next to no chance of being right. But sometimes when we run out of clues and are desperate, we allow ourselves to make wild guesses. Most of the time these wild guesses will be wrong but on those rare occasions when by chance they turn out to be right, there is no prohibition in cryptanalysis that we turn our back on them.
It is not a satisfactory explanation to say that knowledge that is absolutely certain occurs by chance, but it is not an unsatisfactory explanation to say that uncertain knowledge could occur by chance. Many people discover how to crack ciphers on their own but a great many more do not. Those that do, happen to have assembled those bits of uncertain knowledge that enable them to make their discovery. Those that do not, were not as lucky. The uncertain knowledge that human beings had at the beginning could have occurred to them by chance.
In cryptanalysis we have to use our senses: we have to look at the cryptogram to find out what it contains. But our senses are fallible. How can they provide us with those true premises from which knowledge is to be inferred?
We are not using cryptanalysis to illustrate a view of the knowing process which holds that we uncover knowledge by inferring it from true premises provided by the senses. This view, we think, is mistaken. The correct view is that we uncover knowledge by following clues and developing new clues from old. The two views are radically different and should not be confused one with the other.
In cryptanalysis we do not mind that our senses are fallible. Indeed, cryptanalysts are perhaps more conscious than most that our senses make mistakes. When we read a newspaper, we read in plain language. Even then our senses make mistakes: we think we see 'theology' when in fact it is 'theseology'. In cryptanalysis we have to look at cryptograms, which are made up of symbols arranged in unusual ways. When these symbols appear in such an unfamiliar fashion, the chance for mistakes is even greater. Now cryptanalysts know this; it is the kind of situation they meet with day in and day out. They know that when they read cryptograms, they could be making mistakes. They know that when they transcribe cryptograms, they could also be making mistakes. Does this make cryptanalysis impossible? No; in cryptanalysis we can correct mistakes (see Chapter 4), even those made by our senses.
Our senses are fallible: they make mistakes. But cryptanalysis shows us, perhaps more clearly than anything else, that despite the fallibility of our senses we can uncover knowledge, gradually.
All observations are theory-laden. This includes observations in cryptanalysis. This is to say, in cryptanalysis from the same cryptogram we can retrieve different messages. It all depends on how we look at the cryptogram, that is, on what theory we have accepted.
We agree that observations are theory-laden. In cryptanalysis we do not observe just anything; we want to observe only what is relevant; and relevance is determined by theory. But we do not subscribe to the view that from the same cryptogram more than one message can be retrieved through cryptanalysis. For when this happens; when more than one message has been retrieved; by definition it spells failure. At least two messages can be retrieved from the following cryptogram:
MEET ME AT THE T?P
namely, MEET ME AT THE TOP and MEET ME AT THE TIP. But in cryptanalysis we would not say both messages are right. Rather, we would say we do not know yet which is right, if either: we have not at this stage found the right message.
Of course, when we are faced with a cryptogram, we may want to do something else with it instead of cryptanalysing it. For example, we may want to draw a circle around it; or we may want to frame it, put it up on a wall and admire it. But if we want to cryptanalyse this cryptogram, we cannot regard ourselves as having succeeded even when we have extracted from it two different messages.
To explain why things hidden can be known you appeal to cryptanalysis. Let us grant that cryptanalysis works. But that it works is an empirical fact, which has to be explained. How do you explain that cryptanalysis works? You can only do so by appealing to cryptanalysis. But this is circular reasoning!
That things hidden can be known is due to the existence of a method, the theseological method. We use cryptanalysis to illustrate this method because cryptanalysis in some instances is simple. The explanation why the theseological method works does not depend on cryptanalysis, as is implied in the objection; it depends on the fact that structures have characteristics. Clues originate from structures, since they are their characteristics, even though disguised. It is because clues are the characteristics of structures that by following them we can crack ciphers. Cryptanalysis therefore is not explained by cryptanalysis, but by reference to method; which in turn is explained by reference to clues; which are the characteristics of structures. No circular reasoning is involved.
It seems circular reasoning is unavoidable once you try to explain, in the way you do, why cryptanalysis works. You explain why cryptanalysis works by reference to method. Now, as you say, method itself has to be explained. But how do you know this explanation is correct? You can only say it is correct because it can be found to be correct by the theseological method. But this is circular reasoning. You are using the theseological method to justify the theseological method.
Questions such as this is one of the reasons why it is important that we be aware that theseology is not a form of reasoning (see again Chapter 1). The theseological method is the centrepiece of theseology and is also not a form of reasoning. Now a method is a kind of tool. Some tools can be used on itself with happy consequences. Computers can be used to design computers. As a result, we have better computers. No circular reasoning is involved here. When we use a tool on itself we are not engaging in circular reasoning. Now we could have been using the theseological method without understanding why it can produce the kind of results it does. When we use the theseological method to find out why, there is no circular reasoning involved. Why? Because in the first place, the theseological method is not a form of reasoning. Second, using a tool on itself is not circular reasoning.
You think cryptanalysis leads to the truth. I think it does just the opposite. Take your SBR cryptogram for example. You've found the cleartext to be THE THIRD SHIPMENT WILL ARRIVE TWO WEEKS FROM TODAY. So you wait two weeks and expect to intercept this shipment. But there is no shipment two weeks from today because unbeknownst to you the sender and receiver of the cryptogram had an agreement beforehand that all numbers referring to time are to be halved. The shipment arrived one week ago when you were still waiting! Cryptanalysis is useless for finding the truth. Even after you have broken the cryptogram you still cannot tell what the truth is. Indeed, the cryptogram can be designed in such a way as to lead you away from it.
What has been described is quite possible. It is quite possible that the two communicating parties have the kind of agreement suggested. But this just means that the cipher used in this case is more complicated than we thought.
Suppose the cipher is more complicated; suppose it does include this agreement about halving numbers when referring to time; have we totally failed after we have only solved the cryptogram and not found out about the agreement?
We have not. If in addition to solving their cryptogram we had also found out about the agreement, we would have located the truth. As it was, we only managed to solve the cryptogram; we did not find out about the agreement. We were short of the truth but closer than if we had not even solved their cryptogram.
Is it impossible to find out about the agreement? In practice, it may be. That is, in practice it may be impossible until it is too late. For when we later find out that the shipment is not arriving after two weeks, we may start suspecting that there is more to the cipher than we thought.
Is it impossible to find out about the agreement in advance? Not necessarily. If there are clues; if we can enlarge the investigation; it is possible that by following the theseological method we can find out. Of course, if there are no clues the theseological method cannot be applied. The theseological method cannot be expected to work even when there are no clues.
The theseological method is not fool-proof (Chapter 2). Usually people who are familiar with it are not fools. Professional cryptanalysts for example are aware that a cleartext recovered through cryptanalysis can itself be cover for a deeper message. They call this superencryption. Messages superencrypted are more difficult to deal with and success is not promised in advance (success is never promised in advance when using the theseological method), but basically we still deal with them in the same way, that is, by using the theseological method.
Cryptanalysts are aware of the possibility of ciphers within ciphers. People interested in solving mysteries, people such as Sherlock Holmes, are aware of the possibility of mysteries within mysteries. Scientists are aware of the possibility of structures within structures. The real world, theseologisers know, is complicated. In this complicated world, theseologisers are sometimes deceived despite their partial successes.
If there can be ciphers within ciphers and messages within messages, how do we know when we have recovered the real message? Suppose your SBR cryptogram has enabled you to break up a drug ring. You may think then that your decipherment is correct. But that drug ring could be a decoy, to direct your attention away from another drug ring, which is also a decoy for yet another drug ring and so on and so on, indefinitely. This way you will never catch the real drug ring, even though you catch all the decoys. You cannot exhaust an infinite series.
Does it matter if you do not catch the real drug ring if you can catch all the decoys that your SBR cryptogram leads to? With every new drug ring you catch, you are more certain that your decipherment of the SBR cryptogram is correct. And with every new drug ring you uncover, you know that all the former ones are decoys. Now these decoys are not the real drug ring, but if they are to be good decoys, they will have to look very much like a real drug ring. For example, the drugs you find in every case are real drugs. They have to be if they are to be good decoys. The real drug dealers want you to think you have found the real drug ring so you would stop going after them. You would not think you have found the real drug ring unless the drugs found are real ….
I think our discussion at this point has an important lesson for us. Those detectives who do not want to waste time on decoys will never catch the real drug ring. But those who track down one decoy after another will have served their community in their limited way whether they catch the real drug dealers or not.
In real life drug dealers cannot design infinite deceptions, but perhaps Descartes' Evil Demon can, since in Descartes' hypothesis, the Evil Demon is as powerful as God. But even if there were an Evil Demon, it seems it should still be worth our while to peel back his deceptions layer after layer. For suppose there is a drug dealer who can design infinite deceptions. If we can peel back her deceptions layer by layer, this drug dealer will never make money from drug dealing. For she will spend all her time creating decoys and all these decoys will be smashed. She cannot make money from smashed decoys. But she will not be able to make money from the real drug ring either since an infinite series of decoys has no last decoy and the real drug ring can only come after the last decoy. So, despite the power she has in being able to create infinite deceptions, this drug dealer is a failure in her own eyes.
I do not know why we should be talking about deceptions here when our interest is in knowledge. But since we have started, let me suggest another possible case. It is again about the SBR cryptogram. Suppose this time the agreement between the two corresponding parties is just the following: 'When you see the three letters SBR, come home.' SBR in this case is a code. It means 'Come home!' Now after you have intercepted the SBR cryptogram, you work on it and discover the plaintext THE THIRD SHIPMENT WILL ARRIVE TWO WEEKS FROM TODAY. But this means nothing at all to the corresponding parties. They are only interested in the first three letters, SBR. The 'message' about the shipment is just there to throw you off your tracks. While you are waiting for the third shipment, the person who received the SBR code has already gone home. So my question to you is, how useful is the theseological method where knowledge is concerned? Here in this case, you have been completely deceived.
We are interested in deception because when we are deceived we are taking some falsehood to be the truth, and this we want to avoid when searching for knowledge. In the case you bring up, there are two things to be said. 1)That we can crack ciphers does not mean that people can never keep secrets from us. The SBR code you have suggested cannot be broken (let us assume). Neither can a one-time pad. 2)In the case you mentioned, you say we are completely deceived. This is not true. Cryptanalysts know that messages they decipher can be blinds. When we obtain the message THE THIRD SHIPMENT WILL ARRIVE TWO WEEKS FROM TODAY, if we are any good as cryptanalysts, we will not say that we are absolutely certain that this is the true message. Will we wait for the shipment to arrive? Yes; if we have no other information to go on and the shipment is important to us. Are we completely deceived? No; because if after two weeks no shipment arrives, we will have reason to suspect as one of the possibilities that the message is a blind.
The theseological method will not make it possible for us to know everything. It will not prevent us from being deceived sometimes. Some people think, if we have a method that leads to knowledge, one day we will know everything. I do not share this view.
Part II: Knowledge
To understand why knowledge of the hidden is possible, we should start with examples like tables and chairs and ask why knowledge of such things is possible. Tables and chairs are obvious: we see them; they are right in front of us. When we have understood why knowledge of the obvious is possible, we can then go on to ask why knowledge of the hidden is possible. But you have not done this. Instead, you start into why knowledge of the hidden is possible right away. Why?
It is natural for us to think, the process by which the obvious is known is simple, the process by which the hidden is known is more complex. It is common sense to start with the simple before going on to the complex. So we often tell ourselves, find out how the obvious is known before finding out how the hidden is known. This, I have found, turns out to be a mistake. Nothing in fact is obvious; everything is hidden. This is what discussion in this area has told us, discussion that has been going on for over two thousand years. Tables and chairs are not obvious: they are as much hidden as the letters of the alphabet in the Morse Code. Those who have internalised the Morse Code hear the letters of the alphabet 'directly'. We see tables and chairs 'directly' only because we have internalised the cipher by which we recognise these things. People have expressed this wish very often: Would it that there are some things we know directly! For then, they say, we should be certain of these things, certain that we know them for what they are without any possibility of error. It is often said this is how God knows. Gods knows directly. Indeed, some think that God is one with what he knows: His knowledge is so direct that in His case there is no separation between the subject who knows and the object known. But human beings are in a sadder situation. Human experience has been, there is nothing human beings know of which they can be absolutely certain, nothing in which the possibility of error can be ruled out in advance. This seems to indicate there is nothing human beings know which they know directly. Everything they know (fallibly) is hidden.
People usually think, we should find out how we know the obvious before we try to find out how we know the hidden. They follow their own advice. As a result they never get around to finding out how we know the hidden.
There is no answer to the question how we know the obvious. There is no answer because there is nothing obvious. All our knowledge is of the hidden, the reason why even in the best instances our knowledge is uncertain and always open to the possibility of error.
The only way to understand why knowledge is possible is to start out by trying to understand why knowledge of the hidden is possible. If we start out by trying to understand why knowledge of the obvious is possible, we will not get anywhere. There is no explanation why knowledge of the obvious is possible because nothing is obvious.
There is no such thing as a scientific method. A scientific method is supposed to tell us how to make discoveries. But there cannot be a method for making discoveries because making discoveries requires bright ideas and there is no method for having bright ideas. Do you not agree?
It is true that there is no method for having bright ideas, but it is a mistake to think there is no method for making discoveries. When we have broken a cipher we have discovered something that we originally did not know. We crack ciphers, we make discoveries, by following a method, the theseological method, the same method used by scientists and detectives and any one else who follows clues and develops new clues from old. But that there is a method does not mean discoveries are easy. Discoveries are difficult, as any one who has tried to break a cipher will know, or any one who has tried to solve a crime. They are difficult because they require bright ideas, which is another word for ingenuity. However, that discoveries require bright ideas or ingenuity does not mean there is no method. There are two kinds of methods (see again Chapter 2): foolproof and non-foolproof. Foolproof methods do not require ingenuity; non-foolproof methods do. There is no foolproof method for making discoveries but there is a non-foolproof method: the theseological method. A foolproof method guarantees that even a fool can obtain the right results. A non-foolproof method only guarantees that the results will be right if the method is followed. Whether the method can be followed depends on whether the person attempting to follow it is sufficiently ingenious. The theseological method is a non-foolproof method. It tells us to follow clues and develop new clues from old. Both require ingenuity. But if we are able to do both over and over again we will discover things we originally did not know.
You say clues lead us to knowledge of the hidden. I find this hard to believe. Clues merely suggest ideas or theories, but the important question is, are these ideas or theories true? You cannot tell whether they are true by where they come from! Some people have bright ideas by drinking coffee, but their ideas can't be true just because they drink coffee.
I say we uncover knowledge of the hidden by following clues and developing new clues from old. There are two things we have to do over and over again. We cannot just follow clues, we also have to develop new clues from old. When old clues lead to new clues, the interpretation of the old clues are likely to be right. Clues do not just suggest ideas. The emergence of new clues allows us to evaluate the ideas suggested by old clues.
We cannot compare following clues to drinking coffee. Clues originate from the hidden structures we want to know; coffee comes from coffee beans. If we drink coffee and do not pay attention to clues, the ideas we propose after drinking coffee are no more than wild guesses. But ideas proposed in response to clues have a chance of being right. This is what clues do: they narrow down the field.
Not all clues are the same. Some are vague and some, revealing. If we know nine letters of a ten-letter word, we will have a pretty good idea what the tenth letter is.
Whether an idea is likely to be true does depend on where it 'comes' from. We want them to come from clues. We allow wild guesses only when we are desperate.
We infer from the known to the unknown. This is how we expand our knowledge, is it not?
This is a common view, that we extend our knowledge from the known to the as-yet unknown by reasoning. By reasoning we enlarge our knowledge, so it is said. But this view, though common, is mistaken. In fact, we do not expand our knowledge by inferring from the known to the unknown. Reason is not that powerful. If we reason deductively, we can never get beyond what we already know. Some say that besides deductive reasoning, there is also ampliative reasoning (sometimes also called inductive), which allows us to go beyond what we already know. But no one has been able to show why ampliative reasoning is ever valid. No one has been able to do this, Karl Popper has pointed out, because ampliative reasoning is never valid. I think Popper is right. But since we do not expand our knowledge by reasoning, that ampliative reasoning is never valid is a fact that we can calmly accept. How do we expand our knowledge if not by ampliative reasoning? We do so by following the theseological method, that is, by following clues and developing new clues from old. The theseological method is not a form of reasoning. It tells us to follow clues and develop new clues from old, not what conclusions to draw from what premises (see again Chapter 1).
How can there be a method for expanding our knowledge if there is no ampliative reasoning, that is, if ampliative reasoning is never valid?
The common view before Popper was that we expand our knowledge by reasoning ampliatively from knowledge provided us by our senses. It was commonly believed then that this was the method to use. But Popper says ampliative reasoning is never valid. Since this method depends on ampliative reasoning, this so-called method cannot work. Therefore, Popper says, there is no method. But there is an unstated assumption here. The assumption is that if there is a method this method has to depend on ampliative reasoning. But why should we accept this assumption? Can there not be a method that does not depend on ampliative reasoning? There was a time when people assumed that if we were to fly God would have to give us wings. Since God had never given us wings, they concluded that we could never fly. But we have flown! Not by growing wings but by inventing aeroplanes.
Growing wings is not the only way to fly. In fact, we cannot grow wings. If we want to fly, we have to think of other ways. We cannot make ampliative reasoning valid. We can no more do this than we can make ourselves grow wings. But we do not have to rely on ampliatively reasoning to enlarge our knowledge. Instead, we follow the theseological method. The theseological method does not depend on ampliative reasoning, no more than human flight depends on human beings growing wings.
You say the theseological method is the same as the scientific method. Perhaps you should not do that. You discovered the theseological method by examining one example in cryptanalysis from beginning to end. But you have not examined a single example from science from beginning to end. How do you know that in science we also follow the same method?
There are a number of things to be said in answer to this question:
1)Cryptanalysis is usually regarded to be both an art and a science. Inasmuch as cryptanalysis is at least in part a science, it seems to me there is some legitimacy in calling the theseological method the scientific method.
2)Historically, interest in scientific method stems in large part from the desire for knowledge of things hidden. Back at the beginning scientists were asking, is this kind of knowledge possible? How do we bring it about? Is there a method?
Now if there is, as I think there is, it hardly matters what we call it, whether theseological or scientific. Whatever we call it, the method is still worthy of attention. I call it the theseological method to draw attention to the role clues play in the exercise of this method. But if we want to draw attention to its historical connection, we can call it the scientific method: it is the method pioneering scientists were looking for.
3)If you were Watson, every morning when you see Sherlock Holmes you would not examine him from head to toe to make sure he is the same person. If he looks like the Sherlock Holmes you know, he is Sherlock Holmes.
Nowadays much about the way scientists work is known. For example, we know that scientists follow clues: they are interested in repeating patterns and unique occurrences——such as why chlorine has an atomic weight half way between whole numbers; we know they engage in a process of elimination from time to time; we know they sometimes speculate as to what the world might be like; …. In all appearances scientists seem do all of the things a person practising the art of detection would do. When this is the case it is not, I submit, unreasonable to suggest that the method they use is the theseological method. Now it is possible that scientists do not use the theseological method and still manage to uncover knowledge of things hidden. But so far I have not seen any clue that would suggest that this is the case.
Knowledge has to be built on a solid foundation, that is, some truth of which we are absolutely certain. Without such a foundation; no knowledge can be obtained. Is this not so?
Cryptanalysis does not depend on a solid foundation. In cryptanalysis we follow clues and develop new clues from old. Clues, especially at the beginning, are vague and uncertain; yet they can lead to knowledge. Of course, this knowledge we obtain by following clues is uncertain and may be inaccurate, but if we can develop new clues, we can render it more and more certain, and more and more accurate if it is not already accurate.
To uncover knowledge of the hidden we have to use our senses. But our senses are fallible. How can we depend on them for any kind of knowledge, method or no method?
It is true that our senses are fallible; nevertheless they can help us uncover knowledge. We uncover knowledge by following the theseological method. The theseological method does not require that we should have absolute certainty at any point. For the method to work, nothing is required to provide us with absolute certainty, neither the senses nor anything else.
Because our senses are fallible, we will sometimes make mistakes. But the theseological method allows us to correct mistakes. When mistakes are minor, they can be corrected in the course of the investigation. When mistakes are major, progress will become difficult; it may even stop altogether; at which point we can backtrack and try to locate our mistakes. We will have located them and corrected them if we can advance again (see again Chapter 4).
Since there is no requirement for absolute certainty in the theseological method; since the method can correct mistakes; that we have to use our senses and that they are fallible do not prevent us from uncovering knowledge.
How can we correct mistakes without certainty? If you do not know what the truth is; if you are not certain; how will you know what departs from it? If you are not certain that 2 + 2 = 4, how will you know 5 is the wrong answer? When you correct mistakes without certainty, you could be making things worse rather than better. If you substitute a vague recollection that 2 + 2 = 5 for certainty, you will reject 4 as the right answer!
It is counterintuitive that we can correct mistakes without knowing for certain what the truth is. But this we actually do in the theseological process, a reason (as I have stressed) why we should make every attempt to directly acquaint ourselves with it. In the theseological process, if we make serious mistakes the whole process will come to a halt. If after re-doing some of the steps we are able to advance again——and to keep on advancing, it is likely that we have both located and corrected the mistakes (see again Chapter 4).
We make mistakes when we interpret the information gained through our senses. If we do not interpret, we cannot make mistakes. To obtain knowledge, therefore, we should refrain from all interpretation and confine ourselves only to describing what our senses tell us.
We cannot uncover knowledge of things hidden without interpreting the information provided by our senses. Our senses provide us with clues, among other things. Clues have to be interpreted if we are to find out what they mean. If we do not interpret, we cannot find out. True, when we interpret we sometimes make mistakes but mistakes can be corrected when we are following the theseological method.
You say there is a method for uncovering knowledge of things hidden. But how do you know there is anything hidden?
You do not have to know for certain that there is something hidden before you try to look for it. If we had to know for certain that there is something hidden before we can try, no method would ever help. But we do not have to know for certain. Indeed we could merely suspect. For example, in the death of a certain person we could suspect foul play. But after investigation we discover no foul play was involved; death was due to suicide. We could suspect, and even suspect the wrong thing; but after the investigation we will find out what is likely to be the case. There may be nothing hidden; but if after investigation we find out not only that there is something hidden, but what this something is, then it is likely that there is something hidden.
Knowledge comes either from the senses or from reason, or a combination of the two. Now philosophers have argued that knowledge cannot come from the senses, or reason, or a combination of the two. How then can we uncover knowledge of things hidden?
Knowledge comes neither from the senses nor from reason, nor from a combination of the two. Knowledge is uncovered by following the theseological method.
Those who say knowledge comes from the senses think that the senses can provide us directly with truths that are certain. Those who say that knowledge comes from reason think reason can do the same. Those who say knowledge comes from the senses and reason working together think that the two working together can provide, directly, truths that are certain. But in following the theseological method our senses and our reason play only supporting roles. In their supporting roles, they are not required to provide us with certainties either individually or in concert. In following the theseological method we do not start with certainties nor arrive at certainties; we start with uncertainties and we approach certainty asymptotically.
When I look at a tree I have the image of a tree. But how do I know that it is the true image? I can look again but all that I will obtain is another image of the tree. What the tree is like in itself, I will never know.
To crack a cipher I do not have to look at the actual cipher. The cipher I am trying to crack could have been locked up in a safe and the key thrown away. Without opening the safe, without retrieving the key to the safe, I could still find out how the cipher works. And when I do find out, I can tell that the cipher I have recovered corresponds to the cipher that is still locked up in the safe. I can tell without putting the two side by side and making the comparison. From this we see that to know is very different from looking or having an image. One can uncover knowledge of some things without ever looking at them.
If we are to know, we have to have a criterion of truth. Without a criterion of truth, it is impossible to know.
It is false that to know, we have to have a criterion of truth. By following the theseological method we can uncover knowledge. But in following the theseological method we do not use a criterion of truth. Instead, we evaluate results by asking whether they are arrived at by following clues, and whether they lead to new clues. This is to say, we evaluate results by seeing whether they are arrived at by following the theseological method. When there is a method, there is no need for a criterion.
Scientists are rational, so are cryptanalysts, so is Sherlock Holmes. When they think a theory is true, their decision must be based on some criteria. By looking to actual cases in which these people make this kind of evaluation, we should be able to discover the criteria they use. What criteria have you found? How do you justify them? How do you justify those criteria by which you justify the criteria you have found?
Cryptanalysis is a rational activity. In cryptanalysis not every theory we advance is equally good. In cryptanalysis we evaluate theories too. But this evaluation is not based on criteria. In cryptanalysis we evaluate theories by the theseological method, that is, by seeing whether they, the theories, are based on clues and whether they lead to more and more new clues.
It is true that science is rational but we cannot infer from this that scientists determine whether their theories are true or good by criteria. It is a mistake to think that the only rational way to evaluate is by criteria. Scientists follow clues and develop new clues from old. They follow the theseological method. They, therefore, evaluate theories the same way we do in cryptanalysis.
Because of the common but mistaken assumption that rational evaluation is necessarily evaluation by criteria, much effort has been put into looking for those supposed criteria by which scientists determine whether their theories are true. None has been found. None has been found because there is none to start with.
If we evaluate theories by criteria, the question will arise as to how the criteria are to be justified, and this will lead to an infinite regress. But since we do not evaluate theories by criteria, there is no infinite regress.
In cryptanalysis we evaluate theories by seeing whether they lead to more and more new clues. Is this not evaluation by criteria?
We have to be careful here. In cryptanalysis when a theory leads to new clues, it is likely to be right. But we cannot be certain. So we wait and see if it leads to more clues. In the theseological process we have to evaluate results over and over again; we can never arrive at an absolute, final verdict. But in traditional discussions about criteria, criteria are expected to give final verdicts. According to Descartes, once we can see that an idea is clear and distinct we know it is true, so true that even the Evil Demon cannot make us doubt. Now I do not wish to argue about words. In my view what is important is to understand how results are evaluated in the theseological process. Is it done once and for all? Or does it take place over and over again, as the investigation continues? If my theory about the knowing process is correct, I have no objection if it is rephrased using some other words, provided that we do not in the process introduce confusion.
Knowledge is the result of our imposing order upon the world; is it not? Take the theories in science for example. These are, indisputably, human creations. We create these theories and then look at the world through these theories we have created. By doing so we introduce order into the world. But since different people can come up with different theories, there is no such thing as a true theory; there are just different ways of imposing order upon the world.
Suppose we are given a cryptogram containing altogether 157 letters. On the surface these 157 letters do not make any sense. But suppose we have managed to re-arrange these 157 letters in such a way as to produce a message that makes sense, can we be said to have solved the cryptogram? That is, broken the cipher?
We cannot. Rearranging the 157 letters simply with the view to forming a message that makes sense——this is anagramming. Anagramming is not cryptanalysis. In anagramming one can rearrange the given set of letters any way one pleases, so long as one produces, each time, a 'message' that makes sense and no letter is left over. There are no other rules. This is different from cryptanalysis. In cryptanalysis one has to discover not just the hidden message but also the set of rules (the cipher) which transforms the original message into the cryptogram.
In anagramming, from the same set of letters many messages (anagrams) can be formed. In cryptanalysis, to any cryptogram there is one and only one plaintext. (If we have found more than one, we know by this very fact that we have not succeeded.)
In anagramming we impose order onto a set of letters that apparently has no order. We do this any way we please. In cryptanalysis we do not impose order on a set of letters that apparently has no order. The cryptogram only appears to have no order. In fact there is a message behind it, a message encrypted by a cipher. In cryptanalysis we want to recover both the message and the cipher.
In anagramming, different people are likely to impose different orders on the same set of letters (since so many different messages can be formed). In cryptanalysis different people, if they succeed, will recover the same message and the same cipher from the same cryptogram.
In cryptanalysis we have to advance theory. These theories we form on our own. But that we do this does not mean we are imposing order upon the world. Not every one who advances theory is trying to impose order upon the world. Some, through the theories they advance, are trying to elicit order that is present in the world. This is what cryptanalysts do. This is what scientists do when they follow the theseological method. In that many of the theories scientists have advanced have led to more and more new clues, they, scientists, seem to have met with some success in their attempts at elicitation.5
Scientists look at the world through the theories they have created. They then justify their theories by the facts, that is, by how the world appears to them. They go around in a circle. But then so does everybody else. All theories, whether in science or in cryptanalysis or in crime detection, are self-validating, that is, always true in the views of those who advance them.
In cryptanalysis, after we have proposed a hypothesis, the next thing we do is interpret the ‘world’ we are investigating in terms of that hypothesis. For example, suppose there are many Ss in the cryptogram and our hypothesis is that ciphertext letter S stands for cleartext letter T. Once we have put forward this hypothesis, very naturally we will translate all the Ss in the cryptogram to Ts (see below). There is no mystery in this; it is a perfectly rationally thing to do.
But some will ask, by doing this are we not going around in a circle? Are we not prejudging the hypothesis we put forward? For in cracking the cipher the cryptogram is our evidence. If we translate all the Ss in the evidence to Ts, then of course the evidence will agree with our hypothesis! This is to say, if we always interpret the evidence in accordance with the hypotheses we put forward; if we see the world in terms of the theories we have created; whatever hypotheses we put forward will always be right.
But clearly, this cannot true. It cannot be the case that in cryptanalysis every hypothesis we put forward will be correct. It is common experience in cryptanalysis that many of the hypotheses we put forward are wrong. In cryptanalysis we interpret the evidence in accordance to the hypotheses we put forward but this does not mean every hypothesis we put forward will be regarded as right by us.
Why is this the case?
If we look to our cryptanalytic example we will see the reason.
Here the hypothesis is that S stands for T. All the Ss in the ciphertext have been translated to Ts in the cleartext. At this point, will we say that the hypothesis——that S stands for T——is correct?
Clearly, we do not. We do not know yet the rest of the message!
What does this show?
It shows that when following the theseological method we do not determine that a hypothesis is correct because it agrees with evidence after interpretation.
What do we do instead? How do we determine whether our hypothesis is correct?
We see first whether our hypothesis is prompted by clues. Then we wait and see if it leads to new clues. If the hypothesis is prompted by clues, if has a chance of being right. If it leads to new clues, its chance increases.
Scientists create theories and look at the world through the theories they have created. This does not mean theories are self-validating. Scientists follow the theseological method. Their theories are in response to clues. Some of their theories have led to new clues. Others have been found, by scientists themselves, to be wrong. Scientific theories are not self-validating. Neither are theories in cryptanalysis.
Science is exact. Scientific method, if there is one, must be exact too. This is to say, the rules making up the method that leads to knowledge should be like the rules in logic: they should allow of no exceptions. Is the theseological method exact?
The method that leads to knowledge of the hidden I call the theseological method. It is a method used by scientists, detectives, cryptanalysts, among others. If one wants to call it the scientific method, I have no objection.
But it is not an exact method! It allows for exceptions. The theseological method tells us to follow clues. But sometimes those who are familiar with the method will consciously go on a fishing expedition: they will make wild guesses fully aware that they are doing so. Why would they do this? Why would they when they know they should be following clues?
They do this when they run out of clues. A fishing expedition is a desperate measure but one that is sometimes resorted to, consciously.
Will it help?
Once in a blue moon, it does. Wild guesses have little chance of being right but little does not mean none. If the person making the wild guesses is lucky, one of the wild guesses may lead to new clues and in that way resuscitate an investigation that has stalled or begin one in which no clues have been found.
The theseological method allows for exceptions: we are allowed to make wild guesses sometimes. In this sense, the theseological method is not exact.
Perhaps it is exact in a different sense? Perhaps it is exact in the sense that the rules can be so clearly stated that we can know when we are following them and when not?
The theseological method has two rules. They both mention clues. Are clues always so clear that we know when we are following them or when we have developed them?
Any one who has had experience with clues will know they are not. Clues are the characteristics of structures, disguised. Things in disguise are hard to recognise. Clues are clear only from hindsight, when the disguise has been removed.
In using the theseological method we try to follow its rules. But because clues are things in disguise, we often fail despite good intentions. The rules in the theseological method are not so clear that we always succeed in following them.
Rules in the theseological method are not exact in more ways than one. In more ways than one, they are unlike the rules in logic. If we look for rules like those in logic in the theseological method we will not find them.
To any puzzling situation there is an infinite number of possible explanations. We can eliminate some of these as false and that is the best we can do: there is no way we can determine which of these possible explanations is the true one.
It is not true that we cannot do better. Sometimes we can. Sometimes, besides eliminating false explanations, we can also home in on the true one gradually. We can on those occasions when we have enough clues. We may never reach the true explanation, but we can come closer and closer. We can do so when we can follow the theseological method. In our SBR example, SB is puzzling. Why SB? What does it stand for? We have found an explanation by following the theseological method. SB, we have found, stands for TH. The theseological method asks us to follow clues and develop new clues from old. A clue could be vague. A vague clue cannot pinpoint the structure of which it is a clue but it can narrow down the range in which this structure can be found. When we develop new clues from old, we narrow down this range further. SB cannot be just anything; there is not an infinite number of possibilities for SB; There are a few decipherments of SB which are more likely to be correct than others. As SB leads to new clues, what SB stands for becomes even clearer.
The theseological method tells us to follow clues. When we are following clues we are reasoning. Following clues is a kind of reasoning. But reasoning cannot tell us what is true and what is false. Reasoning can only tell us whether we have arrived at a contradiction. Thus when we follow the theseological method, the most we can tell is that we have not arrived at a contradiction; we cannot tell what is really the case; do you not agree?
I do not agree. Although we do a lot of reasoning when we follow clues, following clues, we have said, is not just a matter of reasoning (Chapter 1). Before we can follow clues we have to detect them. Detecting clues requires more than just reasoning (Chapter 3). When Sherlock Holmes follows clues, he is following the theseological method. By following this method he can reconstruct what happened during the crime (Chapter 2). Both Sherlock Holmes and his audiences are interested in what is the case, not just in the absence of contradictions.
You say that when we have made serious mistakes in an investigation we will soon hit the brick wall. I do not see why this has to be the case. As you would admit, in an investigation there is nobody standing next to us holding a high whip, telling us what we should do and what not. In an investigation we are our own masters: we formulate our own hypotheses and we determine by ourselves whether these hypotheses are correct. So long as we don't involve ourselves in logical contradictions, we should be free to do whatever we want. Why then should an investigation ever come to a stop if we don't want it to? True, from time to time we will meet with difficulties, but what difficulties can we meet with which we cannot overcome with a little imagination? How then can we ever be forced to admit that we have hit a brick wall? Or if we have, what will stop us from driving through it? You said there was a brick wall in the study of blackbody radiation? Who is to tell? After Planck's discovery, we say there was. But somebody else could have come along and dealt with the problem in a different way. If this had occurred there would not have been a brick wall.
It is true that in an investigation there is nobody standing next to us holding a high whip, but this does not mean we can do anything we like in an investigation. Theseology is an empirical art (Chapter 3). If we want to understand this empirical art, we have to experiment. Can we really find out things hidden? I have suggested that we should experiment——for example, by making up a cipher of our own and challenge others to break it. I would not suggest to any one that they commit a crime and wait for Sherlock Holmes to come along.
Unfortunately, some do find out the hard way why things hidden can be known. There was once a government minister who while in power advised those officials working for him to deny and deny and deny whenever they did not want to admit publicly to a mistake that they had made, or rather that he had made. When later he was charged with the murder of his wife, he followed his own advice. He denied and denied and denied. He was still denying when he was sent to prison.
Can we refuse to admit that we have hit a brick wall when we have? Can we plough through a brick wall? In this instance too, I suggest we experiment. Make a serious mistake in cracking a cipher and see if we can still proceed without admitting that we have made a mistake. Of course, if you are not interested in cracking the cipher you do not have to admit to the mistake. If you are not interested in cracking the cipher you can do whatever you like.
Any theory can be defended come what may. For to show that a theory is false, we have to show that it clashes with experience. But theories are never confronted with experience one by one; instead, they are confronted in groups. When a group of theories clashes with experience, it is up to us which theory to replace in order to restore harmony between theory and experience. If, therefore, there is one theory in the group we want never to abandon we are at liberty never to do so: we can always make changes elsewhere in the group.
This thesis that any theory can be defended come what may is true if one’s intention is to win an argument at all costs. For in that case everything else can be sacrificed in defence of that one theory, including interest in the truth. But if one’s intention is to uncover knowledge, this thesis is false. To uncover knowledge one has to follow clues and develop new clues from old. In the process, if one holds on to a theory that is seriously defective, one will deprive oneself of the chance to detect clues or develop new clues.
There is also this important point. In an investigation a correct theory will harmonise with experience, but this does not mean a theory that harmonises with experience is necessarily true. There is more we can ask of a theory than that it should harmonise with experience. In addition to harmonising with experience, we want a theory to help advance the investigation. If we defend a theory come what may, by doing so we could prevent the investigation from advancing, and thus prevent ourselves from finding out more.
When we are looking for knowledge, we are looking for something we do not yet know. If we know already there is no need to look. But it is impossible to look for what we do not know. For how are we to do it? Where do we start? How will we know we have found it when we do not even know what we are looking for?
This argument can be found in Plato’s dialogue Meno, and is nowadays called the Meno Paradox. It is supposed to show that knowledge is impossible, or rather, that the search for knowledge is impossible. But since we regularly search for knowledge and succeed (to various degrees), the argument has to be fallacious.
Why fallacious? Where is the mistake?
Here is the mistake. The argument has overlooked an important point: we do not look for what we do not know the same way we look for what we already know. This needs explanation, which I now provide.
When we are looking for something we already know, what we do depends on what that something is. If I am looking for my glasses, I will try to remember when and where I was last wearing them. If I am looking for a map of Canada, I will look up an atlas. In these cases too, when I have found what I am looking for I will know. I am familiar with my glasses: they have a certain shape, they enable me to see; so when I have found them a brief examination will assure me that I have. Similarly with the map of Canada. We know what a map of Canada looks like, we know what we are likely to find on it, and usually a map of Canada carries the logo, 'Map of Canada'. When we have found a map of Canada, we should have no problem determining that we have.
But we do not look for what we do not know in the same way. Nor do we decide whether we have succeeded in the same way. Let’s illustrate this with the way we crack a cipher (cryptanalysis).
When we are trying to crack a cipher we are looking for something we do not know. We do not know how the cipher works; that’s why we want to engage in cryptanalysis. But obviously we would not go about this the same way we look for something we already know. For example, we would not start by enlisting the help of our enemies on the ground that while we do not know what we are looking for, they do. We would not say to them, ‘We don’t know this cipher we are looking for; you do. Tell us therefore what it is like so that we can look for it. Where did you use it last? Is it still on your desk or has someone put it back in the vault? Also, kindly give us a photocopy so that when we have found your cipher we can check our results, to make sure we are right.’ We do not go about cracking a cipher this way. We crack ciphers on our own, in the dark, in secret. We do not let our enemies know; we do not communicate with them; we do not ask them for directions; we do not expect they would help. If they would answer questions; if they would give us copies; if they would help us determine whether our results are correct——if they would do all these things, there would have been no need to engage in cryptanalysis in the first place.
But if this is not the way to crack a cipher, how do we actually do it? Can we crack a cipher without first knowing what cipher it is? Can we crack a cipher without ever having been provided with a photocopy? Autographed to ensure that it is genuine? Can we look for what we do not know by ourselves, staying always on our side of the fence, never crossing over? Can we uncover knowledge by looking at the world only from our point of view, barred as we are from God’s?
Obviously we can. There have been innumerable cases in which cryptanalysts have broken ciphers without having been given copies, autographed or otherwise. In the Middle Ages cryptanalysis was called the Black Art: it was thought possible only with supernatural help, help in this case provided not by God, but by the devil. But nobody believes cryptanalysis is a Black Art any more; there is nothing supernatural about cryptanalysis; it is a perfectly rational activity.
How do cryptanalysts find what they do not know?
Cryptanalysts find what they do not know by following clues and by developing new clues from old. Following clues and developing new clues from old——this is the way to crack ciphers, this is the way to find what we do not know. We do not find what we do not know by doing the impossible, by first knowing what we do not know, that is, by knowing and not knowing at the same time.
Ciphers are created by human beings. The structures we find in nature——are they not created by a god?
Not all structures are created: some are, and some not. Ciphers are created by human beings; viruses are not: they are the products of evolution.
There are breakable ciphers and unbreakable ciphers. An unbreakable cipher does not have a structure. It is possible that parts of nature have no structure. But one cannot argue to the existence of a god both from the presence and absence of structure in nature.
Suppose I have broken a cipher and I want to know who its author is. I cannot do this simply by argument. I have to carry out an investigation, that is, I have to follow clues and develop new clues from old. This means this person who created the cipher have to have characteristics; she cannot be beyond all characterisation, as some people believe some gods are.
Sherlock Holmes is a product of his time, and his method, which you call the theseological method, is nothing more than a means of oppression——of the poor by the rich, of the powerless by the powerful. For who are the criminals in real life in Victorian England? They are not usually the gentle folk that you find in these stories, but those people these stories seldom mention: the underclass. You say the theseological method is the same as the scientific method. What do scientists do? They put nature 'on the rack', as Bacon says. Look also at the language you use in cryptanalysis. You speak about breaking a cipher. What is to break but to use force?
There is nothing inherent in the theseological method that says it can only be used as a means of oppression. Most of the criminals in Victorian England were poor and powerless but there are lots of crimes committed by the rich and powerful at the present time and they are caught (at least sometimes) by using the same method.
Is the method violent? Certainly that there is such a method, a method that allows us to uncover the hidden, is unexpected, counterintuitive. In the Middle Ages cryptanalysis was thought humanly impossible. Those who could crack ciphers, it was thought, must have been in league with the devil. Even Descartes thinks that if we are to know; if we are to have knowledge about nature; we will have to be more powerful than the Evil Demon. Perhaps it is from this kind of history that we get the idea that the theseological method is violent; it is violent because it is 'unnatural'——it is counterintuitive that there should be such a method, and what is counterintuitive is unnatural. Is it really violent? Is there any violence involved in solving our SBR cryptogram? I do not see any.
How did people solve crimes not so long ago? In fact how do many people solve crimes even now? They snatch some poor souls in the middle of the night and torture them until they confess. Now the theseological method produces better results than torture. Instead of false confessions (common when people are tortured), the theseological method leads to accounts that are much closer to the truth. Perhaps this is another reason why some people think it is violent. If it produces better results than torture, it must be more violent than torture. But clearly, this reasoning is absurd.
Knowledge is power. Sherlock Holmes has power over criminals——he can send them to prison, for example——because he can find out what they have done. Should we therefore not look for knowledge because it gives us power, power not only over nature but people? There can be dangerous corners when we are looking for knowledge, corners at which we have to slow down or stop if we are to prevent a catastrophe, but few I think would take it as a good general policy to give up the search for knowledge altogether and think it better that we should live in ignorance.
Myths spring up when knowledge is absent. There have been many myths surrounding the knowing process. These myths can be dispelled but only if we succeed in understanding the knowing process the way it ought to be understood.
You say that when we want to find out things hidden we should follow clues and develop new clues from old. But look at the wrongful convictions that have occurred. When so many people can be sent to prison or even executed as a result of following clues, how can any one reasonable believe that clues can lead us to the truth?
In writing this book I am not defending every one who follows clues. On the contrary I am suggesting that we should be more critical of the theseological process than we have been in the past. Clues lead us towards the truth only if we follow them properly, and I think it should have been clear from this book that it is not easy to be following clues properly. The theseological process is complex and when we are engaging in this complex process it is easy to make mistakes. In order that we can avoid these mistakes we have to understand this process; which is what this book is trying to promote.
Moreover, how do we find out that people have been wrongfully convicted? Do we simply take them at their word? Most people convicted of crimes say they are innocent. To find out whether a person has been wrongfully convicted we have to investigate, which means following clues and developing new clues from old.
When more people have a proper understanding of the theseological process, there should be fewer wrongful convictions. Keep in mind that verdicts in a court of law are often rendered, not by judges, not by lawyers, not by detectives, but by juries.
Perhaps the theseological method is culture- specific? Conan Doyle got his idea for Sherlock Holmes from Joseph Bell, one of his professors when he was in medical school at Edinburgh. Sherlock Holmes himself says his method is no different from the scientific method. Now science is a western creation, so perhaps this method works only in a western cultural context? Take it away from its native culture and it would not make any sense at all.
It is true that the kind of science we have grows up in the west, but secret writings can be found in many places in the world dating from some 4,000 years ago.6 Now when people use secret writings, there will always be others who want to decipher them. It seems therefore that the art of detection is not something that only the west could have discovered; it could have been discovered——in the sense of being practised——over and over again in different parts of the world. Indeed, children quite often discover how to crack simple ciphers on their own; they do not have to be taught.
Possibly, therefore, the theseological method is not unique to the west; it could be present in many cultures. It is possible, however, that while it is practised in other cultures, less importance is attached to it in view of the difference in value structures.
1After they have led to other clues.
2Despite his doctoring of the results of the experiments to make them into a clearer clue, as Wendy Lai reminds me.
3Descartes thinks human beings cannot know without God’s help. Many philosophers nowadays still think that knowledge is possible only if we are able to see things from God’s point of view.
4We can add more detail to this answer; see Lai .
5Given a cryptogram we could either try to cryptanalyze it or form anagrams from it; it is up to us what we do. If we decide to form anagrams, our enemies will not be there to stop us. But the two activities are not the same; anagramming cannot be substituted for cryptanalysis. This is to say, after we have taken notice of the world we could try to impose order upon it or elicit the orders inherent in it. We may not succeed in the latter, or not fully succeed, but we should not deceive ourselves into thinking the two activities are the same, nor that one can substitute for the other.
6See David Kahn, The Codebreakers. New York: Scribners, 1967.