Stochastic Approach on Hash Cracking

Stochastic Approach on Hash Cracking Goran Martinovic, Luka Horvat, Josip Balen Faculty of Electrical Engineering, Osijek, Croatia goran.martinovic@et...
Author: Hilary Powell
0 downloads 0 Views 502KB Size
Stochastic Approach on Hash Cracking Goran Martinovic, Luka Horvat, Josip Balen Faculty of Electrical Engineering, Osijek, Croatia [email protected]

Abstract—Hash functions are used everywhere today, from wireless communications to password storage. Some of them are proven to be mathematically insecure, but still the only way to crack most of them is a brute-force or dictionary attack. Brute-force is usually discarded as an option since it consumes huge amounts of time. Dictionary attack is a regular approach on hash cracking. The attack is performed by selecting words from a dictionary and comparing their hashed value to the obtained value. Word selection is usually done sequentially, which indicates a very long execution time if the searched word is located at the end of the dictionary. This paper introduces a stochastic approach on word selection in a dictionary attack and shows that the approach is desirable if the searched word is found in the second half of the wordlist. Keywords - dictionary attack; randomization; stochastic approach

hash

cracking;

I. INTRODUCTION Hash value is a fixed-size bit array that “presents” an arbitrary block of data. Hash value is created with a hash function using a block of data as its input. Functions are made so that any change to the input sequence should induce a change in the hash value. Most frequently used hash methods exploit bitwise operations to produce message digest. An important property of the hash function is that there should be no two different inputs that can induce the same hash value (collision resistance). Also, hashing functions are one-way functions. Therefore, there should be no analytic way for obtaining an input sequence from its hashed value. However, some of today’s hashing techniques have been proven as collision irresistible and analytical ways for their cracking have been found. In [1], authors explain how to crack an MD5 hash using collision attacks, and in [2], even newer hashing standards are exploited. Still, hashing is considered the safest way for storage and transfer of delicate information such as passwords. Most standards apply numerous iterations of hashing sequences to produce the final value and store or transfer it, which complicates the analytical cracking process. Breaking hashes is usually brought down to preimage attacks that can take huge amounts of both time and processing power. Preimage attacks, also known as brute forcing, are usually unpractical because their complexity depends on the digest size (search takes 2L steps, where L is the number of bits digest is made of). Hash attacks can be online or offline. In [3], authors explain the background of undetectable online password guessing attacks in authentication based protocols, and in [4], authors expand the Dolev – Yao model to give the theoretical background of offline dictionary attacks.

This paper concentrates on the offline hash cracking method with the hash value obtained. When this is the case, an attacker usually has the time to conduct his attack in a variety of ways. Loads of free tools are available online [5] and all of them support parallelism. Some can use the power of graphics cards or other devices, as described in [6] and [7], which can improve the cracking rate by a couple of hundred times. Online services [8] offer password cracking on the desired set of characters – for cash. Those tools usually use dictionaries (wordlists) or pre-computed rainbow tables and brute forcing is rarely considered. Although wordlists are usually sorted, binary searches and similar ones are out of the question due to hashing. Therefore, most hash cracking techniques rely on grabbing lines sequentially from the wordlist, performing certain hashing on them, and comparing hashed values to the captured one. This methodology is known as a dictionary attack. Lines grabbing is done sequentially which leads to a worse case running time of N steps, where N is the size of the wordlist. This process is equal to any array search algorithm, except most of the processing power here is spent on the hashing. The idea given here is quite straightforward: What if a randomized word selection can find the desired word in fewer steps than a sequential one in a dictionary attack? The goal of this paper is to show when the stochastic passphrase search will outperform the sequential one and why. The rest of this paper is organized as follows. First, a dictionary attack is described in Section II. Arguments for using the stochastic approach are given in Section III. In Section IV, an algorithmic solution is described. Experimental results and their evaluation are given in Section V. Section VI concludes this paper. II. DICTIONARY ATTACK A dictionary attack is a technique usually used to crack password-protected security systems, like computers or servers. An attack is performed by grabbing words from dictionaries and entering these words as a possible passphrase. Dictionaries are wordlists with a prearranged list of words and words grabbing is done sequentially. Dictionaries can be arranged in a way that phrases with higher probability of success are in the beginning, and vice versa. Used dictionaries in this work are arranged alphabetically. Dictionary attacks have a high efficiency if the passphrase is an ordinary word. Furthermore, dictionary attacks are often used in finding the key necessary to decrypt an encrypted message or document. In breaking cryptographic hash functions, a dictionary attack is performed by selecting a word from a dictionary,

performing certain hashing on it and comparing the hashed value to the obtained value. In [4], three different examples of dictionary attacks are presented: 

Native vote protocol: If the sender encrypts his vote with the public key of the receiver, only the receiver can decrypt the message with his own private key and register the vote. However, if the attacker knows a finite set of values that vote can take, than he encrypts these values with the public key and compares encrypted values with the vote. The attack is successful if the vote equals one of the encrypted values.



Handshake protocol: A sender generates a random number and sends it encrypted with a password to a receiver. Both a sender and a receiver know the password. After the receiver decrypts the message, he computes the sent number plus one, and returns it to the sender. If the password is chosen poorly, the attacker tries to decrypt both messages by guessing the password. The attack is successful if it finds two values such that the second value equals the first value plus one.



Enhanced Kerberos protocol: A sender and a receiver communicate over the server which provides a secret key. The password between the receiver and the sender is assumed to be very strong but the password between the sender and the server is set up by the user and it could come from a dictionary. However, this protocol implements some protections against dictionary attacks using the nonces, the confounder and the timestamps. If the clocks of the sender and the server are not well synchronized, the attacker can replay a copy of an eavesdropped message within the clock skew and perform the attack.

III. STOCHASTIC APPROACH A negative side of randomization and its implementation in search algorithms is that they are time consuming. Using the solution that can be found in [9], execution of randomized search could be up to four times longer than sequential one. On a three million words long wordlist, randomized search took on average 1.2883 seconds, while the sequential one took 0.3161 seconds. This difference would be huge if the searching is the only operation executed per search. It must be kept in mind that with every examination, a real cracking program has to carry out a certain amount of hashing (the number of hashing iterations is usually measured in thousands). RAR passwords, for example, apply up to 262144 iterations of SHA1 (Secure Hash Algorithm) [7] and that requires great amounts of processing time – much more than simple operations applied from [9]. Due to aforementioned, it can be assumed that a one whole iteration of grabbing a word, hashing it and comparing to the captured hash value lasts nearly the same as if it is done either sequentially or at random. Since all words in a wordlist must be examined, the number of steps of the stochastic approach in the worst

case is the same as sequential. Furthermore, probability P(X) that a word will be at line 5 of a wordlist and at line N-5 is equal in both approaches, but the probability of a stochastic search algorithm to hit any line at the beginning of the search is the same for all lines. This is due the wordlist being sorted alphabetically, not by decreasing probability of success, and the search algorithm not being biased towards certain parts of the wordlist (meaning that the algorithm works on the principle of discrete uniform distribution). Random variable X has a discrete uniform distribution over set S = {1, 2, ..., k} (where k is a natural number) if (1) holds. f(x) = P(X = x) = 1/k, x  S

(1)

P(X  S) = 1 Wordlist is countable and every word in this list is unique, therefore every word is a discrete variable. This induces the same overall running times in both searching approaches when hashing is included. But, if a word that the program is searching is located in the second half of wordlist, the best case running time of a sequential approach is N/2, while that of a stochastic approach is still 1. Note that sequential and stochastic algorithms search the whole wordlist with the same probabilities of each element to be picked. However, the wordlist is sorted and sequential search always starts from the beginning and it always performs search from the first to the last word. Contrary to this, stochastic search retrieves words randomly. Therefore, if the word we are searching for is closer to the end of the wordlist, at the beginning of the search best case running time of a classical approach is very high while stochastic is still 1. However, if the word being searched for is found at the beginning of the sorted wordlist, the probability that a sequential approach will find it first is greater than the one of the stochastic approach. The probability that random search will find the searched word in less than X steps at the beginning of the wordlist is less than X/N, where X is the number of steps required for a sequential approach to find the word. IV. ALGORITHMIC SOLUTION When comparing stochastic and sequential approaches on hash cracking, two different metrics were observed. In the first case, we assumed that the time difference between random and sequential search can be negligible due to its insignificance compared to time consumption of hashing iterations. Therefore, only plaintext words are compared. As measure of efficiency an average number of iterations needed to found the correct word is used. In the second case, experiment was conducted with included MD5 hashing with 30 hash iterations. The metrics was an average time needed to found the correct value. The solutions were developed under C++ language and a pseudo code can be seen in Algorithm 1. A publicly available wordlist is taken and loaded into RAM as a dynamic array of strings (variable str in Algorithm 1) where each element is one word (one line

Algorithm 1 WordSearch (wordlist, test_case) str ← wordlist major_counter ← 0 for i ← 0 to 10000 do x ← rand() mod(test_case) answer ← hash(str[x]) for j ← 1 to N do hit ← hash(str[j]) if (answer = hit) break for k ← 1 to N do temp ← rand() mod(N) hit ← hash(temp) if(answer = hit) break _temp ← str[k] str[k] ← str[temp] str[temp] ← _temp major_counter ← (j - k) + major_counter return (major_counter/10000)

of the average execution duration for the stochastic and sequential approaches is used in the comparison. The search process comprises nine test cases. In the first, searched word is located in the first 1/5 of the wordlist. In the next test case, the word is located in the first 2/5 of the wordlist, and so on. In the last test case the searched word is located in the last 1/5 of the whole wordlist. Table I shows locations of the searched word for all nine test cases. Ranges in Table I have been selected in order to show the relation of the position of the searched word in the wordlist to the difference of steps taken by each process or in the time domain for the comparing the time needed to find the searched value. Since the wordlist is sorted, it can be assumed that when the searched word is near the beginning of the wordlist, sequential search will be faster. In contrast to this, randomized search should show increasingly better performance than sequential if the searched word is located at the end of the wordlist. TABLE I. Test case

from the list). The program allocated 230 MB of RAM to hold 6 million words. The first problem encountered was a limit of C++ rand() function that even after some redefinition of constants in libraries, could not return a value larger than 0x7FFF or 3276710. Therefore, problem solution was designed in the way that every digit is randomized individually with values 0 – 9. Consequently, a 7-digit number can be generated with certainty that no values are missed. The next problem was redundancy of random values but a solution to this problem is proposed in [9]. With implementation of these lines all obstacles were removed, and a solution was ready to be coded. For a sequential solution, for loop that checks elements in a sorted wordlist from index 1 to N was used. When the searched word is found, the overall number of for loop iterations is assigned to the corresponding counter in the case when metrics is an average number of iterations. In the case when metrics is time, the duration of each execution was measured. A stochastic solution basically also runs in the for loop that goes from 1 to N, but this time the current state of iteration is not used to check elements, rather the index for checking is randomized, as shown in Algorithm 1. After the element is checked, it is swapped with the element indexed by the current iteration of the for iteration, and therefore this value could be checked only once. After the element is found, the overall number of for iterations is assigned to the corresponding counter or the duration of each execution was measured. At the end, a major_counter is calculated as a difference between the overall number of sequential iterations and the overall number of random iterations needed to find the searched word. This whole procedure is repeated for 10,000 times in order to obtain greater precision in measurement. The average value of major_counter is used in comparison of the sequential and the stochastic process in the case when metrics is the average number of iterations. In the case when metrics is time, the difference

RANGES OF PICKED WORDS

Parts of wordlist taken for selection

1

1/5

1/5

1/5

1/5

1/5

2

1/5

1/5

1/5

1/5

1/5

3

1/5

1/5

1/5

1/5

1/5

4

1/5

1/5

1/5

1/5

1/5

5

1/5

1/5

1/5

1/5

1/5

6

1/5

1/5

1/5

1/5

1/5

7

1/5

1/5

1/5

1/5

1/5

8

1/5

1/5

1/5

1/5

1/5

9

1/5

1/5

1/5

1/5

1/5

V. EXPERIMENTAL RESULTS Five sizes of wordlists have been used for the testing purposes, containing: 

0.3 x 106 words



0.6 x 106 words



1 x 106 words



3 x 106 words



6 x 106 words

For each size of the wordlist there were nine test cases. Therefore, in each of 10,000 iterations of a certain test case a word the program is searching for is located in the particular range of the current wordlist. Still, the scope of both sequential and stochastic search is the whole wordlist.

In Fig. 1, results of comparing an average number of iterations in all five sizes of wordlists are shown. On the X axis there are test cases from 1 to 9, and on the Y axis an average value of major_counter is shown. Considering major_counter showing an average difference of the number of iterations needed for the sequential search and the number of iterations needed for the stochastic search, it can be seen when the stochastic process outperforms the sequential one and vice versa. The value of major_counter in the first four test cases for all five wordlists indicates that if the searched word is located in the first 4/5 of wordlist, the sequential process finds the searched word in less iterations than the stochastic process does. The sequential process has the best results in all five test cases in comparison with when the word is located in the first 1/5 of the wordlist. In the first test case with 0.3 x 106 words, the sequential process has 41.20% better search results while in the biggest wordlist with 6 x 106 words search results are 37.71% better. In the test case 5, when the domain is the whole wordlist, both sequential and stochastic processes have almost equal results. This means that if we do not know in which part of the wordlist the searched word is located, the average number of iterations needed to find the searched word is nearly the same for both processes. In test cases 6, 7, 8 and 9 it can been seen that if the searched word is located in the last 4/5 of the wordlist, the stochastic process shows better results. If the searched word is not located only in the first 1/5 of the wordlist (test case 6), the stochastic process already shows

5.24% better results in 0.6 x 106 words big wordlist to 13.12% in the wordlist with 6 x 106 words. In test case 9 when the word is located in the last fifth of the wordlist, the stochastic process shows around 40% better results in the wordlist with 0.3 x 106, 3 x 106 and 6 x 106 words, 30.53% for 1 x 106 words and 22.65% for a 0.6 x 106 words big wordlist. From this we can conclude that our hypothesis of the stochastic process outperforming the sequential one when the searched word is not located at the beginning of the wordlist is valid. Fig. 1 f) shows experimental results for all five sizes of the wordlist. The difference in gradients is due to the size of the wordlists. From this line graph we can conclude that experimental results for all sizes of wordlists show almost linear distribution through test cases. For bigger wordlist sizes, the number of iterations of the sequential process increases with the wordlist size and it is much greater than the number of stochastic iterations if the searched word is located at the end of the wordlist. From this we can conclude that for large wordlists a significant increase in performance could be obtained with the stochastic process if the searched word is closer to the end of the wordlist. In test case 9 of the largest tested wordlist, the stochastic process is faster for an average of over 2 million steps or 39.35%. Due to the long execution time, in the time domain experiment only wordlist with 1 x 106 words was used. As shown in Fig. 2 and similar as in Fig. 1, sequential

Figure 1. a) - e) Major_counter values for different wordlists sizes through cases. f) Overall results of testing.

Time difference in miliseconds

1 000 000 words

1000

500 0 1

2

3

4

5

6

7

8

9

-500

therefore, it decreases the search time. This difference of steps taken, increases as the size of the wordlist increases. On a regular basis, stochastic search is not better or faster – it is as fast as the sequential one on a large pattern. This paper gives another point of view on the problem of an offline dictionary attack, and another type of an approach to performing it. This solution could be implemented in any search method with linear time complexity such as simple array searches, string matching algorithms [10] or rainbow cracking techniques.

-1000

ACKNOWLEDGMENT -1500 Number of test case

This work was supported by research project grant No. 165-0362980-2002 from the Ministry of Science, Education and Sports of the Republic of Croatia.

Figure 2. Comparison in the time domain

REFERENCES approach shows better results in the first four test cases. In the test case 5 results are almost similar. When the searched word is located in the second half of the wordlist, stochastic approach shows shorter average time needed to find the corrected value with included MD5 hashing. These results confirm the hypothesis that due to the long duration of hashing iterations, sequential and stochastic approach lasts nearly the same in every execution. It can be concluded that a stochastic approach outperforms the sequential one in cases when the searched word is located in the second half of the wordlist. However, since the most users (attackers) do not know in which part of the wordlist the searched word is located, the stochastic approach does not guarantee a lower number of iterations than the sequential approach. VI. CONCLUSION Dictionary attack is one of techniques used in hash cracking. An attack is performed by selecting words from a dictionary and comparing their hashed value to the obtained value. As the dictionary is sorted, the word selecting is done sequentially and if the desirable passphrase is located at the end of the dictionary, worst case running time is often very long. This paper shows that randomization can be desirable in some search cases. Experimental results show that when the searched word is placed in the second half of the wordlist stochastic search needs fewer steps to find it and

[1]

[2]

[3]

[4]

T. Xie, D. Feng, “How To Find Weak Input Differences For MD5 Collision Attacks,” Cryptology ePrint Archive, Report 2009/223, May 2009, available at http://eprint.iacr.org/2009/223 K. Aoki, J. Guo, K. Matusiewicz, Y. Sasaki, and L. Wang, “Preimages for Step-Reduced SHA-2,” Proc. of 15th Int. Conf. on the Theory and Application of Cryptology and Information Security: Advances in Cryptology, Springer, Berlin, Nov. 2008, pp. 578-597. Y. Ding, P. Horster, “Undetectable On-line Password Guessing Attacks,” SIGOPS Operating Systems Review, October 1995, Vol. 29, No. 4, pp. 77-86. S. Delaune, F. Jacquemard, “A Theory of Dictionary Attacks and its Complexity,” Proc. of 17th IEEE Computer Security Foundations Workshop, Cambridge, UK, 28-30 June 2004, pp. 215.

[5]

Top 10 Password Crackers, available http://sectools.org/crackers.html, accessed: November 2010.

[6]

Y. S. Dandass, “Using FPGAs to Parallelize Dictionary Attacks for Password Cracking,” Proc. of 41st Annual Hawaii Int. Conf. on System Sciences, Hawaii, USA, 7-10 January 2008, pp. 485-485.

[7]

G. Hu, J. Ma, B. Huang, “Password Recovery for RAR Files Using CUDA,” Proc. of 8th IEEE Int. Conf. on Dependable, Autonomic and Secure Computing, Chengdu, China, 12-14 Dec., 2009, pp. 486-490. Question Defense, Online Hash Cracker, available at http://tools.question-defense.com, accessed: November 2010.

[8] [9]

at

C++ Notes: Example - Shuffle Array, available at http://www.fredosaurus.com/notes-cpp/misc/random-shuffle.html, accessed: December 2010. [10] F. Zhao, Q. Liu, “A string matching algorithm based on efficient hash function,” Proc. of Int. Conf. on Information Engineering and Computer Science, Wuhan, China, 19-20 December, 2009, pp. 14.

Suggest Documents