A Markov Game Theory-based Risk Assessment Model for Network Information System

Cui Xiaolin, Tan Xiaobin, Zhang Yong,Xi Hongsheng

Department of Automation University of Science and Technology of China Hefei, Anhui, 230027, P.R.China E-mail: cuixl@mail.ustc.edu.cn
AbstractRisk assessment is a very important tool to acquire a present and future security status of the network information system. Many risk assessment approaches consider the present system security status, while the future security status, which also has an impact on assessing the system risk, is not taken into consideration. In this paper we propose a novel risk assessment model based on Markov game theory. In this model, all of the possible risk in the future will impact on the present risk assessment. The farther away from now, the smaller impact on the risk assessment it has. After acquiring the system security status, we proposed an automatic generated reinforcement scheme which will provide a great convenience to the system administrator. A software tool is developed to demonstrate the performance of the risk assessment of a network information system and a simulation example shows the effectiveness of the proposed model. Keywords- risk assessment; Markov game theory; threat transmission

vulnerability, and could discover the hidden risks. But the model needs to the relationship between every two different vulnerability. So the relationship is very complex when the number of vulnerability in network information systems is large. In this paper, we present a Markov game theory-based risk assessment model for the network information system. In this model, we use a Markov chain to describe the spreading process of potential threats so as to assess the system risk. By using the Markov chain, we can simulate the threat propagation and discover the hidden risk. And we utilize another Markov chain to depict the repair process implemented by the system Administrator aiming at the system vulnerability, so as to make the amount of vulnerabilities utilized by threats be less and make the system safer. The rest of the paper is organized as follows. In Section 2, we briefly describe the related work. Section 3 presents our framework for risk assessment. In Section 4, we describe the Markov game theory-based risk assessment model for the network information system. Section 5 shows our experiment results and discussions. Section 6 concludes the paper. II. RELATED WORK

I.

INTRODUCTION

With the rapid development of the Internet, the amount and species of malicious codes also ever-increasing, which bring serious threat to the network security. To counter these threats, we need to identify those vulnerabilities that are susceptible to these malicious codes. Vulnerability is the absence or weakness of a safeguard in some asset or resource. This absence or shortcoming makes a threat or attack potentially more harmful or costly and more likely to occur. In order to understand the affects from these threats to the network information system security, we need to take a risk assessment on the network information system and improve the security situation of the system based on the assessment results. On the current information security risk assessment methods can be classified two categories: One is the traditional risk assessment methods, such as FTA [1], FMECA [2], HAZOP [3], Markov [4] and so on; The other is the modern methods of risk assessment, such as CORAS [5], RSDS [6], CRAMM [7], COBIT [8] and so on. However, these methods did not give the specific application for the network information system and not advise how to repair the vulnerability of the network information system. Zhang et al. [9] provides a risk propagation model for assessing network information systems, and give the specific application. This model considered the relationship between different

The purpose of risk assessment is to understand the present and future system risks, access the security threats and the degree of influence probably engendered from these risks, and provide the basis for security strategy identification, establishment and safe operation of the information system. In order to achieve the purpose, many countries and organizations have established the risk assessment audit standards, such as: CC [10], SSE-CMM [11], ISO/IEC 17799[12], BS 7799 [13], ISO 13335 [14], IATF [15] and GB/T 20984-2007 [16]. GB/T [16] puts forward the principle of risk calculation:

RISK = R( A, T , V ) = R ( L(T , V ), F (I a , Va ))

(1)

Where R is the function of security risk calculation; A, T, V denote asset, threat and vulnerability respectively; Ia denotes the value of asset acted security event; Va shows the harm extent of vulnerability; L denotes the probability of security event induced threats which utilize the vulnerability of asset; F is the loss took place after safety time.

This work is supported by the National 863 High-tech Program of China (No. 2006AA01Z449) and the 42nd National Science Foundation for Postdoctoral Scientists of China (No. 20070420738).

operation on system vulnerability by system administrator need to do so as to reduce the system risk to an acceptable range. III. FRAMEWORK FOR RISK ASSESSMENT

Figure 1.

Principle of Risk Assessment

Threat involves malicious code and network attack. In this paper we primarily consider the threats of malicious code. Malicious code is divided into five categories: Trojan horses, worms, viruses, spyware and corpse network client programmer. Vulnerability is consisted of management vulnerability and hardware and software vulnerability. Asset is described by Confidentiality, integrality and availability, and includes the importance of network location. In the network information system, for each asset, the threat induces risk by utilizing its vulnerability. In order to conduct risk assessment, first of all we should identify the threat. For an asset, the threats come from two aspects: the existing threats and potential threats. The existing threats are the threats which exist in asset and the potential ones mean that exist in network information system and the Internet rather than the asset. Although the potential threat does no harm to system at the present stage, it will endanger the asset by spreading through LAN and the Internet. Now, there are many threat identification methods [17], vulnerability identification methods [18] and asset identification methods [19].These methods provide the basic data for risk assessment. But how these data will integration is a very important task. Shen et al. [20] puts forward the Markov game theoretic data fusion approach. The Markov (stochastic) game method is used to estimate the belief of each possible cyber attack pattern and give the corresponding defensive strategy. Based on the principles of risk assessment and the Markov (stochastic) game method, this paper have proposed a Markov game theory-based risk assessment model (MGTBRAM). The model uses a Markov chain to describe the spreading process of potential threats so as to assess the system risk, and utilizes another Markov chain to depict the repair process done by the system Administrator aiming at the system vulnerability, so as to make the amount of vulnerabilities utilized by threats be less and make the system more safe. On the one hand, threats acting on vulnerability can induce risk and the risk will be larger and larger by threat spreading; on the other hand, the risk will be smaller and smaller by the system Administrators repairing the vulnerability. Thus we can establish a game relation between threats and vulnerabilities. In this paper we mainly discuss the following two aspects: 1) the changes of system risks in the next period of time if the system administrator doesnt repair any vulnerability. 2) The repair

Based on above principle of risk assessment, we suggest a novel design approach to risk assessment system, illustrated in Figure 2. This framework gives precise mathematical model to describe network risk. Especially, it gives a practical security reinforcement scheme used to guide people to improve network security. It is composed of five main modules. We will discuss them in details. Threat identification module: It detects malicious codes in each asset of the network system and stores these data in a database in a certain format. These data include threat name, threat type, asset IP, related vulnerability, probability of the threat spread and so on. Vulnerability identification module: it detects vulnerability in each asset of the network system and stores these data in a database in a certain format. These data include vulnerability name, vulnerability type, asset IP, harm extent of vulnerability and so on. Asset identification module: it detects the network system and evaluates the value of each asset by the importance of asset. Then it stores the value of assets and the IP of assets in a database. When assessing risk, the IP of assets will link to threat and vulnerability. MGTBRAM module: the data of above three modules will input into this module. By calculating, the output (max risk and reinforcement scheme) will input into risk assessment module. This module is the core of risk assessment and will be discussed in section 4. Risk assessment module: in this module, we will calculate the value at risk for each vulnerability, and recorded as R(i), i=0,1, , n. We assume that we have repaired each vulnerability according to the repair scheme, and calculate the value at risk for each vulnerability again. The new value will be recorded residual risk as RR(i), i=0, 1,, n. In order to give administrator a repair scheme list, we recorded eliminate risk as ER (i), i=0, 1, , n.
ER (i ) = R (i) RR (i) i = 0,1, ,n

(2)

Figure 2.

The Framework of Risk Assessment

Let ER(i) list in descending order and we will select several forefront ER as the suggestion to administrator. The administrator could use the least operation to minimize the system risk. The Markov game theory- based risk assessment model provides a new idea for the automatic generation of a reinforcement scheme. MARKOV GAME THEORY-BASED RISK ASSESSMENT MODEL Plays --- Two sides of Game are threat agent and vulnerability agent. The threat agent increases the risk by threat spreading and the vulnerability agent decreases the risk by system administrators repairing the vulnerability. State Space --- For the threat agent (denoted by t), at the moment of k, the threat state (TS) of the n-th asset is expressed as snt(k) and its value is 0 or 1: 0 denotes no threat and 1 denotes a threat. The threat state of the whole system at the moment of k is: IV.

For the vulnerability agent, we say the system administrator repairing one vulnerability as its one action. The repair scheme of system administrator about the vulnerability is described in the state transition rule. For simplifying the repair process of vulnerability, we assume that repair one vulnerability of asset once. At the moment of k, the repair process of the system administrator about the vulnerability of asset is label as uv(k), where v is the vulnerability agent. The State Transition Rule --- As time changes, the state of each asset in the network system also constantly changes. We use p(sk+1|sk,ut,uv) to describe the law of changing state, where sk+1, sk denote the states of the (k+1)-th and k-th moment separately, ut(k) and uv(k) denotes the actions took by the threat and vulnerability agent separately. At the moment of k, the threat and vulnerability agent will adopt corresponding actions according with their own strategy sets (Transmission Strategy Sets and Repair Strategy Sets) separately. Figure 3. shows the Markov game process.

st (k ) = ( s1t (k ), s2t (k ),

t , sn ( k ))

(3)

For example, assume that there are three assets, so (1,0,0) denotes the first asset has the threat and the second and third dont have the threat. Similarity for the vulnerability agent (denoted by v), at the moment of k, the vulnerability state (VS) of the n-th asset is expressed as snv(k) and its value is 0 or 1: 0 denotes no vulnerability and 1 denotes a vulnerability. The vulnerability state of the whole system at the moment of k is:
v s v (k ) = ( s1v (k ), s2 (k ), v , sn ( k ))

Figure 3. Markov Game Process

(4)

The risk state (RS) of network information system at the moment of k as s(k):

Damage Function --- When a asset has a threat, the threat will do some damage to the asset and the longer time the threat exists, the more the damages are. We label the damage value at the moment of k in the network system as V(s(k)) and in this system the damage value in unit time of the threat at the moment of k is R(s(k)). Thus:
V (s(k )) = R(s(k )) +
t v p(s(k + 1) | s(k ), u , u )V (s(k + 1)) (6) s(k + 1)

s(k ) = (s k ) * s k ), s k ) * s k ), , s k ) * s k ))

t 1(

v 1(

t 2(

v 2(

t n(

v n(

(5)

Action Space --- For the threat agent, we say one spread of the threat as its one action. The threat can be transmitted to other assets with certain probability which is given in the state transition rule. For simplifying the spread process of threat, we assume that the threat is spread to one asset once. In order to distinguish the main spread threat, we define source asset s and destination asset d. To one threat, source assets are internet or the assets which have the threat. And destination assets are ones which have no this threat. At the moment of k, the action is the process of threat spreading from source asset to destination asset. At the moment of k, threats through a variety of ways from the source of assets to the purpose of dissemination of assets via various ways is labeled as ut(k) t is the threat agent. Threats have many spreading modes and we mainly consider the following several modes: removable Storage, EMail and downloads, shared directories and so on.

Where s(k+1) is the system state at the moment of k, which is related with s(k), ut, uv, so we need to sum all possible states of s(k+1). is a discount factor. p(sk+1|sk,ut,uv) is the state transition probability. That is probability of the system risk state changed into s(k+1), when system risk state is s(k) and the actions of the threat, vulnerability agent are separately.
R( s(k )) = VA s (k )

(7)

V denotes the harm extent of vulnerability and A is the value of asset. In order to facilitate to calculate the risk value of system, we only consider the changes of system state within n steps. According to the Damage Function, we can calculate the system risk induced from each threat and accumulate all these risks to get the aggregate risk value Vsys of the system.

Vsys = Vi
i

(8)

Risk Assessment --- When the risk assessment, the system administrator not to take any repair scheme. The vulnerability state remains unchanged. Therefore, the process of Markov game regresses into the process of Markov decision. Given a speed of threat transmission, we select a transmission strategy of threat to make the system risk values be largest and utilize this largest risk value to assess the risk.
V ( s ( k ) ) = m ax V t , v ( s ( k ) ) t T

VDS The vulnerability state (MS07-037) (VS) which is related to the threat we have detected is sv(k). sv(k)=(1,0,0). And the harm extent of vulnerability is 3. ADS: The value of assets divided into five grades. The small network system has 3 assets and their values are 1, 2 and 3. That is A= (1,2,3). RAS: In this subsystem, we assume that is 0.8. And we will calculate the risk value by 4 steps because the impact to risk value after 4th step can be ignored. The result of calculation is that the value at risk is 9.72307 and we will repair the vulnerability at the first asset. In order to illustrate the superiority of our model, we compared the two sets of data: the result of calculation by the Markov game model (MGM) and the result of calculation by the traditional assessment model (TAM).
VS:(1,0,1) 45 40 35 30 25 20 15 10 5 0 VS:(1,1,1)

(9)

Repair Scheme --- When system administrator repair the vulnerability, the transmission strategy of threat will change accordingly to maximize the risk affects of system from the threat. In such a case, the system administrator needs to select the best repair scheme to minimize the system risk. *(s(k)) is the best repair scheme.
* ( s (k )) = arg min max Vt ,v ( s (k ))
v V t T

(10)

The system risk value will be reduced to the maximum as long as system administrator operates according with the best repair scheme.
V ( s (k )) = min max Vt , v ( s( k )) v V t T

value at risk

(000) (001) (010) (011) (100) (101) (110) (111) threat state

Figure 4. Traditional Assessment model

(11)
VS:(1,0,1) 45 40 35 30 25 20 15 10 5 0 VS:(1,1,1)

V.

EXPERIMENTS AND DISSCUSSIONS

value at risk

To evaluate our game theoretic approach for risk assessment, we have constructed a Risk Assessment Platform (RAP).In the platform, there are four subsystems. They are Malicious Code Detection Subsystem (MCDS), Vulnerability Detection Subsystem (VDS), Asset Detection Subsystem (ADS) and Risk Assessment Subsystem (RAS). MCDS will detect the malicious code in the network system. The results of malicious code detection include threat name, threat type, asset IP, related vulnerability, probability of the threat spread and so on. VDS will detect the vulnerability in the network system. The results of vulnerability detection include vulnerability name, vulnerability type, asset IP, harm extent of vulnerability and so on. ADS will detect the asset in the network system. The results of asset detection include asset name, asset IP, and value of asset. The results of the three subsystems will save in database for RAS. We have detected a small network system and gained following data: MCDS: We have detected many threats from the small network system, but we will use a threat (Trojan.Mybot-6307) as an example. The threat state (TS): st(k)=(1,0,0) and the transmission probability is 0.2.

(000) (001) (010) (011) (100) (101) (110) (111) threat state

Figure 5.

Markov Game model

Traditional Assessment Model 30 value at risk 25 20 15 10 5 0

Markov Game Model

(000) (001) (010) (011) (100) (101) (110) (111) threat state

Figure 6.

Two model comparison

From Figure 4, we can see that the two VS have the same value at risk when threat state is (0,0,1), (0,1,1), (1,0,1). The

traditional model cannot distinguish the risk of different vulnerability state. But from Figure 5, we can distinguish them. When the vulnerability state is (1,1,1), there are greater risks in the small network system. So we think that the Markov game model can discover the potential risks. From Figure 6, we can also see the value at risk obtained by MGM is greater than the value obtained by TAM. It is because the value at risk obtained by MGM contained the potential risks. From Figure 4, Figure 5 and Figure 6, it is clear that the performance of Markov game model is better than the traditional assessment model. By the Markov game model, we can not only get more comprehensive value at risk, but also give the best system repair scheme. In this experiment, we obtained the repair table (Figure 7) for all threat state and all vulnerability state. No matter what threat state and vulnerability state, we can easily find a repair strategy from the repair table. In Figure 6, 1 denotes that we should repair the first asset. 2 and 3 is similar to 1. For example, we have detected that the threat state is (1,1,0) and the vulnerability state is (0,1,1), so we will repair the vulnerability of the second asset.
Threat state
000 001 1 3 2 3 1 3 2 3 010 1 3 2 2 1 3 2 2 011 1 3 2 3 1 3 2 3 100 1 3 2 3 1 1 1 1 101 1 3 2 3 1 3 1 3 110 1 3 2 2 1 1 2 2 111 1 3 2 3 1 3 2 3

ACKNOWLEDGMENT This work is supported by the National 863 High-tech Program of China (No. 2006AA01Z449) and the 42nd National Science Foundation for Post-doctoral Scientists of China (No. 20070420738). REFERENCES
[1] [2] IEC 1025: 1990 Fault Tree Analysis (FTA), 1990 Bouti, A. and Ait Kadi, D., "A State of the Art Review of FMEA/FMECA", International Journal of Reliability, Quality and Safety Engineering. January 1949: 515-543. Redmill F, Chudleigh M, Catmur J, "HAZOP and Software HAZOP", Wiley, 1999. B. Littlewood, "A reliability Model for Systems with Markov Structure", Applied Statistics, 1997, 24(2):172~177. CORAS IST-2000-25031 Web Site, http://www.nr.no/coras. 24 February 2003. Reactive System Design Support, "RSDS", http://www.kcl.ac.uk.2002. Commission of the European Communities Security Investigations Projects: Project S2014, Risk Analysis, CRAMM Evaluation, 1993. Control Objectives for Information and Related Technology, http://www.isaca.org/COBIT, 2003. J ZHANG Yong-Zheng etc., "Research on Network Node Correlation in Network Risk Assessment", China Journal of Computers, February 2007: 234~240. The International Organization for Standardization, Common Criteria for Information Technology Security Evaluation, ISO/IEC15408: 1999(E), 1999. SSE-CMM Model Description Document, Version 2.0, 1999, http://www.sse-cmm.org. International Organization for Standardization, Code of Practice for Information Security Management, ISO/IEC 17799:2000, December 2000. BSI/DISC Committee BDD/2, BS7799 Code of Practice for Information Security Management, 1999. J International Organization for Standardization, ISO/IEC TR 13335, Guidelines for the Management of IT Security (GMITS), 1996-2001. J National Security Agency, Information Assurance Technical Framework (IATF), Version 3.0, 2000, http://www.iatf.net. Information security technology, Risk assessment specification for information security, GB/Z 20984-2007. WU Bing etc., "Network-based malcode detection technology", Journal on Communications, November 2007: 87~91. Ritchey, R.W etc., "Using model checking to analyze network vulnerabilities", IEEE Conference on Security and Privacy, May 2000. Beaudoin, L etc., "Asset Valuation Technique for Network Management and Security", IEEE Conference on Data Mining Workshops, December 2006. Dan Shen etc., "Adaptive Markov Game Theoretic Data Fusion Approach for Cyber Network Defense", IEEE Conference on Military Communications, October 2007.

[3] [4] [5] [6] [7] [8] [9]

Vulnerability state

000 001 010 011 100 101 110 111

1 3 2 3 1 3 2 3

[10]

[11] [12]

[13] [14]

Figure 7.

Repair table

[15] [16] [17] [18] [19]

VI.

CONCLUSIONS

In this paper, we present a Markov game theory-based risk assessment model for the network information system. Using the model, we can know the risk condition of the network system, which include the risk from the potential threats. And we can get the automatic generation of a remedial scheme and provide network administrators with a convenient. Experiment results on Risk Assessment Platform approve the effectiveness of our proposed method. In future, we will research the spread of the threat (Trojan horses, worms, viruses) and use it in our model in order to improve our model.

[20]