Cyber Security product management framework Towards a universal terminology standard for knowledge intensive services

Tim Bervoets 15-02-2016

Introduction Society is becoming increasingly dependent on computer systems (e.g. government, critical infrastructure, industry). Therefore there is a growing need for effective cyber security, which is defined as the protection of information systems from theft or damage as well as from disruption or misdirection of the services they provide (Gasser, 1988). Cyber security service providers like Fox-IT aim to help protect organizations against IT threats by improving (or outsourcing) their incident prevention, detection and response processes. Also, intelligence is being used increasingly to counter threats proactively by enriching these security processes with contextual and predictive information. Different cyber security companies offer many seemingly similar services (and products), which can impact client security processes differently (e.g. penetration tests, vulnerability assessment). The exact service differences are important to understand for clients (e.g. service value) and cyber security companies (e.g. product management, benchmarking). Unfortunately, these differences are difficult to grasp using high-level service terminology (i.e. incident prevention, detection, response; too broad) and low-level terminology (i.e. technologies, domain knowledge; too detailed, complex and knowledge intensive). Therefore, a universal mid-level service terminology is needed for product managers to analyze and compare services conveniently and uniformly across different cyber security organizations. High-level description can only capture a very broad, entangled and ambiguous concept of how cyber security services work. In other words, cyber security services cannot be uniquely identified using high-level terminology. To illustrate this, consider the following. What exactly is a penetration test? Notice that there does not exist something like a typical penetration test service that always only impacts prevention, detection or response capacity, or some combination. For example, some service providers offer a penetration test service that mostly impacts a client’s capacity for prevention (e.g. goal to sneak in undetected), while others impact detection (e.g. goal to be detected at different security levels) or response (e.g. goal to cause an incident that is difficult to 1

trace or restore) or combinations and in varying degrees. Furthermore, a vulnerability test may be part of a penetration test service or other services, or it can be a service on its own, or all three, thereby impacting client prevention, detection and response capabilities differently. These examples demonstrate the limitations of high-level service description terminology. High-level terminology can capture the overall goal or function of cyber security services, but it fails to differentiate between specific service goals and features. Using low-level service description terminology, one can say that different cyber security companies (e.g. Onsight, DearBytes, Digital Investigation, SurfRight) offer different services for different markets based on different technologies and domain expertise. However, the sheer complexity at this level (e.g. tools, data analysis techniques, domain knowledge) makes it practically impossible to analyze cross connections or communicate effectively about services, because of a lack of overview and a lack of pragmatic focus (i.e. product management and client perspective). Both description levels capture important service aspects and both have well-established terminology standards. However, there are two main problems. Firstly, there does not exist a direct relationship between low-level ‘technical’ and high-level ‘functional’ terminology, contrary to what may be conveniently assumed among technicians. In other words, a penetration test is not solely about having the best tools and the ‘smartest’ testers testing client prevention, detection and response capacities. The actual service value depends heavily on how the test is designed and organized at the level of tasks. Secondly, the cyber security field has not yet established a universal terminology standard for mid-level (i.e. task level) service description. Up till now, services are being functionally decomposed into tasks using either non-standardized task terminology (e.g. unstructured lists of definitions) or general task analysis standards which lack power and universality. General task analysis standards suffer important limitations for the analysis of knowledge intensive services and tasks (Schreiber et al, 1999). For example, they do not describe task functions in a domain-independent terminology (i.e. no decoupling of domain knowledge/technologies and task functions), and the emphasis is mostly on representing ‘external’ control (i.e. state-transitions; see Modern Structured Analysis and OMT) instead of on ‘internal’ control of the reasoning process (i.e. reasoning steps and strategies). General task analysis standards also fail to describe communication aspects in a separate communication model. In this paper it is argued that knowledge intensive services like cyber security services need to be functionally decomposed into knowledge intensive tasks. This approach empowers product management (or similar functions) to contribute most effectively to knowledge intensive product/service development and marketing. Product management is defined here as an organizational lifecycle function within a company dealing with the planning, forecasting, and production, or marketing of a product or products at all stages of the product lifecycle. The current approach empowers a product manager to communicate unambiguously and pragmatically with clients, business managers and analysts. He can do this on the appropriate level of analysis using a language that captures the universal characteristics of cyber security services and products. It also empowers him to have a comprehensive overview and to do in-depth benchmarking analyses without him necessarily having to be a technical domain expert. Instead the product manager can allocate his valuable (yet limited) cognitive resources fully to marketing and analyses related tasks and knowledge. This emphasizes that, in addition to a universal terminology standard, product managers also need to have strong information and knowledge analysis skills to easily elicit domain knowledge from available domain experts or documentation. This paper presents a universal midlevel terminology standard and framework for cyber security service analyses. 2

Framework The current paper introduces a customized product management framework that draws on insights from the field of knowledge engineering (KE; Schreiber et al, 1999). The basic idea is that cyber security services essentially consist of knowledge intensive tasks for which useful and reliable KE task templates are available (e.g. classification, assessment, diagnosis, monitoring, design; see Task type Analysis

Input System observations

Output System characterization

Knowledge System model

Features System description given.

Classification

Object features

Object class

Diagnosis

Symptoms/ complaints

Fault category

Feature-class associations Model of behavior

Assessment

Case description

Decision class

Criteria, norms

Monitoring

System data

Discrepancy class

Normal behavior

system

Prediction

System data

System state

Model of behavior

system

Set of classes is predefined. Form output varies (causal chain, state, component) and depends on use made of it (troubleshooting). Assessment is performed at one particular point in time (cf. monitoring) System changes over time. Task is carried out repeatedly. Output state is a system description at some future point in time.

system

is

Table 1. Analysis task templates overview

Task type Synthesis

Input Requirements

Output System structure

Knowledge System elements, constraints, preferences

Features System description needs to be generated

Design

Requirements

Artifact description

May include creative design of components.

Configuration design

Requirements

Artifact description

Assignment

Two object sets, requirements Goals, requirements Job activities, resources, time slots, requirements Requirements

Mapping set 1  set 2 Action plan

Components, constraints, preferences Components, Skeletal designs, constraints, preferences Constraints, preferences Actions, constraints, preferences Constraints, preferences

Model elements, template models, constraints, preferences

May include “synthesis”

Planning Scheduling

Modeling

Schedule = activities allocated to time slots of resources Model

Subtype of design in which all components are predefined. Mapping need not be one-to-one. Actons are (partially) ordered in time. Time-oriented character distinguishes it from assignment. creative

Table 2. Synthesis task templates overview

3

table 1 and 2). Each task type differs in the input and knowledge it uses, and the output it generates. For example a diagnosis task uses symptoms/complaints as input, a model of system behavior as knowledge and it generates a fault as output, whereas an assessment task uses case descriptions as input, criteria/norms as knowledge and a decision as output. Furthermore, the same task type can have varying task type features, namely input types, knowledge types and output types. For example, a diagnosis task can have as input type a complaint, a hypothetical complaint or a yet unknown (but existing) complaint, as knowledge type a causal model (i.e. literal often invisible causality) or a manifestation model (i.e. practical observables and indicators), and as output type a causal chain, faulty system state or faulty component. Consequently, the following hypothesis is proposed. All possible cyber security services can be uniquely identified using this fixed set of task types and task feature types. In other words, these tasks are the only means by which any cyber security service provider is capable of improving (or outsourcing) a client’s capacity for incident prevention, detection or response. These tasks are also the only contexts in which specific technologies and domain knowledge are applied by any cyber security provider. This implies that there is a direct hypothetical relationship between mid-level and high-level terminology, and between mid-level and low-level terminology. The mid-level terminology is thus compatible with other levels of analysis and universal across different cyber security providers. The framework can be applied to redefine existing cyber security services using unambiguous terminology. This empowers the product manager to communicate unambiguously and pragmatically with clients, business managers and analysts on the appropriate level of analysis, using a universal language for service description. See table 3. for a list of preliminary examples. Let us consider the following example. From a client’s perspective, a penetration test is functionally essentially a diagnosis task, because the cyber security service provider uses extensive knowledge (i.e. causal model, manifestation model) about the client’s organizational systems with the aim of achieving a ‘complaint’ (i.e. incident). In other words, a penetration test takes a hypothetical complaint (e.g. R&D data theft incident) as input type. This is very similar to a regular car diagnosis task when there is an actual complaint. Someone complains that his car engine doesn’t work and figures out that it needs a new fuse after trying different hypotheses. The difference with pen testing is that the tester is actually hypothesizing all the different ways the engine can be caused to stop working. Ultimately, the penetration test diagnosis task output provides valuable knowledge about faulty system states (e.g. data is leaking, server is unavailable), causal chains (e.g. social engineering  desktop control  server access) and faulty system components (e.g. the firewall firmware contains a vulnerability) at each level of organizational systems. Notice that this doesn’t mean that a penetration test necessarily needs to be named a ‘diagnosis’ when communicating with clients (even though that is what it essentially is at its core). It can also be named a ‘pen test assessment’ as long as the actual service is functionally treated as essentially a diagnosis of a client’s organizational system(s). In fact, the entire pen test service can be composed of many additional secondary tasks (e.g. classification, planning, assessment, design) and it can still be named a ‘pen test assessment’ conventionally whilst essentially being a diagnosis task with a hypothetical complaint as input type.

4

Service name Penetration test

Task type Diagnosis

Input type Hypothetical complaint from: - Prevention process - Detection process - Response process

Output type Causal chain(s) System state(s) Faulty component(s)

Forensics

Diagnosis Classification

Actual complaint from: - Response process

Causal chain(s) Attacker(s) identity

Encryption

Design

Artifact encryption

Data diode

Assessment

Training

Design

Security Operations Center

Monitoring Design

Security rating

Classification

Requirements for: - Prevention process Case descriptions from: - Prevention process - Detection process Requirements for: - Prevention process - Detection process - Response process System data from: - Prevention process - Detection process Requirements for: - Prevention process - Detection process - Response process Object features from: - Prevention process - Detection process - Response process

Instructive decision Commissive decision Declarative decision Artifact description: - curriculum (what,why) - skills (how,when)

Knowledge type Causal model {Policies, People Networks Devices Applications Data} Manifestation model Causal model {Policies, People Networks Devices Applications Data} Manifestation model System element(s) {Data} Norms {Data} System element(s) {People}

Discrepancy class Artifact description class: - infrastructure design and configuration

Norms and system elements {Networks Devices Applications Data}

Object rating

Feature-class associations for {Policies, People Networks Devices Applications Data}

Table 3. Cyber security service definition examples

The framework can also be used to optimize existing services by focusing on service composition (i.e. number of tasks, the order of tasks, the type of tasks, number of reasoning steps per task, order of reasoning steps per task) and the features of each task (i.e. input types, knowledge types and output types). This empowers the product manager to have a comprehensive pragmatic overview from which in-depth benchmarking analyses can be performed without him necessarily having to be a technical domain expert. Lastly it can be used to discover completely new services (innovations) and inform market positioning. The next sections demonstrate briefly how it can be applied for service optimization and innovation purposes. Note that a full analysis is beyond the scope of this paper as the aim is merely to illustrate how the framework can be applied for innovation analysis. Service innovation can be achieved bottom-up by focusing on all possible combinations of task type(s) and task feature types, and asking how any particular combination can create value for clients in terms of high-level goals (i.e. incident prevention, detection and response). Low-level 5

technical feasibility is completely discarded at this stage. See again for example table 3. Remember, the total number of knowledge intensive task types is fixed (11 total) as well as the total number of task feature types per task type. This means that innovation is no longer an ‘ill defined problem’, because all necessary information is available to solve the problem and all necessary criteria are available for the solution. Notice again that the problem is framed in such a way that it is entirely independent of technological aspects (e.g. feasibility) or domain knowledge aspects (e.g. availability of expertise). It has become a straight forward exercise of trying all principally possible type combinations through means-end analysis, while limiting the problem space by focusing first on combinations that heuristically make more sense than others. This solution process is called forward chaining, whereby one searches for a solution from the initial state onwards. Rating service example. A Dutch Cyber security provider Onsight offers a security rating service called IntelliRating. This service is essentially a classification task with the goal to impact a clients’ capacity for incident prevention, because it enables one to know one’s own security rating and the security rating of (potential) partner organizations, and take decisions thereupon. This obviously has great value and would have been easily identified using the framework by simply looking at the classification task type. A competitor could offer a similar security rating service, using improved detection and recognition of the same ‘object features’ (i.e. knowledge type) or adding completely new object features at the same level of organizational analysis (e.g. devices, applications and data) or at another level of analysis (e.g. people, policies). This competitor could also offer a security rating service on a completely different level of organizational analysis, for example by rating the security level of countries or branches, or even within organizations at the level of people by basing ratings on employees’ social psychological profiles, or even policy rating. Furthermore, this competitor could also offer the organization security rating service in combination with an assessment task using ratings at different levels of organizational analysis (i.e. policies rating, people rating, networks safety rating, devices rating, applications rating, data rating) as case types and matching these with norms to produce a concrete decision or recommendation as output. This example illustrates that by looking at just one task type (i.e. classification) and task feature types (e.g. object features), one can easily identify areas for service innovation. Penetration test example. One can offer a service that first performs a classification of the organization in terms of an overall security rating (i.e. similar to Onsight IntelliRating) based on configuration data and security events related object features as input (e.g. spam, social media, botnets, ddos, malware code, user behaviors). The output is a simple security rating value between 250 (i.e. very bad) and 900 (i.e. very good). Based on this initial rating one can plan a pen test diagnosis which focuses on an area that needs most security according to the client or alternatively first do an additional, more general vulnerability assessment in all major areas of the client’s IT infrastructure to identify key focus areas. Notice that the assessment task is importantly different from a diagnosis task. The assessment takes case descriptions (e.g. router specs, server age, database configuration specs, people profiles) as input and uses knowledge about rules, heuristics, norms and laws (i.e. not a causal model) to produce a decision as output (e.g. replacement, updating or configuration of a router, desktop, server, function, or even a person). The assessment output can add value to the service by providing information about ‘known vulnerabilities’. This information can be exploited by the diagnosis task with the aim of diagnosing the organization’s capacity for incident detection. Another good reason to incorporate an assessment task is that in reality, attackers have vulnerability scanners too. Given that it is difficult to predict where and when real scans will be performed by real attackers it may be valuable to dedicate a team (or person) to doing nothing but 6

full scale vulnerability assessment and reporting back to a separate team (or person) focusing on the diagnosis task. Similarly, we might add a dedicated monitoring task, which uses monitoring tools like cameras, microphones and data sniffing tools to continuously listen in on as many people as possible inside the company and report back to diagnosis upon hitting on something ‘useful’. Both the pen test diagnosis and the vulnerability assessment will produce valuable reports that address specific client requirements so that the client really knows something about the organization’s capacity to prevent, detect and respond to IT threats. Both the diagnosis report and assessment report eventually feed into a design proposal aimed at adapting (i.e. redesigning) the organization systems effectively. The execution of the proposal is planned in accordance with specific client requirements and practical constraints. See table 4. Other cyber security services (i.e. not only rating and penetration test services) can be analyzed similarly, including intelligence related services.

Knowledge: Organizational Systems Policies

People

Networks

Devices Applications

Data

Diagnosis output

Input: Incident prevention Design (planning) output

Sales and marketing functions are a dangerous entry point for delivery of malware John turns out to not strictly follow company policies regarding his marketing function. Veronica uses the same (weak) password for many different devices and applications. Michael’s social psychological profile with respect to specific traits, attitudes, beliefs and values, made him very susceptible to social engineering, and he has a trustful friendship with one of the R&D guys. The network segments for marketing and R&D are connected. Pivoting has extended successfully all the way from a marketing desktop to R&D file server and database. The R&D server is 20 years old and contains known vulnerabilities Marketing desktop AV was not sandboxing the malware excel sheet with macro that successfully injected script into Windows powershell. The R&D database is not storing data encrypted

Adjust marketing function policies around email use and periodic password renewal, restrict marketing user rights to access particular network devices and applications Veronica and Michael will be coached or trained to overcome their weaknesses. John will need to be replaced for making 3 dangerous, professional errors.

The R&D network segment needs to be physically separated from the marketing and sales segment. Replace the old R&D server by a new one Install a new AV from a different brand

Use transparent encryption to encrypt the entire database, and also use File system encryption.

Table 4. Simplified penetration test for incident prevention example

7

Conclusion

This paper introduces a product management framework for cyber security service analysis. The framework represents a universal mid-level service description terminology standard. The main hypothesis is that all possible (existing and new) cyber security services can be uniquely identified using the fixed set of task types and task type features provided in the framework. The framework can thus be applied to redefine existing cyber security services using unambiguous terminology. The framework can also be used to optimize existing services by focusing on service composition and the features of each task. Lastly it can be used to discover completely new services and to aid market positioning. It is argued that knowledge intensive services like cyber security services need to be functionally decomposed into knowledge intensive tasks, using reliable task templates from the field of knowledge engineering. This approach empowers a product manager to communicate unambiguously and pragmatically with clients, business managers and analysts on the appropriate level of analysis, using a language that captures the universal characteristics of cyber security services. It also empowers him to have a comprehensive overview from which in-depth benchmarking analyses can be performed without him necessarily having to be a technical domain expert. Instead product managers can allocate valuable (yet limited) cognitive resources to marketing and service analyses related tasks and knowledge. As a consequence, the current framework emphasizes that product managers also need to have strong information- and knowledge analysis skills that allow them to easily elicit domain knowledge from available domain experts or documentation. It is thus highly recommended that cyber security product managers have a background in knowledge engineering, information analysis and marketing. The framework is also relevant for product managers working outside the domain of cyber security, as well as for other (yet similar) organizational functions. Future research can focus on using the current framework to analyze other services than the pen testing or rating service example, or even beyond services that impact client capacity for incident prevention, detection and response. One possibility is to investigate for example how intelligence data can be productized or utilized to increase client awareness of attacks in the (branch) environment. This marketing solution may address the problem of cyber security value perception, which holds that ever improving prevention measures are causing a decrease in true positive threat detection and thus a decrease in perceived service necessity by clients. Future research can also focus on expanding the current framework with a universal standard for knowledge intensive service communication. It is recommended to devise a synthesis of theory of planned behavior, speech act theory and communication grounding theory, to maximize communication effectiveness and impact. One limitation of the current paper is that the knowledge intensive task templates are taken from a knowledge engineering methodology (i.e. commonKads) which has not yet been compared to other existing methodologies. Although there exist different knowledge engineering methodologies, the task templates themselves are based on extensive model-based tasks literature. See for example Benjamins (1993) for a review on model-based diagnosis.

8

References

M. Gasser (1988). Building a Secure Computer System (PDF). Van Nostrand Reinhold. p. 3. ISBN 0442-23022-2. G. Schreiber, H. Akkermans, A. Anjewierden, R. de Hoog, N. Shadbolt, W. van de Velde, B. Wielinga (1999). Knowledge Engineering and Management: The CommonKADS Methodology. p.125. ISBN 9780262193009 Benjamins, V. R. (1993). Problem Solving Methods for Diagnosis. Ph.D. thesis, University of Amsterdam, Amsterdam.

9