Using Data Integration for Security Testing

    Sébastien SALVA and Loukmen REGAINIA*,

ICTSS 2017, Saint-Petersburg

10 October 2017

Introduction

  • Software security is essential and should be considered as early as possible
    1.  Security is not a common skill in developement teams
    2. Developers lack guidance for secure development and security tests 
  • Developers are provided with a plethora of documents:
    1. Security patterns express a set of reusable solutions to recurrent security problems
    2. Attack patterns present information about attacks (steps, countermeasures, etc.) in a form comprehensive for both professionals and academicians 

Plan

  1. Security patterns and ADTrees

  2. Data acquisition and integration 

  3. Method steps

  4. Evaluation

  5. Conclusions and perspectives

Introduction

  • The  number of these documentation is continually growing making difficult:
    1.  The choice of the appropriate solution 
    2. Testing  the chosen solution
  • ->pragmatic approach to guide developers in choosing security solutions and writing concrete test cases
  • Main features:
    • Security data acquisition in a data-store putting together attacks and patterns
    • Generation of Attack Defense Trees 
    • Generation and the execution of GWT (Given, When, Then) test cases

Security patterns

  • Reusable solution to a recurrent security problem 
  • Use of a security pattern requires an instantiation phase
  • Number of patterns is growing (176)
  • A security pattern is often related with other patterns

Data Integration

Data acquisition method associating security patterns, attacks, etc. [Crisis17]

  • 26 security patterns
  • 217 countermeasures (21 clusters)
  • 448 attack techniques
  • 66 Security principles
  • 209 Attack steps
  • 215 hierarchically organized attacks

Title Text

Attack Defense Trees

  • "Attack Defense Trees ADTrees are graphical representations of possible measures an attacker might take in order to attack a system and the defenses that a defender can employ to protect the system " [Kordy et al. 14]
  • An ADTree can be expressed with
\wedge^{o/p} ~~~~ Conjunction
∧o/p    Conjunction\wedge^{o/p} ~~~~ Conjunction
\vee^{o/p}~~~~Disjunction
∨o/p    Disjunction\vee^{o/p}~~~~Disjunction
\vec{\wedge}^{o/p}~~~Sequential~Conjunction~(SAND)
∧⃗o/p   Sequential Conjunction (SAND)\vec{\wedge}^{o/p}~~~Sequential~Conjunction~(SAND)
c^{o/p}~~~Countermeasures
co/p   Countermeasuresc^{o/p}~~~Countermeasures

Opponent "o"

Proponent"p"

Algebra (ADTerms):

Data Integration

  • Application Context
  • Prerequisites
  • Test architecture
  • 43 Patterns consequences
  • 209 GWT test Cases
  • 632 Generic procedures (Reusable for all applications in a specific context (Web, mobile, etc.))

Meta-model extension for test case generation.

Step 1: Initial ADTree

Denoted~T
Denoted TDenoted~T

Method steps

Step 2: ADTree Generation

Step 3: Security pattern choice

A~BADStep~ b ~is ~an ~ADTerm ~of ~the ~from ~c^p(st,sp),
A BADStep b is an ADTerm of the from cp(st,sp),A~BADStep~ b ~is ~an ~ADTerm ~of ~the ~from ~c^p(st,sp),
defense(b) = \{sp_1~|~b=c^p(st,sp_1)\}\cup
defense(b)={sp1 ∣ b=cp(st,sp1)}∪defense(b) = \{sp_1~|~b=c^p(st,sp_1)\}\cup
\{sp_1,...,sp_m~|b=c^p(st,\wedge^o(sp_1,...,sp_m))\}
{sp1,...,spm ∣b=cp(st,∧o(sp1,...,spm))}\{sp_1,...,sp_m~|b=c^p(st,\wedge^o(sp_1,...,sp_m))\}

Basic Attack Defence Step (BADStep)

Keep security pattern conjunctions

Step 3: Security pattern choice

Denoted~T(Att)~ for ~the ~attack ~Att~\in~T
Denoted T(Att) for the attack Att ∈ TDenoted~T(Att)~ for ~the ~attack ~Att~\in~T

Step 4: Final ADTree generation

Step 5: Test suite generation

Step 5: Test suite generation

A~test~case~TC(b)~is~generated~for~each~b~\in~BADStep(s)
A test case TC(b) is generated for each b ∈ BADStep(s)A~test~case~TC(b)~is~generated~for~each~b~\in~BADStep(s)
\vec{\wedge}^{p}~(c^p~(\vee^p(tech1,tech2,tech3),~\wedge^o(Audit~Interceptor,~Secure~Logger)),
∧⃗p (cp (∨p(tech1,tech2,tech3), ∧o(Audit Interceptor, Secure Logger)),\vec{\wedge}^{p}~(c^p~(\vee^p(tech1,tech2,tech3),~\wedge^o(Audit~Interceptor,~Secure~Logger)),
(c^p~(\vee^p(tech1,tech2,tech3,tech4),~\wedge^o(Secure~Logger,~Input~Guard,~Output~Guard)),
(cp (∨p(tech1,tech2,tech3,tech4), ∧o(Secure Logger, Input Guard, Output Guard)),(c^p~(\vee^p(tech1,tech2,tech3,tech4),~\wedge^o(Secure~Logger,~Input~Guard,~Output~Guard)),
...)
...)...)
SC(T_f)~has~the~form~:~\vec{\wedge}^{p}~(st_1,...,st_n)
SC(Tf) has the form : ∧⃗p (st1,...,stn)SC(T_f)~has~the~form~:~\vec{\wedge}^{p}~(st_1,...,st_n)

BADStep

Test scenario extraction from ADTree using ADTerms

Step 5: Test suite generation

Step 6: Test case completion

Step 7: Test suite execution

  • Test cases are executed and a set of verdicts are obtained:
F~ stands~ for~ the ~partition ~P~ (\{Fail_{sp}~ | sp_i ~\in ~ defense(b)~ \})~\backslash~\phi
F stands for the partition P ({Failsp ∣spi ∈ defense(b) }) \ ϕF~ stands~ for~ the ~partition ~P~ (\{Fail_{sp}~ | sp_i ~\in ~ defense(b)~ \})~\backslash~\phi
Local~verdicts:~Verdict(TC(b)||IUT)~=~V~with~V~\in~
Local verdicts: Verdict(TC(b)∣∣IUT) = V with V ∈ Local~verdicts:~Verdict(TC(b)||IUT)~=~V~with~V~\in~
\{VUL,~ NVUL,~ VUL/VIOLATE, ~NVUL/VIOLATE, ~INCONCLUSIVE\}
{VUL, NVUL, VUL/VIOLATE, NVUL/VIOLATE, INCONCLUSIVE}\{VUL,~ NVUL,~ VUL/VIOLATE, ~NVUL/VIOLATE, ~INCONCLUSIVE\}

Step 6: Test case completion

+

Step 7: Test suite execution

Final Verdicts :

Methodology Evaluation

Part 1 (1h30):

  • CAPEC Documentation of SQL injection and XSS attacks
  • How to manually perform the attacks
  • Security patterns documentation

Part 2(1h30):

  • CAPEC Documentation of SQL injection and XSS attacks
  • How to manually perform the attacks
  • Security patterns documentation
  • ADTrees
  • GWT test cases

Questions:

  • Q7 : Was it easy to write test cases ?
  • Q8 : How long did you take for writing tests ?
  • Q9 : How confident are you about your tests ?
  • Q10 : Provide your test cases ?

Criteria:

  • C1: Comprehensibility
  • C2: Effectiveness
  • C3: Efficiency

Methodology Evaluation results

  • C1: Comprehensibility

Methodology Evaluation results

  • C2: Effectiveness

Conclusions

We presented:

  • A data-store gathering
    • 215 attacks, 209 steps, 388 GWT tests, 632 generic procedures;
  • Automated generation of GWT and ADtrees
  • Data-Stores and Tools are available in

Perspectives

  • Complete the approach with Runtime-verification to test other properties; (Security test+ behavior + pattern properties);
  • Collaborating with MITRE-CAPEC to link security patterns with Attack patterns;

http://regainia.com/research/companion.html

Perspectives

  • Complete the approach with Runtime-verification to test other properties; (Security test+ behavior + pattern properties);
  1. LTL Behaviral properties
  2. SP behaviors
  3. application traces
  4. detecting security pattern behavioral occurences

 

Perspectives

Reference Architecture (Example: IOT azure microsoft)

Perspectives

Reference Architecture (Example: IOT azure microsoft)

Security

Data integration for security reference architecture building

Thank you for your attention 

Step 4: Final ADTree generation

Each node of the initial ADTree denoted with attack $$Att \in T$$ is automatically substituted with its ADTree $$T(Att)$$ The final ADTree is denoted $$T_f$$ whose root is the root of $$T$$

Methodology Evaluation results

  • C3: Efficiency

Step 5: Test suite generation

T_f~is ~defined ~with ~an ~ADTerm ~\iota(T_f)
Tf is defined with an ADTerm ι(Tf)T_f~is ~defined ~with ~an ~ADTerm ~\iota(T_f)
SC(T_f)~is~the~set ~of~ scenarios~being~extracted~from~T_f
SC(Tf) is the set of scenarios being extracted from TfSC(T_f)~is~the~set ~of~ scenarios~being~extracted~from~T_f
An~attack~scenario~s~\in~SC(T_f)~is~an~ADTerm~over~BADStep(s)
An attack scenario s ∈ SC(Tf) is an ADTerm over BADStep(s)An~attack~scenario~s~\in~SC(T_f)~is~an~ADTerm~over~BADStep(s)
A~test~case~TC(b)~is~generated~for~each~b~\in~BADStep(s)
A test case TC(b) is generated for each b ∈ BADStep(s)A~test~case~TC(b)~is~generated~for~each~b~\in~BADStep(s)

Attack Patterns

  • An attack pattern describes:
    • Techniques, context, indicators, prerequisits, etc.
    • The sequencing of the attack steps
    • The potential countermeasures
  • CAPEC (Common Attack Pattern Enumeration and Classification) is the main resource of attack patterns :
    • The unification of attack descriptions 
    • A data base of more than 500 detailed attack patterns

Step 2: ADTree Generation

Scenario: Attack x step y

   Given: prerequisites

   When: Actions  

   Then: Assertions

  1.   One Then section: Attack step success;

  2.   Many Then sections: Presence of security pattern consequence;

Generic Procedure:

Reusable for all applications in a specific context (Web, mobile, etc.) Completed with guidelines :

  • Resources required -CAPEC-
  • Attack step techniques -CAPEC-
  • Outcomes/Indicators -CAPEC-
  • Patterns consequences 
A~test~case~TC(b)~has~the~form:
A test case TC(b) has the form:A~test~case~TC(b)~has~the~form:

Step 5: Test suite generation

Attack Defense Trees

  • An ADTree is also defined with ADTerms
  • An ADTerm, expresses two types of roles (opponent "o") and (proponent "p") in addition to four types of relations
\wedge^{o/p} ~~~~ Conjunction
∧o/p    Conjunction\wedge^{o/p} ~~~~ Conjunction
\vee^{o/p}~~~~Disjunction
∨o/p    Disjunction\vee^{o/p}~~~~Disjunction
\vec{\wedge}^{o/p}~~~Sequential~Conjunction~(SAND)
∧⃗o/p   Sequential Conjunction (SAND)\vec{\wedge}^{o/p}~~~Sequential~Conjunction~(SAND)
c^{o/p}~~~Countermeasures
co/p   Countermeasuresc^{o/p}~~~Countermeasures
Made with Slides.com