Deadline–Budget Constrained Scheduling

Presented by: Elaheh Barati

elaheh@wayne.edu

Wayne State University

"Low-time complexity budget–deadline constrained workflow scheduling on heterogeneous resources"

Subtitle

Hamid Arabnejad ,  Jorge G. Barbosa,  Radu Prodan

LIACC, Departamento de Engenharia Informática, Faculdade de Engenharia, Universidade do Porto, Portugal

University of Innsbruck, Institute of Computer Science, Innsbruck, Austria

Future Generation Computer Systems 55 (2016)

Content

• Introduction
• Problem definition
• Proposed algorithm
• Experimental results
• Conclusions
• Future work

Utility computing is a  

service provisioning model that provides:

• computing resources
• infrastructure management to consumers as they need them
• a payment model that charges for usage

Utility computing is a  

service provisioning model that provides:

• computing resources
• infrastructure management to consumers as they need them
• a payment model that charges for usage

 Service-oriented grid and cloud computing, have become the basis for providing these services

• Utility computing has been rapidly moving towards a pay-as-you-go model
  • computational resources or services have different prices with different performance and Quality of Service (QoS) levels
    • Users consume services and resources when they need them and pay only for what they use

Cost and time become two of the most relevant user concerns

Challenge: The cost/time trade-off problem for scheduling workflow applications

Scheduling consists of:

• defining an assignment
• mapping of the workflow tasks onto resources

The scheduling problem belongs to a class of problems known as NP-complete

Research on workflow QoS aware scheduling:

• Single objective approach:
  • Optimizing one QoS parameter, such as time
  • Constraining another QoS parameter, such as cost
•   Bi-objective approach:
  • Optimizing two QoS parameters, such as time and cost simultaneously
  • Constraints are related to processor availability and load

Workflow scheduling to satisfy multiple QoS parameters

multi-objective scheduling algorithms

search-based strategies

meta-heuristic methods

Need significantly high planning costs in terms of the time consumed to produce good results

 A low-time complexity heuristic

Deadline–Budget Constrained Scheduling (DBCS)

 constrained to two QoS parameters time and cost

Objective

Find a feasible schedule map that satisfies the user-defined deadline and budget constraint values

Implementing a mechanism to control the time and cost consumption by each task when producing a schedule solution

Problem definition

• System model
• Application model
• Scheduling problem

• The target utility computing platform: a set of heterogeneous resources that provide services for different capabilities and costs
• Processor price is defined
  • the most powerful processor has the highest cost and the less powerful processor has the lowest cost

System Model

• In a utility grid
  • The resource price is defined and charged per time unit 

System Model

• In a utility grid
  • The resource price is defined and charged per time unit

System Model

• In a cloud environment
  •  the granularity of charging resource usage varies
    •   Amazon EC2: charging per hour

• In a utility grid
  • The resource price is defined and charged per time unit

System Model

• In a cloud environment
  •  the granularity of charging resource usage varies
    •   Amazon EC2: charging per hour

No cost benefit of adding resources for workflows that run for less than an hour

System Model

• A heterogeneous resource has a set of processors available
• Each processor has a price per time unit

In this paper

Application Model

A typical workflow application can be represented by a Directed Acyclic Graph (DAG)

 Tasks of the workflow

t1

t2

t3

Application Model

A typical workflow application can be represented by a Directed Acyclic Graph (DAG)

 Tasks of the workflow

t1

t2

t3

 data dependencies

Application Model

A typical workflow application can be represented by a Directed Acyclic Graph (DAG)

 Tasks of the workflow

t1

t2

t3

 data dependencies

n × n matrix of communication data

entry task

exit task

Application Model

 The amount of data that must be transferred from task ti to task tj 

The average communication time between the tasks ti and tj :

Average bandwidth among all processor pairs

Average latency

Application Model

Execution Time to complete task ti on processor pj

Due to heterogeneity, each task may have a different execution time on each processor

Number of resources in processor set P

The average execution time of task ti :

Application Model

For entry task : EST = 0

Earliest Start Time of the task ti on processor pj

is Zero if :  Parent  and child are assigned to the same machine

Earliest time at which processor pj is ready

Application Model

Earliest Finish Time (EFT) of a task ti on processor pj

Application Model

data storage cost of task ti

The financial cost

cost of transferring the data required for task ti

Application Model

Assigned Cost of task ti : ti is already assigned to a processor 

Scheduling Problem

For a given workflow G, a scheduling is defined by a function schedG : T → P which assigns to each task ti ∈ T a processor pj ∈ P, subject to:

• Processor constraint: no processor executes more than one task at the same time
• Precedence constraint:  represented edges of the workflow
• Budget constraint:
• Deadline constraint​:

Task Selection

Processor Selection

Deadline–Budget Constrained Scheduling algorithm

Length of the longest path from task ti to the exit node

 filtered processor set based on

maximum available budget for the current task that can be consumed by its assignment 

Text

 how much the finish time of the current task is closer to the task sub-deadline

Text

how much the actual cost on pj is less than the cost on the processor  that results in the earliest finish time

Time Complexity

O(n.p)

• p is the number of available resources
• n is the number of tasks in the workflow application

• O(n.p): for calculating FT and Cost for the current task among all processors
• O(p): for calculating the Quality measure

The total time is O(n.p + n(n.p + p))

Experimental Results

Workflow structure

• Randomly generated
  • random DAG generation: total 10000
• Real-world application

Experimental Results

Simulation Platform

• SimGrid toolkit: provides the required fundamental abstractions for the discrete-event simulation of parallel applications in distributed environments.
• Use three sites:
  • Rennes site has normal distribution of CPU power among its clusters
  • Sophia site contains a higher number of low-speed processors
  • Lille site has a higher number of fast processors

Experimental Results

Budget and deadline parameters

Lowest and highest execution time of the application

 critical parent of task ti

the minimum and the maximum execution time for task ti on the fastest and the slowest processor among all sites

Experimental Results

Budget and deadline parameters

in the range [0 .. 1]

Experimental Results

Performance metric

Planning Successful Rate:

Percentage of valid schedules

Experimental Results

randomly generated workflows

Experimental Results

randomly generated workflows

Experimental Results

 Results for real world applications

Conclusion

• The DBCS algorithm was presented for budget and deadline constrained scheduling
• It achieves a similar performance to higher-time complexity algorithms with a time complexity of the heuristic algorithms of the order O(n^2 .p) for n tasks and p processors

Future work

• Extend the algorithm to consider the dynamic concurrent DAG scheduling problem
  •  allow users to execute concurrent workflows that might not be able to start together but can share resources
  • the total time and cost for the user can be minimized to meet their deadlines and budgets