Scheduling Algorithms for Hierarchical Fog Networks

The evolution of IoT devices has lead to the generation of huge amounts of data which needs to be monitored, processed, and analysed [32]. Due to ample computation power and storage volume, the cloud has become a competent model to execute user applications [7]. The cloud data center is used to process the application data, which is managed by cloud service providers, such as Amazon, Google, or Microsoft Azure. A significant delay in processing the applications has been observed at the cloud data center, owing to its distant geographic location from the users. This delay may be unacceptable for applications with stringent response times, such as smart healthcare [11]. To overcome this delay constraint, system designers are exploring fog computing [13, 38].

In general, a cloud only architecture may not be suitable for latency sensitive applications, due to the distance issue highlighted above. Fog computing offers an architecture comprising of a number of fog devices/micro data centres, in close proximity to data generation sources, such as smartphones and sensors. Hence, the generated data can be processed on the fog devices in a timely manner. It has been observed that fog devices have limited computation power and storage capacity [10]. Hence, they are able to handle jobs with modest cpu requirements, e.g. interactive jobs. Therefore, a flat fog only architecture may not be suitable for processing jobs with less modest execution requirements. This seems to suggest a hierarchical fog-cloud architecture, consisting of multiple fog-tiers and a cloud data center. Accordingly, we consider a hierarchy of fog devices in our architecture [24]. As an example, consider a 2-tier fog-cloud hierarchy, where tier-1 fog devices have a lower execution capacity than tier-2 fog devices. On the flip side, tier-2 fog devices are located at a greater geographical distance than tier-1 fog devices from the users. Thus, there exists an interesting trade off between execution capacity and propagation delay of tier-1 and tier-2 fog nodes. The proposed architecture is depicted in Fig.1. Note that this model can be extended to accommodate a larger number of fog tiers, based on application requirements.

As a use case to drive the proposed algorithms, let us discuss a “smart home”. Typically, a smart home uses various internet connected devices that allows monitoring and supervision. Due to the large number of these devices, such smart homes will generate huge amounts of data per hour. As per the computer business review [25], it has been reported that UK smart households will generate 26 million GBs of data every week. Some of this data may need real-time responses. Due to network congestion and delayed response times, the cloud data center may not be a suitable option for processing such time-critical data. Handling some of this data at the nearby fog layer may assure real-time response times to users. A smart home scenario can involve a number of sensors, such as: motion sensors, water sensors, temperature sensors, light sensors, weather sensors etc. A smart home uses the data provided by these sensors to perform various functions such as warning for gas leaks, emergency health services, maintaining the thermostat, managing lights, maintaining the laundry cycle of the washing machine, domestic grocery shopping based on shopping history etc. These jobs will have diverse execution requirements. An example of a job with relatively modest cpu requirements is activating a burglar alarm or a fire detection system. If the alarm isn’t generated quickly, then there is a possibility of theft or causalities. It may not be wise to process such data at the cloud data center, due to the significant latency involved. Such jobs should be sent to tier-1 fog devices located in proximity to the user. On the other hand, jobs with medium cpu requirements could be related to maintaining the comfort of the home, such as roller shutter automation, maintaining air-conditioning, switching lights on/off etc. These are examples of jobs with moderate cpu requirements that may be executed at tier-2 fog devices. Lastly, the jobs with high cpu requirements can be executed at the cloud data center. Examples of these jobs could be automatically recording TV programs for subsequent viewing of users. Besides offering reduced response times for job execution, another advantage of fog devices is that they are geographically distributed without a single point of failure, so the service may not be disrupted in the case of device failure.

The main contributions of this paper can be summarised as :

• •

  We propose scheduling algorithms for hierarchical fog-cloud architectures with multiple layers of fog devices between the users and the cloud data center. Specifically, we consider flat-tier, 2-tier, 3-tier and 4-tier fog architectures. We formulate our research problem by jointly minimising the completion time as well as the energy consumption on an n-tiered fog-cloud architecture.

• •

  In order to minimise the completion times of the jobs, we propose two hierarchical scheduling algorithms: FiFSA (Hierarchical First Fog Scheduling Algorithm), and EFSA (Hierarchical Elected Fog Scheduling Algorithm).

• •

  In order to demonstrate the performance of our proposed algorithms, we conduct extensive simulations and develop a working prototype. We run the simulations over a widely accepted simulator - iFogSim, using a real-life Alibaba trace. We observe the system performance by varying a number of parameters such as: job load, delay, and MIPS (Million of Instructions Per Second). We compare our proposed algorithms to another related algorithm for hierarchical fog-cloud architectures - Longest Time First (LTF).

• •

  We evaluate and discuss the impact of various fog-cloud hierarchies on the performance of FiFSA and EFSA.

The rest of this paper is organized as follows. Section II discusses related work. The model, notation, and problem formulation is described in section III. The proposed algorithms are discussed in section IV. Section V discusses the simulation and experimental results. Finally, section VI concludes the paper and discusses future work.

Cloud computing is a model that provides “on-demand” computer services, such as data storage and computation power on a flexible, budget-friendly environment over the internet [45, 4]. Cloud computing uses a “pay as you go” model, due to which expenses such as hardware costs, manpower costs can be saved by administrators. Even though the cloud offers many advantages, there are multiple challenges: downtime, security, privacy, resource provisioning, power and energy management. On the other hand, IoT has gained widespread popularity with a huge increase in the number of sensors, actuators, and mobile devices connecting billions of things across the world [21, 3]. The IoT field is approximated to be worth 11 trillion dollars per year by the end of 2025 [29]. By that time, the deployment of IoT devices is expected to cross 1 trillion. IoT visions a new environment which will impact our day to day life by providing services such as smart healthcare [16], smart environmental monitoring[5], and so on. This implies that a large number of applications will be processed and managed by IoT in the future [24, 34]. The main requirements in IoT applications are: low latency and fast processing. It may not be possible to achieve low latency by employing cloud computing alone. The evolution of fog computing has made it possible to meet the demands of the IoT model [9]. Fog computing extends the services of the cloud closer to the edge of the network, bringing them closer to the users. The various advantages of fog computing have been discussed in a Cisco white paper [13]. There are several potential use cases that can benefit from fog computing: smart buildings, autonomous driving, aerial drones etc. The reference architecture of these use cases has been proposed and discussed by the Open Fog Consortium [19]. In [37], the authors propose an architecture of placing storage and computing nodes close to the users. This architecture mitigates the computation limitation of mobile devices by allowing resource-hungry applications to use cloud-like service for computation. Fog devices are geographically distributed, which helps in maintaining the performance of the system, even in the case of decreased battery power. The modelling and simulation of fog and edge computing environments can be carried out using simulators, such as - iFogSim [20]. By using iFogSim, one can evaluate the end-to-end latency, achieve Quality of Service (QoS), simulate various resource management strategies, measure power consumption. The simulator follows the sense $\rightarrow$ process $\rightarrow$ actuate model during the simulation of application scenarios. The importance of fog computing in IoT scenarios has been discussed in [48, 35].

Multiple papers have explored scheduling on edge computing resources. A run time IoT data placement strategy for latency reduction is proposed in [31]. In iFogStor, an exact solution is found by using an Integer Linear Programming (ILP) based approach. A heuristic based solution using geographical zoning is proposed in iFogStorZ. This reduces the overall computation time. In [22], a survey and a mobility aware scheduling algorithm is proposed. A genetic algorithm is used as a heuristic to provide the possible solution of resource provisioning in the fog-cloud architecture [41]. In [36], the authors provide a comparison of cloud computing and fog computing paradigms. The authors focus on investigating the suitability of fog computing in the IoT environment by providing a network model. In [39], the authors use game theory to provide near optimal resource allocation, while increasing the user’s quality of experience. The authors propose a hybrid computation offloading approach which minimises the total cost of the system with non orthogonal multiple access [27]. In [26], the authors propose two different matching algorithms to solve the resource allocation problem in two different tiers. In [49], a scheduling and resource allocation algorithm is proposed. This algorithm uses Lyapunov optimization to increase the average network throughput of the system. The authors use game theory to formulate a unified multi-tier cost model in a user scheduling problem and prove the occurrence of nash equilibrium [28]. As the fog nodes have limited computation power, the authors have used a competitive game approach to allocate resources in order to fulfill the user’s requests in [40]. In [30], the authors highlight the benefits of fog computing for future IoT industry 4.0 applications. They propose an energy efficient adaptive fog cloud architecture as per application requirements. In [2], the authors focus on minimising the latency by using genetic algorithms for IoT applications. However, all these works consider a flat architecture i.e. one layer of fog devices between the edge of the network and the cloud. Hence, they cannot be directly applied to multi-tier fog architectures, which are more complex.

There has been some research in multi-tier hierarchical fog cloud scheduling. In [33], Peixoto et al. propose a component based scheduler for a multi-tier fog cloud architecture. They study the effect of shifting the application components to the cloud so as to minimize the impact on network usage. However, they consider only two tiers of cloudlets/fog nodes in their work, and measure the results using only simulations. We consider a larger number of fog tiers and also report results for a real-life testbed. In [12], the authors propose multi-tier real time scheduling algorithms by considering two kinds of priorities – low and high. We consider non-real time jobs in our work. In [18], the authors propose a WALL scheme in which they sort the jobs in descending order based on their workloads (maximum workloads $\rightarrow$ minimum workloads). Next, they sequentially assign each job to a potential cloudlet/fog node that incurs the minimum response delay. Their work is limited to simulations conducted on a smaller number of tiers. In [42, 12, 17], the authors propose a Branch and Bound (BnB) algorithm for a multi-tier fog cloud architecture. The proposed algorithm is expensive computationally, and is hard to scale to large applications. This limitation has been addressed by [17], where the authors report simulation results for a two-tier fog cloud architecture. In [46], the authors propose a latency minimisation algorithm where the tasks are optimally divided and assigned to the nodes on multiple layers. However, their results are restricted only to simulations. On the other hand, we report results for a working prototype testbed.

We observe that various aspects of multi-tier fog computing have been discussed in the above literature. An effective mapping and scheduling of jobs in a multi-tier fog network still faces several challenges, specifically in terms of the cost model. To the best of our knowledge, no work has proposed algorithms for multi-tier fog networks while minimising the execution cost, which comprises of two components: completion time and energy consumption. Most work multi-tier hierarchical fog networks consists of fewer tiers in their architecture and propose algorithms that may not scale well. Along with this, they have performed either simulations or test-bed for the implementation. Our research work fills the void by simulating over multi tiers and giving a detailed analysis on the multi tier performance. We proposed algorithms that can scale well over large systems and validated our results on a hierarchical test-bed.

In this section, we discuss the problem formulation. Table I depicts the notation used in the rest of this paper. We consider a hierarchical fog-cloud architecture comprising of users ${(u_{1},u_{2},....)}$ at the lowest level. The $i^{th}$ user is denoted by $u_{i}$. The users are connected to m tier-1 fog devices given by $FD_{1}$ = {$fd_{1}^{1}$, $fd_{1}^{2}$,……,$fd^{m-1}_{1}$,$fd^{m}_{1}$}. The tier-1 fog devices are followed by r tier-2 fog devices given by $FD_{2}$ = {$fd^{1}_{2}$, $fd^{2}_{2}$,……,$fd^{r-1}_{2}$,$fd^{r}_{2}$}. The tier-2 fog devices are further connected to p tier-n fog devices, such as, $FD_{n}$ = {$fd^{1}_{n}$, $fd^{2}_{n}$,……,$fd^{p-1}_{n}$,$fd^{p}_{n}$}. Note that, n can be any tier such as, tier-3, tier-4, and so on. At the topmost level, we have a cloud data center cdc $\in$ C, where C is the set of all cloud data centers. The cloud data center cdc has the maximum execution execution capacity among all the tiers. The execution capacity of fog devices in $FD_{1}$ is less than fog devices in $FD_{2}$, which is further less than $FD_{n}$. A similar hierarchical fog architecture has been outlined in [19]. The order of execution capacity is: $cdc>FD_{n}>....>FD_{2}>FD_{1}$. The units of execution capacity is MIPS, i.e., Millions of Instructions per Second. We have used MIPS as the unit of execution capacity, as the iFogSim simulator [20] measures the computational capability of execution devices in MIPS. iFogSim is a well known simulator that has been used to simulate fog, edge, and cloud networks[8]. A smart home example scenario is shown in Fig. 2. In the considered smart home scenario, the jobs which require a response within milliseconds, such as burglar alarm, gas leaks, etc. can be executed on the fog devices in $FD_{1}$. The jobs such as maintaining overall temperature of the home, switching lights on/off can be scheduled at fog devices in $FD_{2}$. Next, the weekly/monthly grocery shopping by tracking the consumption of food of the individuals can be executed at higher tiers $FD_{n}$ with the nodes having higher computational capacity, where n = 3. Finally, the entertainment services such as recording some TV series to watch later can use the services of the cloud data center.

We consider a job set J, that can be executed on fog devices in $FD_{1}$, $FD_{2}$, …., $FD_{n}$, or the cdc. We considere a single cdc in our work. However, our work can be easily extended to multiple cloud data centers. The $i^{th}$ fog device at tier-1, tier-2, and tier-n is given by $fd^{i}_{1}$, $fd^{i}_{2}$, and $fd^{i}_{n}$ respectively. Likewise, the $k^{th}$ job is given by $j^{k}$. The delay between $j^{k}$ and $fd^{i}_{1}$ is given by $d(j^{k},fd^{i}_{1})$. Correspondingly, the delay between $j^{k}$ and $fd^{i}_{2}$ is given by $d(j^{k},fd^{i}_{2})$, and so on. Lastly, $d(j^{k},cdc)$ gives the delay between $j^{k}$ and cdc. We assume that the fog devices within each tier are homogeneous in nature, i.e., no “intra-level heterogeneity” has been considered. This implies that all fog devices in $FD_{1}$ have identical execution capacities. This holds true for fog devices in $FD_{2}$, and fog devices in $FD_{n}$ as well. However, all tiers, i.e., fog devices in $FD_{1}$, fog devices in $FD_{2}$, fog devices in $FD_{n}$, and cloud data center cdc are heterogeneous with respect to each other. In other words, we consider “inter-level heterogeneity” in our model. Intuitively, jobs with modest $\Delta_{j^{k}}$ are executed at tier-1 fog devices by default, as they can be executed faster owing to the distance proximity from the user. Generally, the jobs with moderate $\Delta_{j^{k}}$ are assigned to tier-2, or tier-n fog devices. Finally, the jobs with large cpu requirements $\Delta_{j^{k}}$ are assigned to the cdc.

Job $j^{k}$ can be assigned to fog devices in $FD_{1}$, $FD_{2}$, $FD_{n}$, or the cdc, depending upon it’s execution cost and/or sufficient spare capacity in the fog device or the cloud. Let $x_{tik}$ be the variable which denotes whether $j^{k}$ is assigned to a fog device or to the cloud data center:

In eq. (1) above, $fd^{i}_{t}$ can be $i^{th}$ fog device present at $t^{th}$ tier ($t=1,2,....,n$).

We need to make sure that each $j^{k}$ is assigned to a single fog device only. The following equation ensures this constraint:

The begin time bt of $j^{k}$ on a fd $\in$ FD is denoted by $bt(j^{k},fd)$. We assume that all jobs are independent of each other, so no precedence constraints are present among the jobs. Hence, theoretically a job may begin at time 0. The execution time of $j^{k}$ is represented by $et(j^{k},fd)$. The completion time of $j^{k}$ $\in$ J on fd $\in$ $FD$ is designated by $ct(j^{k},fd)$. The completion time of $j^{k}$ can be defined as:

The minimum completion time $ct_{min}(j^{k})$ of a job can be calculated as:

In eq. (4) above, $fd^{i},fd^{x}\in FD$, $j^{k}\in J$ and $x\neq i$.

Next, we model the network usage $NU$ of the system. This quantity is defined as the total data sent and received by various network devices: fog devices in $FD_{1}$, $FD_{2}$, $FD_{n}$, and the cloud data center cdc. Mathematically, $NU$ can be defined as:

In eq. (5) above, $dp(j^{k})$ is the data processed by $j^{k}$, $ad(j^{k})$ is the aggregate delay that occurs during the submission and execution of $j^{k}$, and $st$ is the time for which the simulation was run.

Let us assume that the maximum amount of jobs that can be processed at fd is $\mu_{i}$ and that the job arrival rate at fd is $\lambda_{i}$. Note that, fd can be any fog device present at tier-1, tier-2, or tier-n. We follow an M/M/1 queuing system to process user jobs at each fog device [43]. The average completion time cost $act(j^{k},fd)$ for $j^{k}$ at fd can be calculated as:

Likewise, we can model act of cdc as:

Let the average job execution capacity of fd be $acp(fd)$. We can derive the average energy consumption cost $aec(j^{k},fd)$ as follows:

The energy consumption of a fog device is directly proportional to the fog device’s computation power/execution capacity.

Similarly, $aec(j^{k},cdc)$ can be defined as follows:

As each fog device has a limited resource capability, this means that the offloaded job’s $\Delta_{j^{k}}$ must not exceed it’s $\mu_{i}$.

Here $\delta_{i}$ is the remaining spare capacity of fd $\in$ FD.

We assume that $\Delta_{j^{1}}$, $\Delta_{j^{2}}$,…, $\Delta_{j^{n}}$ be the cpu requirements of the jobs which are being offloaded to fog devices in $FD_{1}$. The probability that $fd_{1}^{i}$ can run the $j^{k}$ with $\Delta_{k}$ is given by $P(\Delta_{j^{k}}\leq\mu_{fd_{1}^{i}})$. Hence, the probability that all tier-1 fog devices can schedule jobs with different cpu requirements can be defined as follows:

In case $P(\Delta_{j^{k}}\geq\mu_{fd_{1}^{i}})$, this means tier-1 fog devices are unable to schedule the jobs based on their cpu requirement. In this case, the jobs are sent to the higher tiers for execution. Likewise, we can define the probability for all tier-2 fog devices, and tier-n fog devices. For the sake of brevity, we omit these equations.

The overall cost for all tier-1 fog devices can be written as:

The overall cost for all tier-2 fog devices can be written as:

A similar formulation can be carried out for all tier-n fog devices.

Likewise, we model the cost for the cloud:

The optimisation problem that we attempt to solve in this work is modelled as follows:

Given a set of jobs $J$, a set of fog devices $FD_{1}$, $FD_{2}$, $FD_{n}$, and a cloud data center cdc, with inter-level heterogeneity, schedule the jobs onto fog tiers/cdc by using FiFSA & EFSA such that the overall cost is minimised. Formally,

We now describe the working of the two proposed algorithms for scheduling jobs on multi-tiered fog-cloud architectures : FiFSA and EFSA.

The steps involved in the working of FiFSA are described below:

1. 1.

   In the first step, all the jobs are arranged in increasing order of $\Delta_{j^{k}}$ in Q.

2. 2.

   The $\mu$ of fog devices in $FD_{1}$ is compared with $\Delta_{j^{k}}$. If the current fog device has spare capacity to run $j^{k}$, then the job is scheduled on $fd_{1}^{i}$. The value of flag is set as 1.

3. 3.

   Otherwise, the $\mu$ of fog devices in $FD_{2}$ is compared with $\Delta_{j^{k}}$. If the fog device can run the job, then $j^{k}$ is scheduled on $fd_{2}^{i}$.

4. 4.

   Otherwise, we check the capacity of tier-3 fog devices, and so on till we obtain an appropriate fog tier to run $j^{k}$.

5. 5.

   Finally, if none of the $FD_{1}$, $FD_{2}$, or $FD_{n}$ are capable of scheduling $j^{k}$, the job is then scheduled on the cloud data center cdc.

6. 6.

   Calculate $Co_{sys}$.

Note that while selecting the fog device to schedule a job, we evaluate all fog devices present at that tier sequentially. Once we identify a fog device which has sufficient spare capacity, we assign the job to the fog device and set the variable flag$(j^{k})=1$, and move on the next job. If no fog device at tier-1 is able to schedule the job, we proceed to the tier-2 fog layer, and so on.

We now discuss the functioning of the proposed scheduling algorithm EFSA. Here, we create different queues for each fog-tier according to $\Delta_{j}$ of the jobs. The steps involved in the working of EFSA are described below:

1. 1.

   The $\mu$ of fog devices in $FD_{1}$ is compared with $\Delta_{j^{k}}$. If the job’s cpu requirement is within the execution capacity of fog devices in $FD_{1}$, then the job is populated in $Q_{1}$. However, if the cpu requirement of job $j^{k}$ is more than the capacity of that fog-tier, then it is moved to the next tier, else it is placed in the queue of the same tier.

2. 2.

   Otherwise, $\Delta_{j^{k}}$ is compared with the capacity $\mu$ of fog node in $FD_{2}$. If the job can be run with the execution capacity of tier-2 fog devices, then job $j^{k}$ is added to $Q_{2}$.

3. 3.

   Likewise, we populate the jobs in $Q_{n}$ by measuring the $\Delta_{j^{k}}$ with the execution capacity of fog devices present at the tier-n $FD_{n}$.

4. 4.

   The remaining jobs are scheduled on the cloud data center $cdc$.

We use the min-min scheduler to execute the jobs present at fog devices in $FD_{1}$, fog devices in $FD_{2}$, and on fog devices in $FD_{n}$. The rationale behind choosing the min-min scheduler is that it finds the job with smallest completion time and assigns it to the fastest fog device. This fog device is selected by using the procedure MinMinNode($Q_{1}$, $cl$, $ct$, $FD_{1}$).

The Min-Min procedure works as follows:

1. 1.

   For each $Q_{1}$, $Q_{2}$, and $Q_{n}$ present at each tier $FD_{1}$, $FD_{2}$, and $FD_{n}$ respectively, we use the Min-Min scheduler for the job execution at each tier separately [23].

2. 2.

   The Min-Min heuristic works in N iterations, where N is the number of jobs which need to be scheduled.

3. 3.

   This heuristic consists of two steps, as outlined below:

   • •

     In the first step, the heuristic finds the minimum completion time MCT of each unassigned job over the set of fog devices. It finds the best fog device i.e. the fog device which can finish the job in the earliest time. The algorithm takes into account the current load $cl(fd_{1}^{i})$ as well as completion time $(ct)$ while finding the best fog device for job scheduling.

   • •

     In the second step, the heuristic chooses the unassigned job which has the minimum MCT among all jobs. This job will be assigned to the best fog device found in the first step.

   • •

     The scheduled job is removed from the queue, and is not considered in the rest of the iterations.

4. 4.

   The above process is repeated until all jobs present in different queues have been scheduled.

5. 5.

   Finally, $Co_{sys}$ is calculated.

In FiFSA, we first sort the jobs in an increasing order of cpu requirement. We note that the fog devices at low tiers have modest execution capacities. Hence, it may not be advisable to schedule the jobs with high cpu requirements on lower tiers. Once we identify a fog device having sufficient spare capacity, we schedule the job on it. Otherwise, we keep checking fog devices on higher tiers. For the second algorithm, EFSA use the MinMin scheduler. MinMin is a fast and simple algorithm that considers smaller jobs first, and assigns them to fast processors, which in turn minimises the completion time. This aligns well with our hierarchical architecture, as the fog devices at lower tiers have lower execution capacity as compared to fog devices present at higher tiers.

We now discuss the results of the simulations that have been carried out to analyse the performance of the proposed algorithms : FiFSA and EFSA. We consider various simulation scenarios and evaluate the performance for each scenario. The considered sample scenarios align with our fog architecture introduced in Fig 1. The jobs may execute on : tier-1 fog devices, tier-2 fog devices, tier-3 fog devices, tier-4 fog devices, and the cloud data center cdc. We have considered several tiers of fog devices to measure the performance of the algorithms on different architectures. The proposed model can be extended to a larger number of tiers, as per the requirements of the application. In FiFSA, the jobs execute on the respective tiers as per their $\Delta$. If $\mu$ of a tier-1 fog device is sufficient to meet the requirements of $j^{k}$, then $j^{k}$ will execute on tier-1, otherwise it will be dispatched to tier-2 for execution. Again, $\mu$ of a tier-2 fog device is checked, if it is sufficient, then job can execute on tier-2. If it’s not sufficient, then further tiers are checked for job execution. If the job is still unscheduled on all examined fog tiers, it is executed on the cdc. We compare the results of four algorithms: FiFSA, EFSA, LTF, and cdc-only. In cdc-only, the jobs execute only on the cloud data center, without considering the fog devices. The EFSA algorithm schedules the job on the fog device by finding the MinMinNode onto various fog tiers. We use MinMinNode and elected fog device interchangeably in the paper. The LTF (Longest Time First) algorithm has been proposed In [47]. LTF assigns the job with the longest execution time to the fog node which executes it in the minimum time i.e fastest node.

There is a significant propagation delay involved in executing the jobs on cloud data centers. This delay can be reduced significantly by incorporating fog devices. Application requirements, such as smart transportation, often necessitate the consideration of a hierarchical fog-cloud model, with many layers of fog nodes. In this paper, we propose two algorithms that schedule jobs on such multi-tiered fog networks: FiFSA and EFSA. We take job characteristics such as cpu requirement into account, while scheduling them on various fog tiers (up to 4 fog tiers) and the cloud data center. The idea is to allocate the jobs on appropriate fog devices or the cloud data center, while minimising the overall cost. We compared the performance of both algorithms with cdc-only and the LTF algorithm, for a number of scenarios. The simulation results on a real-life workload and a testbed show that EFSA offers the best performance in terms of cost, completion time and network usage. In the future work, we aim to integrate multiple cloud data centers. Additionally, we plan to work on both ”inter-level heterogeneity” and ”intra-level heterogeneity” in our model.