Optimising resource allocation for virtual network functions in SDN/NFV-enabled MEC networks

Network function virtualisation (NFV), software deﬁned networks (SDNs), and mobile edge computing (MEC) are emerging as core technologies to satisfy increasing number of users’ demands in 5G and beyond wireless networks. SDN provides clean separation of the control plane from the data plane while NFV enables the ﬂexible and on-the-ﬂy creation and placement of virtual network functions (VNFs) and are able to be executed within the various locations of a distributed system. In this paper, VNF placement and resource allocation (VNFPRA) problem is considered which involves placing VNFs opti-mally in distributed NFV-enabled MEC nodes and assigning MEC resources efﬁciently to these VNFs to satisfy users’ requests in the network. Current solutions to this problem are slow and cannot handle real-time requests. To this end, an SDN-NFV infrastructure is proposed to tackle the VNFPRA problem in wireless MEC networks. Our aim is to minimise the overall placement and resource cost and also to minimise the total number of VNF migrations. A genetic based heuristic algorithm is proposed. The superior performance of the proposed solution is conﬁrmed in comparison with four existing algorithms, i.e. resource utilisation-single objective evolutionary algorithm (RU-SOEA), genetic non-bandwidth link allocation algorithm (GA-NBA), random-ﬁt placement algorithm (RFPA), and ﬁrst-ﬁt placement algorithm (FFPA). The results demonstrate that a coordinated placement of VNFs in SDN/NFV enabled MEC networks can satisfy the objective of overall reduced cost. Simulation results also reveal that the proposed scheme approximates well with the optimal solution returned by Gurobi and also achieves reduction on overall cost compared to other methods.

able to be executed within the various locations of a distributed system and, in our case, in the NFV-enabled MEC nodes.
In cloud environments, cloud providers propose their infrastructure, which is composed of a single or multiple data centers (DCs) containing computing and networking equipments. NFV, as an evolving technology, follows the similar concept and takes it to a higher level by leveraging virtualisation technology to virtualise network functions; yet, it has brought new challenges to network operators such as (i) finding the most appropriate placement of these VNFs into the MEC environment (ii) optimal resource allocation to these VNFs in NFV-enabled MEC nodes. Finding the best placement of a single VNF may not be as challenging as allocating resources (i.e. CPU, memory, storage, and bandwidth) for a set of VNFs in MEC nodes. Specifically, when a service function chain (SFC) consists of one VNF, it is a matter of finding a MEC node which has enough resources to minimise the costs. However, when an SFC comprises of multiple VNFs, the placement problem turns into even more difficult since VNFs are limited to some certain order as well as link and VNF delays. SFC [5] is a capability that uses software defined network (SDN) capabilities to create a service chain of connected network services (e.g. firewall, network address translation) and connect them in a virtual chain.
MEC nodes are usually distributed in different locations. The energy price for each MEC node varies depending on its location; hence, the operational cost (i.e. cost of running VNFs) is not the same. Moreover, from service providers' (SPs') perspective, they aim at provisioning service requests and increase their total revenue. This can only be accomplished by allocating resources for VNFs in MEC nodes such that the service request is provisioned at a reasonable cost to the end-users.
The combination of SDN, NFV, and MEC technologies are widely considered to be essential in the development of 5G wireless networks. MEC, together with SDN and NFV will play to address the challenges that 5G aims to undertake. We consider MEC nodes to be any NFV-enabled network nodes at which MEC services can be hosted with allocated VNFs. MEC involves placing computing power in new locations. This computation may involve network functionality and end-user applications. None of the studies on the integration of MEC and SDN explores the use of NFV to deploy specific VNFs to offer final user services, such as VoIP, video streaming, or web browsing. In [6], a recent placement scheme applicable to MEC in NFV environment has been proposed to minimise access latency and maximise service availability for ultra-reliable low-latency communications (uRLLC) services. The authors proposed a genetic algorithm and compared with CPLEX implementation. Their approach did not consider failure recovery in case of physical machine (PM) breakdowns, especially at the edge and for critical services, such as the connected car. Although the work proposed in [7] considered NFV resource allocation (NFV-RA) problem, it did not consider transmission latency. Also in the aforementioned latency-aware NFV-RA works, the hierarchical network resource allocation has not been considered yet. Hence, their solutions are not fit for the hierarchical network that contains both MEC nodes and DCs. Thus, these solutions may not be appropriate to implement in dynamic network resource allocation systems with a wide range of resource demands and services.
To benefit from the advantages of SDN, NFV, and MEC, VNFs need to be provisioned with sufficient resources in edge servers without impacting network quality of service (QoS). This paper proposes an SDN/NFV-enabled MEC model for optimal virtual network function placement and resource allocation (VNFPRA) with the goal to minimise both the overall deployment cost and resource cost. A single VNF might be enough to handle certain amount of requests before it becomes overloaded. But, when there is a sudden spike of requests, one VNF may not be enough to process all the received requests. In such case, we investigate the impact of replicating VNFs from neighbouring MEC nodes or from remote cloud with the help of centralised SDN controller. When receiving a VNF request, NFV-enabled MEC nodes either process the request and interact directly with the user if the requested VNF is available in it, if not, SDN controller decides whether the requested VNF should be forwarded to a neighbouring MEC node. Whenever a new VNF request from user equipment (UE) enters, the first NFV-enabled MEC node receives a description packet, which is forwarded to the SDN controller. This packet contains the description of the VNF, the bandwidth and the destination node that the UE requires. If the required VNF is not in the NFVenabled MEC node, the controller finds the optimal route and resource allocation along the route such as bandwidth for the transmission of VNF.
The main contributions in this work are as follows: (i) We propose a novel SDN/NFV-enabled MEC infrastructure for low-cost MEC resource allocation framework; (ii) We formulate the VNFPRA problem as a mixed integer program (MIP) problem by taking into account the possibility of NFV-enabled MEC service instantiations and propose a genetic based heuristic solution which is theoretically proven to be sound and complete. Simulation results reveal that the proposed scheme approximates well with our optimal solution returned by Gurobi and also achieves reduction on overall cost compared to other methods.
The paper is organised as follows. Section 2 addresses the related work. Section 3 describes the proposed system model. Section 4 addresses the problem formulation. In Section 5, we present details of cost-aware VNFPRA for NFV-enabled MEC. Section 6 describes the proposed genetic algorithm for VNF migration (GA-VNFM). In Section 7 and 8, we show our simulation results and conclude the paper, respectively.

RELATED WORK
Recently, with the advent of SDN [8] and NFV [9], the concept of NFV-enabled MEC emerged whereby services can be hosted at any conventional network node that has virtualised resources (e.g. switches, routers, access points etc.). Within this new model, the on-demand network resource allocation in the form of VNFs becomes achievable. This has the advantage of reducing investments in specialised hardware, while allowing flexible scaling of the capacity of the functions [10]. Additionally, routers and switches can use their unused computational resources to implement these VNFs [11]. NFV has become an essential technology for delivering network services to the end users. The network services accessed by users are becoming more dynamic and diverse with the rapid growth of user data traffic [12]. NFV can also be interconnected with other technologies such as internet of things (IoT), cloud computing, and MEC. It can also provide the necessary flexibility to offer diverse and dynamic network services through the service chaining (SC) model. SDN and NFV are complementary in nature. They have intersected objectives of substituting proprietary and closed network elements (switches) with commodity hardware that can be controlled by standardised software [13].
Existing works in the literature consider delay/latency perceived in the overall communication during the VNF placement and SC deployment. This overall communication delay is expressed as the propagation delay. In such scenarios, the delay is either considered explicitly in the objective function with the ultimate goal to be minimised [14] or it is taken as a constraint satisfying a maximum tolerable delay [15]. The delay minimisation has been studied in a multicloud environment. Authors in [16] considered only the distance between the cloud provider and the end user. In this case, the end user may be able to select among a set of available SPs to find the best offer between the latency perceived and deployment cost. On the other hand, the authors in [17] used a random cloud selection approach which combined with a predictive model with the aim to keep the delay below a threshold.
Existing techniques also consider load balancing features and deployment costs when placing VNFs and SCs. The goal is to increase the availability of the resources, providing essential scalability, and increasing the number of SCs in cloud infrastructure to maximise the revenue of the SP. In terms of load balancing there is a high interest in resource usage and path optimisation [18]. Deployment cost is usually expressed as the communication and computational resources that a SC needs, in order to be provisioned. Most of the existing procedures try to consolidate as many of the VNFs as possible in the same server [19], or rack in case of a data center. The authors in [20] proposed a microeconomic approach inspired from producer-consumer theory to model the interactions between virtual machines (VMs) and physical resources. Rankothge et al. in [21] provided a multi-objective genetic algorithm (GA)-inspired approach to predict dynamic resource utilisation and energy consumption in data centers. The results showed that the proposed placement algorithm not only reduces the memory and CPU usage, but also reduces the energy consumption of the data center as well.
Management and resource orchestration are essential for the NFV-MEC deployments. One of the first efforts to define a double-tier NFV-MEC architecture associated to the European Telecommunications Standards Institute (ETSI) NFV management and orchestration (MANO) specifications was proposed in [22] by improving the resource orchestration for MEC functions and IoT deployments (e.g. Cloud RAN). The MEC architecture and open research issues were also surveyed in [23] by emphasising the necessity to extend the NFV ecosystem at the edge and far edge scenarios to deal with 5G and IoT network deployments. Yi et al. in [24] addressed SFC provisioning problem and designed two meta-heuristic algorithms to make a trade-off between optimal network performance and time complexity. Luizelli et al. in [25] presented a metaheuristic algorithm using variable neighbourhood search (VNS) to minimise allocated resources, while meeting network flow requirements.
The resources in MEC nodes are restricted compared to that in remote data centers [26]. By combining NFV and MEC technologies these resources can be utilised efficiently because network operators can assign resources for VNFs according to the MEC resource usage requirement. Cost reduction is one of the foremost goals of NFV. Most latency-aware NFV-RA methodologies emphasise on minimising cost without sacrificing service latency requirements. Authors in [27] developed a dynamic cost minimisation model by considering delay into the cost objective in order to jointly optimise three steps in NFV-RA, including VNFs-chain composition, VNFs-scheduling, and VNFs-forward graph embedding. In [28] a mixed integer linear programming (MILP) model with latency constraints is proposed for the cost minimisation in metro core networks. Another model is designed in [29] to minimise the same cost in the previous model to upgrade the solutions by adding virtualisation latency into constraints. A low latency VNF placement algorithm is designed in data center networks to reduce service latency [30]. In [31] a mixed integer quadratically constrained programming model is developed to minimise resource consumption for VNF placement while producing specific latency.
Only few works have addressed the problem of VNF placement in a mixed environment that includes edge and central clouds. Authors in [32] [33] introduced an efficient content delivery network as a service platform and VNF placement and proposed optimisation strategies over an edge-central carrier cloud infrastructure by taking into account QoS requirements. Their main design goal is to provide an optimal tradeoff solution between the cost efficiency and quality of experience (QoE) to optimise resource utilisation for an appropriate set of VNFs. Usually, VNFs are placed either in physical machines within central cloud infrastructures only [34], or in MEC servers [35].
Although there are some methodologies for managing NFV and edge resources, there is still a chain of open issues and further gaps with regard to their well-organised integration and edge infrastructure management in NFV ecosystems. In order to address these gaps, we have taken a mixed environment which is consistent with current state-of-the-art SDN and NFV architectures. This paper addresses the interaction of the three technologies (SDN, NFV, and MEC) and builds a new framework for optimal VNF placement and resource allocation to minimise the overall deployment and resource cost. It also focuses on how to minimise the total number of VNF migrations and maximise the total utility of the MEC system.

SYSTEM MODEL
The system model consists of a control plane and a data plane as shown in Figure 1. We assume that the network is made of N small base stations (BS) and each BS is attached with an NFV-enabled MEC node connected directly to each other that are capable of being controlled by an SDN controller. As in SDN-enabled technologies, only the centralised controller has the full view of the global network topology and it is a responsive agent, which will react based on the requests that it receives. We also assume that the SDN controller can communicate with the MANO layer of the NFV. As stated in [36], VNF manager is the critical component of the NFV architecture, also called MANO. In this system, we consider each MEC server has VNFs which are managed by a VNF manager located in the core network. VNF manager can start and stop VNFs based on local demand. By doing this, we can reduce deployment costs, power consumption as well as improve service readiness to operate a network component. The controller is assumed to have sufficient resources to run all possible VNFs to satisfy user's demand. By using appropriate cost function, the controller finds the best possible route of the incoming requests. It may happen that for the chosen route there is no feasible VNF placement found or there is no possible route to migrate VNFs. In such cases the SDN controller signals the NFV MANO layer to migrate the required VNF containers appropriately to the required NFV-enabled MEC nodes. The controller also pushes some flow rules for the request in the respective NFV-enabled MEC nodes. The rules are added in sequence starting from the destination node and the last rule is added at the source node in the path. After this, the incoming requests are switched across the data plane.
In order to effectively manage the network, there are two main decisions the controller needs to make: (i) VNF placement, i.e. how to allocate VNFs and assign optimal set of resources by considering the resource availability in MEC nodes. The cost incurs in deploying VNF services and delivering them to the end-users must be minimised for cost-efficiency, and (ii) Resource allocation, i.e. how the computational capabilities available at NFV-enabled MEC nodes are allocated to the VNFs. We use (, ) to denote the network with N users, where  = {1, 2, ⋯, V } is the set of NFV-enabled MEC nodes and  is the set of links between the NFV-enabled MEC nodes. In this paper, we assume that any end user can be within the coverage of one MEC node. Total demands that each VNF may satisfy are limited. For simplicity, initially we assume that one end user sends only one request to the MEC node and one VNF can handle K number of requests. Requests sent from users should be first received by the NFV-enabled MEC node j , where j ∈  and it interacts directly with the user if the requested VNF is available in it. But, when there is a sudden spike of requests and j -th MEC node cannot process these requests because of capacity limit, it can replicate required VNFs from the neighbouring MEC nodes or from remote cloud by VNF manager. Each MEC node j is attributed with a vector of available resource capacities W (e.g. CPU, Memory). We consider that each NFV-enabled MEC node j , has maximum computing capacity C max j and maximum storage capacity S max j , which is the maximum capacity to compute and store VNFs. We denote the set of VNFs of different types as  = {1, 2, ⋯, F }. Each VNF i ∈  on MEC node, if activated, consumes some physical resources (i.e. DRAM memory, CPU cycles) and has a predetermined resource requirement of computing capacity c i (CPU cycles) and storage capacity s i (memory) to process it.
We summarise the notation and symbols used in this paper in Table 1 unless otherwise specified.

PROBLEM FORMULATIONS
Initially we migrate the VNFs from remote cloud and store them in distributed MEC nodes. This migration requires placement cost. After that we replicate the VNFs from neighbou ring MEC nodes if needed rather than replicating from remote cloud. This replication may cause link usage cost (replication cost) to replicate VNFs from adjacent MEC nodes. When a MEC node does not have the VNF application requested by some users, we should determine whether the application VNF should be replicated from another MEC node or not. Let l ( j, k) denote the link between MEC node j and MEC node k and every link l ∈  has capacity c l and per unit link usage cost of u l j →k,i . We denote by y l j →k,i the decision variable stating whether the link is used to replicate VNF i from NFV-enabled MEC node k to NFV-enabled MEC node j . Bandwidth demand for replicating VNF i over the link l is Based on above assumptions, we address the VNF placement and resource allocation problem subject to various constraints such as the VNF requirement of each request and the resources available at each NFV-enabled MEC node to deploy the VNFs. We need to place VNFs at the NFVenabled MEC node such that the overall resource cost is minimised. We consider three different types of costs incurred as follows:

VNF placement/allocation cost
This is the cost incurred if any VNF is placed/hosted in NFVenabled MEC node. The optimal VNF placement involves determining how many VNF instances are needed to meet the end user's demand and on which servers they should be deployed. This cost can be defined as where C j i is the cost of placing VNF i at NFV-enabled MEC node j . a i j is the allocation variable where

Resource/computation cost
This is the cost per utilised processor and memory of the NFVenabled MEC node. This cost can be expressed as where R j i is the number of VNF requests for VNF i from MEC node j , which is known or given. C j W is the cost of resource W (e.g. CPU, memory) at MEC node j .

Link usage/replication cost
This is the cost per utilised capacity of the virtual links. This can be defined as ∑ where u l j →k,i is the link usage cost for replicating VNF i from MEC node k to MEC node j over the link l .
The objective function minimises the overall resource cost. These are the placement cost incurred if required VNF is placed/hosted in NFV-enabled MEC node, cost per utilised memory and CPU of the NFV-enabled MEC node, and the cost per utilised capacity of the links, which is defined as where (4a) and (4b) represent that the sum of computing capacity of NFV-enabled MEC node j for activating VNFs that cannot exceed the maximum computing capacity C max

COST-AWARE VNF PLACEMENT AND RESOURCE ALLOCATION FOR NFV-ENABLED MEC
To find the optimum placement of MEC nodes required for the deployment of VNFs, the problem addressed in this paper is MIP problem that takes into consideration the virtualisation of MEC applications. Since the allocation of VNFs is known to be NP-hard [37], we propose a heuristic based genetic algorithm for VNF placement and resource allocation (GA-VNFPRA) which is a heuristic algorithm used to search for an optimal solution by simulating the natural evolutionary process (selection, mutation and crossover). The algorithm optimises the placement of VNFs which are distributed on the NFV-enabled MEC nodes.
The detailed explanation of the proposed heuristic algorithm for our optimisation problem is as follows.

Chromosome representation
In our problem, each solution of VNF placement is represented by a chromosome. A chromosome signifies a possible VNF placement solution. We implement a matrix representation for a chromosome. We use a binary encoding method to represent the assignment VNF placement solution. The placement matrix solution {a i j } is a m × n matrix if there are m VNFs and n NFV-enabled MEC nodes. If the ith VNF is deployed on the j th MEC node, the entry a i j is set to 1 to guarantee that the VNF i is deployed only once on the MEC node j and other elements in the ith row must be equal to 0. This binary encoding technique contributes to simplify the crossover and mutation operation between two matrices. Figure 2 shows an example of the proposed chromosome for our optimisation problem. In this example, there are 10 VNFs. It describes that the VNF1 is deployed on MEC node 3, VNF2 is deployed on node 2, VNF3 is deployed on node 1 (MEC n1), VNF4 is deployed on node 1 etc.

Population initialising
We generate initial population I p which grows in each generation of genetic algorithm. First, we generate the accessible set  j i for each VNF i in MEC node j .  j i represents the set of all MEC nodes from where VNF i can be migrated. Then, the MEC node calculates the accessible probability P  i,k for each VNF i in MEC node k, which is used to identify the target MEC node where the VNF can be migrated. MEC node calculates the accessible probability for each VNF i, that represents how likely VNF can be migrated to another MEC node. First of all, we generate the accessible set for each VNF and the previous best solution to get the initial number of populations. Then we calculate the accessible probability for each VNF i. If previous best solution is available then we use it as a seed with the accessible probability. The MEC node with higher accessible probability has higher chances for getting selected. We calculate P  i,k by the following equation: where s j i,k is the variable which represents if VNF i can be migrated from MEC node j to MEC node k.

Fitness function
The fitness function evaluates how well an individual matches the objective. We should obtain an optimal placement solution considering our main objective, minimising the overall resource cost. In order to be consistent with the final objective, the fitness of chromosome is calculated based on overall cost. The fitness function evaluates how a chromosome matches our main objective. In our Algorithm 1, fitness is defined as where i is an individual in the population and the objective function is defined in (4). Considering the objective function in (4), we calculate the placement and resource cost of all the VNFs mapped on each MEC node according to the matrix placement solution. Then, we get the best fitness value for this solution. Moreover, the individuals with higher fitness value should be nominated as parents to generate new populations for obtaining better chromosomes for next generation. Algorithm 1 solves our optimisation problem by generating a group of random chromosomes a a , a b , a c , and a d and evaluate each chromosome by (4)-(4f). Two best chromosomes a a and a b whose resource cost in (4) is lower are selected to produce another group of random chromosomes by mutation and recombination. This technique lasts until the acquired result reaches convergence or until the number of generations reaches the maximum number of generations.

GENETIC ALGORITHM FOR VNF MIGRATION (GA-VNFM)
Next key challenge is to find an optimal MEC node for VNF migration (VNFM). The goal of VNFM is to minimise the total number of VNF migrations and to maximise the total utility of the MEC system.
In VNF migration, we have to assign the VNFs to new MEC nodes so that the total utility of the MEC system is maximised. This is a NP-hard problem and thus, there is no algorithm that can provide an assured optimal solution in polynomial time. For this reason, we propose a GA-VNFM that can ensure near-optimal solution by reducing the total number of VNF migrations. The proposed Algorithm 2 allows us to identify the candidate MEC node for VNFM. The SDN controller makes  a a and a b into a c and a d ; insert a c and a d into  k+1 ; endfor / / Mutate: decision on possible MEC nodes based on the load of the target MEC node. If the load of MEC node is not considered, then the total number of VNF migration will be increased. In the following subsection we present different parts of GA-VNFM.

Chromosome representation
In our problem, each solution of VNFM is represented by a set of chromosomes denoted as  = {C 1 , C 2 , ⋯, C N } of length N . Each bit in a chromosome represents a VNF and the value of this bit is used to identify the target MEC node where the VNF can be migrated. Next, we use Algorithm 2 to generate the initial population and then we have to select two individuals for the crossover after the calculation of fitness value. After selecting two individuals, we perform mutation operation on the new offspring. Then, we replace the individuals i.e. y c and y d with the lowest fitness by the offsprings y a and y b with the highest fitness value.

Fitness function
For VNF migration in our model we can use the following fitness function to evaluate all the individuals in a population: Here, F n is the fitness value for nth individual and U j n,i is the utility of VNF i in MEC node j , which is defined by the following equation: where P j n,i is the minimum normalised task processing time of VNF i for the n-th individual when VNF i is migrated from MEC node j to MEC node k. When VNF i is not migrated, it means the selected MEC node doesn't have the required VNF to migrate. In such case the SDN controller signals the NFV MANO layer to migrate the required VNF appropriately to the required NFV-enabled MEC node. We calculate P j n,i by this equation where  j i is the accessible set for each VNF i in MEC node j and  p is the set of individuals.
Moreover, T j n,i is the minimum normalised transfer time of VNF i for the nth individual from MEC node j to k. Using the below equation we calculate minimum VNF transfer time as where L j n,i is the minimum normalised VNF load association degree between MEC node j to k, which helps the system to predict the load of the MEC node. Minimum normalised VNF load association can be calculated as ALGORITHM 2 Genetic algorithm for VNF migration Input: Fitness function.
Pick out two individuals based on P s n .

10.
Replace y c and y d in  P with y a and y b respectively.

Return the best individual.
We use min-max normalisation technique to calculate P j n,i , T j n,i , and L j n,l .

Genetic operators
Once fitness value for each individual is calculated, next we have to choose two individuals for the crossover operation. At the start of the selection process, the selection probability P s n for each individual n is calculated using this equation: Here, an individual with greater P s n has greater chance to be selected. Next, a mutation operation is performed on the new offspring with the mutation probability P m and then we reduce the offspring which are not compatible with VNF's accessible set. To retain the stochastic property of VNFPRA problem, we replace an individual from  P with the offsprings such that the offspring fitness values are higher than the individual fitness values. The whole procedure is listed in Algorithm 2.

PERFORMANCE EVALUATION
In this section, we assess the performance of our proposed methods for VNFPRA. We develop a simulation model using Python programs to evaluate the performance of the proposed algorithms. Each MEC node is randomly deployed in the network. The system initially generates the target users, each of them is randomly distributed in the coverage area of different   Figure 3. The simulation settings are shown in Table 2.
In this work, first of all we compare the proposed MIP model, implemented using the Gurobi optimizer [40] , with the proposed GA-VNFPRA to show how close our solutions are to the optimal values returned by MIP. In particular, we used the Gurobi with PyCharm to execute the MIP algorithm. Gurobi is an optimisation solver that is available for a number of programming languages. The Gurobi distribution includes a Python interpreter and a basic set of Python modules and these are sufficient for building and running optimisation models. Each PyCharm project also have a virtual environment (venv) associated with it. The gurobipy package is available to the Python interpreter of that venv. The parameters used in genetic algorithm are listed in Table 3. Furthermore, the proposed heuristic based GA-VNFPRA algorithm is studied in detail to achieve the near optimal placement and cost effectiveness for VNFs. Regarding MIP approaches, they are trying to go through every possible solution to extract the optimal one but they are char- acterized by an increased complexity for large scale experimentations. In contrast to MIP, the genetic algorithm is very simple and easy to implement by simply translating the pseudocode into a Python code. For optimal solutions, we consider the VNF placement with/without dynamic virtual link bandwidth allocation. It is not surprising that MEC nodes running a higher number of virtual functions may experience larger traffic loads, causing congestion on their outgoing virtual links, which, in turn, become bottlenecks and ultimately affect the transmission delays along those links. The reality is that not all virtual links should be allocated for the same share of the underlying physical capacity. Rather, a more (traffic-aware) proportional allocation of bandwidth is suitable. The obtained results assert that a traffic-aware link bandwidth allocation in GA-VNFPRA is able to reduce the overall cost. Indeed, this is achieved by allocating more bandwidth for the links which need it most, i.e. for those MEC nodes with higher loads. Such allocation is able to reduce the transmission delays: higher capacity results in lower costs. Furthermore, to better illustrate the efficiency of our proposed approaches, we compare them against the following baseline algorithms: (1) Resource utilisation-single objective evolutionary algorithm (RU-SOEA): The RU-SOEA is an evolutionary algorithm designed to minimise the size of the robot body and reducing the number of collision between the robot and the obstacle during the climbing motion presented in [38]. We use RU-SOEA approach to compare it with our proposed approach by designing fitness function in such a way that the lesser is the fitness score, the best the placement solution is. Once fitness score is obtained, it will be compared to the fittest placement fitness score. If current fitness score is better than the fittest score, current solution will be kept as the fittest one. The simulation will be repeated until the maximum number of generation is reached. In our simulations GA-VNFPRA and RU-SOEA both methods are able to produce optimised results with the capability to performing better placement solutions. However, one of the solutions from GA-VNFPRA is outperforming the fittest solution from RU-SOEA. The reason is in RU-SOEA each evolutionary run generates single solution, the second solution will only be generated after the first solution is obtained; these processes will be repeated for obtaining other solutions. While the advantage of GA-VNFRA is its capability in generating a complete set of optimal solutions in a single run with provides users a choice of best placement solutions This algorithm places each VNF at the first available MEC node that has enough capacity to host it. (5) Gurobi optimizer: This is an exact algorithm [40] which derives the optimal solution to (4).
We use Python programming to verify the effectiveness of the proposed algorithm. In this section, a comparison of the two approaches is presented via simulation. Figure 4 presents a comparison of the objective function values obtained for our proposed GA-VNFPRA and optimal solution returned by Gurobi. MIP algorithm is used as a benchmark since it always has the exact optimal solution but is not implementable for large-scale systems (e.g. MIP needs more execution time than GA). When the number of iterations increases, both the objective value of the proposed heuristic GA-VNFPRA and the Gurobi are gradually decreased and eventually tend to converge. Since we are solving a minimisation problem, lower values for a specific placement and resource allocation combination mean better performance. Our proposed GA-VNFPRA is shown to approximate well with the optimal solutions provided by the Gurobi. However, after 2500 iterations, the proposed GA-VNFPRA converges to a local optimum and no further improvements can be achieved while Gurobi converges after 4000 iterations, this is because it increases the particular solution computational time  for large scale experimentations and thus we can only use it as a benchmark approach or an offline solution.
In Figure 5, we evaluate the performance of the proposed scheme and four existing approaches to find VNF placement cost. It depicts the cost per VNF placement with increasing number of users. It shows that our proposed GA-VNFPRA strategy is beneficial in terms of VNF placement cost. The proposed algorithm finds the maximum number of VNFs that can be placed in MEC nodes and achieve the minimum cost. Here, we can also see how the cost of the random allocation spikes in comparison with the GA-VNFPRA and other baseline algorithms.
In Figure 6, we applied the proposed algorithm to find the minimum computation cost. We can see that our proposed heuristic approach has an adverse effect in terms of computation cost and it performs better than RU-SOEA and other baseline approaches. The GA-VNFPRA performs well as number of users grows by allocating optimal computing resources to VNFs and link bandwidth allocation at much lower cost. This is because in GA-VNFPRA, we provide a heuristic approach  to organise the distribution of VNFs and to find suitable MEC nodes with maximum resources to run on top of it with the cheapest cost. If the MEC node has enough resources to host the VNFs, then the program automatically search to another valid and available MEC node to run these VNFs in it. Figure 7 demonstrates that our approach for VNFPRA consist of finding the most appropriate MEC nodes, determine the shortest paths to both the selected content nodes and the enduser with the aim of minimising link usage cost. As our proposed framework has more than one path between most of the (source, destination) node pairs, and these paths mostly use different links, the GA-VNFPRA tries to maximise the number of used links, so that the system is encouraged to use different paths to route traffic between each (source, destination) node pair. This results in distributing traffic over different paths, and reduces the average link utilisation and minimises the link usage cost. It can be seen that with the increasing number of users our proposed GA-VNFPRA gives better results compare to baseline approaches.
In Figure 8, we applied the proposed algorithm to evaluate the objective function and overall cost. In the figure, the overall cost increases with increasing number of users. RFPA performs worse than GA-VNFPRA and RU-SOEA, because there is a greater probability for RFPA to choose nodes with less remaining resources to serve VNFs. Since the GA-VNFPRA considers to reduce the resource consumption and adopt an adjustment process to cope with constraint violations, the advantage of GA-VNFPRA becomes more and more obvious with the increasing number of users.
In Figure 9, we evaluate the total number of MEC nodes required with increasing number of requests from users. The number of MEC nodes needed to facilitate the incoming requests increase rapidly for GA-NBA, RFPA and FFPA while GA-VNFPRA algorithm requires minimum MEC nodes to process requests and also it approximates well with the optimal solution. This is because GA-VNFPRA takes account of the resources of MEC nodes, load of the server, and the load of the links during the selection of VNFs to be placed. RU-SOEA also effectively distributes the incoming requests among available MEC nodes. RFPA doesn't consider the best MEC nodes with efficient available resources, which means the allocation of CPU and memory resource is not optimal. When there are too many VNF requests from users, the optimal allocation of resources in MEC nodes for VNFs will become more important.
In Figure 10, we study the performance of VNF migration for increasing number of requests using our proposed GA-VNFM. GA-VNFM requires minimum VNF migrations while in baseline approaches, the numbers of VNF migrations rapidly increase with the number of requests from users because the baseline algorithms do not consider the load of target MEC node. Compared to these approaches the number of VNF migration in our proposed GA-VNFM steadily goes to a stable state. This is because it always selects the appropriate MEC node for each VNF.
In Figure 11, we measure the average execution time overhead added by the additional VNFs in MEC nodes. We ran a series of experiments to measure how the additional deployment of VNFs affect the CPU, memory, and network  performance. GA-VNFPRA algorithm can decide MEC node and network allocations for hundreds of VNFs based on the CPU core cost, bandwidth cost and the remaining resource of MEC nodes. Nodes that incur less cost and have more remaining resources are more possible to be chosen. Results show that time needed to solve the VNFPRA problem increases dramatically for GA-NBA when dealing with large number of VNFs. On the other hand, GA-VNFPRA shows better scalability and seems more convenient for real network scenarios since it provides a near optimal solution within a reasonable time. The initial solution provided by genetic approach can impact the overall cost. The GA-VNFPRA solution provides relatively smaller execution-time as compare to the baseline methods.

CONCLUSION
A framework based on SDN and NFV MANO architecture is presented for deploying VNFs in distributed NFV-enabled MEC nodes. Furthermore, a VNFPRA problem is formulated that requires meeting the optimal placement requirements of VNFs in SDN/NFV-enabled MEC nodes to reduce the deployment and resource cost. This paper proposes a solution for this problem based on two algorithms (i) an optimal solution formulated as a MIP problem and (ii) a genetic based heuristic algorithm. We also proposed a GA-VNFM algorithm to minimise the total number of VNF migrations. Results show the higher performance of the proposed methods in comparison with four existing algorithms in literature. The results also validate the fact that a coordinated placement of VNFs in SDN, NFV, and MEC can satisfy the objectives of overall reduced cost.