PAGE: A Partition Aware Engine for Parallel Graph Computation
Abstract:
Graph partition quality affects the overall performance of
parallel graph computation systems. The quality of a graph partition is
measured by the balance factor and edge cut ratio. A balanced graph partition
with small edge cut ratio is generally preferred since it reduces the expensive
network communication cost. However, according to an empirical study on Giraph,
the performance over well partitioned graph might be even two times worse than
simple random partitions. This is because these systems only optimize for the
simple partition strategies and cannot efficiently handle the increasing
workload of local message processing when a high quality graph partition is
used. In this paper, we propose a novel partition aware graph computation
engine named PAGE, which equips a new message processor and a dynamic
concurrency control model. The new message processor concurrently processes
local and remote messages in a unified way. The dynamic model adaptively
adjusts the concurrency of the processor based on the online statistics. The
experimental evaluation demonstrates the superiority of PAGE over the graph
partitions with various qualities.
Algorithm:
Ø
Encryption.
Ø
Upload Algorithm.
Text File upload.
Ø
File Split Techniques.
Split File.
Ø
Graph partition, Edge Cut Ratio.
Graph computation.
Key points:
1.
File Uploading.
2.
Encrypt key.
3.
File split.
EXISTING SYSTEM
A good balanced graph partition even leads to a decrease of the
overall performance in existing systems. The Page Rank algorithm on six
different partition schemes of a large web graph dataset, and apparently the
overall cost of Page Rank per iteration increases with the quality improvement
of different graph partitions. Lots of existing parallel graph systems are
unaware of such effect of the underlying partitioned sub graphs, and ignore the
increasing workload of local message processing when the quality of partition
scheme is improved. Therefore, these systems handle the local messages and
remote messages unequally and only optimize the processing of remote messages.
The existing graph systems still cannot effectively utilize the benefit of high
quality graph partitions.
PROPOSED SYSTEM
Since the message processing pipeline satisfied the producerconsumer
model, several heuristic rules are proposed by considering the
producer-consumer constraints. To balance the workload, the proposed strategies
repartition the graph according to the online workload. Thus the quality of
underlying graph partition changes along with repartitioning. To address this
problem, we proposed a partition aware graph computation engine named PAGE that
monitors three high-level key running metrics and dynamically adjusts the
system configurations. In the adjusting model, we elaborated two heuristic
rules to effectively extract the system characters and generate proper
parameters.
Advantage
v Easy to Upload files
v Provide files.
System architecture
MODULE DESCRIPTION
MODULE
Case Study and Data
Collection
v User
v Admin Authentication
MODULE DESCRIPTION
Case Study and Data Collection
We consider a case study of a web-based collaboration application
for evaluating performance. The application allows users to store, manage, and
share documents and drawings related to large construction projects. The
service composition required for this application includes: Firewall (x1),
Intrusion Detection (x1), Load Balancer (x1), Web Server (x4), Application
Server (x3), Database Server (x1), Database Reporting Server (x1), Email Server
(x1), and Server Health Monitoring (x1). To meet these requirements, our
objective is to find the best Cloud service composition
USER
A balanced graph partition with small edge cut ratio is generally
preferred since it reduces the expensive network communication cost. However,
according to an empirical study on Graph, the performance over well partitioned
graph might be even two times worse than simple random partitions. A good
balanced partition (or high quality partition) usually has a small edge cut and
helps improve the performance of systems. Because the small edge cut reduces
the expensive communication cost between different sub graphs, and the balance
property generally guarantees that each sub graph has similar computation
workload.
Upload File
The user can upload the file to DB. And
the Admin can allow the data to store the DB.
Split File
The user can upload the file to DB . The
files are split and store the database.
Edge Cut Ratio
A balanced graph partition with small edge cut ratio is generally
preferred since it reduces the expensive network communication cost.
Admin Authentication
Prominent examples include web graphs, social networks and other
interactive networks in bioinformatics. The up to date web graph contains
billions of nodes and trillions of edges. Graph structure can represent various
relationships between objects, and better models complex data scenarios. The
method consists of: reliability estimation, weak point recommendation and weak
point strengthening steps, as defined by the overview. In the rest of this
section, we briefly describe each of the stated steps.
Accept user
The admin can accept the new user request and also black the
users.
Allow user file
The users can upload the file to cloud.
And the admin can allow the files to cloud then only the file can store the
cloud.
No comments:
Post a Comment