Date of Award
8-2009
Document Type
Thesis
Degree Name
Master of Science (MS)
Legacy Department
Computer Engineering
Committee Chair/Advisor
Ligon, Walter B
Committee Member
Hoover , Adam W
Committee Member
Brooks , Richard R
Abstract
One of the challenges that the design of parallel file system for
HPC(High Performance Computing) has to face today is maintaining the
scalability to handle the I/O generated by parallel applications that
involve accessing directories containing a large number of entries and
performing hundreds of thousands of operations per second. Currently,
highly concurrent access to large directories is poorly supported in
parallel file systems. As a result, it is important to build a
scalable directory service for parallel file systems to support
efficient concurrent access to larger directories.
In this thesis we demonstrate a scalable directory service designed
for parallel file systems(specifically for PVFS) that can achieve high
throughput and scalability while minimizing bottlenecks and
synchronization overheads. We describe important concepts and goals in
scalable directory service design and its implementation in the
parallel file system simulator--HECIOS. We also explore the simulation
model of MPI programs and the PVFS file system in HECIOS, including
the method to verify and validate it. Finally, we test our scalable
directory service on HECIOS and analyze the performance and
scalability based on the results.
In summary, we demonstrate that our scalable directory service can
effectively handle highly concurrent access to large directories in
parallel file systems. We are also able to show that our scalable
directory service scales well with the number of I/O nodes in the
cluster.
Recommended Citation
Wu, Yang, "A Study for Scalable Directory in Parallel File Systems" (2009). All Theses. 625.
https://open.clemson.edu/all_theses/625