文档介绍：The Google File System (GFS)
Introduction
Design ponent failures are the norm
1000s ponents
Bugs, human errors, failures of memory, disk, connectors, networking, and power supplies
Monitoring, error detection, fault tolerance, automatic recovery监控,错误检测,容错,自动恢复
Files are huge by traditional standards
Multi-GB files mon
Billions of objects
Introduction
Design constraints
Most modifications are appends
Random writes are practically nonexistent
Many files are written once, and read sequentially
Two types of reads
Large streaming reads
Small random reads (in the forward direction)
Sustained bandwidth more important than latency
高度可用的带宽比低延迟更加重要
File system APIs are open to changes
Interface Design
Not pliant 不遵守POSIX规范
Additional operations
Snapshot
Record append
Architectural Design
A GFS cluster
A single master 在逻辑上只有一个master
Multiple chunkservers per master
Accessed by multiple clients
Running modity Linux machines
A file 数据以文件形式存储在Chunk Server上
Represented as fixed-sized chunks(数据块)
Labeled with 64-bit unique global IDs
Stored at chunkservers
3-way Mirrored across chunkservers
GFS chunkserver
Linux file system
Architectural Design (2)
GFS Master
GFS chunkserver
Linux file system
GFS chunkserver
Linux file system
Application
GFS client
chunk location?
chunk data?
Architectural Design (3)
Master server
Maintains all metadata
Name space, access control, file-to-chunk mappings, garbage collection, chunk migration
GFS clients
Consult master for metadata
Access data from chunkservers
Does not go through VFS
No caching at clients and chunkservers due to the frequent case of streaming
客户端大部分是流式顺序读写,并不存在大量的重复读写,缓存文件数据对提高系统整体性能的作用不大,服务器不缓存是因为Linux操作系统会把经常访问的数据存放在内存中
Single-Master Design
Simple
Master answers only chunk locations
A client typically asks for multiple chunk locations in a single request
The master also predicatively provide chunk locations immediately following those requested
Chunk Size
64 MB
Fewer chunk location requests to the master