What is the role of GFS master during read and write operations?

[shivajikobardan]

I am really confused by this question as there are multiple possible answers of this question, so I am asking this.
There is very minimum role of GFS in writing and reading operations..

You can either go through these or just tell me what is the correct answer if you are bored to go through all of these.


https://static.googleusercontent.com/media/research.google.com/en//archive/gfs-sosp2003.pdf

1) My try-: https://we.tl/t-DG1QxAFMFJ
2) https://www.uio.no/studier/emner/matnat/ifi/INF5100/h10/undervisningsmateriale/gfs.pdf
https://imgur.com/a/NPLwQEL
3) https://ameyashanbhag.gitbooks.io/p...lytics/content/hadoop/google-file-system.html
read and write operations on GFS
4)https://courses.cs.washington.edu/courses/cse550/14au/notes/lect13.pdf
https://imgur.com/a/XjwWzjR
5) https://imgur.com/a/oipZ1Sr
6)https://imgur.com/a/uru4FkI
https://imgur.com/a/Q74IH7b
http://wwwusers.di.uniroma1.it/~ste...s_18-19/Schedule_files/Google File System.pdf
7) https://cs.stanford.edu/~matei/courses/2015/6.S897/slides/gfs.pdf slide 17
8)https://imgur.com/a/eZKHCRd
https://wiki.ubc.ca/images/8/81/GFS.pdf
9)https://imgur.com/a/GM3ZUpX
10) https://imgur.com/a/BOOGoSa

 

[MathLover1]

here I highlighted what I believe the role is:

The architecture of a GFS cluster; the master maintains state information about all system components. The master controls a number of chunk servers. A chunk server runs under Linux and uses metadata provided by the master to communicate directly with an application. The data flow is decoupled from the control flow.

What is GFS in cloud computing?
Image result for What is the role of GFS master during read and write operations?
Google File System (GFS or GoogleFS, not to be confused with the GFS Linux file system) is a proprietary distributed file system developed by Google to provide efficient, reliable access to data using large clusters of commodity hardware.

The GFS node cluster is a single master with multiple chunk servers that are continuously accessed by different client systems. ... Stored data is divided into large chunks (64 MB), which are replicated in the network a minimum of three times. The large chunk size reduces network overhead.

The Google file system (GFS) is a distributed file system (DFS) for data-centric applications with robustness, scalability, and reliability [8]. GFS can be implemented in commodity servers to support large-scale file applications with high performance and high reliability.

In fact, for a better performance, GFS decouple the data flow from the control flow in that each server forwards the data to its “ closest ” machine in the network topology that has not got it, by pipelining over TCP connections at the rate of 80 ms per 1 MB.

What happens if a client fails in GFS?
Back to your question, how GFS deal with replica write failure? If any error is encounter during replication, the failure of mutation is reported to the client. The client must handle the error and retry the mutation. The inconsistent chunks will be garbage collected during regular scan in chunk servers
What is chunk handle?
terminology. • chunk—fixed-size piece of file. • chunk server—holds chunks. • master—coordinates chunk servers. • chunk handle—ID of a chunk (64 bit, globally unique)
Can clients read stale data in GFS?
Stale replicas are not given to clients when they ask the master for the location of a chunk, and they are not involved in mutations either. However, because a client caches the location of a chunk, it may read from a stale replica before the information is refreshed.

 

[shivajikobardan]

here I highlighted what I believe the role is:

The architecture of a GFS cluster; the master maintains state information about all system components. The master controls a number of chunk servers. A chunk server runs under Linux and uses metadata provided by the master to communicate directly with an application. The data flow is decoupled from the control flow.

What is GFS in cloud computing?
Image result for What is the role of GFS master during read and write operations?
Google File System (GFS or GoogleFS, not to be confused with the GFS Linux file system) is a proprietary distributed file system developed by Google to provide efficient, reliable access to data using large clusters of commodity hardware.

The GFS node cluster is a single master with multiple chunk servers that are continuously accessed by different client systems. ... Stored data is divided into large chunks (64 MB), which are replicated in the network a minimum of three times. The large chunk size reduces network overhead.

The Google file system (GFS) is a distributed file system (DFS) for data-centric applications with robustness, scalability, and reliability [8]. GFS can be implemented in commodity servers to support large-scale file applications with high performance and high reliability.

In fact, for a better performance, GFS decouple the data flow from the control flow in that each server forwards the data to its “ closest ” machine in the network topology that has not got it, by pipelining over TCP connections at the rate of 80 ms per 1 MB.

What happens if a client fails in GFS?
Back to your question, how GFS deal with replica write failure? If any error is encounter during replication, the failure of mutation is reported to the client. The client must handle the error and retry the mutation. The inconsistent chunks will be garbage collected during regular scan in chunk servers
What is chunk handle?
terminology. • chunk—fixed-size piece of file. • chunk server—holds chunks. • master—coordinates chunk servers. • chunk handle—ID of a chunk (64 bit, globally unique)
Can clients read stale data in GFS?
Stale replicas are not given to clients when they ask the master for the location of a chunk, and they are not involved in mutations either. However, because a client caches the location of a chunk, it may read from a stale replica before the information is refreshed.

Thanks lot for response and I got some perspective. But I am still confused what should I write in role of GFS master during read and write operations

 

[MathLover1]

Thanks lot for response and I got some perspective. But I am still confused what should I write in role of GFS master during read and write operations

this might be helpful

The Master Node

●Responsible for all system-wide activities
○ managing chunk leases, reclaiming storage space, load-balancing
● Maintains all file system metadata
○ Namespaces, ACLs, mappings from files to chunks, and current locations of chunks
○ all kept in memory, namespaces and file-to-chunk mappings are also stored persistently in
operation log
● Periodically communicates with each chunkserver in HeartBeat messages
○ This let’s master determines chunk locations and assesses state of the overall system
○ Important: The chunkserver has the final word over what chunks it does or does not have on
its own disks – not the master
●For the namespace metadata, master does not use any per-directory data
structures – no inodes! (No symlinks or hard links, either.)
○ Every file and directory is represented as a node in a lookup table, mapping pathnames to
metadata. Stored efficiently using prefix compression (< 64 bytes per namespace entry)
● Each node in the namespace tree has a corresponding read-write lock to
manage concurrency
○ Because all metadata is stored in memory, the master can efficiently scan the entire state
of the system periodically in the background
○ Master’s memory capacity does not limit the size of the system