What is the role of GFS master during read and write operations?

Discussion in 'Other Advanced Math' started by shivajikobardan, Jan 10, 2022.

  1. shivajikobardan

    shivajikobardan

    Joined:
    Jan 8, 2022
    Messages:
    41
    Likes Received:
    6
  2. shivajikobardan

    MathLover1

    Joined:
    Jun 27, 2021
    Messages:
    2,989
    Likes Received:
    2,884
    here I highlighted what I believe the role is:

    The architecture of a GFS cluster; the master maintains state information about all system components. The master controls a number of chunk servers. A chunk server runs under Linux and uses metadata provided by the master to communicate directly with an application. The data flow is decoupled from the control flow.

    What is GFS in cloud computing?
    Image result for What is the role of GFS master during read and write operations?
    Google File System (GFS or GoogleFS, not to be confused with the GFS Linux file system) is a proprietary distributed file system developed by Google to provide efficient, reliable access to data using large clusters of commodity hardware.

    The GFS node cluster is a single master with multiple chunk servers that are continuously accessed by different client systems. ... Stored data is divided into large chunks (64 MB), which are replicated in the network a minimum of three times. The large chunk size reduces network overhead.

    The Google file system (GFS) is a distributed file system (DFS) for data-centric applications with robustness, scalability, and reliability [8]. GFS can be implemented in commodity servers to support large-scale file applications with high performance and high reliability.

    In fact, for a better performance, GFS decouple the data flow from the control flow in that each server forwards the data to its “ closest ” machine in the network topology that has not got it, by pipelining over TCP connections at the rate of 80 ms per 1 MB.

    What happens if a client fails in GFS?
    Back to your question, how GFS deal with replica write failure? If any error is encounter during replication, the failure of mutation is reported to the client. The client must handle the error and retry the mutation. The inconsistent chunks will be garbage collected during regular scan in chunk servers
    What is chunk handle?
    terminology. • chunk—fixed-size piece of file. • chunk server—holds chunks. • master—coordinates chunk servers. • chunk handle—ID of a chunk (64 bit, globally unique)
    Can clients read stale data in GFS?
    Stale replicas are not given to clients when they ask the master for the location of a chunk, and they are not involved in mutations either. However, because a client caches the location of a chunk, it may read from a stale replica before the information is refreshed.
     
    MathLover1, Jan 10, 2022
    #2
  3. shivajikobardan

    shivajikobardan

    Joined:
    Jan 8, 2022
    Messages:
    41
    Likes Received:
    6
    Thanks lot for response and I got some perspective. But I am still confused what should I write in role of GFS master during read and write operations
     
    shivajikobardan, Jan 12, 2022
    #3
  4. shivajikobardan

    MathLover1

    Joined:
    Jun 27, 2021
    Messages:
    2,989
    Likes Received:
    2,884
    this might be helpful

    The Master Node

    Responsible for all system-wide activities
    ○ managing chunk leases, reclaiming storage space, load-balancing
    Maintains all file system metadata
    ○ Namespaces, ACLs, mappings from files to chunks, and current locations of chunks
    ○ all kept in memory, namespaces and file-to-chunk mappings are also stored persistently in
    operation log
    Periodically communicates with each chunkserver in HeartBeat messages
    ○ This let’s master determines chunk locations and assesses state of the overall system
    Important: The chunkserver has the final word over what chunks it does or does not have on
    its own disks – not the master
    ●For the namespace metadata, master does not use any per-directory data
    structures – no inodes
    ! (No symlinks or hard links, either.)
    ○ Every file and directory is represented as a node in a lookup table, mapping pathnames to
    metadata. Stored efficiently using prefix compression (< 64 bytes per namespace entry)
    ● Each node in the namespace tree has a corresponding read-write lock to
    manage concurrency
    ○ Because all metadata is stored in memory, the master can efficiently scan the entire state
    of the system periodically in the background

    Master’s memory capacity does not limit the size of the system

     

    Attached Files:

    MathLover1, Jan 12, 2022
    #4
Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.
Similar Threads
There are no similar threads yet.
Loading...