Changes between Version 1 and Version 2 of VolunteerDataArchival


Ignore:
Timestamp:
Nov 23, 2011, 9:27:08 AM (13 years ago)
Author:
davea
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • VolunteerDataArchival

    v1 v2  
    3131   We don't consider direct client-to-client communication.
    3232
     33Recovering from the failure of a host, using techniques like replication,
     34involves uploading data from a 2nd host, then downloading it to a 3rd host.
     35Each of these steps may take days.
     36This, for volunteer storage the ratio
     37
     38 average time to failure / average time to recover
     39
     40may be fairly small (like 100).
     41In other distributed storage systems (such as RAIDs) this ratio may
     42be on the order of 100,000.
     43Thus, these systems can modeled as a sequence of individual
     44failures and recoveries.
     45
     46Volunteer storage, on the other hand, must be modeled as process
     47in which multiple recoveries may be in progress at the same time,
     48and new failures may occur during these recoveries.
    3349There are two basic techniques for achieving reliable storage using
    3450unreliable resources:
    3551
    36  * '''Replication''': a file
     52'''Replication''': a file is divided into N pieces,
     53and each piece is stored on M hosts.
     54If a replica is lost, and there another replica,
     55that replica is uploaded to the server, then downloaded to another host.
    3756
    38  * '''Coding''': with Reed-Solomon coding, a file is divided into N 'packets',
    39   and an additional K checksum packets are generated.
    40   The original data can be reconstructed from any N of these N+K packets.
     57
     58
     59'''Coding''': with Reed-Solomon coding, a file is divided into N 'packets',
     60and an additional K checksum packets are generated.
     61The original data can be reconstructed from any N of these N+K packets.
     62
     63In