From FIL, Arweave to Walrus, Shelby: How far is the road to the popularization of Decentralization storage?

Original author: @BlazingKevin_, the Researcher at Movemaker

Storage has long been one of the top narratives in the industry. As the leading project in the previous bull market, Filecoin's market cap once exceeded 10 billion USD. Arweave, as a comparable storage protocol, positioned itself with permanent storage as its selling point, reaching a market cap of up to 3.5 billion USD. However, as the availability of cold data storage has been debunked, the necessity of permanent storage is now questioned, leaving a big question mark over whether the decentralized storage narrative can hold up. The emergence of Walrus has reignited the long-dormant storage narrative, and now Aptos, in collaboration with Jump Crypto, has launched Shelby, aiming to elevate decentralized storage in the hot data sector to a new level. So, can decentralized storage make a comeback and provide widespread use cases? Or is it just another topic hype? This article analyzes the evolution of the decentralized storage narrative through the developmental trajectories of Filecoin, Arweave, Walrus, and Shelby, attempting to seek an answer to this question: How far is the path to the popularization of decentralized storage?

Filecoin: Storage is the surface, mining is the essence

Filecoin is one of the earliest emerging altcoins, and its development direction naturally revolves around decentralization, which is a common characteristic of early altcoins - that is, finding the significance of decentralization in various traditional sectors. Filecoin is no exception; it links storage with decentralization, naturally leading to the disadvantages of centralized storage: the trust assumption on centralized data storage service providers. Therefore, what Filecoin does is shift from centralized storage to decentralized storage. However, certain aspects sacrificed in the process of achieving decentralization became the pain points envisioned to be solved by later projects like Arweave or Walrus. To understand why Filecoin is merely a mining coin, one needs to understand the objective limitations of its underlying technology, IPFS, which is not suitable for hot data.

IPFS: Decentralized Architecture, Yet Stalled at Transmission Bottlenecks

IPFS (InterPlanetary File System) was launched around 2015, aiming to disrupt the traditional HTTP protocol through content addressing. The biggest drawback of IPFS is its extremely slow retrieval speed. In an era where traditional data service providers can achieve millisecond-level responses, retrieving a file from IPFS still takes several seconds, making it difficult to promote in practical applications, which also explains why it is rarely adopted by traditional industries, except for a few blockchain projects.

The underlying P2P protocol of IPFS is mainly suitable for "cold data," which refers to static content that does not change often, such as videos, images, and documents. However, when it comes to handling hot data, such as dynamic web pages, online games, or artificial intelligence applications, the P2P protocol does not have a significant advantage over traditional CDNs.

However, although IPFS itself is not a blockchain, its directed acyclic graph (DAG) design philosophy aligns closely with many public chains and Web3 protocols, making it inherently suitable as a foundational building framework for blockchains. Therefore, even if it lacks practical value, it is sufficient as a foundational framework that supports blockchain narratives. Early altcoin projects only needed a functional framework to embark on their journey, but as Filecoin developed over time, the inherent shortcomings brought by IPFS began to hinder its progress.

The Logic of Mining Coins Under Storage Cloaks

The original intention of IPFS is to allow users to store data while also being a part of the storage network. However, without economic incentives, it is difficult for users to voluntarily use this system, let alone become active storage nodes. This means that most users will only store files on IPFS but will not contribute their own storage space or store others' files. It is against this backdrop that Filecoin was born.

In the token economic model of Filecoin, there are mainly three roles: users are responsible for paying fees to store data; storage miners receive token incentives for storing user data; and retrieval miners provide data when users need it and receive incentives.

This model has potential malicious space. Storage miners may fill junk data after providing storage space to obtain rewards. Since this junk data will not be retrieved, even if it is lost, it will not trigger the penalty mechanism for storage miners. This allows storage miners to delete junk data and repeat this process. Filecoin's proof of replication consensus can only ensure that user data has not been privately deleted, but it cannot prevent miners from filling junk data.

The operation of Filecoin largely depends on miners' continuous investment in the token economy, rather than on actual demand from end users for distributed storage. Although the project is still undergoing iterations, at this stage, the ecosystem of Filecoin aligns more with the definition of a "mining coin logic" rather than an "application-driven" storage project.

Arweave: Succeeded by Long-Termism, Failed by Long-Termism

If Filecoin's design goal is to build an incentivized, verifiable decentralized "data cloud" shell, then Arweave takes a different extreme direction in storage: providing the capability for permanent data storage. Arweave does not attempt to build a distributed computing platform; its entire system revolves around a core assumption—important data should be stored once and remain permanently on the network. This extreme long-termism makes Arweave fundamentally different from Filecoin in terms of mechanisms, incentive models, hardware requirements, and narrative perspectives.

Arweave uses Bitcoin as a learning object, attempting to continuously optimize its permanent storage network over a long cycle measured in years. Arweave does not care about marketing, nor does it care about competitors and market trends. It is simply moving forward on the path of iterating its network architecture, even if it goes unnoticed, because this is the essence of the Arweave development team: long-termism. Thanks to long-termism, Arweave was highly sought after during the last bull market; and because of long-termism, even if it falls to the bottom, Arweave may still survive several rounds of bull and bear markets. The only question is whether there will be a place for Arweave in the future of decentralized storage. The intrinsic value of permanent storage can only be proven through time.

Since version 1.5 of the Arweave mainnet to the recent version 2.9, although it has lost market discussions, it has been committed to enabling a wider range of miners to participate in the network at minimal cost and incentivizing miners to maximize data storage, continuously enhancing the robustness of the entire network. Arweave has taken a conservative approach, fully aware that it does not align with market preferences, does not embrace the miner community, and the ecosystem is completely stagnant. It upgrades the mainnet at minimal cost while continuously lowering the hardware threshold without compromising network security.

A Review of the Upgrade Journey from 1.5 to 2.9

The Arweave 1.5 version exposed a vulnerability that allowed miners to rely on GPU stacking rather than actual storage to optimize block production chances. To curb this trend, version 1.7 introduced the RandomX algorithm, limiting the use of specialized computing power and instead requiring general-purpose CPUs to participate in mining, thereby weakening the centralization of computing power.

In version 2.0, Arweave adopts SPoA, transforming data proofs into a concise path structured as a Merkle tree, and introduces format 2 transactions to reduce synchronization burdens. This architecture alleviates network bandwidth pressure, significantly enhancing the collaborative capabilities of nodes. However, some miners can still evade the responsibility of holding real data through centralized high-speed storage pool strategies.

To correct this bias, 2.4 introduced the SPoRA mechanism, which incorporates global indexing and slow hash random access, requiring miners to genuinely hold data blocks to participate in effective block production, thereby weakening the effects of power stacking from a mechanistic perspective. As a result, miners began to focus on storage access speed, promoting the application of SSDs and high-speed read-write devices. 2.6 introduced hash chain control over block production rhythm, balancing the marginal benefits of high-performance devices and providing fair participation space for small and medium-sized miners.

Subsequent versions further enhance network collaboration capabilities and storage diversity: 2.7 introduces collaborative mining and pool mechanisms to improve the competitiveness of small miners; 2.8 launches a composite packaging mechanism that allows large capacity low-speed devices to participate flexibly; 2.9 introduces a new packaging process in the format of replica_ 2 _ 9, significantly improving efficiency and reducing computational dependency, completing the closed loop of data-driven mining models.

Overall, Arweave's upgrade path clearly presents its long-term strategy oriented towards storage: while continually resisting the trend of computing power centralization, it consistently lowers the barriers to participation, ensuring the possibility of the protocol's long-term operation.

Walrus: Is Embracing Hot Data Hype or Hidden Potential?

Walrus, in terms of design philosophy, is completely different from Filecoin and Arweave. The starting point of Filecoin is to create a decentralized and verifiable storage system, at the cost of cold data storage; the starting point of Arweave is to build an on-chain library of Alexandria that can permanently store data, at the cost of too few scenarios; the starting point of Walrus is to optimize the storage overhead of hot data storage protocols.

Magic modification of error correction codes: cost innovation or new wine in an old bottle?

In terms of storage cost design, Walrus believes that the storage expenses of Filecoin and Arweave are unreasonable, as both adopt a fully replicated architecture. Their main advantage lies in the fact that each node holds a complete copy, which provides strong fault tolerance and independence between nodes. This type of architecture ensures that even if some nodes go offline, the network still maintains data availability. However, this also means that the system requires multiple copies for redundancy to maintain robustness, thereby increasing storage costs. Particularly in Arweave's design, the consensus mechanism itself encourages node redundancy storage to enhance data security. In contrast, Filecoin is more flexible in cost control, but the downside is that some low-cost storage may carry a higher risk of data loss. Walrus attempts to find a balance between the two, controlling replication costs while enhancing availability through structured redundancy, thus establishing a new compromise path between data availability and cost efficiency.

The Redstuff created by Walrus is a key technology for reducing node redundancy, originating from Reed-Solomon (RS) coding. RS coding is a very traditional erasure coding algorithm, which is a technique that allows for the duplication of datasets by adding redundant fragments (erasure code) to reconstruct the original data. It is frequently used in everyday life, from CD-ROMs to satellite communications to QR codes.

Erasure coding allows users to take a block, for example, 1 MB in size, and then "expand" it to 2 MB in size, where the additional 1 MB is special data known as erasure coding. If any byte in the block is lost, users can easily recover those bytes through the code. Even if up to 1 MB of the block is lost, you can recover the entire block. The same technology allows computers to read all data from a CD-ROM, even if it has been damaged.

The most commonly used is RS coding. The implementation method is to start with k information blocks, construct the relevant polynomial, and evaluate it at different x-coordinates to obtain the coding blocks. Using RS erasure codes, the probability of randomly sampling large chunks of data loss is very low.

For example: Divide a file into 6 data blocks and 4 parity blocks, totaling 10 pieces. As long as any 6 of them are retained, the original data can be completely restored.

Advantages: Strong fault tolerance, widely used in CD/DVD, fault-tolerant disk arrays (RAID), and cloud storage systems (such as Azure Storage, Facebook F 4).

Disadvantages: Decoding calculations are complex and have high overhead; not suitable for frequently changing data scenarios. Therefore, it is usually used for data recovery and scheduling in off-chain centralized environments.

Under a decentralized architecture, Storj and Sia have adjusted traditional RS coding to meet the practical needs of distributed networks. Walrus has also proposed its own variant based on this — the RedStuff coding algorithm — to achieve lower costs and a more flexible redundancy storage mechanism.

What is the biggest feature of Redstuff? By improving the erasure coding algorithm, Walrus can quickly and robustly encode unstructured data blocks into smaller fragments, which are distributed and stored in a storage node network. Even if up to two-thirds of the fragments are lost, the original data block can be quickly reconstructed using partial fragments. This is possible while maintaining a replication factor of only 4 to 5 times.

Therefore, it is reasonable to define Walrus as a lightweight redundancy and recovery protocol redesigned around a decentralized scenario. Compared to traditional erasure codes (such as Reed-Solomon), RedStuff no longer pursues strict mathematical consistency, but instead makes realistic trade-offs regarding data distribution, storage verification, and computational costs. This model abandons the immediate decoding mechanism required for centralized scheduling and instead adapts to a more dynamic, marginalized network structure by verifying through on-chain Proof whether nodes hold specific data copies.

The core design of RedStuff is to split data into two categories: primary slices and secondary slices. Primary slices are used to recover the original data, and their generation and distribution are subject to strict constraints, with a recovery threshold of f+1, requiring 2f+1 signatures as a validity endorsement. Secondary slices, on the other hand, are generated through simple operations such as XOR combinations, serving the purpose of providing elastic fault tolerance and enhancing the overall system robustness. This structure essentially reduces the requirement for data consistency—allowing different nodes to temporarily store different versions of data, emphasizing a practical path towards "eventual consistency." Although similar to the lenient requirements for retroactive blocks in systems like Arweave, achieving some effect in reducing network burden, it simultaneously weakens the guarantees of data availability and integrity.

It is important to note that while RedStuff has achieved effective storage in low-computational and low-bandwidth environments, it essentially remains a "variant" of erasure coding systems. It sacrifices a certain degree of data read determinism in exchange for cost control and scalability in a decentralized environment. However, it remains to be seen whether this architecture can support large-scale, high-frequency interactive data scenarios at the application level. Furthermore, RedStuff has not truly broken through the long-standing coding computation bottleneck of erasure coding; rather, it has avoided the high coupling points of traditional architectures through structural strategies. Its innovation is more reflected in engineering-side combinatorial optimization, rather than a fundamental disruption at the algorithmic level.

Therefore, RedStuff is more like a "reasonable modification" made for the current decentralized storage reality. It indeed brings improvements in redundancy costs and operational loads, allowing edge devices and non-high-performance nodes to participate in data storage tasks. However, in scenarios with large-scale applications, general computing adaptation, and higher consistency requirements, its capability boundaries are still quite apparent. This makes Walrus's innovation more like an adaptive transformation of the existing technological system, rather than a decisive breakthrough in promoting the migration of the decentralized storage paradigm.

Sui and Walrus: Can High-Performance Public Chains Drive the Practicality of Storage?

The official research article from Walrus shows its target scenario: "The original intention of Walrus's design is to provide a solution for storing large binary files (Blobs), which are the lifeblood of many decentralized applications."

The so-called large blob data usually refers to binary objects that are large in size and have an irregular structure, such as videos, audio, images, model files, or software packages.

In the context of cryptocurrency, it refers more to images and videos in NFTs and social media content. This also constitutes the main application direction of Walrus.

• Although the article also mentions the potential uses of AI model dataset storage and Data Availability Layer (DA), the phase-out of Web3 AI has left very few relevant projects, and the number of protocols that will truly adopt Walrus in the future may be quite limited.
• In terms of the DA layer, whether Walrus can serve as an effective alternative remains to be seen, and its feasibility can only be verified after mainstream projects like Celestia rekindle market interest.

Therefore, the core positioning of Walrus can be understood as a hot storage system for serving content assets such as NFTs, emphasizing dynamic invocation, real-time updates, and version management capabilities.

This also explains why Walrus needs to rely on Sui: with the high-performance chain capabilities of Sui, Walrus is able to build a high-speed data retrieval network, significantly reducing operational costs without developing its own high-performance public chain, thus avoiding direct competition with traditional cloud storage services in terms of unit costs.

According to official data, the storage cost of Walrus is approximately one-fifth that of traditional cloud services. Although it appears to be tens of times more expensive compared to Filecoin and Arweave, its goal is not to pursue extremely low costs but to build a decentralized hot storage system that can be used in real business scenarios. Walrus itself operates as a PoS network, with the core responsibility of verifying the honesty of storage nodes and providing the most basic security guarantees for the entire system.

As for whether Sui truly needs Walrus, the discussion currently remains more at the level of ecological narrative. If the primary use case is financial settlement, Sui does not urgently require off-chain storage support. However, if it hopes to support more complex on-chain scenarios in the future such as AI applications, content assetization, and composable agents, then the storage layer will be indispensable in providing context, context, and indexing capabilities. High-performance chains can handle complex state models, but these states need to be bound to verifiable data in order to build a trustworthy content network.

Shelby: Dedicated Fiber Network Fully Unlocks Web3 Application Scenarios

Among the biggest technical bottlenecks faced by current Web3 applications, "reading performance" has always been a difficult shortcoming to overcome.

Whether it's video streaming, RAG systems, real-time collaboration tools, or AI model inference engines, they all rely on low-latency and high-throughput access to hot data. Decentralized storage protocols (from Arweave and Filecoin to Walrus) have made progress in data persistence and trustlessness, but because they operate on the public internet, they are always constrained by high latency, unstable bandwidth, and uncontrollable data scheduling.

Shelby attempted to address the issue at its root.

Firstly, the Paid Reads mechanism directly reshapes the "read operation" dilemma in decentralized storage. In traditional systems, reading data is almost free, and the lack of effective incentive mechanisms leads to service nodes generally being lazy in responding and cutting corners, resulting in a user experience that is far behind Web2.

Shelby links user experience directly to service node revenue by introducing a pay-per-read model: the faster and more reliably a node returns data, the more rewards it can earn.

This model is not an "incidental economic design," but rather the core logic of Shelby performance design — without incentives, there is no reliable performance; with incentives, there is sustainable improvement in service quality.

Secondly, one of the biggest technological breakthroughs proposed by Shelby is the introduction of the Dedicated Fiber Network (, which is equivalent to building a high-speed rail network for the instant reading of hot data in Web3.

This architecture completely bypasses the public transport layer that Web3 systems generally rely on, directly deploying storage nodes and RPC nodes on a high-performance, low-congestion, physically isolated transport backbone. This not only significantly reduces the latency of inter-node communication but also ensures the predictability and stability of the transmission bandwidth. The underlying network structure of Shelby is closer to the dedicated line deployment model between AWS internal data centers, rather than the "upload to a miner node" logic of other Web3 protocols.

![From Filecoin, Arweave to Walrus, Shelby: How far is the popularization of decentralized storage?])https://img-cdn.gateio.im/webp-social/moments-3ada47b20ce02ea4adf3262c874011bf.webp(

Source: Shelby White Paper

This network-level architectural inversion makes Shelby the first truly capable decentralized hot storage protocol that can deliver a Web2-level user experience. Users can read a 4K video, call the embedding data of a large language model, or trace back a transaction log on Shelby without having to endure the second-level latency that cold data systems typically suffer from, but instead can achieve sub-second response times. For service nodes, the dedicated network not only improves service efficiency but also significantly reduces bandwidth costs, making the "pay-per-read" mechanism genuinely economically viable, thus incentivizing the system to evolve towards higher performance rather than higher storage capacity.

It can be said that the introduction of dedicated fiber optic networks is the key support for Shelby to "look like AWS, but at its core is Web3." It not only breaks the natural opposition between decentralization and performance but also opens up the possibility for Web3 applications to be truly implemented in areas such as high-frequency reading, high-bandwidth scheduling, and low-cost edge access.

In addition, between data persistence and cost, Shelby adopts the Efficient Coding Scheme built by Clay Codes, achieving storage redundancy as low as <2 x through the optimal coding structure of MSR and MDS in mathematics, while still maintaining a persistence of 11 nines and 99.9% availability. While most Web3 storage protocols are still stuck at redundancy rates of 5 x to 15 x today, Shelby is not only more efficient technically but also more competitive in cost. This also means that for dApp developers who truly value cost optimization and resource scheduling, Shelby provides a realistic option that is "both cheap and fast."

Summary

Looking at the evolution route from Filecoin, Arweave, Walrus to Shelby, we can clearly see: the narrative of decentralized storage has gradually shifted from a technological utopia of "existence is justification" to a realistic approach of "usability is justice." Early Filecoin drove hardware participation through economic incentives, but real user needs were long marginalized; Arweave chose extreme permanent storage, yet appeared increasingly isolated amidst a silent application ecosystem; Walrus attempted to find a new balance between cost and performance, but still left questions in the construction of landing scenarios and incentive mechanisms. It wasn't until Shelby appeared that decentralized storage first proposed a systematic response to "Web2-level usability"—from dedicated fiber optic networks at the transmission layer, to efficient erasure coding designs at the computation layer, and to the pay-per-use incentive mechanism, these capabilities, originally exclusive to centralized cloud platforms, began to be reconstructed in the Web3 world.

The emergence of Shelby does not mean the end of problems. It has not resolved all challenges: issues such as developer ecology, permission management, and terminal access still lie ahead. However, its significance lies in opening up a possible path of "performance without compromise" for the decentralized storage industry, breaking the binary paradox of "either censorship-resistant or user-friendly."

The path to the popularization of decentralized storage will ultimately not rely solely on conceptual hype or token speculation, but must move towards an application-driven stage that is "usable, integrable, and sustainable." In this stage, whoever can first solve the real pain points of users will reshape the narrative of the next round of infrastructure. From mining coin logic to usage logic, Shelby's breakthrough may signify the end of an era - and the beginning of another.

About Movemaker

Movemaker is the first official community organization authorized by the Aptos Foundation, jointly initiated by Ankaa and BlockBooster, focusing on promoting the construction and development of the Aptos ecosystem in Chinese-speaking regions. As the official representative of Aptos in the Chinese-speaking area, Movemaker is committed to building a diverse, open, and prosperous Aptos ecosystem by connecting developers, users, capital, and numerous ecological partners.

Disclaimer:

This article/blog is for reference only, representing the author's personal views and does not reflect the position of Movemaker. This article does not intend to provide: )i( investment advice or investment recommendations; )ii( offers or solicitations to buy, sell, or hold digital assets; or )iii( financial, accounting, legal, or tax advice. Holding digital assets, including stablecoins and NFTs, carries a high level of risk, with significant price volatility, and they may even become worthless. You should carefully consider whether trading or holding digital assets is suitable for you based on your financial situation. If you have specific questions, please consult your legal, tax, or investment advisor. The information provided in this article (including market data and statistics, if any) is for general reference only. Reasonable care has been taken in compiling this data and charts, but no responsibility is accepted for any factual errors or omissions contained herein.