Special thanks for @Olshansky helping me edit this piece
TL;DR
- A new proposal enables the Pocket Network to have custom minting rates on a per-chain basis.
- This proposal solves the cold start problem and incentivizes higher quality-of-service for users
- Other RPCs, such as LLM inference, can now be hosted on Pocket Network, opening up the total addressable market.
The Proposal
Pocket Network had one of its most impactful governance proposals passed recently by the c0d3r team.
It allows for unique minting on a per-chain basis. Until now, every chainid on Pocket has had the same mint rate. We have been limited to the cold start problem on a per-chain basis causing node runners to face one of two issues:
- Lack of traffic to generate profitability
- Earn rewards that are disproportionately expensive relative to the cost of supporting the underlying chain
Now that the network can customize mint rates for chains depending on the cost of physically (hardware and personnel) maintaining them, node runners are incentivized to support more expensive chains. Previously, if these chains had little traffic, there was little financial incentive in maintain them. In most cases, this resulted in one or two node providers monopolizing them on the network without the need to optimize for quality of service. Higher rewards will breed competition, leading to a higher quality-of-service, which will ultimately trickle down to benefit the end user.
This is critical for more expensive chains such as Solana that can require upwards of 512GB of RAM! For example, the network can tune rewards to be more specific for archival nodes as well as open the door for the network to feasibly host Solana archival without depending on 100’s of millions of requests to be profitable. As Pocket Network opens up the ability for new Gateways to join the network permissionlessly with the Shannon upgrade, the ability to configure unique rewards per chain creates an opportunity for new markets to emerge.
Expanding the Market
This proposal opens up other types of non-web3 native services for Pocket. Pocket can support any open source service or public database. Specifically, Pocket will now be able to support open source LLMs hosted by node runners. Hosting an open-source model such as Llama 2 is simple and inexpensive. Unlike a blockchain which is stateful and needs to continuously sync with the network, LLMs are stateless and only need periodic updates, done manually, when the model is updated. However, the compute costs of running inference on these models is much more expensive than an RPC that reads state from a blockchain. To do so quickly and with a high quality-of-service, node runners may require GPUs, which are very high in demand in today’s market.
Up until OpenAI’s release of ChatGPT, most costs associated with AI were put into model training. With the advent of sufficiently generalizable base models, like GPT4, new products can be built solely through contextual inference. Companies like OpenAI and Anthropic provide fine-tuning API’s, but empirically, it looks like the market is trending towards building products on top of contextual inference. When you use ChatGPT, you are inferring OpenAI’s trained model. While close source models, like GPT4, can’t be supported on the network, many open-source alternatives, such as Llama 2 can be set up by node runners with lots of experience in infrastructure. In the same way that chains require periodic updates, models will also be periodically updated without the overhead of needing to stay in sync all the time.
This proposal enables companies like Grove, and anyone else on Pocket Network, to serve endpoints to AI products backed by LLMs. There are already a couple POC’s in the pocket ecosystem like GPokT and PNYX. These applications use a variety of data sources to synthesize answers, including but not limited to Pocket GitHub repositories, discord channels, telegram, X, and other documentation. These early applications will allow us to understand the quality of service needed to support AI applications in production.
Limitations of hosting open source models on Pocket Network
Pocket is really good at serving deterministic data. For example, an RPC request on an Ethereum full node will always provide the same answer, assuming finality has been reached on Ethereum and the node is synched. In contrast, querying uncommitted data from a mempool or a node that fell behind will temporarily provide an inconsistent answer depending who is queried and when.
Querying information from LLM’s cannot be expected to be deterministic. It could vary on the configuration of the LLM, the hardware it runs on, alongside many other factors. Members of the Pocket ecosystem are already designing mechanisms to address this problem. For example, Gateways could query multiple LLM providers for every query and either provide all the responses, the best one, or a summary, depending on what the end user is looking for. Though not perfect, it is an open problem in the AI space, and Pocket Network could create an incentive that’ll attract researchers to improve and iterate on potential solutions.
Pocket Network’s TAM Today
There are conservatively 100B serviceable paid daily rpc requests in the web3 market today. Assuming a cost of ($3 – $6 per million) requests, today’s Web3 RPC TAM amounts to about $100M – $200M annually. Though there are fewer new L1 chains coming online, Vitalik’s Rollup Centric Roadmap from 2020 is slowly being realized with the advent of new Data Availability layers (Celestia, EigenDA, Near, Optimism, etc…) upon which application-specific rollups are being built, each of which will require its own RPC entrypoint. I strongly believe that in 5 years, we should expect the web3 RPC market to be about 1T RPC requests per day.
We can do a TAM/SAM/SOM exercise to understand what Pocket Network can capture from the AI market. While difficult to measure, the Appendix estimates that in a $40B market, approximately $1M per day is spent on maintaining the infrastructure needed to run LLM inference, putting the SAM at around $365M today. If Pocket were to capture 10% of that SAM, that would put Pocket’s SOM at $36.5m annually. These are extremely conservative estimates that assume no growth of the AI market past 2023.
Opening Pocket Network to the LLM Inference market would conservatively increase our market by about 30% today.
Pocket Network continues to build the foundation to enable most infrastructure needs to run through decentralized protocols. As we see protocols like Gensyn focus on the training portion of the AI market, Pocket Network will complement their use cases perfectly as we integrate other middleware protocols more deeply. There is potential for a world where these protocols are so seamlessly interconnected that node runners are consistently updating their models as users query and infer them, bringing AI embedded more deeply into the web3 space and into our lives, with the guardrails provided by blockchains.
Appendix
[1] Bloomberg estimates the global AI market is approximately $40B today and expected to grow to $1.3T by 2032.
[2] Sam Altman publicly said that OpenAI is on track to make ~ $1.3B in revenue in 2023, just a year after launching ChatGPT.
[3] This is largely driven by ChatGPT subscriptions, which has 180M users, $80M revenue/month and costs ~$700K per day to run as of August 2023.
[4] $700K per day does not account for OpenAI’s API offerings or any other models on the market. In addition, new companies that provide hosted solutions of open-source models are popping up every day.
Rounding $700K up to $1M is a very conservative estimate of the pure cost needed to maintain the infrastructure of the AI LLM inference market today, in a rapidly growing market.