This milestone serves the purpose of setting the methodology plan for the rest of the project. The milestone includes reviewing of the suggested methodology and the code that needs to be implemented in order to get the right results. This is the right time to make amendments to the suggested methodology. The project description above includes several items that need to be measured and benchmarked. In this milestone we need to dive deep into each of these items and surface details that might change performance drastically.

Detailed breakdown of methodology plan

This milestone is the collection of scripts to be used to for the experiment. The purpose of this milestone is to identify whether we have enough detail at the methodology and development level, or if more detail needs to be integrated in the system, whether we will be able to extract the information we need by the final software and results, or if any readjustment needs to be done to the strategy followed. Although results are not expected at this stage, we need to have a clear idea of what results will be collected when the scripts are executed.

Experiment scripts/PoC

These results and the initial development help in order to judge whether we’re on the right track, or if adjustments need to be made. The purpose of this milestone is to identify whether we have enough detail, or if more detail needs to be integrated in the system, whether we will be able to extract the information we need by the final software and results, or if any readjustment needs to be done to the strategy followed.

Initial Results / Proof of Concept

Anyone should be able to reproduce the experiments done by the team. This is especially important for this project, as the dynamics and structure of the IPFS network changes over time and hence, engineering teams, or community members should be able to re-run the experiments in the future. <ol><li>Any code developed for this project should be migrated, or at least linked from the main GH repository:&nbsp;<a href="https://github.com/protocol/network-measurements." rel="noopener noreferrer" target="_blank" style="color: blue;">https://github.com/protocol/network-measurements.</a></li><li>The code should include detailed documentation/instructions on how to use the developed software, as well as contact details of the team.</li></ol> This milestone is expected before the end of the project and is not expected to be the final software. Updates can be done to the software and documentation after this milestone too. The milestone is needed in order for our team to carefully review the code, the code comments, as well as the HOWTO guide and provide feedback/ask for revisions in case things are not clear.

Reproducible Experiments & Stable Results

We expect a final report that details the main findings which address the&nbsp;<a href="https://github.com/protocol/network-measurements/blob/master/RFMs.md#rfm-7---distribution-of-dht-lookup-times" rel="noopener noreferrer" target="_blank" style="color: rgb(17, 85, 204);">Request for Measurements</a>&nbsp;(RFM), the methodology used,&nbsp;as well as any conclusions made and open issues for future investigation. The report is not expected to be long - think of a relatively detailed blogpost. The Final Report should be posted on the GH repository in the designated folder.

Final Report

With this measurement we are looking to understand how the DHT performs in terms of lookup latencies, depending on several parameters, such as the specific file being searched (e.g., popular vs unpopular), the location of nodes in the network, the size of the file, the time required to perform a connection, and the load of the nodes. The result of this measurement should be a distribution of the time required to perform DHT lookups, as well as a breakdown of latencies in the different stages of the content routing process. Ideally, we should have the option (e.g., in a dashboard-style tool) to chose to see the distribution of DHT lookups of the last 7 days, which will output a histogram with the time invested to perform a lookup in the DHT by the different probes. In order to achieve higher confidence in the results, we want to have nodes spun up in different locations across the planet, which publish content that is then retrieved from other locations. We want to find out, among others:<ul><li>distribution of DHT lookup latencies over time (e.g., below 500ms, below 1s, below 2s, above 10s etc), in a CDF, or similar.</li><li>correlation between slow lookups and geolocation or ASN. This will give us an insight on whether some part of the wider internet is slower, or if a particular node is either on a slow connection or overloaded.</li><li>stability of measurements over time, i.e., is a node consistently being involved in slow lookups?</li><li>the time spent in the different stages of the content lookup process: bitswap discovery, DHT lookup, content fetching (through Bitswap).</li></ul> These results will give us a good sense of the state of the network at a specific epoch, as well as the time spent in each of the content lookup and routing stages. We should think of this as a "snapshot" of the network at a particular point in time. Additionally, we could show the distribution of lookups in a specific probe, i.e. in a specific section of the network, in order to have a sense of the DHT lookup times to be expected for someone in the probe's premises.<h4> </h4>Relevant Request for Measurement:&nbsp;<a href="https://github.com/protocol/network-measurements/blob/master/RFMs.md#rfm-7---distribution-of-dht-lookup-times" rel="noopener noreferrer" target="_blank" style="color: rgb(17, 85, 204);">https://github.com/protocol/network-measurements/blob/master/RFMs.md#rfm-7---distribution-of-dht-lookup-times</a>

DHTs are complex structures that enable content routing in decentralised P2P environments. We would like to measure the performance of IPFS's libp2p Kademlia DHT from different vantage points in the network and have a breakdown of latencies

Please login to apply for this grant!