Week 10 & 11
During these two weeks, I’ve worked hard on paginating values on the DHT.
Value pagination
As explained on my post on data persistence, we’ve had network traffic issues. The solution we have found for this is to use the queries (see also this) to filter data on the remote peer we’re communicating with. The queries let us select fields of a value instead of fetching whole values. This way, we can fetch values with unique ids. The pagination is the process of first selecting all value ids for a given hash, then making a separate “get” request packet for each of the values.
This feature makes the DHT more friendly with UDP. In fact, UDP packets can be dropped when of size greater than the UDP MTU. Paginating values will help this as all UDP packets will now contain only one value.
Week 12
I’ve been working on making the “put” request lighter, again using queries. This is a key feature which will make it possible to enable data persistence. In fact, it enables us to send values to a peer only if it doesn’t already have the value we’re announcing. This will substantially reduce the overall traffic. This feature is still being tested. The last thing I have to do is to demonstrate the reduction of network traffic.
