Reminder: One more day to go for early bird registration..
Homa was first introduced to us in John Ousterhout's exciting 0x16 keynote[1]. John argued in the keynote that it was time to replace TCP with Homa to avoid what he labelled as the "datacentre tax".
For the uninformed: Homa is message-oriented like UDP or SCTP as opposed to TCP being stream-oriented. There is no notion of connection in Homa, only request/response RPC messaging. A single Homa socket can send/receive RPCs to many peers. One advantage that TCP has by virtue of using the streaming approach is it can pipeline very large messages (think 500KB across MTU sized packets on the wire) - whereas a message oriented approach will have to copy bytes to/from user space and DMA-able packet buffers in parrallel.
In his experiments with an mlx5 based 25G NIC, Dr. Ousterhout found that TCP infact outperformed Homa with large messages. At 500KB message size TCP was able to achieve 17-19 Gbps throughput, Homa was only able to hit 10-11Gbps. How dare TCP beat Homa in any metric! So John set out to fix this situation ;->. When he was done with the preliminary work Homa was able to hit the same rate as TCP (17-19Gbps).
In this talk, he will go over the approach he took in the kernel code to allow the kernel to manage user mmaped buffers, changes made to both recvmsg() and sendmsg() to make this happen.
1. https://netdevconf.info/0x16/sessions/keynote/keynote-ousterhout.html
cheers, jamal