Re6st (also called Re6stnet and pronounced resist) creates a resilient, scalable, ipv6 network on top of an existing ipv4 network, by creating tunnels on the fly, and then routing targeted traffic through these tunnels.
Nexedi's two Re6st networks
- Two Re6st networks at Nexedi: Trial and Production network
- 100+ nodes in Production network, and more to come
- Networks spanning countries all around the world, including China
- Networks need to stay connected despite China's firewall
- TSN networks are planned in the future
- Exterior nodes choose wrong BGP gateways
- Only one IP per node and one range per network supported
- Multicast not supported
- OpenVPN traffic gets blocked by China's Firewall
To solve these issues, a roadmap for re6st with new features and fixes has been planned and will be presented here.
- Represents a geographical zone
- Every node belongs to one community
- One BGP gateway associated to each community
- Example: Japan Community, Europe Community, Roaming Community, etc...
Since our re6st networks spans multiple continents, we need our gateways in each geographical zone to be used for the nodes in that zone. Currently, if a machine with native IPv6 in japan tries to access a re6st node in japan, it will go through our Paris gateway, which leads to high latencies. The problem only exists in one direction, from exterior to re6st.
Communities will be implemented by assigning one AS sub-prefix per community. We will have to run a BGP daemon to advertise each community's gateway.
More information, progress and todo: P-VIFIB-Re6stnet.Communities
TSN with re6st
TSN projects currently under development at Nexedi:
- Autonomous cooperative drone swarm
- Industrial automation: PLC with synchronized edge nodes controlling motors
- 5G fronthaul through ethernet
Support multiple ranges
A TSN enabled re6st node should support multiple classes of traffic with different latency, jitter, and bandwidth requirements:
- General purpose traffic (ssh, sending data to Wendolin server, etc...)
- Keep-alive messages (for PLC)
- Sending each other's position (transponder, for drones)
- Safety circuit (to stop everything if something goes wrong)
To implement this multiplexing:
- each traffic class will get an IPv6 range assigned to it.
- Messages within a traffic class should not get routed to an address of another traffic class, unless there is no other way
- Ideally, we want traffic classes to be mapped to hardware RX and TX queues if possible
- Congestion should be spread equally for the transponder or keep-alive traffic class
PIM sparse Multicast
Multicast is necessary for the transponder or keep-alive traffic classes in order not to waste network ressources.
- PIM sparse multicast should be functionnal in re6st
- With multicast we can do OPC-UA PubSub
Having multiple VPNs is necessary to ensure connectivity stays through China, FOU and / or GRE tunnels needs to be implemented on re6st besides openVPN.
Investigate re6st perfomance issues
We need to investigate why after 3 hops re6st becomes 3 times slower than IPv4 in some cases, such as during the Basel prospect.
We are forced to use TCP openVPN right now because of fragmentation issues with UDP openVPN, this causes perfomances issues and needs to be changed.