kgess-mirror/matrix-pinecone
1
0
Fork 0
mirror of https://github.com/matrix-org/pinecone.git synced 2026-01-11 19:46:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Initial commit

Browse source
This commit is contained in:
Neil Alexander 2021-05-05 15:49:58 +01:00
commit 30c2fbab2c
No known key found for this signature in database
GPG key ID: A02A2019A2BB0944
63 changed files with 9059 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
.DS_Store
/pinecone

177
LICENSE Normal file
View file

@ -0,0 +1,177 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS

3
NOTICE Normal file
View file

@ -0,0 +1,3 @@
Pinecone
Peer-to-peer overlay routing for the Matrix ecosystem
Copyright 2021 The Matrix.org Foundation C.I.C.

67
README.md Normal file
View file

@ -0,0 +1,67 @@
# Pinecone
Pinecone is an experimental overlay routing protocol suite which is the foundation of the current P2P Matrix demos. It is designed to provide end-to-end encrypted connectivity between devices at a global scale over any compatible medium (currently TCP, WebSockets, Bluetooth Low Energy etc), allowing multi-hop peer-to-peer connectivity between devices even in places where there is no Internet connectivity.
Pinecone builds two virtual topologies: a globally agreed spanning tree, like [Yggdrasil](https://github.com/yggdrasil-network/yggdrasil-go), and a virtual line (or snake) arranged sequentially by public key. This gives some rise to the routing scheme perhaps being called SNEK (Sequentially Networked Edwards Key) routing, but perhaps we can come up with a better acronym. 🐍
Intersecting paths between keyspace neighbours provide the bulk of the routing knowledge, whilst the spanning tree provides greedy routing for some bootstrap and path setup traffic. In addition, Pinecone also implements source routing and an active pathfinder, although these are currently not used by the P2P Matrix demos.
Pinecone is incredibly experimental still. There might be bugs, vulnerabilities or architectural problems that we don't yet know about. If you spot anything that doesn't look right, we are very happy to welcome issues and pull requests, or you can join us in [#p2p:matrix.org](https://matrix.to/#/#p2p:matrix.org).
## Requirements
Go 1.14 or later.
## Questions
### Is there any documentation?
Aside from this readme and the comments in the code, not yet. There will be soon!
### Is there a protocol specification?
Not yet. We'll write one eventually, but in the meantime it is quite likely we will make a number of breaking protocol changes so it doesn't make sense to document the protocol itself too deeply yet.
### Does Pinecone perform well?
Generally yes! However, this implementation isn't terribly well optimised yet so there will almost certainly be improvements that can be made.
### Will Pinecone scale?
It's a primary goal of ours to build something that will scale up to Internet-like proportions in order to help us with our plans for Matrix world domination. We think Pinecone should scale well, but ultimately we will find out and make changes accordingly.
### Is Pinecone secure?
All traffic transported over Pinecone is end-to-end encrypted using TLS, therefore intermediate nodes will not be able to inspect packet contents. Many of the protocol messages are also cryptographically signed for authenticity. The protocol is still in its infancy, however, so there may be theoretical attacks that we don't know about yet.
### Can I run Pinecone on my platform?
This implementation is written in [Go](https://golang.org) which has excellent support for a number of platforms and cross-compilation for mobile devices. We've successfully seen Pinecone running on macOS, Linux, Windows, Android and iOS. We aren't aware of any specific reasons that it wouldn't work on other platforms supported by Go.
### Why not Yggdrasil?
We did in fact experiment with Yggdrasil in earlier P2P Matrix demos, and in many ways, Pinecone is directly inspired by Yggdrasil. However, the spanning tree topology alone is not a suitable routing scheme for highly dynamic networks. Peerings that represent parent-child relationships on the spanning tree can result in entire parts of the coordinate system becoming temporarily invalidated, interrupting connectivity.
### Why not libp2p?
We experimented with that too! libp2p worked well for local-only scenarios but currently assumes in many places that nodes will be directly routable to each other over (either over a LAN or the Internet), which is not necessarily going to be the case for P2P Matrix. Other components that would be useful (such as NAT traversal and overlay routing) are still quite early. If Pinecone overlay routing is a success then we will hopefully to be able to assist with adding support into libp2p in the future.
### Does Pinecone provide anonymity?
No, it is not a goal of Pinecone to provide anonymity. Pinecone packets will be routed using the most direct paths possible (in contrast to Tor and friends, which deliberately send traffic well out of their way) and Pinecone packets do contain source and destination information in their headers currently. It is likely that we will be able to seal some of this information, in particular the source addresses, to reduce traffic correlation, but this is not done today.
### Does Pinecone work on mobile devices?
Yes, we actually have two P2P Matrix demos for Android and iOS today. Node mobility is an important problem for us to solve, as not many routing schemes today respond well to topology changes. We believe that the SNEK routing within Pinecone should respond well to topology changes.
### What is a static peer?
Pinecone nodes can automatically discover each other depending on the platform (for example, Android and iOS nodes can discover each other over Bluetooth without any further configuration). However, you might not be physically close to any other Pinecone nodes, so instead you can join the wider Pinecone network by connecting to a static peer over the Internet or an existing network. Traffic to distant nodes will be routed over this peer connection.
### Can I run my own static peer?
Sure, the `cmd/pinecone` binary will help you to do that. You will need to provide the `-listen` command line argument to specify which port to accept connections on, and you will probably also want to specify the `-connect` flag to connect your node to an existing peer so that your node is not isolated from the rest of the world. Unless, of course, isolation is what you are aiming for.
### Does Pinecone work through firewalls or NATs?
Yes. Pinecone peering connections look like regular TCP or WebSocket connections and will work fine through firewalls or NATs. If you make an outbound connection to a static node, you will still be able to receive incoming Pinecone traffic over that peering.

96
cmd/pinecone/main.go Normal file
View file

@ -0,0 +1,96 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"crypto/ed25519"
"flag"
"fmt"
"log"
"net"
"os"
"net/http"
_ "net/http/pprof"
"github.com/matrix-org/pinecone/multicast"
"github.com/matrix-org/pinecone/router"
"github.com/matrix-org/pinecone/sessions"
)
func main() {
pk, sk, err := ed25519.GenerateKey(nil)
if err != nil {
panic(err)
}
logger := log.New(os.Stdout, "", 0)
if hostPort := os.Getenv("PPROFLISTEN"); hostPort != "" {
logger.Println("Starting pprof on", hostPort)
go func() {
_ = http.ListenAndServe(hostPort, nil)
}()
}
pineconeRouter := router.NewRouter(logger, "router", sk, pk, nil)
_ = sessions.NewSessions(logger, pineconeRouter)
pineconeMulticast := multicast.NewMulticast(logger, pineconeRouter)
pineconeMulticast.Start()
listen := flag.String("listen", "", "address to listen on")
connect := flag.String("connect", "", "peer to connect to")
flag.Parse()
if connect != nil && *connect != "" {
go func() {
conn, err := net.Dial("tcp", *connect)
if err != nil {
panic(err)
}
port, err := pineconeRouter.AuthenticatedConnect(conn, "", router.PeerTypeRemote)
if err != nil {
panic(err)
}
fmt.Println("Outbound connection", conn.RemoteAddr(), "is connected to port", port)
}()
}
go func() {
listener, err := net.Listen("tcp", *listen)
if err != nil {
panic(err)
}
fmt.Println("Listening on", listener.Addr())
for {
conn, err := listener.Accept()
if err != nil {
panic(err)
}
port, err := pineconeRouter.AuthenticatedConnect(conn, "", router.PeerTypeRemote)
if err != nil {
panic(err)
}
fmt.Println("Inbound connection", conn.RemoteAddr(), "is connected to port", port)
}
}()
select {}
}

1000
cmd/pineconesim/graphs/graph-250-2.txt Normal file
View file

File diff suppressed because it is too large Load diff

500
cmd/pineconesim/graphs/graph-250.txt Normal file
View file

@ -0,0 +1,500 @@
171 176
224 246
167 174
100 147
75 227
104 115
88 223
41 160
14 31
124 221
65 78
76 88
115 227
147 237
95 129
116 209
123 250
161 237
181 226
30 171
52 220
44 138
25 125
189 211
158 244
98 181
32 146
101 216
228 247
44 190
20 82
76 87
21 211
137 173
103 192
163 190
80 206
82 222
137 153
2 38
38 203
101 231
4 48
42 143
133 247
36 79
188 216
162 220
51 100
39 57
51 131
154 208
134 176
20 196
44 161
42 231
13 250
46 129
96 222
152 160
188 249
140 195
160 185
175 201
113 127
37 175
21 108
20 78
49 164
76 128
120 134
14 217
46 70
40 137
63 249
41 115
21 156
149 220
118 144
34 151
119 167
30 64
19 93
49 221
139 186
56 77
99 189
69 225
28 231
29 239
148 205
22 185
20 34
64 179
155 236
39 238
129 232
21 134
27 95
113 142
44 151
74 226
129 130
161 182
9 15
223 228
3 65
49 95
28 79
137 211
68 111
36 128
32 96
27 167
40 157
109 168
74 154
52 165
149 238
6 38
197 199
38 163
171 215
20 134
13 36
49 207
187 206
102 166
67 203
43 219
94 229
52 172
129 218
73 97
43 104
19 164
122 220
73 75
28 46
76 240
86 220
81 114
144 158
49 192
10 147
143 193
58 135
72 173
236 246
187 189
75 171
216 241
136 224
204 249
169 221
66 246
32 74
55 164
42 197
102 126
181 207
91 190
104 134
39 182
106 150
88 217
122 149
150 173
54 108
92 215
4 87
28 104
17 213
168 176
56 181
3 44
111 244
73 117
45 100
167 185
185 205
100 236
101 250
61 216
62 222
41 173
223 230
3 67
129 186
107 156
17 180
119 212
48 54
29 204
137 209
143 155
69 95
179 199
92 105
41 234
63 126
13 215
17 28
4 165
121 197
72 171
32 173
123 176
108 113
60 129
87 184
145 164
3 217
48 138
20 67
47 85
54 233
72 111
45 86
48 198
136 193
176 189
131 146
53 167
179 190
46 241
212 217
60 139
28 117
47 203
58 133
68 204
5 85
3 125
48 231
43 77
79 92
101 132
1 55
89 134
47 166
29 131
87 217
153 170
84 153
6 40
54 175
212 224
163 200
113 227
82 173
103 193
68 209
78 237
12 67
99 109
9 162
90 182
12 210
31 57
27 73
11 166
137 205
32 166
117 182
177 193
8 126
90 228
231 241
67 236
27 50
122 124
146 182
47 146
110 180
39 72
14 41
80 240
47 248
109 216
30 201
33 174
22 115
113 120
164 214
46 117
50 85
201 204
34 236
162 176
191 200
3 118
60 125
84 226
70 137
25 140
73 229
90 184
195 226
109 183
214 247
21 118
68 191
162 209
160 202
29 220
15 25
64 89
86 172
108 249
143 235
113 166
87 135
111 122
168 194
204 245
105 206
176 182
135 229
107 244
85 176
143 165
9 38
79 177
22 244
17 38
13 88
200 232
149 170
112 122
153 244
222 243
1 157
136 235
80 86
68 249
20 59
106 203
196 241
140 239
95 221
184 221
29 180
56 133
94 150
130 131
7 159
11 207
31 65
15 246
177 199
44 69
14 92
16 232
50 100
159 209
91 137
34 163
183 207
23 187
198 209
54 191
56 123
29 74
130 180
63 205
207 210
96 164
48 213
144 179
181 210
46 93
177 210
137 160
66 223
42 51
24 48
22 60
195 228
10 169
160 219
131 150
104 118
117 156
178 231
35 137
31 55
84 132
8 110
14 191
131 211
96 126
207 239
106 194
31 50
1 36
100 133
54 137
20 146
65 171
189 208
118 190
48 166
15 205
29 211
136 161
28 227
20 230
4 120
107 179
231 244
116 120
71 187
98 120
37 65
37 200
91 151
92 144
90 220
171 214
213 227
108 112
37 163
38 51
14 101
144 190
235 246
154 249
58 193
82 238
69 141
56 142
152 163
141 147
208 237
183 231
40 64
190 209
10 25
31 86
170 200
117 207
28 235
10 77
83 192
111 238
53 135
25 220
19 146
87 99
8 10
151 163
50 198
119 160
22 235
33 73
110 126
166 193
134 246
67 125
15 176
105 109
205 215
189 244
27 217
167 184
81 196
128 170
61 177
59 236
10 236
225 228
133 147
51 98
120 232
114 124
1 2
156 242
94 225
115 156
74 153
38 99
47 147
121 196
74 77
16 40
118 169
161 166
81 153
60 141
103 125
85 245
17 198
104 161
102 191
180 199
59 209
114 129
8 189
76 156

125
cmd/pineconesim/graphs/sim-medium.txt Normal file
View file

@ -0,0 +1,125 @@
27 83
4 48
32 93
68 77
73 97
70 96
12 17
42 87
61 79
16 44
40 57
25 100
29 82
48 56
12 87
10 98
15 59
36 41
38 92
74 89
30 69
4 21
82 83
15 86
34 97
87 96
12 19
74 90
25 98
41 48
65 75
2 85
36 80
51 88
23 91
33 64
18 97
40 86
40 58
4 64
17 40
41 59
70 72
50 93
59 99
33 72
29 55
56 70
6 61
57 83
41 54
37 56
49 88
22 50
6 87
11 14
56 96
11 19
29 39
44 78
22 76
68 87
13 40
28 64
4 85
30 71
15 22
12 14
36 81
1 4
56 67
79 93
19 42
18 94
89 92
37 62
27 97
39 47
6 15
8 43
11 86
27 80
24 88
36 86
2 25
60 96
11 44
1 65
12 47
5 75
57 79
23 55
28 54
29 54
16 57
8 40
5 32
11 85
1 29
16 83
23 45
31 32
4 75
50 77
61 89
3 58
51 97
16 18
1 24
41 91
83 92
88 92
16 99
53 93
80 87
75 90
4 25
3 19
53 71
66 81
40 97
24 69
40 63
13 27
54 70

5
cmd/pineconesim/graphs/sim.txt Normal file
View file

@ -0,0 +1,5 @@
a b c d e f g h i j k l m n o p q r s t u v w x y z
f h u r q f l m n v u n
f i j l p q r n v g h
a b r u r s y m v p
p j g h e f s r n k m l

414
cmd/pineconesim/main.go Normal file
View file

@ -0,0 +1,414 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"bufio"
"encoding/hex"
"flag"
"fmt"
"log"
"os"
"runtime"
"sort"
"strings"
"sync"
"text/template"
"time"
"github.com/gorilla/websocket"
"github.com/matrix-org/pinecone/cmd/pineconesim/simulator"
"github.com/matrix-org/pinecone/router"
"github.com/matrix-org/pinecone/util"
"net/http"
_ "net/http/pprof"
)
func main() {
go func() {
_ = http.ListenAndServe(":65432", nil)
}()
filename := flag.String("filename", "cmd/pineconesim/graphs/sim.txt", "the file that describes the simulated topology")
flag.Parse()
file, err := os.Open(*filename)
if err != nil {
panic(err)
}
defer file.Close()
scanner := bufio.NewScanner(file)
scanner.Split(bufio.ScanLines)
nodes := map[string]struct{}{}
wires := map[string]map[string]bool{}
for scanner.Scan() {
tokens := strings.Split(strings.TrimSpace(scanner.Text()), " ")
for _, t := range tokens {
nodes[t] = struct{}{}
}
for i := 1; i < len(tokens); i++ {
a, b := tokens[i-1], tokens[i]
if _, ok := wires[a]; !ok {
wires[a] = map[string]bool{}
}
if _, ok := wires[b][a]; ok {
continue
}
wires[a][b] = false
}
}
log := log.New(os.Stdout, "\u001b[36m***\u001b[0m ", 0)
sim := simulator.NewSimulator(log)
configureHTTPRouting(sim)
sim.CalculateShortestPaths(nodes, wires)
for n := range nodes {
if err := sim.CreateNode(n); err != nil {
panic(err)
}
}
for a, w := range wires {
for b := range w {
if a == b {
continue
}
log.Printf("Connecting %q and %q...\n", a, b)
err := sim.ConnectNodes(a, b)
wires[a][b] = err == nil
if err != nil {
continue
}
}
}
/*
rand.Seed(time.Now().UnixNano())
maxintv, maxswing := 5, int32(10)
var swing atomic.Int32
// Chaos disconnector
go func() {
for {
if swing.Load() > -maxswing {
parentloop:
for a, w := range wires {
for b, s := range w {
if !s {
continue
}
if err := sim.DisconnectNodes(a, b); err == nil {
wires[a][b] = false
swing.Dec()
break parentloop
}
}
}
}
time.Sleep(time.Second * time.Duration(rand.Intn(maxintv)))
}
}()
// Chaos connector
go func() {
for {
if swing.Load() < maxswing {
parentloop:
for a, w := range wires {
for b, s := range w {
if s {
continue
}
if err := sim.ConnectNodes(a, b); err == nil {
wires[a][b] = true
swing.Inc()
break parentloop
}
}
}
}
time.Sleep(time.Second * time.Duration(rand.Intn(maxintv)))
}
}()
*/
log.Println("Configuring HTTP listener")
go func() {
for {
time.Sleep(time.Second * 15)
log.Println("Starting pathfinds...")
tasks := make(chan pair, len(nodes)*len(nodes))
for from := range nodes {
for to := range nodes {
tasks <- pair{from, to}
}
}
close(tasks)
numworkers := runtime.NumCPU() * 16
var wg sync.WaitGroup
wg.Add(numworkers)
for i := 0; i < numworkers; i++ {
go func() {
for pair := range tasks {
log.Println("Pathfind from", pair.from, "to", pair.to)
if err := sim.Pathfind(pair.from, pair.to); err != nil {
log.Println("Pathfind from", pair.from, "to", pair.to, "failed:", err)
}
}
wg.Done()
}()
}
wg.Wait()
log.Println("All pathfinds finished, repeating shortly...")
}
}()
select {}
}
type pair struct{ from, to string }
func configureHTTPRouting(sim *simulator.Simulator) {
wsUpgrader := websocket.Upgrader{}
http.DefaultServeMux.HandleFunc("/simws", func(w http.ResponseWriter, r *http.Request) {
var n *simulator.Node
nodeID := r.URL.Query().Get("node")
if nodeID != "" {
n = sim.Node(nodeID)
} else {
for id, node := range sim.Nodes() {
if node != nil {
n, nodeID = node, id
break
}
}
}
if n == nil {
w.WriteHeader(404)
return
}
c, err := wsUpgrader.Upgrade(w, r, nil)
if err != nil {
return
}
conn := util.WrapWebSocketConn(c)
if _, err = n.AuthenticatedConnect(conn, "websocket", router.PeerTypeRemote); err != nil {
return
}
log.Printf("WebSocket peer %q connected to sim node %q\n", c.RemoteAddr(), nodeID)
})
http.DefaultServeMux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
tmpl := template.Must(template.ParseFiles("./cmd/pineconesim/page.html"))
nodes := sim.Nodes()
wires := sim.Wires()
totalCount := len(nodes) * len(nodes)
dhtConvergence := 0
pathConvergence := 0
for _, nodes := range sim.DHTConvergence() {
for _, converged := range nodes {
if converged {
dhtConvergence++
}
}
}
for _, paths := range sim.PathConvergence() {
for _, converged := range paths {
if converged {
pathConvergence++
}
}
}
data := PageData{
AvgStretch: "TBD",
DHTConvergence: "TBD",
PathConvergence: "TBD",
}
if totalCount > 0 {
data.DHTConvergence = fmt.Sprintf("%d%%", (dhtConvergence*100)/totalCount)
data.PathConvergence = fmt.Sprintf("%d%%", (pathConvergence*100)/totalCount)
}
roots := map[string]int{}
nodeids := []string{}
for n := range nodes {
nodeids = append(nodeids, n)
}
sort.Strings(nodeids)
for _, n := range nodeids {
node := nodes[n]
public := node.PublicKey()
entry := Node{
Name: n,
Port: node.ListenPort,
Coords: fmt.Sprintf("%v", node.Coords()),
Key: hex.EncodeToString(public[:2]),
IsRoot: node.IsRoot(),
}
rootkey := node.RootPublicKey()
entry.Root = hex.EncodeToString(rootkey[:2])
if predecessor := node.Predecessor(); predecessor != nil {
entry.Predecessor = hex.EncodeToString(predecessor[:2])
}
if successor := node.Successor(); successor != nil {
entry.Successor = hex.EncodeToString(successor[:2])
}
data.Nodes = append(data.Nodes, entry)
data.NodeCount++
roots[entry.Root]++
}
for range wires {
data.PathCount++
}
switch r.URL.Query().Get("view") {
case "logical":
for id, n := range nodes {
for id2, n2 := range nodes {
p := n.Predecessor()
s := n.Successor()
if p != nil && p.EqualTo(n2.PublicKey()) {
data.Links = append(data.Links, Link{
From: id,
To: id2,
Enabled: true,
})
}
if s != nil && s.EqualTo(n2.PublicKey()) {
data.Links = append(data.Links, Link{
From: id,
To: id2,
Enabled: true,
})
}
}
}
case "physical":
fallthrough
default:
for a, w := range wires {
for b, conn := range w {
data.Links = append(data.Links, Link{
From: a,
To: b,
Enabled: conn != nil,
})
// If we find any external nodes, let's show those too...
if _, ok := nodes[b]; !ok {
data.Nodes = append(data.Nodes, Node{
Name: b,
IsExternal: true,
})
}
}
}
}
for n, r := range roots {
data.Roots = append(data.Roots, Root{
Name: n,
References: r * 100 / len(data.Nodes),
})
}
var stretchT float64
var stretchC int
for a, aa := range sim.Distances() {
for b, d := range aa {
if a == b {
continue
}
dist := Dist{
From: a,
To: b,
Real: "TBD",
Observed: "TBD",
Stretch: "TBD",
}
dist.Real = fmt.Sprintf("%d", d.Real)
if d.Observed >= d.Real {
dist.Observed = fmt.Sprintf("%d", d.Observed)
}
if d.Observed >= d.Real {
stretch := float64(1)
if d.Real > 0 && d.Observed > 0 {
stretch = float64(1) / float64(d.Real) * float64(d.Observed)
}
dist.Stretch = fmt.Sprintf("%.2f", stretch)
stretchT += stretch
stretchC++
}
//data.Dists = append(data.Dists, dist)
}
}
if stretch := stretchT / float64(stretchC); stretch >= 1 {
data.AvgStretch = fmt.Sprintf("%.2f", stretch)
}
_ = tmpl.Execute(w, data)
})
}
type Node struct {
Name string
Port string
Coords string
Key string
Root string
Predecessor string
Successor string
IsRoot bool
IsExternal bool
}
type Link struct {
From string
To string
Enabled bool
}
type PageData struct {
NodeCount int
PathCount int
Nodes []Node
Links []Link
Roots []Root
Dists []Dist
AvgStretch string
DHTConvergence string
PathConvergence string
}
type Root struct {
Name string
References int
}
type Dist struct {
From string
To string
Real string
Observed string
Stretch string
}

198
cmd/pineconesim/page.html Normal file
View file

@ -0,0 +1,198 @@
<html>
<head>
<title>Pinecone Sim</title>
<script type="text/javascript" src="https://visjs.github.io/vis-network/standalone/umd/vis-network.min.js"></script>
<style type='text/css'>
body {
font-family: -apple-system, sans-serif;
}
.row {
display: flex;
justify-content: space-between;
flex-direction: row;
flex-wrap: nowrap;
padding: 1em;
}
.column {
flex-direction: column;
}
.column#left {
order: 1;
width: auto;
padding-right: 2em;
}
.column#right {
order: 2;
flex-shrink: 100;
}
#main.row #left.column,
#main.row #right.column {
height: calc(100vh - 100px);
overflow-y: scroll;
}
#main.row div#canvas {
height: calc(100vh - 100px);
background-color: #f6f6f6;
}
#main.row div#left {
order: 1;
flex-basis: auto;
}
#main.row div#right {
order: 2;
flex-basis: 100%;
}
#title div#right a {
color: black;
text-decoration: none;
padding: 0.3em;
margin-left: 0.1em;
margin-right: 0.1em;
border: 1px solid black;
border-radius: 0.5em;
}
td, th {
text-align: center;
padding-left: 4px;
padding-right: 4px;
}
</style>
</head>
<body>
<div class="row" id="title">
<div class="column" id="left">
<strong>Pinecone Simulator</strong> is running with {{.NodeCount}} nodes
</div>
<div class="column" id="right">
Topology view:
<a href="/?view=physical">Physical</a>
<a href="/?view=logical">Logical</a>
</div>
</div>
<div class="row" id="main">
<div class="column" id="left">
<h4>Statistics</h4>
<table>
<tr>
<td>Total number of paths:</td>
<td>{{.PathCount}}</td>
</tr>
<tr>
<td>Average stretch of paths:</td>
<td>{{.AvgStretch}}</td>
</tr>
<tr>
<td>DHT full convergence:</td>
<td>{{.DHTConvergence}}</td>
</tr>
<tr>
<td>Path full convergence:</td>
<td>{{.PathConvergence}}</td>
</tr>
</table>
<h4>Node Summary</h4>
<table>
<tr>
<th>Name</th>
<th>Port</th>
<th>Coords</th>
<th>Root</th>
<th>↓</th>
<th>Key</th>
<th>↑</th>
</tr>
{{range .Nodes}}
{{if not .IsExternal}}<tr>
<td>{{.Name}}</td>
<td><code>{{.Port}}</code></td>
<td><code>{{.Coords}}</code></td>
<td><code>{{.Root}}</code></td>
<td><code>{{.Predecessor}}</code></td>
<td><code>{{.Key}}</code></td>
<td><code>{{.Successor}}</code></td>
{{end}}</tr>{{end}}
</table>
<h4>Tree Building</h4>
<table>
<tr>
<th>Root Key</th>
<th>Convergence</th>
</tr>
{{range .Roots}}<tr>
<td><code>{{.Name}}</code></td>
<td>{{.References}}%</td>
</tr>{{end}}
</table>
<h4>Path Lengths</h4>
<table>
<tr>
<th>From</th>
<th>To</th>
<th>Actual</th>
<th>Observed</th>
<th>Stretch</th>
</th>
{{range .Dists}}<tr>
<td>{{.From}}</td>
<td>{{.To}}</td>
<td>{{.Real}}</td>
<td>{{.Observed}}</td>
<td>{{.Stretch}}</td>
</tr>{{end}}
</table>
</div>
<div class="column" id="right">
<div id="canvas" />
</div>
</div>
<script>
var nodes = new vis.DataSet([
{{range .Nodes}}
{{if .IsRoot}}
{ id: "{{.Name}}", label: "{{.Name}}", color: "#63cfa1" },
{{else if .IsExternal}}
{ id: "{{.Name}}", label: "{{.Name}}", color: "#dddddd" },
{{else}}
{ id: "{{.Name}}", label: "{{.Name}}", color: "#9fc2f7" },
{{end}}
{{end}}
]);
var edges = new vis.DataSet([
{{range .Links}}
{{if .Enabled}}
{ from: "{{.From}}", to: "{{.To}}", color: "black" },
// {{else}}
// { from: "{{.From}}", to: "{{.To}}", color: "lightgrey", dashes: true },
{{end}}
{{end}}
]);
var container = document.getElementById("canvas");
var data = {
nodes: nodes,
edges: edges,
};
var options = {
interaction:{
dragView: true,
zoomView: true,
},
physics:{
enabled: true,
}
};
var network = new vis.Network(container, data, options);
</script>
</body>
</html>
</body>
</html>

104
cmd/pineconesim/simulator/interface.go Normal file
View file

@ -0,0 +1,104 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package simulator
import (
"fmt"
"net"
"strings"
"github.com/matrix-org/pinecone/types"
)
func (sim *Simulator) LookupCoords(target string) (types.SwitchPorts, error) {
node, ok := sim.nodes[target]
if !ok {
return nil, fmt.Errorf("node %q not known", target)
}
return node.Coords(), nil
}
func (sim *Simulator) LookupNodeID(target types.SwitchPorts) (string, error) {
for id, n := range sim.nodes {
if n.Coords().EqualTo(target) {
return id, nil
}
}
return "", fmt.Errorf("coords %v not known", target)
}
func (sim *Simulator) LookupPublicKey(target types.PublicKey) (string, error) {
for id, n := range sim.nodes {
if n.PublicKey().EqualTo(target) {
return id, nil
}
}
return "", fmt.Errorf("public key %s not known", target.String())
}
func (sim *Simulator) ReportNewLink(c net.Conn, source, target types.PublicKey) {
sourceID, err := sim.LookupPublicKey(source)
if err != nil {
return
}
if _, err = sim.LookupPublicKey(target); err == nil {
return
}
sim.wiresMutex.Lock()
defer sim.wiresMutex.Unlock()
if _, ok := sim.wires[sourceID]; !ok {
sim.wires[sourceID] = map[string]net.Conn{}
}
sim.wires[sourceID][target.String()] = c
}
func (sim *Simulator) ReportDeadLink(source, target types.PublicKey) {
sourceID, err := sim.LookupPublicKey(source)
if err != nil {
return
}
if _, err = sim.LookupPublicKey(target); err == nil {
return
}
sim.wiresMutex.Lock()
defer sim.wiresMutex.Unlock()
if _, ok := sim.wires[sourceID]; !ok {
return
}
delete(sim.wires[sourceID], target.String())
}
func (sim *Simulator) ReportDistance(a, b string, l int64) {
if strings.Compare(a, b) > 0 {
a, b = b, a
}
sim.distsMutex.Lock()
defer sim.distsMutex.Unlock()
if _, ok := sim.dists[a]; !ok {
sim.dists[a] = map[string]*Distance{}
}
if _, ok := sim.dists[a][b]; !ok {
sim.dists[a][b] = &Distance{}
}
sim.dists[a][b].Observed = l
if sim.dists[a][b].Real == 0 {
na, _ := sim.graph.GetMapping(a)
nb, _ := sim.graph.GetMapping(b)
path, err := sim.graph.Shortest(na, nb)
if err == nil {
sim.dists[a][b].Real = path.Distance
}
}
}

88
cmd/pineconesim/simulator/links.go Normal file
View file

@ -0,0 +1,88 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package simulator
import (
"fmt"
"net"
"github.com/matrix-org/pinecone/router"
)
func (sim *Simulator) ConnectNodes(a, b string) error {
sim.nodesMutex.RLock()
na := sim.nodes[a]
nb := sim.nodes[b]
sim.nodesMutex.RUnlock()
if na == nil || nb == nil {
return fmt.Errorf("invalid node pair")
}
sim.wiresMutex.RLock()
wa := sim.wires[a][b]
wb := sim.wires[b][a]
sim.wiresMutex.RUnlock()
if wa != nil || wb != nil {
return fmt.Errorf("already connected")
}
c, err := net.Dial(na.l.Addr().Network(), na.l.Addr().String())
if err != nil {
return fmt.Errorf("net.Dial: %w", err)
}
if _, err := nb.AuthenticatedConnect(c, "sim", router.PeerTypeRemote); err != nil {
return fmt.Errorf("nb.AuthenticatedConnect: %w", err)
}
/*
pa, pb := net.Pipe()
go func() {
if _, err := na.Connect(pa, nb.PublicKey(), "sim", router.PeerTypeRemote); err != nil {
return //fmt.Errorf("nb.AuthenticatedConnect: %w", err)
}
}()
go func() {
if _, err := nb.Connect(pb, na.PublicKey(), "sim", router.PeerTypeRemote); err != nil {
return //fmt.Errorf("nb.AuthenticatedConnect: %w", err)
}
}()
*/
sim.wiresMutex.Lock()
defer sim.wiresMutex.Unlock()
if sim.wires[a] == nil {
sim.wires[a] = map[string]net.Conn{}
}
sim.wires[a][b] = c
//sim.wires[a][b] = pa
sim.log.Printf("Connected node %q to node %q\n", a, b)
return nil
}
func (sim *Simulator) DisconnectNodes(a, b string) error {
sim.wiresMutex.RLock()
wire := sim.wires[a][b]
if wire == nil {
wire = sim.wires[b][a]
}
sim.wiresMutex.RUnlock()
if wire == nil {
return fmt.Errorf("no wire found")
}
sim.wiresMutex.Lock()
sim.wires[a][b] = nil
sim.wiresMutex.Unlock()
return wire.Close()
}

70
cmd/pineconesim/simulator/nodes.go Normal file
View file

@ -0,0 +1,70 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package simulator
import (
"crypto/ed25519"
"fmt"
"net"
"github.com/matrix-org/pinecone/router"
)
func (sim *Simulator) Node(t string) *Node {
sim.nodesMutex.Lock()
defer sim.nodesMutex.Unlock()
return sim.nodes[t]
}
func (sim *Simulator) CreateNode(t string) error {
l, err := net.Listen("tcp", ":0")
if err != nil {
return fmt.Errorf("net.Listen: %w", err)
}
_, port, err := net.SplitHostPort(l.Addr().String())
if err != nil {
return fmt.Errorf("net.SplitHostPort: %w", err)
}
pk, sk, err := ed25519.GenerateKey(nil)
if err != nil {
return fmt.Errorf("ed25519.GenerateKey: %w", err)
}
n := &Node{
Router: router.NewRouter(sim.log, t, sk, pk, sim),
l: l,
ListenPort: port,
}
sim.nodesMutex.Lock()
sim.nodes[t] = n
sim.nodesMutex.Unlock()
go func(n *Node) {
for {
c, err := l.Accept()
if err != nil {
continue
}
if _, err = n.AuthenticatedConnect(c, "sim", router.PeerTypeRemote); err != nil {
continue
}
}
}(n)
sim.log.Printf("Created node %q (listening on %s)\n", t, l.Addr())
//sim.log.Printf("Created node %q\n", t)
return nil
}

64
cmd/pineconesim/simulator/pathfind.go Normal file
View file

@ -0,0 +1,64 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package simulator
import (
"context"
"fmt"
"time"
)
func (sim *Simulator) Pathfind(from, to string) error {
fromnode := sim.nodes[from]
tonode := sim.nodes[to]
ctx, cancel := context.WithTimeout(context.Background(), time.Second*5)
defer cancel()
dhtSuccess := false
pathfindSuccess := false
//sim.log.Println("DHT search from", from, "to", to)
//public := tonode.PublicKey()
defer func() {
sim.pathConvergenceMutex.Lock()
if _, ok := sim.pathConvergence[from]; !ok {
sim.pathConvergence[from] = map[string]bool{}
}
sim.pathConvergence[from][to] = pathfindSuccess
sim.pathConvergenceMutex.Unlock()
sim.dhtConvergenceMutex.Lock()
if _, ok := sim.dhtConvergence[from]; !ok {
sim.dhtConvergence[from] = map[string]bool{}
}
sim.dhtConvergence[from][to] = dhtSuccess
sim.dhtConvergenceMutex.Unlock()
}()
/*if _, _, err := fromnode.DHTSearch(ctx, public[:], false); err != nil {
sim.log.Println("DHT search from", from, "to", to, "failed:", err)
} else {
dhtSuccess = true
}*/
_, err := fromnode.Pathfind(ctx, tonode.PublicKey())
if err != nil {
return fmt.Errorf("fromnode.r.Pathfind: %w", err)
} else {
pathfindSuccess = true
}
return nil
}

38
cmd/pineconesim/simulator/realpaths.go Normal file
View file

@ -0,0 +1,38 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package simulator
import (
"github.com/RyanCarrier/dijkstra"
)
func (sim *Simulator) CalculateShortestPaths(nodes map[string]struct{}, wires map[string]map[string]bool) {
sim.log.Println("Building graph")
sim.graph = dijkstra.NewGraph()
sim.maps = make(map[string]int)
for n := range nodes {
sim.maps[n] = sim.graph.AddMappedVertex(n)
}
for a, aa := range wires {
for b := range aa {
if err := sim.graph.AddMappedArc(a, b, 1); err != nil {
panic(err)
}
if err := sim.graph.AddMappedArc(b, a, 1); err != nil {
panic(err)
}
}
}
}

81
cmd/pineconesim/simulator/simulator.go Normal file
View file

@ -0,0 +1,81 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package simulator
import (
"log"
"net"
"sync"
"github.com/RyanCarrier/dijkstra"
)
type Simulator struct {
log *log.Logger
nodes map[string]*Node
nodesMutex sync.RWMutex
graph *dijkstra.Graph
maps map[string]int
wires map[string]map[string]net.Conn
wiresMutex sync.RWMutex
dists map[string]map[string]*Distance
distsMutex sync.RWMutex
pathConvergence map[string]map[string]bool
pathConvergenceMutex sync.RWMutex
dhtConvergence map[string]map[string]bool
dhtConvergenceMutex sync.RWMutex
}
func NewSimulator(log *log.Logger) *Simulator {
sim := &Simulator{
log: log,
nodes: make(map[string]*Node),
wires: make(map[string]map[string]net.Conn),
dists: make(map[string]map[string]*Distance),
pathConvergence: make(map[string]map[string]bool),
dhtConvergence: make(map[string]map[string]bool),
}
return sim
}
func (sim *Simulator) Nodes() map[string]*Node {
sim.nodesMutex.RLock()
defer sim.nodesMutex.RUnlock()
return sim.nodes
}
func (sim *Simulator) Wires() map[string]map[string]net.Conn {
sim.wiresMutex.RLock()
defer sim.wiresMutex.RUnlock()
return sim.wires
}
func (sim *Simulator) Distances() map[string]map[string]*Distance {
sim.distsMutex.RLock()
defer sim.distsMutex.RUnlock()
return sim.dists
}
func (sim *Simulator) PathConvergence() map[string]map[string]bool {
sim.pathConvergenceMutex.RLock()
defer sim.pathConvergenceMutex.RUnlock()
return sim.pathConvergence
}
func (sim *Simulator) DHTConvergence() map[string]map[string]bool {
sim.dhtConvergenceMutex.RLock()
defer sim.dhtConvergenceMutex.RUnlock()
return sim.dhtConvergence
}

32
cmd/pineconesim/simulator/types.go Normal file
View file

@ -0,0 +1,32 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package simulator
import (
"net"
"github.com/matrix-org/pinecone/router"
)
type Node struct {
*router.Router
l net.Listener
ListenPort string
}
type Distance struct {
Real int64
Observed int64
}

16
go.mod Normal file
View file

@ -0,0 +1,16 @@
module github.com/matrix-org/pinecone
go 1.14
require (
github.com/RyanCarrier/dijkstra v1.0.0
github.com/gorilla/websocket v1.4.2
github.com/kr/text v0.2.0 // indirect
github.com/neilalexander/utp v0.1.1-0.20210315133037-94aa85fe0436
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e // indirect
go.uber.org/atomic v1.7.0
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110
golang.org/x/sys v0.0.0-20201231184435-2d18734c6014
gopkg.in/check.v1 v1.0.0-20200902074654-038fdea0a05b // indirect
gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776 // indirect
)

87
go.sum Normal file
View file

@ -0,0 +1,87 @@
github.com/RoaringBitmap/roaring v0.4.7/go.mod h1:8khRDP4HmeXns4xIj9oGrKSz7XTQiJx2zgh7AcNke4w=
github.com/RyanCarrier/dijkstra v1.0.0 h1:aMzaxaXb3B/ZJ2EqM8qdmd19c4ZuPuN1VZzbmGWIbCY=
github.com/RyanCarrier/dijkstra v1.0.0/go.mod h1:5agGUBNEtUAGIANmbw09fuO3a2htPEkc1jNH01qxCWA=
github.com/RyanCarrier/dijkstra-1 v0.0.0-20170512020943-0e5801a26345 h1:fgSpoKViTSqRb4hjDNj10ig5wUvO0CayCzFdLf6fuRM=
github.com/RyanCarrier/dijkstra-1 v0.0.0-20170512020943-0e5801a26345/go.mod h1:OK4EvWJ441LQqGzed5NGB6vKBAE34n3z7iayPcEwr30=
github.com/albertorestifo/dijkstra v0.0.0-20160910063646-aba76f725f72 h1:uGeGZl8PxSq8VZGG4QK5njJTFA4/G/x5CYORvQVXtAE=
github.com/albertorestifo/dijkstra v0.0.0-20160910063646-aba76f725f72/go.mod h1:o+JdB7VetTHjLhU0N57x18B9voDBQe0paApdEAEoEfw=
github.com/anacrolix/envpprof v0.0.0-20180404065416-323002cec2fa/go.mod h1:KgHhUaQMc8cC0+cEflSgCFNFbKwi5h54gqtVn8yhP7c=
github.com/anacrolix/envpprof v1.0.0/go.mod h1:KgHhUaQMc8cC0+cEflSgCFNFbKwi5h54gqtVn8yhP7c=
github.com/anacrolix/envpprof v1.1.1 h1:sHQCyj7HtiSfaZAzL2rJrQdyS7odLqlwO6nhk/tG/j8=
github.com/anacrolix/envpprof v1.1.1/go.mod h1:My7T5oSqVfEn4MD4Meczkw/f5lSIndGAKu/0SM/rkf4=
github.com/anacrolix/log v0.3.0 h1:Btxh7GkT4JYWvWJ1uKOwgobf+7q/1eFQaDdCUXCtssw=
github.com/anacrolix/log v0.3.0/go.mod h1:lWvLTqzAnCWPJA08T2HCstZi0L1y2Wyvm3FJgwU9jwU=
github.com/anacrolix/missinggo v1.1.2-0.20190815015349-b888af804467/go.mod h1:MBJu3Sk/k3ZfGYcS7z18gwfu72Ey/xopPFJJbTi5yIo=
github.com/anacrolix/missinggo v1.2.1 h1:0IE3TqX5y5D0IxeMwTyIgqdDew4QrzcXaaEnJQyjHvw=
github.com/anacrolix/missinggo v1.2.1/go.mod h1:J5cMhif8jPmFoC3+Uvob3OXXNIhOUikzMt+uUjeM21Y=
github.com/anacrolix/missinggo/perf v1.0.0 h1:7ZOGYziGEBytW49+KmYGTaNfnwUqP1HBsy6BqESAJVw=
github.com/anacrolix/missinggo/perf v1.0.0/go.mod h1:ljAFWkBuzkO12MQclXzZrosP5urunoLS0Cbvb4V0uMQ=
github.com/anacrolix/sync v0.2.0 h1:oRe22/ZB+v7v/5Mbc4d2zE0AXEZy0trKyKLjqYOt6tY=
github.com/anacrolix/sync v0.2.0/go.mod h1:BbecHL6jDSExojhNtgTFSBcdGerzNc64tz3DCOj/I0g=
github.com/anacrolix/tagflag v0.0.0-20180109131632-2146c8d41bf0/go.mod h1:1m2U/K6ZT+JZG0+bdMK6qauP49QT4wE5pmhJXOKKCHw=
github.com/anacrolix/utp v0.1.0/go.mod h1:MDwc+vsGEq7RMw6lr2GKOEqjWny5hO5OZXRVNaBJ2Dk=
github.com/bradfitz/iter v0.0.0-20140124041915-454541ec3da2/go.mod h1:PyRFw1Lt2wKX4ZVSQ2mk+PeDa1rxyObEDlApuIsUKuo=
github.com/bradfitz/iter v0.0.0-20190303215204-33e6a9893b0c/go.mod h1:PyRFw1Lt2wKX4ZVSQ2mk+PeDa1rxyObEDlApuIsUKuo=
github.com/bradfitz/iter v0.0.0-20191230175014-e8f45d346db8 h1:GKTyiRCL6zVf5wWaqKnf+7Qs6GbEPfd4iMOitWzXJx8=
github.com/bradfitz/iter v0.0.0-20191230175014-e8f45d346db8/go.mod h1:spo1JLcs67NmW1aVLEgtA8Yy1elc+X8y5SRW1sFW4Og=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/docopt/docopt-go v0.0.0-20180111231733-ee0de3bc6815/go.mod h1:WwZ+bS3ebgob9U8Nd0kOddGdZWjyMGR8Wziv+TBNwSE=
github.com/dustin/go-humanize v0.0.0-20180421182945-02af3965c54e/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/glycerine/go-unsnap-stream v0.0.0-20180323001048-9f0cb55181dd/go.mod h1:/20jfyN9Y5QPEAprSgKAUr+glWDY39ZiUEAYOEv5dsE=
github.com/glycerine/goconvey v0.0.0-20180728074245-46e3a41ad493/go.mod h1:Ogl1Tioa0aV7gstGFO7KhffUsb9M4ydbEbbxpcEDc24=
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/btree v0.0.0-20180124185431-e89373fe6b4a/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/gopherjs/gopherjs v0.0.0-20181103185306-d547d1d9531e/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gorilla/websocket v1.4.2 h1:+/TMaTYc4QFitKJxsQ7Yye35DkWvkdLcvGKqM+x0Ufc=
github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/huandu/xstrings v1.0.0/go.mod h1:4qWG/gcEcfX4z/mBDHJ++3ReCw9ibxbsNJbcucJdbSo=
github.com/huandu/xstrings v1.2.0 h1:yPeWdRnmynF7p+lLYz0H2tthW9lqhMJrQV/U7yy4wX0=
github.com/huandu/xstrings v1.2.0/go.mod h1:DvyZB1rfVYsBIigL8HwpZgxHwXozlTgGqn63UyNX5k4=
github.com/jtolds/gls v4.2.1+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mattomatic/dijkstra v0.0.0-20130617153013-6f6d134eb237 h1:acuCHBjzG7MFTugvx3buC4m5rLDLaKC9J8C9jtlraRc=
github.com/mattomatic/dijkstra v0.0.0-20130617153013-6f6d134eb237/go.mod h1:UOnLAUmVG5paym8pD3C4B9BQylUDC2vXFJJpT7JrlEA=
github.com/mschoch/smat v0.0.0-20160514031455-90eadee771ae/go.mod h1:qAyveg+e4CE+eKJXWVjKXM4ck2QobLqTDytGJbLLhJg=
github.com/neilalexander/utp v0.1.1-0.20210315133037-94aa85fe0436 h1:1gP/M2jAATYqlKjSbDBlB+EZmlAjzF50pxb5ILor1cs=
github.com/neilalexander/utp v0.1.1-0.20210315133037-94aa85fe0436/go.mod h1:ylsx0342RjGHjOoVKhR/wz/7Lhiusonihfj4QLxEMcU=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWbfPhv4DMiApHyliiK5xCTNVSPiaAs=
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/philhofer/fwd v1.0.0/go.mod h1:gk3iGcWd9+svBvR0sR+KPcfE+RNWozjowpeBVG3ZVNU=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/ryszard/goskiplist v0.0.0-20150312221310-2dfbae5fcf46/go.mod h1:uAQ5PCi+MFsC7HjREoAz1BU+Mq60+05gifQSsHSDG/8=
github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
github.com/smartystreets/goconvey v0.0.0-20181108003508-044398e4856c/go.mod h1:XDJAKZRPZ1CvBcN2aX5YOUTYGHki24fSF0Iv48Ibg0s=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.1/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/tinylib/msgp v1.0.2/go.mod h1:+d+yLhGm8mzTaHzB+wgMYrodPfmZrzkirds8fDWklFE=
github.com/willf/bitset v1.1.9/go.mod h1:RjeCKbqT1RxIR/KWY6phxZiaY1IyutSBfGjNPySAYV4=
go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw=
go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110 h1:qWPm9rbaAMKs8Bq/9LRpbMqxWRVUAQwMI9fVrssnTfw=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201231184435-2d18734c6014 h1:joucsQqXmyBVxViHCPFjG3hx8JzIFSaym3l3MM/Jsdg=
golang.org/x/sys v0.0.0-20201231184435-2d18734c6014/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20200902074654-038fdea0a05b h1:QRR6H1YWRnHb4Y/HeNFCTJLFVxaq6wH4YuVdsUOr75U=
gopkg.in/check.v1 v1.0.0-20200902074654-038fdea0a05b/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776 h1:tQIYjPdBoyREyB9XMu+nnTclpTYkz2zFM+lzLJFO4gQ=
gopkg.in/yaml.v3 v3.0.0-20200615113413-eeeca48fe776/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

314
multicast/multicast.go Normal file
View file

@ -0,0 +1,314 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package multicast
import (
"context"
"crypto/ed25519"
"encoding/binary"
"encoding/hex"
"fmt"
"log"
"net"
"sync"
"time"
"github.com/matrix-org/pinecone/router"
"github.com/matrix-org/pinecone/types"
"go.uber.org/atomic"
"golang.org/x/net/ipv6"
)
const MulticastGroupAddr = "[ff02::114]"
const MulticastGroupPort = 60606
type Multicast struct {
r *router.Router
log *log.Logger
ctx context.Context
cancel context.CancelFunc
id string
started atomic.Bool
interfaces sync.Map // -> *multicastInterface
listener net.Listener
dialer net.Dialer
tcpLC net.ListenConfig
udpLC net.ListenConfig
}
type multicastInterface struct {
context context.Context
cancel context.CancelFunc
net.Interface
}
func NewMulticast(
log *log.Logger, r *router.Router,
) *Multicast {
public := r.PublicKey()
m := &Multicast{
r: r,
log: log,
id: hex.EncodeToString(public[:]),
}
m.tcpLC = net.ListenConfig{
Control: m.tcpOptions,
}
m.udpLC = net.ListenConfig{
Control: m.udpOptions,
}
m.dialer = net.Dialer{
Control: m.tcpOptions,
Timeout: time.Second * 5,
}
return m
}
func (m *Multicast) Start() {
if !m.started.CAS(false, true) {
return
}
m.ctx, m.cancel = context.WithCancel(context.Background())
var err error
m.listener, err = m.tcpLC.Listen(m.ctx, "tcp", "[::]:0")
if err != nil {
panic(err)
}
m.log.Println("Listening on", m.listener.Addr())
go m.accept(m.listener)
go func() {
for {
select {
case <-m.ctx.Done():
return
default:
}
intfs, err := net.Interfaces()
if err != nil {
panic(err)
}
for _, intf := range intfs {
unsuitable := intf.Flags&net.FlagUp == 0 ||
intf.Flags&net.FlagMulticast == 0 ||
intf.Flags&net.FlagPointToPoint != 0
if v, ok := m.interfaces.Load(intf.Name); ok {
if unsuitable {
mi := v.(*multicastInterface)
mi.cancel()
m.interfaces.Delete(intf.Name)
}
} else {
if !unsuitable {
ctx, cancel := context.WithCancel(context.Background())
mi := &multicastInterface{ctx, cancel, intf}
go m.start(mi)
}
}
}
time.Sleep(time.Second * 10)
}
}()
}
func (m *Multicast) Stop() {
if !m.started.CAS(true, false) {
return
}
if m.cancel != nil {
m.cancel()
}
if m.listener != nil {
m.listener.Close()
}
}
func (m *Multicast) accept(listener net.Listener) {
for {
conn, err := m.listener.Accept()
if err != nil {
m.log.Println("m.listener.Accept:", err)
return
}
tcpaddr, ok := conn.RemoteAddr().(*net.TCPAddr)
if !ok {
m.log.Println("Not TCPAddr")
return
}
if !m.started.Load() {
return
}
if _, err := m.r.AuthenticatedConnect(conn, tcpaddr.Zone, router.PeerTypeMulticast); err != nil {
//m.log.Println("m.s.AuthenticatedConnect:", err)
continue
}
}
}
func (m *Multicast) start(intf *multicastInterface) {
groupAddrPort := fmt.Sprintf("%s:%d", MulticastGroupAddr, MulticastGroupPort)
addr, err := net.ResolveUDPAddr("udp6", groupAddrPort)
if err != nil {
//m.log.Printf("net.ResolveUDPAddr (%s): %s, ignoring interface\n", intf.Name, err)
return
}
listenString := fmt.Sprintf("[::]:%d", MulticastGroupPort)
conn, err := m.udpLC.ListenPacket(m.ctx, "udp6", listenString)
if err != nil {
//m.log.Printf("lc.ListenPacket (%s): %s, ignoring interface\n", intf.Name, err)
return
}
sock := ipv6.NewPacketConn(conn)
if err := sock.JoinGroup(&intf.Interface, addr); err != nil {
//m.log.Printf("sock.JoinGroup (%s): %s, ignoring interface\n", intf.Name, err)
return
}
addr.Zone = intf.Name
ifaddrs, err := intf.Addrs()
if err != nil {
//m.log.Printf("intf.Addrs (%s): %s, ignoring interface\n", intf.Name, err)
return
}
var srcaddr net.IP
for _, ifaddr := range ifaddrs {
srcaddr, _, err = net.ParseCIDR(ifaddr.String())
if err != nil {
continue
}
if !srcaddr.IsLinkLocalUnicast() || srcaddr.To4() != nil {
continue
}
break
}
if srcaddr == nil {
return
}
m.log.Printf("Multicast discovery enabled on %s (%s)\n", intf.Name, srcaddr.String())
m.interfaces.Store(intf.Name, intf)
go m.advertise(intf, conn, addr)
go m.listen(intf, conn, &net.TCPAddr{
IP: srcaddr,
Zone: addr.Zone,
})
}
func (m *Multicast) advertise(intf *multicastInterface, conn net.PacketConn, addr net.Addr) {
defer m.interfaces.Delete(intf.Name)
//defer m.log.Println("Stop advertising on", intf.Name)
tcpaddr, _ := m.listener.Addr().(*net.TCPAddr)
portBytes := make([]byte, 2)
binary.BigEndian.PutUint16(portBytes, uint16(tcpaddr.Port))
ticker := time.NewTicker(time.Second * 5)
first := make(chan struct{}, 1)
first <- struct{}{}
ourPublicKey := m.r.PublicKey()
for {
select {
case <-m.ctx.Done():
return
case <-intf.context.Done():
return
case <-ticker.C:
case <-first:
}
_, err := conn.WriteTo(
append(ourPublicKey[:], portBytes...),
addr,
)
if err != nil {
//m.log.Println("conn.WriteTo:", err)
intf.cancel()
continue
}
}
}
func (m *Multicast) listen(intf *multicastInterface, conn net.PacketConn, srcaddr net.Addr) {
defer m.interfaces.Delete(intf.Name)
//defer m.log.Println("Stop listening on", intf.Name)
dialer := m.dialer
//dialer.LocalAddr = srcaddr
dialer.Control = m.tcpOptions
buf := make([]byte, 512)
ourPublicKey := m.r.PublicKey()
neighborKey := types.PublicKey{}
publicKey := buf[:ed25519.PublicKeySize]
listenPort := buf[ed25519.PublicKeySize : ed25519.PublicKeySize+2]
for {
select {
case <-m.ctx.Done():
return
case <-intf.context.Done():
return
default:
}
n, addr, err := conn.ReadFrom(buf)
if err != nil || n < ed25519.PublicKeySize+2 {
//m.log.Println("conn.ReadFrom:", err)
intf.cancel()
continue
}
copy(neighborKey[:], publicKey)
if neighborKey.EqualTo(ourPublicKey) {
continue
}
udpaddr, ok := addr.(*net.UDPAddr)
if !ok {
continue
}
if m.r.IsConnected(neighborKey, udpaddr.Zone) {
continue
}
tcpaddr := &net.TCPAddr{
IP: udpaddr.IP,
Port: int(binary.BigEndian.Uint16(listenPort)),
Zone: udpaddr.Zone,
}
if !m.started.Load() {
return
}
peer, err := dialer.Dial("tcp6", tcpaddr.String())
if err != nil {
//m.log.Println("dialer.Dial:", err)
continue
}
if !m.started.Load() {
peer.Close()
return
}
if _, err := m.r.AuthenticatedConnect(peer, udpaddr.Zone, router.PeerTypeMulticast); err != nil {
m.log.Println("m.s.AuthenticatedConnect:", err)
continue
}
}
}

62
multicast/platform_darwin.go Normal file
View file

@ -0,0 +1,62 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// +build darwin
package multicast
import "C"
import (
"fmt"
"syscall"
"golang.org/x/sys/unix"
)
func (m *Multicast) multicastStarted() { // nolint:unused
}
func (m *Multicast) udpOptions(network string, address string, c syscall.RawConn) error {
var reuseport error
var recvanyif error
control := c.Control(func(fd uintptr) {
reuseport = unix.SetsockoptInt(int(fd), unix.SOL_SOCKET, unix.SO_REUSEPORT, 1)
recvanyif = unix.SetsockoptInt(int(fd), syscall.SOL_SOCKET, 0x1104, 1) // SO_RECV_ANYIF
})
switch {
case reuseport != nil:
return fmt.Errorf("SO_REUSEPORT: %w", reuseport)
case recvanyif != nil:
return fmt.Errorf("SO_RECV_ANYIF: %w", recvanyif)
default:
return control
}
}
func (m *Multicast) tcpOptions(network string, address string, c syscall.RawConn) error {
var recvanyif error
control := c.Control(func(fd uintptr) {
recvanyif = unix.SetsockoptInt(int(fd), syscall.SOL_SOCKET, 0x1104, 1) // SO_RECV_ANYIF
})
switch {
case recvanyif != nil:
return fmt.Errorf("SO_RECV_ANYIF: %w", recvanyif)
default:
return control
}
}

45
multicast/platform_other.go Normal file
View file

@ -0,0 +1,45 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// +build !darwin
package multicast
import (
"fmt"
"syscall"
"golang.org/x/sys/unix"
)
func (m *Multicast) multicastStarted() {
}
func (m *Multicast) udpOptions(network string, address string, c syscall.RawConn) error {
var reuseport error
control := c.Control(func(fd uintptr) {
reuseport = unix.SetsockoptInt(int(fd), unix.SOL_SOCKET, unix.SO_REUSEPORT, 1)
})
switch {
case reuseport != nil:
return fmt.Errorf("SO_REUSEPORT: %w", reuseport)
default:
return control
}
}
func (m *Multicast) tcpOptions(network string, address string, c syscall.RawConn) error {
return nil
}

61
router/callback.go Normal file
View file

@ -0,0 +1,61 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
import (
"sync"
"github.com/matrix-org/pinecone/types"
)
type callbacks struct {
r *Router
mutex sync.RWMutex
connected func(port types.SwitchPortID, publickey types.PublicKey, peertype int)
disconnected func(port types.SwitchPortID, publickey types.PublicKey, peertype int, err error)
}
// SetConnectedCallback defines a function that will be called when a new peer
// connection is successfully established.
func (r *Router) SetConnectedCallback(f func(port types.SwitchPortID, publickey types.PublicKey, peertype int)) {
r.callbacks.mutex.Lock()
defer r.callbacks.mutex.Unlock()
r.callbacks.connected = f
}
// SetDisconnectedCallback defines a function that will be called either when
// an existing peer connection disconnects. An error field contains additional
// information about why the peering connection was closed.
func (r *Router) SetDisconnectedCallback(f func(port types.SwitchPortID, publickey types.PublicKey, peertype int, err error)) {
r.callbacks.mutex.Lock()
defer r.callbacks.mutex.Unlock()
r.callbacks.disconnected = f
}
func (c *callbacks) onConnected(port types.SwitchPortID, publickey types.PublicKey, peertype int) {
c.mutex.RLock()
defer c.mutex.RUnlock()
if c.connected != nil {
c.connected(port, publickey, peertype)
}
}
func (c *callbacks) onDisconnected(port types.SwitchPortID, publickey types.PublicKey, peertype int, err error) {
c.mutex.RLock()
defer c.mutex.RUnlock()
if c.disconnected != nil {
c.disconnected(port, publickey, peertype, err)
}
}

168
router/conn.go Normal file
View file

@ -0,0 +1,168 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
import (
"fmt"
"net"
"time"
"github.com/matrix-org/pinecone/types"
)
// SourceAddr implements net.Addr, containing a source-routed
// path to another node.
type SourceAddr struct {
types.SwitchPorts
}
func (a *SourceAddr) Network() string {
return "ps"
}
func (a *SourceAddr) String() string {
return fmt.Sprintf("path %v", a.SwitchPorts)
}
// GreedyAddr implements net.Addr, containing a greedy-routed
// set of destination coordinates to another node.
type GreedyAddr struct {
types.SwitchPorts
}
func (a *GreedyAddr) Network() string {
return "pg"
}
func (a *GreedyAddr) String() string {
return fmt.Sprintf("coords %v", a.SwitchPorts)
}
// ReadFrom reads the next packet that was delivered to this
// node over the Pinecone network. Only traffic packets will
// be returned here - no protocol messages will be included.
// The net.Addr returned will contain the appropriate return
// path based on the mechanism used to deliver the packet.
// If the packet was delivered using greedy routing, then the
// net.Addr will contain the source coordinates. If the packet
// was delivered using source routing, then the net.Addr will
// contain the source-routed path back to the sender.
func (r *Router) ReadFrom(p []byte) (n int, addr net.Addr, err error) {
frame := <-r.recv
switch frame.Type {
case types.TypeGreedy:
addr = &GreedyAddr{frame.Source}
case types.TypeSource:
addr = &SourceAddr{frame.Source} // TODO: should get the remainder of the path
case types.TypeVirtualSnakeBootstrap:
addr = &frame.SourceKey
case types.TypeVirtualSnake:
addr = &frame.SourceKey
default:
r.log.Println("Not expecting non-source/non-greedy frame")
return
}
n = len(frame.Payload)
copy(p, frame.Payload)
return
}
// WriteTo sends a packet into the Pinecone network. The
// packet will be sent as a traffic packet. The net.Addr must
// be one of the Pinecone address types (e.g. GreedyAddr or
// SourceAddr), as this will dictate the method of delivery
// used to forward the packet.
func (r *Router) WriteTo(p []byte, addr net.Addr) (n int, err error) {
timer := time.NewTimer(time.Second * 5)
defer func() {
if !timer.Stop() {
<-timer.C
}
}()
switch ga := addr.(type) {
case *GreedyAddr:
select {
case <-timer.C:
return 0, fmt.Errorf("router appears to be deadlocked")
case r.send <- types.Frame{
Version: types.Version0,
Type: types.TypeGreedy,
Destination: ga.SwitchPorts,
Source: r.Coords(),
Payload: p,
}:
return len(p), nil
}
case *SourceAddr:
select {
case <-timer.C:
return 0, fmt.Errorf("router appears to be deadlocked")
case r.send <- types.Frame{
Version: types.Version0,
Type: types.TypeSource,
Destination: ga.SwitchPorts,
Payload: p,
}:
return len(p), nil
}
case *types.PublicKey:
select {
case <-timer.C:
return 0, fmt.Errorf("router appears to be deadlocked")
case r.send <- types.Frame{
Version: types.Version0,
Type: types.TypeVirtualSnake,
DestinationKey: *ga,
SourceKey: r.PublicKey(),
Payload: p,
}:
return len(p), nil
}
default:
err = fmt.Errorf("unknown address type")
return
}
}
// LocalAddr returns a net.Addr containing the greedy routing
// coordinates for this node.
func (r *Router) LocalAddr() net.Addr {
public := r.PublicKey()
return &public
}
// SetDeadline is not implemented.
func (r *Router) SetDeadline(t time.Time) error {
return nil
}
// SetReadDeadline is not implemented.
func (r *Router) SetReadDeadline(t time.Time) error {
return nil
}
// SetWriteDeadline is not implemented.
func (r *Router) SetWriteDeadline(t time.Time) error {
return nil
}

389
router/dht.go Normal file
View file

@ -0,0 +1,389 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
import (
"context"
"crypto/rand"
"errors"
"fmt"
"net"
"sort"
"sync"
"time"
"github.com/matrix-org/pinecone/types"
"github.com/matrix-org/pinecone/util"
)
type dhtEntry interface {
PublicKey() types.PublicKey
Coordinates() types.SwitchPorts
SeenRecently() bool
}
type dht struct {
r *Router
finger map[types.PublicKey]dhtEntry
fingerMutex sync.RWMutex
sorted []dhtEntry
sortedMutex sync.RWMutex
requests sync.Map
}
// setup sets up the DHT struct.
func newDHT(r *Router) *dht {
d := &dht{
r: r,
finger: make(map[types.PublicKey]dhtEntry),
}
return d
}
// table returns a set of known DHT neighbours to try querying.
func (d *dht) table() []dhtEntry {
d.sortedMutex.RLock()
defer d.sortedMutex.RUnlock()
results := make([]dhtEntry, 0, len(d.sorted))
for _, n := range d.sorted {
if n.SeenRecently() {
results = append(results, n)
}
}
if len(results) > 8 {
return results[:8]
}
return results
}
// getCloser returns a list of all nodes that we know about,
// either from the finger table or from the predecessor or
// successor which are closer to the given public key. It is
// possible for this list to be empty if we don't know of any
// closer nodes.
func (d *dht) getCloser(k types.PublicKey) []dhtEntry {
results := append([]dhtEntry{}, d.sorted...)
sort.SliceStable(results, func(i, j int) bool {
return util.DHTOrdered(d.r.public, results[i].PublicKey(), results[j].PublicKey())
})
return results
}
// search starts an iterative DHT search. It will start
// the search by looking at nodes which we know about, and
// we will continue to ask nodes as we get closer to our
// destination. This function blocks for the duration of
// the search. The address returned will be a set of coords
// for greedy routing.
func (d *dht) search(ctx context.Context, pk types.PublicKey, stopshort bool) (types.PublicKey, net.Addr, error) {
// If the search is for our own key then don't do anything
// as we already know what our own coordinates are.
if d.r.public.EqualTo(pk) && !stopshort {
return pk, &GreedyAddr{d.r.Coords()}, nil
}
// Each time we visit a node, we'll insert it into this
// map. We specifically put our own key in here because
// we shouldn't ever be directed back to ourselves as a
// part of a search.
var visited map[types.PublicKey]struct{}
// The nexthop function processes the next step of the
// search by contacting the next hop. This is recursive
// and will continue until either there are no branches
// to follow or we've reached our destination.
var nexthop func(ctx context.Context, sc types.SwitchPorts, via, pk types.PublicKey) (types.PublicKey, types.SwitchPorts, error)
nexthop = func(ctx context.Context, sc types.SwitchPorts, via, pk types.PublicKey) (types.PublicKey, types.SwitchPorts, error) {
// Check that we haven't exceeded the supplied context.
select {
case <-ctx.Done():
return via, nil, ctx.Err()
default:
}
// Check that we haven't already visited this node before.
// If we have then we won't contact it again.
if _, ok := visited[via]; ok {
return via, nil, fmt.Errorf("this node has already been visited")
}
// Mark the node as visited.
visited[via] = struct{}{}
// Send the request to the remote node. This will hopefully
// return the node's own public key and a list of closer
// nodes.
pubkey, results, err := d.request(sc, pk)
if err != nil {
return via, nil, fmt.Errorf("d.request: %w", err)
}
// If the node's public key is equal to the public key that
// we're looking for, then we've successfully completed the
// search! Return the coordinates that we contacted.
if pubkey.EqualTo(pk) {
return pubkey, sc, nil
}
// Sort the resulting closer nodes by distance to the key
// that we're searching for. This means we'll try nodes that
// are closer in keyspace first, to hopefully shorten the
// path that the search takes as far as possible. This is
// also important because it means that, if the remote node
// tried to send us deliberately out of our way, that we
// will not try those distant nodes until we've at least
// tried better ones first.
sort.SliceStable(results, func(i, j int) bool {
return util.DHTOrdered(pk, results[i].PublicKey, results[j].PublicKey)
})
// Process each of the results.
for _, result := range results {
if result.PublicKey.EqualTo(pk) && stopshort {
return via, sc, nil
}
if k, c, err := nexthop(ctx, result.Coordinates.Copy(), result.PublicKey, pk); err == nil {
// If there's no error then that means we've successfully
// reached the destination node. Return the coordinates.
// This will be passed back up the stack to the caller.
return k, c, nil
}
}
// If we've reached this point then none of the nodes that
// were given to us returned satisfactory results. It may
// be because they didn't know any closer nodes, or they
// timed out/just didn't respond in a timely fashion.
return via, nil, errors.New("search has reached dead end")
}
// Start the search by looking at our own predecessor,
// successor and finger table.
initial := d.table()
sort.SliceStable(initial, func(i, j int) bool {
return util.DHTOrdered(pk, initial[i].PublicKey(), initial[j].PublicKey())
})
for _, n := range initial {
// Each time we start a search using one of our direct
// neighbors, we'll reset the list of visited nodes. This
// means that we don't accidentally kill searches based
// on the results of a previous one. We insert our own
// public key here so that if anyone refers us back to
// ourselves, we will abort the search via that peer.
visited = map[types.PublicKey]struct{}{
d.r.public: {},
}
// Send the search via this peer. This is a recursive
// function so will block until either the search fails
// or completes. If the search fails, continue onto the
// next peer and try that instead.
key, found, err := nexthop(ctx, n.Coordinates(), n.PublicKey(), pk)
if err != nil {
continue
}
// We successfully found the destination, so return the
// greedy coordinates as a net.Addr.
return key, &GreedyAddr{found}, nil
}
return types.PublicKey{}, nil, errors.New("search failed via all known nodes")
}
// request sends a DHT query to a remote node on the network. To
// do this we need to know what the coordinates of the remote node
// are. The function will return the public key of the node queried
// and any closer DHT nodes that they know about, if possible.
func (d *dht) request(coords types.SwitchPorts, pk types.PublicKey) (types.PublicKey, []types.DHTNode, error) {
// Create a new request context.
dhtReq, requestID := d.newRequest()
defer dhtReq.cancel()
defer close(dhtReq.ch)
defer d.requests.Delete(dhtReq.id)
// Build the request query.
req := types.DHTQueryRequest{}
copy(req.PublicKey[:], pk[:])
copy(req.RequestID[:], requestID[:])
// Marshal it into binary so that we can send the request out
// to the network.
var buffer [65535]byte
n, err := req.MarshalBinary(buffer[:])
if err != nil {
return types.PublicKey{}, nil, fmt.Errorf("res.MarshalBinary: %w", err)
}
// Send the request frame to the switch. The switch will then
// forward it onto the destination as appropriate.
d.r.send <- types.Frame{
Source: d.r.Coords(),
Destination: coords,
Type: types.TypeDHTRequest,
Payload: buffer[:n],
}
// Wait for a response that matches our search ID, or for the
// timeout to kick in instead.
select {
case res := <-dhtReq.ch:
return res.PublicKey, res.Results, nil
case <-dhtReq.ctx.Done():
return types.PublicKey{}, nil, fmt.Errorf("request timed out")
}
}
// dhtRequestContext represents an ongoing DHT request.
type dhtRequestContext struct {
id [8]byte
ctx context.Context
cancel context.CancelFunc
ch chan types.DHTQueryResponse
}
// newRequest creates a new dhtRequestContext and returns it, along
// with the randomised ID for the search.
func (d *dht) newRequest() (*dhtRequestContext, [8]byte) {
var requestID [8]byte
for {
// Try generating a random search ID.
n, err := rand.Read(requestID[:8])
if n != 8 || err != nil {
panic("failed to generate new request ID")
}
// If we've picked a search ID that's already in use then
// keep trying until we hit one that is new.
if _, ok := d.requests.Load(requestID); ok {
continue
}
// Build a new timeout and response channel. If we get a
// DHT response back, it will be sent into the matching
// channel.
ctx, cancel := context.WithTimeout(context.Background(), time.Second*5)
ch := make(chan types.DHTQueryResponse, 1)
req := &dhtRequestContext{requestID, ctx, cancel, ch}
d.requests.Store(requestID, req)
return req, requestID
}
}
func (d *dht) insertNode(node dhtEntry) {
id := node.PublicKey()
// TODO: Check if the node is important to the functioning
// of the DHT, e.g. by being in optimal ring positions to
// reduce the path of the search.
d.fingerMutex.Lock()
if n, ok := d.finger[id]; ok {
if _, ok = n.(*Peer); !ok {
d.finger[id] = node
}
} else {
d.finger[id] = node
}
d.fingerMutex.Unlock()
d.rebuildSorted()
}
func (d *dht) deleteNode(id types.PublicKey) {
d.fingerMutex.Lock()
delete(d.finger, id)
d.fingerMutex.Unlock()
d.rebuildSorted()
}
func (d *dht) rebuildSorted() {
d.sortedMutex.Lock()
defer d.sortedMutex.Unlock()
d.sorted = make([]dhtEntry, 0, len(d.finger))
d.fingerMutex.RLock()
for _, n := range d.finger {
d.sorted = append(d.sorted, n)
}
d.fingerMutex.RUnlock()
sort.SliceStable(d.sorted, func(i, j int) bool {
return util.DHTOrdered(d.r.public, d.sorted[i].PublicKey(), d.sorted[j].PublicKey())
})
}
// onDHTRequest is called when the router receives a DHT request.
func (d *dht) onDHTRequest(req *types.DHTQueryRequest, from types.SwitchPorts) {
// Build a response.
res := types.DHTQueryResponse{
RequestID: req.RequestID,
}
copy(res.PublicKey[:], d.r.public[:])
// Look up all nodes that we know about that are closer to
// the public key being searched.
for _, f := range d.getCloser(req.PublicKey) {
node := types.DHTNode{
PublicKey: f.PublicKey(),
Coordinates: f.Coordinates(),
}
res.Results = append(res.Results, node)
}
// Marshal the response into binary format so we can send it
// back.
var buffer [65535]byte
n, err := res.MarshalBinary(buffer[:], d.r.private[:])
if err != nil {
fmt.Println("Failed to sign DHT response:", err)
return
}
// Send the DHT response back to the requestor.
d.r.send <- types.Frame{
Source: d.r.Coords(),
Destination: from,
Type: types.TypeDHTResponse,
Payload: buffer[:n],
}
}
// onDHTResponse is called when the router receives a DHT response.
func (d *dht) onDHTResponse(res *types.DHTQueryResponse, from types.SwitchPorts) {
/*
ctx, cancel := context.WithCancel(d.r.context)
d.insertNode(&dhtNode{
ctx: ctx,
cancel: cancel,
public: res.PublicKey,
coords: from.Copy(),
lastseen: time.Now(),
})
*/
req, ok := d.requests.Load(res.RequestID)
if ok {
dhtReq, ok := req.(*dhtRequestContext)
if ok && dhtReq.id == res.RequestID {
dhtReq.ch <- *res
d.requests.Delete(res.RequestID)
}
}
}

180
router/local.go Normal file
View file

@ -0,0 +1,180 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
import (
"bytes"
"net"
"github.com/matrix-org/pinecone/types"
)
// The writer goroutine is responsible for sending traffic from
// the router to the switch.
func (r *Router) writer(conn net.Conn) {
buf := make([]byte, 65535*3+12)
for {
select {
case <-r.context.Done():
return
default:
frame := <-r.send
frame.Version = types.Version0
n, err := frame.MarshalBinary(buf)
if err != nil {
r.log.Printf("frame.MarshalBinary: %s\n", err)
continue
}
if !bytes.Equal(buf[:len(types.FrameMagicBytes)], types.FrameMagicBytes) {
r.log.Println("Should be sending magic bytes", types.FrameMagicBytes)
continue
}
if _, err = conn.Write(buf[:n]); err != nil {
r.log.Println("s.conn.Write:", err)
continue
}
}
}
}
// The reader goroutine is responsible for receiving traffic
// for the router from the switch.
func (r *Router) reader(conn net.Conn) {
buf := make([]byte, 65535*3+12)
var frame types.Frame
for {
select {
case <-r.context.Done():
return
default:
n, err := conn.Read(buf)
if err != nil {
r.log.Println("s.conn.Read:", err)
continue
}
if _, err = frame.UnmarshalBinary(buf[:n]); err != nil {
r.log.Printf("frame.UnmarshalBinary: %s\n", err)
continue
}
switch frame.Type {
case types.TypeGreedy:
// If the frame doesn't appear as if it's meant to be for
// us then we'll drop it.
if !frame.Destination.EqualTo(r.Coords()) {
r.log.Println("Router received frame that isn't for us")
continue
}
r.recv <- frame
case types.TypeVirtualSnake:
// If the frame doesn't appear as if it's meant to be for
// us then we'll drop it.
if !frame.DestinationKey.EqualTo(r.PublicKey()) {
r.log.Println("Router received frame that isn't for us")
continue
}
r.recv <- frame
case types.TypeSource:
// Check if the source path seems to be finished.
if len(frame.Destination) > 0 {
if frame.Destination[0] != 0 {
r.log.Println("Dropping frame that has invalid next-port")
continue
}
frame.Destination = frame.Destination[1:]
}
r.recv <- frame
case types.TypeDHTRequest:
var request types.DHTQueryRequest
if _, err := request.UnmarshalBinary(frame.Payload); err != nil {
r.log.Println("DHTQueryRequest.MarshalBinary:", err)
continue
}
r.dht.onDHTRequest(&request, frame.Source)
case types.TypeDHTResponse:
var response types.DHTQueryResponse
if _, err := response.UnmarshalBinary(frame.Payload); err != nil {
r.log.Println("DHTQueryResponse.UnmarshalBinary:", err)
continue
}
r.dht.onDHTResponse(&response, frame.Source)
case types.TypePathfind, types.TypeVirtualSnakePathfind:
if len(frame.Payload) == 0 {
continue
}
var pathfind types.Pathfind
if _, err := pathfind.UnmarshalBinary(frame.Payload); err != nil {
r.log.Println("pathfind.UnmarshalBinary:", err)
continue
}
if pathfind.Boundary == 0 {
// The search has been sent to us. Now let's set the boundary and
// send it back. This lets the other end work out how much of the
// body was the path here and how much of it was the path back.
signed, err := pathfind.Sign(r.private[:], 0)
if err != nil {
r.log.Println("pathfind.Sign:", err)
continue
}
signed.Boundary = uint8(len(signed.Signatures))
var buffer [65535]byte
n, err := signed.MarshalBinary(buffer[:])
if err != nil {
r.log.Println("signed.MarshalBinary:", err)
continue
}
switch frame.Type {
case types.TypePathfind:
r.send <- types.Frame{
Destination: frame.Source,
Source: frame.Destination,
Type: types.TypePathfind,
Payload: buffer[:n],
}
case types.TypeVirtualSnakePathfind:
r.send <- types.Frame{
DestinationKey: frame.SourceKey,
SourceKey: frame.DestinationKey,
Type: types.TypeVirtualSnakePathfind,
Payload: buffer[:n],
}
}
} else {
// This is a response to a search that we sent out. It will contain
// both the path we took to the destination (before the boundary)
// and the return path (after the boundary). We can pick which of
// the routes was shorter to reduce stretch.
if len(pathfind.Signatures) < int(pathfind.Boundary) {
continue
}
switch frame.Type {
case types.TypePathfind:
r.pathfinder.onPathfindResponse(&GreedyAddr{frame.Source}, pathfind)
case types.TypeVirtualSnakePathfind:
r.pathfinder.onPathfindResponse(frame.SourceKey, pathfind)
}
}
}
}
}
}

118
router/manhole.go Normal file
View file

@ -0,0 +1,118 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
import (
"encoding/json"
"fmt"
"net/http"
"net/http/pprof"
"time"
"github.com/matrix-org/pinecone/types"
)
// DEBUG STATISTICS
func (r *Router) startManhole() {
mux := http.NewServeMux()
mux.Handle("/debug/pprof/goroutine", pprof.Handler("goroutine"))
mux.Handle("/debug/pprof/heap", pprof.Handler("heap"))
mux.Handle("/debug/pprof/profile", pprof.Handler("profile"))
mux.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
results := map[string]interface{}{}
results["self"] = map[string]string{
"public_key": r.PublicKey().String(),
"coords": r.Coords().String(),
}
results["tree"] = map[string]string{
"root": r.tree.Root().RootPublicKey.String(),
}
ports := map[types.SwitchPortID]map[string]interface{}{}
for _, p := range r.activePorts() {
p.mutex.RLock()
ports[p.port] = map[string]interface{}{
"zone": p.zone,
"public_key": p.public.String(),
"coords": p.coords.String(),
"peer_type": p.peertype,
"queued_proto": fmt.Sprintf("%d/%d", len(p.protoOut), cap(p.protoOut)),
"queued_traffic": fmt.Sprintf("%d/%d", len(p.trafficOut), cap(p.trafficOut)),
"tx_proto_successful": p.statistics.txProtoSuccessful.Load(),
"tx_proto_dropped": p.statistics.txProtoDropped.Load(),
"tx_traffic_successful": p.statistics.txTrafficSuccessful.Load(),
"tx_traffic_dropped": p.statistics.txTrafficDropped.Load(),
"rx_dropped_no_dest": p.statistics.rxDroppedNoDestination.Load(),
}
p.mutex.RUnlock()
}
results["ports"] = ports
r.snake.tableMutex.RLock()
r.snake.descendingMutex.RLock()
r.snake.ascendingMutex.RLock()
results["snake"] = map[string]interface{}{
"predecessor": r.snake.descending,
"successor": r.snake.ascending,
"table": r.snake.table,
}
b, err := json.MarshalIndent(results, "", " ")
r.snake.ascendingMutex.RUnlock()
r.snake.descendingMutex.RUnlock()
r.snake.tableMutex.RUnlock()
if err != nil {
w.WriteHeader(500)
r.log.Println("Failed to marshal manhole JSON:", err)
return
}
_, _ = w.Write(b)
})
if err := http.ListenAndServe(":64000", mux); err != nil {
r.log.Println("Failed to setup manhole:", err)
}
}
func (e *virtualSnakeNeighbour) MarshalJSON() ([]byte, error) {
out := map[string]interface{}{
"public_key": e.PublicKey.String(),
"port": e.Port,
"last_seen": time.Since(e.LastSeen).String(),
}
return json.Marshal(out)
}
func (e virtualSnakeTable) MarshalJSON() ([]byte, error) {
out := []map[string]interface{}{}
for key, value := range e {
entry := map[string]interface{}{
"source_key": key.String(),
"source_port": value.SourcePort,
"last_seen": time.Since(value.LastSeen).String(),
}
if time.Since(value.LastSeen) > virtualSnakePathExpiryPeriod {
entry["expired"] = true
}
out = append(out, entry)
}
return json.Marshal(out)
}
func (e virtualSnakeEntry) MarshalJSON() ([]byte, error) {
out := map[string]interface{}{
"last_seen": time.Since(e.LastSeen).String(),
"source_port": e.SourcePort,
}
return json.Marshal(out)
}

76
router/nexthop.go Normal file
View file

@ -0,0 +1,76 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
import (
"github.com/matrix-org/pinecone/types"
)
// getNextHops is called to determine the next-hop ports for a given
// packet. This function will determine the frame type and automatically
// call the correct function by routing scheme. We might end up using:
// * greedy routing following the spanning tree, for routing to/from
// spanning tree coordinates;
// * source routing, where the path is already known ahead of time;
// * virtual snake routing, for most traffic.
// This function also ensures that root announcements, boostrap and
// path setup messages are processed as necessary.
func (p *Peer) getNextHops(frame *types.Frame, from types.SwitchPortID) types.SwitchPorts {
switch frame.Type {
case types.TypeSTP:
if p.port == routerPort {
return p.r.getSourceRoutedNextHop(p, frame)
} else {
p.r.handleAnnouncement(p, frame)
}
case types.TypeVirtualSnakeBootstrap:
nextHops := p.r.snake.getVirtualSnakeNextHop(p, frame.DestinationKey, true)
if nextHops.EqualTo(types.SwitchPorts{0}) {
p.r.snake.handleBootstrap(p, frame)
} else {
return nextHops
}
case types.TypeVirtualSnakeBootstrapACK:
nextHops := p.r.getGreedyRoutedNextHop(p, frame)
if nextHops.EqualTo(types.SwitchPorts{0}) {
p.r.snake.handleBootstrapACK(p, frame)
} else {
return nextHops
}
case types.TypeVirtualSnakeSetup:
nextHops := p.r.getGreedyRoutedNextHop(p, frame)
defer p.r.snake.handleSetup(p, frame, nextHops)
return nextHops
case types.TypeVirtualSnake, types.TypeVirtualSnakePathfind:
return p.r.snake.getVirtualSnakeNextHop(p, frame.DestinationKey, false)
case types.TypeGreedy, types.TypePathfind, types.TypeDHTRequest, types.TypeDHTResponse:
return p.r.getGreedyRoutedNextHop(p, frame)
case types.TypeSource:
if p.port != 0 {
frame.UpdateSourceRoutedPath(p.port)
}
return p.r.getSourceRoutedNextHop(p, frame)
}
// If we've got to this point, the packet type is unknown and
// we don't know what to do with it, so just drop it.
return nil
}

95
router/nexthop_greedy.go Normal file
View file

@ -0,0 +1,95 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
import (
"math"
"github.com/matrix-org/pinecone/types"
)
// getGreedyRoutedNextHop returns next-hops for the given frame.
// If the frame is destined for us locally then a single switch port 0
// will be returned, causing the packet to be handed over to the router
// (as the router is always connected to switch port 0). Otherwise, zero
// or one next-hop ports can be returned.
func (r *Router) getGreedyRoutedNextHop(from *Peer, rx *types.Frame) types.SwitchPorts {
// If it's loopback then don't bother doing anything else.
if rx.Destination.EqualTo(r.Coords()) {
return types.SwitchPorts{0}
}
// Work out how close our own coordinates are to the destination
// message. This is important because we'll only forward a frame
// to a peer that takes the message closer to the destination than
// we are.
ourDist := int64(r.Coords().DistanceTo(rx.Destination))
if ourDist == 0 {
// It's impossible to get closer so there's a pretty good
// chance at this point that the traffic is destined for us.
// Pass it up to the router.
return types.SwitchPorts{0}
}
// Now work out which of our peers takes the message closer.
bestPeer := types.SwitchPortID(0)
bestDist := int64(math.MaxInt64)
for _, p := range r.activePorts() {
// Don't create routing loops.
if p.port == from.port {
continue
}
// Look up the coordinates of the peer.
p.mutex.RLock()
coords := p.coords
p.mutex.RUnlock()
// Work out what the distance across the tree is to that
// peer.
dist := int64(coords.DistanceTo(rx.Destination))
// If the distance is zero, that's because the peer is the
// destination itself.
if dist == 0 {
return []types.SwitchPortID{p.port}
}
// Otherwise, let's see if this peer just happens to be a
// better candidate for the next-hop.
switch {
case dist > ourDist:
// TODO: is this needed?
// This peer will take the traffic further away from our
// own node.
case dist > bestDist:
// This isn't any closer to the destination than our
// current best candidate next-hop.
default:
// This looks like probably the best next-hop candidate we
// have so far.
bestPeer, bestDist = p.port, dist
}
}
// If we've got an eligible next peer, and it doesn't create a
// routing loop by sending the frame back where it came from,
// then return it.
peers := types.SwitchPorts{}
if bestPeer != 0 {
peers = append(peers, bestPeer)
}
return peers
}

73
router/nexthop_source.go Normal file
View file

@ -0,0 +1,73 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
import "github.com/matrix-org/pinecone/types"
const (
routerPort types.SwitchPortID = 0
)
// getSourceRoutedNextHop returns next-hops for the given frame.
// If the frame is destined for us locally then a single switch port 0
// will be returned, causing the packet to be handed over to the router
// (as the router is always connected to switch port 0). Otherwise, zero
// or one next-hop ports can be returned.
func (r *Router) getSourceRoutedNextHop(from *Peer, rx *types.Frame) types.SwitchPorts {
to := types.SwitchPortID(0)
if len(rx.Destination) > 0 {
// The packet has another destination port, so the
// next-hop is basically the next port in the list.
to = types.SwitchPortID(rx.Destination[0])
} else {
// The packet has no more destination ports so it is
// probably for us locally.
return types.SwitchPorts{0}
}
if from.port == routerPort && rx.Type == types.TypeSTP {
// Only allow broadcast frames if they came from the
// local router and they are spanning-tree maintenance
// frames.
return r.getSourceRoutedBroadcastNextHops()
}
if from.port == to {
// Don't allow a packet to be sent back from where it
// came from. This probably means there's been a loop
// somewhere.
return nil
}
if peer := r.ports[to]; !peer.started.Load() || !peer.alive.Load() {
// Don't try to send packets to a port that has nothing
// connected to it or isn't alive.
return nil
}
return []types.SwitchPortID{to}
}
// getSourceRoutedBroadcastNextHops returns a list of next-hops for
// all active switch ports. This is used for spanning tree root
// announcements.
func (r *Router) getSourceRoutedBroadcastNextHops() (peers types.SwitchPorts) {
for _, peer := range r.ports {
if peer.started.Load() {
peers = append(peers, peer.port)
}
}
return
}

440
router/nexthop_virtualsnake.go Normal file
View file

@ -0,0 +1,440 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
import (
"math"
"sync"
"time"
"github.com/matrix-org/pinecone/types"
"github.com/matrix-org/pinecone/util"
"go.uber.org/atomic"
)
const virtualSnakeSetupInterval = time.Second * 16
const virtualSnakePathExpiryPeriod = virtualSnakeSetupInterval * 2
const virtualSnakeNeighExpiryPeriod = virtualSnakePathExpiryPeriod * 2
type virtualSnake struct {
r *Router
maintainInterval atomic.Int32
maintainNow util.Dispatch
table virtualSnakeTable
tableMutex sync.RWMutex
ascending *virtualSnakeNeighbour
ascendingMutex sync.RWMutex
descending *virtualSnakeNeighbour
descendingMutex sync.RWMutex
}
type virtualSnakeTable map[types.PublicKey]virtualSnakeEntry
type virtualSnakeEntry struct {
SourcePort types.SwitchPortID
LastSeen time.Time
}
type virtualSnakeNeighbour struct {
PublicKey types.PublicKey
Port types.SwitchPortID
LastSeen time.Time
Coords types.SwitchPorts
}
func newVirtualSnake(r *Router) *virtualSnake {
snake := &virtualSnake{
r: r,
maintainNow: util.NewDispatch(),
table: make(virtualSnakeTable),
}
go snake.maintain()
return snake
}
// maintain will run continuously on a given interval between
// every 1 second and virtualSnakeSetupInterval seconds, sending
// bootstraps and setup messages as needed.
func (v *virtualSnake) maintain() {
for {
peerCount := v.r.PeerCount(-1)
if peerCount == 0 {
v.maintainInterval.Store(0)
}
exp := math.Exp2(float64(v.maintainInterval.Load()))
after := time.Second * time.Duration(exp)
select {
case <-v.r.context.Done():
return
case <-time.After(after):
case <-v.maintainNow:
}
if peerCount == 0 {
// If there are no peers connected then we don't need
// to do any hard maintenance work.
continue
}
if after < virtualSnakeSetupInterval {
v.maintainInterval.Inc()
}
v.descendingMutex.RLock()
descending := v.descending
v.descendingMutex.RUnlock()
// Send bootstrap messages into the network. Ordinarily we
// would only want to do this when starting up or after a
// predefined interval, but for now we'll continue to send
// them on a regular interval until we can derive some better
// connection state.
switch {
// TODO: case for when the root has changed?
// TODO: case for when our parent has changed?
case descending == nil:
v.maintainInterval.Store(0)
fallthrough
default: // case descending != nil && time.Since(descending.LastSeen) >= virtualSnakeNeighExpiryPeriod:
if !v.r.PublicKey().EqualTo(v.r.RootPublicKey()) {
ts, err := util.SignedTimestamp(v.r.private)
if err != nil {
return
}
v.r.send <- types.Frame{
Type: types.TypeVirtualSnakeBootstrap,
DestinationKey: v.r.PublicKey(), // routes using keys
Source: v.r.Coords(), // used to send back a setup using ygg greedy routing
Payload: ts,
}
}
}
}
}
// portWasDisconnected is called by the router when a peer disconnects
// allowing us to clean up the virtual snake state.
func (t *virtualSnake) portWasDisconnected(port types.SwitchPortID) {
// Check if our descending node was routable via the disconnected
// port. If they were, clear the entry so that the maintain function
// will send new bootstraps to find the next descending node.
t.descendingMutex.Lock()
if p := t.descending; p != nil && p.Port == port {
t.maintainInterval.Store(0)
t.descending = nil
t.maintainNow.Dispatch()
}
t.descendingMutex.Unlock()
// Check if our ascending node was routable via the disconnected
// port. If they were then clear the entry too, it will get
// re-populated the next time we bootstrap.
t.ascendingMutex.Lock()
if s := t.ascending; s != nil && s.Port == port {
t.ascending = nil
t.maintainNow.Dispatch()
}
t.ascendingMutex.Unlock()
// Finally, check if any of our routing table entries are
// via the port in question. If they were then let's nuke those
// too, otherwise we'll be trying to route traffic into black
// holes.
t.tableMutex.Lock()
for key, value := range t.table {
if value.SourcePort == port {
delete(t.table, key)
}
}
t.tableMutex.Unlock()
}
// getVirtualSnakeBootstrapNextHop will return the most relevant port
// for a given destination public key.
func (t *virtualSnake) getVirtualSnakeNextHop(from *Peer, destKey types.PublicKey, bootstrap bool) types.SwitchPorts {
if from.port != 0 && destKey.EqualTo(t.r.public) {
// The traffic didn't originate locally and appears to be for us.
// Send it straight to the router.
return types.SwitchPorts{0}
}
rootKey := t.r.RootPublicKey()
if util.LessThan(rootKey, destKey) {
// The destination key is higher than the root key - this should
// be impossible since the root node should be the highest end of
// the snake. We will therefore just drop the packet.
return nil
}
bestKey, bestPort := t.r.public, types.SwitchPortID(0)
var candidates types.SwitchPorts
var canlength int
if !bootstrap {
candidates, canlength = make(types.SwitchPorts, PortCount), PortCount
}
newCandidate := func(key types.PublicKey, port types.SwitchPortID) {
if !bootstrap && port != bestPort {
canlength--
candidates[canlength] = port
}
bestKey, bestPort = key, port
}
// Start by looking through all ancestors. This includes the root
// node. These routes are effectively ascending paths to higher
// public keys.
if ancestors, rootPort := t.r.tree.Ancestors(); rootPort != 0 {
for _, ancestorKey := range ancestors {
switch {
case !bootstrap && (util.DHTOrdered(bestKey, ancestorKey, destKey) || ancestorKey.EqualTo(destKey)):
newCandidate(ancestorKey, rootPort)
case !util.LessThan(destKey, bestKey) && util.LessThan(destKey, ancestorKey):
newCandidate(ancestorKey, rootPort)
}
}
}
t.tableMutex.RLock()
// Next, look at the routing table. This includes source-side paths
// from setups only.
for dhtKey, entry := range t.table {
if time.Since(entry.LastSeen) > virtualSnakePathExpiryPeriod {
// The routing table entry hasn't been refreshed recently so we'll
// ignore it. This will eventually get picked up and removed from
// the routing table.
continue
}
switch {
case !bootstrap && destKey.EqualTo(dhtKey):
newCandidate(dhtKey, entry.SourcePort)
case util.DHTOrdered(destKey, dhtKey, bestKey):
newCandidate(dhtKey, entry.SourcePort)
}
}
t.tableMutex.RUnlock()
for _, peer := range t.r.activePorts() {
// Finally, look at our direct peers, in case of any of our direct
// peers end up being a better path than anything else we've come up
// with so far.
peer.mutex.RLock()
peerKey := peer.public
peer.mutex.RUnlock()
switch {
case !bootstrap && peerKey.EqualTo(destKey):
newCandidate(peerKey, peer.port)
case util.DHTOrdered(destKey, peerKey, bestKey):
newCandidate(peerKey, peer.port)
}
}
if bootstrap {
return types.SwitchPorts{bestPort}
} else {
return candidates[canlength:]
}
}
// handleBootstrap is called in response to an incoming bootstrap
// packet. It will update the descending information and send a setup
// message if needed.
func (t *virtualSnake) handleBootstrap(from *Peer, rx *types.Frame) {
// Check if the packet has a valid signed timestamp.
if !util.VerifySignedTimestamp(rx.DestinationKey, rx.Payload) {
return
}
var send *types.Frame
defer func() {
// When we reach the end of handleBootstrap, if a response packet
// has been built, then send it into the network. Doing this in a
// defer means we can defer the mutex unlocks but still not hold
// them for longer than needed.
if send != nil {
t.r.send <- *send
}
}()
t.descendingMutex.Lock()
defer t.descendingMutex.Unlock()
switch {
case rx.DestinationKey.EqualTo(t.r.public):
// We received a bootstrap from ourselves. This shouldn't happen,
// so either another node has forwarded it to us incorrectly, or
// a routing loop has occurred somewhere. Don't act on the bootstrap
// in that case.
case t.descending != nil && t.descending.PublicKey.EqualTo(rx.DestinationKey):
// We've received another bootstrap from our direct descending node.
// Just refresh the record and then send back an acknowledgement.
t.descending.Coords = rx.Source
t.descending.Port = from.port
t.descending.LastSeen = time.Now()
ts, err := util.SignedTimestamp(t.r.private)
if err != nil {
return
}
send = &types.Frame{
Destination: rx.Source,
DestinationKey: rx.DestinationKey,
Source: t.r.Coords(),
SourceKey: t.r.PublicKey(),
Type: types.TypeVirtualSnakeBootstrapACK,
Payload: ts,
}
case t.descending != nil && time.Since(t.descending.LastSeen) >= virtualSnakeNeighExpiryPeriod:
// We already have a direct descending node, but we haven't seen it
// recently, so it's quite possible that it has disappeared. We'll
// therefore handle this bootstrap instead. If the original node comes
// back later and is closer to us then we'll end up using it again.
fallthrough
case t.descending == nil && util.LessThan(rx.DestinationKey, t.r.public):
// We don't know about a descending node and at the moment we don't know
// any better candidates, so we'll accept a bootstrap from a node with a
// key lower than ours (so that it matches descending order).
fallthrough
case t.descending != nil && util.DHTOrdered(t.descending.PublicKey, rx.DestinationKey, t.r.public):
// We know about a descending node already but it turns out that this
// new node that we've received a bootstrap from is actually closer to
// us than the previous node. We'll update our record to use the new
// node instead and then send back a bootstrap ACK.
t.descending = &virtualSnakeNeighbour{
PublicKey: rx.DestinationKey,
Port: from.port,
LastSeen: time.Now(),
Coords: rx.Source,
}
ts, err := util.SignedTimestamp(t.r.private)
if err != nil {
return
}
send = &types.Frame{
Destination: rx.Source,
DestinationKey: rx.DestinationKey,
Source: t.r.Coords(),
SourceKey: t.r.PublicKey(),
Type: types.TypeVirtualSnakeBootstrapACK,
Payload: ts,
}
default:
// The bootstrap conditions weren't met. This might just be because
// there's a node out there that hasn't converged to a closer node
// yet, so we'll just ignore the bootstrap.
}
}
func (t *virtualSnake) handleBootstrapACK(from *Peer, rx *types.Frame) {
// Check if the packet has a valid signed timestamp.
if !util.VerifySignedTimestamp(rx.SourceKey, rx.Payload) {
return
}
var send *types.Frame
defer func() {
// When we reach the end of handleBootstrap, if a response packet
// has been built, then send it into the network. Doing this in a
// defer means we can defer the mutex unlocks but still not hold
// them for longer than needed.
if send != nil {
t.r.send <- *send
}
}()
t.ascendingMutex.Lock()
defer t.ascendingMutex.Unlock()
switch {
case rx.SourceKey.EqualTo(t.r.public):
// We received a bootstrap ACK from ourselves. This shouldn't happen,
// so either another node has forwarded it to us incorrectly, or
// a routing loop has occurred somewhere. Don't act on the bootstrap
// in that case.
case t.ascending != nil && t.ascending.PublicKey.EqualTo(rx.SourceKey):
// We've received another bootstrap ACK from our direct ascending node.
// Just refresh the record and then send a new path setup message to
// that node.
t.ascending.Coords = rx.Source
t.ascending.Port = from.port
t.ascending.LastSeen = time.Now()
ts, err := util.SignedTimestamp(t.r.private)
if err != nil {
return
}
send = &types.Frame{
Destination: t.ascending.Coords,
DestinationKey: t.ascending.PublicKey,
SourceKey: t.r.public,
Type: types.TypeVirtualSnakeSetup,
Payload: ts,
}
case t.ascending != nil && time.Since(t.ascending.LastSeen) >= virtualSnakeNeighExpiryPeriod:
// We already have a direct ascending node, but we haven't seen it
// recently, so it's quite possible that it has disappeared. We'll
// therefore handle this bootstrap ACK instead. If the original node comes
// back later and is closer to us then we'll end up using it again.
fallthrough
case t.ascending == nil && util.LessThan(t.r.public, rx.SourceKey):
// We don't know about an ascending node and at the moment we don't know
// any better candidates, so we'll accept a bootstrap ACK from a node with a
// key higher than ours (so that it matches descending order).
fallthrough
case t.ascending != nil && util.DHTOrdered(t.r.public, rx.SourceKey, t.ascending.PublicKey):
// We know about an ascending node already but it turns out that this
// new node that we've received a bootstrap from is actually closer to
// us than the previous node. We'll update our record to use the new
// node instead and then send a new path setup message to it.
t.ascending = &virtualSnakeNeighbour{
PublicKey: rx.SourceKey,
Port: from.port,
LastSeen: time.Now(),
Coords: rx.Source,
}
ts, err := util.SignedTimestamp(t.r.private)
if err != nil {
return
}
send = &types.Frame{
Destination: t.ascending.Coords,
DestinationKey: t.ascending.PublicKey,
SourceKey: t.r.public,
Type: types.TypeVirtualSnakeSetup,
Payload: ts,
}
default:
// The bootstrap ACK conditions weren't met. This might just be because
// there's a node out there that hasn't converged to a closer node
// yet, so we'll just ignore the acknowledgement.
}
}
// handleSetup is called in response to an incoming path setup packet. Note
// that the setup packet isn't necessarily destined for us directly, but is
// instead called for every setup packet being transited through this node.
// It will update the routing table with the new path.
func (t *virtualSnake) handleSetup(from *Peer, rx *types.Frame, nextHops []types.SwitchPortID) {
if rx.SourceKey.EqualTo(t.r.public) {
return
}
t.tableMutex.Lock()
defer t.tableMutex.Unlock()
// Evict old entries.
for k, v := range t.table {
if time.Since(v.LastSeen) > virtualSnakePathExpiryPeriod {
delete(t.table, k)
}
}
// Check if the setup packet has a valid signed timestamp.
if !util.VerifySignedTimestamp(rx.SourceKey, rx.Payload) {
return
}
// Add a new routing table entry.
// TODO: The routing table needs to be bounded by size, so that we don't
// exhaust available system memory trying to maintain network paths. To
// bound the routing table safely, we may want to make sure that we have
// a reasonable spread of routes across keyspace so that we don't create
// any obvious routing holes.
t.table[rx.SourceKey] = virtualSnakeEntry{
LastSeen: time.Now(),
SourcePort: from.port,
}
}

191
router/pathfinder.go Normal file
View file

@ -0,0 +1,191 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
import (
"context"
"fmt"
"net"
"sync"
"github.com/matrix-org/pinecone/types"
)
// The pathfinder is used to build source routes from either greedy or
// snake routing. Generally we don't use this for actual packet routing
// today, although the simulator uses it to determine the path length vs
// real path length. An application using Pinecone as a library can use
// the pathfinder to switch from greedy/snake routing to source routing
// if desired.
type pathfinder struct {
r *Router
requests sync.Map
}
// newPathfinder returns a new pathfinding instance. This can be used to
// perform an active pathfind between two nodes using either greedy or
// snake routing and convert it to
func newPathfinder(r *Router) *pathfinder {
return &pathfinder{
r: r,
}
}
func (p *pathfinder) pathfind(ctx context.Context, addr net.Addr) (net.Addr, error) {
var returnPath types.SwitchPorts
payload := [2]byte{0, 0}
searchContext := p.newPathfind(ctx, addr)
switch a := addr.(type) {
case *GreedyAddr:
if a.SwitchPorts.EqualTo(p.r.Coords()) {
return &SourceAddr{types.SwitchPorts{}}, nil
}
select {
case p.r.send <- types.Frame{
Version: types.Version0,
Type: types.TypePathfind,
Destination: a.SwitchPorts,
Source: p.r.Coords(),
Payload: payload[:],
}:
case <-ctx.Done():
return nil, ctx.Err()
}
select {
case res := <-searchContext.ch:
if out, in := res.res.ReturnPath(false), res.res.ReturnPath(true); len(in) <= len(out) {
returnPath = in
} else {
returnPath = out
}
if p.r.simulator != nil {
if id, err := p.r.simulator.LookupNodeID(a.SwitchPorts); err == nil {
p.r.simulator.ReportDistance(p.r.id, id, int64(len(returnPath)))
}
}
return &SourceAddr{returnPath}, nil
case <-searchContext.ctx.Done():
return nil, searchContext.ctx.Err()
}
case types.PublicKey:
if a.EqualTo(p.r.PublicKey()) {
return &SourceAddr{types.SwitchPorts{}}, nil
}
select {
case p.r.send <- types.Frame{
Version: types.Version0,
Type: types.TypeVirtualSnakePathfind,
DestinationKey: a,
SourceKey: p.r.PublicKey(),
Payload: payload[:],
}:
case <-ctx.Done():
return nil, ctx.Err()
}
select {
case res := <-searchContext.ch:
if out, in := res.res.ReturnPath(false), res.res.ReturnPath(true); len(in) <= len(out) {
returnPath = in
} else {
returnPath = out
}
if p.r.simulator != nil {
if id, err := p.r.simulator.LookupPublicKey(a); err == nil {
p.r.simulator.ReportDistance(p.r.id, id, int64(len(returnPath)))
}
}
return &SourceAddr{returnPath}, nil
case <-searchContext.ctx.Done():
return nil, searchContext.ctx.Err()
}
default:
return nil, fmt.Errorf("expected *GreedyAddr or *GreedySNEKAddr as net.Addr")
}
}
func (r *Router) signPathfind(frame *types.Frame, from, to *Peer) (*types.Frame, error) {
var pathfind types.Pathfind
signedFrame := frame.Copy()
if _, err := pathfind.UnmarshalBinary(frame.Payload); err != nil {
return nil, fmt.Errorf("pathfind.UnmarshalBinary: %w", err)
}
for _, sig := range pathfind.Signatures[pathfind.Boundary:] {
if r.public.EqualTo(sig.PublicKey) {
// The pathfind frame looped
return nil, nil
}
}
port := to.port
if pathfind.Boundary > 0 {
port = from.port
}
if port == 0 {
return frame, nil
}
signed, err := pathfind.Sign(r.private[:], port)
if err != nil {
return nil, fmt.Errorf("pathfind.Sign: %w", err)
}
var buf [65535]byte
n, err := signed.MarshalBinary(buf[:])
if err != nil {
return nil, fmt.Errorf("signed.MarshalBinary: %w", err)
}
signedFrame.Payload = buf[:n]
return signedFrame, nil
}
// pathfindContext represents an ongoing pathfind.
type pathfindContext struct {
id net.Addr
ctx context.Context
ch chan *pathfindResponse
}
type pathfindResponse struct {
from net.Addr
res *types.Pathfind
}
// pathfind creates a new pathfindContext and returns it.
func (p *pathfinder) newPathfind(ctx context.Context, addr net.Addr) *pathfindContext {
// TODO: use something better than a stringified key
if c, ok := p.requests.Load(addr.String()); ok {
return c.(*pathfindContext)
}
// Build a new timeout and response channel. If we get a
// path response back, it will be sent into the matching
// channel.
ch := make(chan *pathfindResponse, 1)
req := &pathfindContext{addr, ctx, ch}
// TODO: It's not nice to generate a map key in this way.
p.requests.Store(addr.String(), req)
return req
}
func (p *pathfinder) onPathfindResponse(addr net.Addr, res types.Pathfind) {
// TODO: use something better than a stringified key
key := addr.String()
if req, ok := p.requests.Load(key); ok {
if dhtReq, ok := req.(*pathfindContext); ok {
dhtReq.ch <- &pathfindResponse{addr, &res}
p.requests.Delete(key)
}
}
}

434
router/peer.go Normal file
View file

@ -0,0 +1,434 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
import (
"bytes"
"context"
"crypto/ed25519"
"encoding/binary"
"errors"
"fmt"
"io"
"sync"
"time"
"github.com/matrix-org/pinecone/types"
"github.com/matrix-org/pinecone/util"
"go.uber.org/atomic"
)
const (
PeerTypeMulticast int = iota
PeerTypeBluetooth
PeerTypeRemote
)
type Peer struct {
r *Router //
port types.SwitchPortID //
started atomic.Bool // worker goroutines started?
alive atomic.Bool // have we received a handshake?
mutex sync.RWMutex // protects everything below this line
zone string //
peertype int //
context context.Context //
cancel context.CancelFunc //
conn util.BufferedRWC // underlying connection to peer
public types.PublicKey //
trafficOut chan *types.Frame // queue traffic message to peer
protoOut chan *types.Frame // queue protocol message to peer
coords types.SwitchPorts //
announcement *rootAnnouncementWithTime //
advertise util.Dispatch // send switch announcement right now
statistics peerStatistics //
}
type peerStatistics struct {
txProtoSuccessful atomic.Uint64
txProtoDropped atomic.Uint64
txTrafficSuccessful atomic.Uint64
txTrafficDropped atomic.Uint64
rxDroppedNoDestination atomic.Uint64
}
func (s *peerStatistics) reset() {
s.txProtoSuccessful.Store(0)
s.txProtoDropped.Store(0)
s.txTrafficSuccessful.Store(0)
s.txTrafficDropped.Store(0)
s.rxDroppedNoDestination.Store(0)
}
func (p *Peer) PublicKey() types.PublicKey {
if p == nil {
return types.PublicKey{}
}
p.mutex.RLock()
defer p.mutex.RUnlock()
return p.public
}
func (p *Peer) Coordinates() types.SwitchPorts {
if p == nil {
return types.SwitchPorts{}
}
p.mutex.RLock()
defer p.mutex.RUnlock()
return p.coords
}
func (p *Peer) SeenRecently() bool {
if last := p.lastAnnouncement(); last != nil {
return time.Since(last.at) < announcementTimeout
}
return false
}
func (p *Peer) lastAnnouncement() *rootAnnouncementWithTime {
if p == nil {
return nil
}
p.mutex.RLock()
defer p.mutex.RUnlock()
return p.announcement
}
func (p *Peer) start() error {
if !p.started.CAS(false, true) {
return errors.New("switch peer is already started")
}
p.alive.Store(false)
go p.reader()
go p.writer()
if p.port != 0 {
p.advertise.Dispatch()
go p.announcer()
}
return nil
}
func (p *Peer) stop() error {
if !p.started.CAS(true, false) {
return errors.New("switch peer is already stopped")
}
p.alive.Store(false)
p.cancel()
p.r.tree.Remove(p)
return p.conn.Close()
}
func (p *Peer) announcer() {
announce:
for {
select {
case <-p.context.Done():
// The switch peer is shutting down.
return
case <-p.advertise:
if !p.started.Load() {
// If the port isn't started then don't bother sending announcements
// to it. There's probably nothing on the other end.
continue announce
}
announcement := p.r.tree.Root()
for _, sig := range announcement.Signatures {
if p.r.public.EqualTo(sig.PublicKey) {
// For some reason the announcement that we want to send already
// includes our signature. This shouldn't really happen but if we
// did send it, other nodes would end up ignoring the announcement
// anyway since it would appear to be a routing loop.
continue announce
}
}
// Sign the announcement.
var err error
announcement, err = announcement.Sign(p.r.private[:], p.port)
if err != nil {
p.r.log.Println("Failed to sign switch announcement:", err)
continue announce
}
var payload [65535]byte
n, err := announcement.MarshalBinary(payload[:])
if err != nil {
p.r.log.Println("Failed to marshal switch announcement:", err)
continue announce
}
frame := types.GetFrame()
frame.Version = types.Version0
frame.Type = types.TypeSTP
frame.Destination = types.SwitchPorts{}
frame.Payload = payload[:n]
select {
case p.protoOut <- frame:
case <-time.After(time.Second):
frame.Done()
p.advertise.Dispatch()
}
}
}
}
func (p *Peer) reader() {
buf := make([]byte, 65535*3+12)
for {
select {
case <-p.context.Done():
// The switch peer is shutting down.
return
default:
var n int
header, err := p.conn.Peek(12)
if err != nil {
if err != io.EOF {
p.r.log.Println("Failed to peek:", err)
}
_ = p.r.Disconnect(p.port, fmt.Errorf("p.conn.Peek: %w", err))
return
}
if !bytes.Equal(header[:4], types.FrameMagicBytes) {
p.r.log.Println(p.port, "traffic had no magic", types.FrameMagicBytes, "bytes", header, types.FrameType(header[1]))
_, _ = p.conn.Discard(1)
continue
}
header = header[4:]
expecting := 0
switch types.FrameType(header[1]) {
case types.TypeVirtualSnakeBootstrap:
payloadLen := int(binary.BigEndian.Uint16(header[2:4]))
coordsLen := int(binary.BigEndian.Uint16(header[4:6]))
expecting = 10 + coordsLen + payloadLen + ed25519.PublicKeySize
case types.TypeVirtualSnakeBootstrapACK:
payloadLen := int(binary.BigEndian.Uint16(header[2:4]))
dstLen := int(binary.BigEndian.Uint16(header[4:6]))
srcLen := int(binary.BigEndian.Uint16(header[6:8]))
expecting = 12 + dstLen + srcLen + payloadLen + (ed25519.PublicKeySize * 2)
case types.TypeVirtualSnakeSetup:
payloadLen := int(binary.BigEndian.Uint16(header[2:4]))
coordsLen := int(binary.BigEndian.Uint16(header[4:6]))
expecting = 10 + coordsLen + (ed25519.PublicKeySize * 2) + payloadLen
case types.TypeVirtualSnake, types.TypeVirtualSnakePathfind:
payloadLen := int(binary.BigEndian.Uint16(header[2:4]))
expecting = 8 + payloadLen + (ed25519.PublicKeySize * 2)
default:
dstLen := int(binary.BigEndian.Uint16(header[2:4]))
srcLen := int(binary.BigEndian.Uint16(header[4:6]))
payloadLen := int(binary.BigEndian.Uint16(header[6:8]))
expecting = 12 + dstLen + srcLen + payloadLen
}
n, err = io.ReadFull(p.conn, buf[:expecting])
switch err {
case io.EOF, io.ErrUnexpectedEOF:
_ = p.r.Disconnect(p.port, fmt.Errorf("io.ReadFull: %w", err))
return
case nil:
default:
p.r.log.Println("Failed to read:", err)
continue
}
if n < expecting {
p.r.log.Println("Expecting", expecting, "bytes but got", n, "bytes")
continue
}
func() {
frame := types.GetFrame()
defer frame.Done()
if _, err := frame.UnmarshalBinary(buf[:n]); err != nil {
p.r.log.Println("Port", p.port, "error unmarshalling frame:", err)
return
}
if frame.Version != types.Version0 {
p.r.log.Println("Port", p.port, "incorrect version in frame")
return
}
switch frame.Type {
case types.TypeSTP:
p.r.handleAnnouncement(p, frame.Borrow())
default:
sent := false
defer func() {
if !sent {
p.statistics.rxDroppedNoDestination.Inc()
}
}()
for _, port := range p.getNextHops(frame, p.port) {
// Ignore ports that are not good candidates.
dest := p.r.ports[port]
if !dest.started.Load() || (dest.port != 0 && !dest.alive.Load()) {
continue
}
if p.port == dest.port || p.public.EqualTo(dest.public) {
continue
}
switch frame.Type {
case types.TypePathfind, types.TypeVirtualSnakePathfind:
signedframe, err := p.r.signPathfind(frame, p, dest)
if err != nil {
p.r.log.Println("WARNING: Failed to sign pathfind:", err)
continue
}
if signedframe == nil {
continue
}
select {
case dest.trafficOut <- signedframe.Borrow():
dest.statistics.txTrafficSuccessful.Inc()
sent = true
return
default:
dest.statistics.txTrafficDropped.Inc()
signedframe.Done()
continue
}
case types.TypeDHTRequest, types.TypeDHTResponse, types.TypeVirtualSnakeBootstrap, types.TypeVirtualSnakeBootstrapACK, types.TypeVirtualSnakeSetup:
select {
case dest.protoOut <- frame.Borrow():
dest.statistics.txProtoSuccessful.Inc()
sent = true
return
default:
dest.statistics.txProtoDropped.Inc()
frame.Done()
continue
}
case types.TypeGreedy, types.TypeSource, types.TypeVirtualSnake:
select {
case dest.trafficOut <- frame.Borrow():
dest.statistics.txTrafficSuccessful.Inc()
sent = true
return
default:
dest.statistics.txTrafficDropped.Inc()
frame.Done()
continue
}
}
}
}
}()
}
}
}
func (p *Peer) writer() {
buf := make([]byte, 65535*3+12)
send := func(frame *types.Frame) error {
fn, err := frame.MarshalBinary(buf)
frame.Done()
if err != nil {
p.r.log.Println("Port", p.port, "error marshalling frame:", err)
return err
}
if !bytes.Equal(buf[:4], types.FrameMagicBytes) {
panic("expected magic bytes")
}
remaining := buf[:fn]
for len(remaining) > 0 {
n, err := p.conn.Write(remaining)
if err != nil {
if err != io.EOF {
p.r.log.Println("Failed to write:", err)
}
_ = p.r.Disconnect(p.port, fmt.Errorf("p.conn.Write: %w", err))
return err
}
remaining = remaining[n:]
}
p.conn.Flush()
return nil
}
for {
if !p.started.Load() {
return
}
select {
case <-p.context.Done():
return
case frame := <-p.protoOut:
if frame != nil {
_ = send(frame)
}
default:
}
select {
case <-p.context.Done():
return
case frame := <-p.trafficOut:
if frame != nil {
_ = send(frame)
}
case frame := <-p.protoOut:
if frame != nil {
_ = send(frame)
}
}
}
}
func (p *Peer) updateCoords(announcement *types.SwitchAnnouncement) {
// TODO: this entire function is bad, do something about it
if len(announcement.Signatures) == 0 {
p.r.log.Println("WARNING: No signatures from peer on port", p.port)
return
}
public := announcement.Signatures[len(announcement.Signatures)-1].PublicKey
if !p.public.EqualTo(public) {
p.r.log.Println("WARNING: Mismatched public key on port", p.port)
return
}
coords := announcement.Coords()
if len(coords) >= 0 {
coords = coords[:len(coords)-1]
}
p.mutex.Lock()
defer p.mutex.Unlock()
p.coords = coords
}
type peers []*Peer
func (p peers) Len() int {
return len(p)
}
func (p peers) Swap(i, j int) {
p[i], p[j] = p[j], p[i]
}
func (p peers) Less(i, j int) bool {
p[i].mutex.RLock()
p[j].mutex.RLock()
defer p[i].mutex.RUnlock()
defer p[j].mutex.RUnlock()
if p[i].peertype < p[j].peertype {
return true
}
return p[i].port < p[j].port
}

470
router/router.go Normal file
View file

@ -0,0 +1,470 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
import (
"bytes"
"context"
"crypto/ed25519"
"encoding/hex"
"fmt"
"io"
"log"
"net"
"sort"
"sync"
"time"
"github.com/matrix-org/pinecone/types"
"github.com/matrix-org/pinecone/util"
)
// PortCount contains the number of ports supported by this Pinecone
// node. This is, in practice, the limit of concurrent peering
// connections that are supported at one time.
const PortCount = 64
// ProtoBufferSize is the number of protocol packets that a node will
// buffer on a slow port.
const ProtoBufferSize = 16
// TrafficBufferSize is the number of traffic packets that a node will
// buffer on a slow port.
const TrafficBufferSize = 256
// Simulator is not used by normal Pinecone nodes and specifies the
// functions that must be satisfied if running under pineconesim.
type Simulator interface {
ReportDistance(a, b string, l int64)
LookupCoords(string) (types.SwitchPorts, error)
LookupNodeID(types.SwitchPorts) (string, error)
LookupPublicKey(types.PublicKey) (string, error)
ReportNewLink(net.Conn, types.PublicKey, types.PublicKey)
ReportDeadLink(types.PublicKey, types.PublicKey)
}
// Implements net.PacketConn. A Router is an instance of a Pinecone
// node and should only be instantiated using the NewRouter method.
type Router struct {
log *log.Logger //
context context.Context // switch context
cancel context.CancelFunc // switch context shutdown signal
callbacks *callbacks // notify when something happens
ports [PortCount]*Peer // all switch ports
simulator Simulator // is the node running in the sim?
id string // friendly identifier (for sim)
private types.PrivateKey // our keypair
public types.PublicKey // our keypair
tree *spanningTree // Yggdrasil-like spanning tree
snake *virtualSnake // SNEK routing protocol
dht *dht // Chord-like DHT tables
pathfinder *pathfinder // source routing pathfinder
active sync.Map // node public keys that we have active peerings with
send chan types.Frame // local node -> network
recv chan types.Frame // local node <- network
}
// NewRouter instantiates a new Pinecone Router instance. The logger
// is where all log output will be sent for this node. The ID is a
// friendly string that identifies the node, but is not used at the
// protocol level and is not visible outside externally. The private
// and public keys are the primary identity of the node and therefore
// must be unique for each node. These keys are also used to sign
// protocol messages.
func NewRouter(log *log.Logger, id string, private ed25519.PrivateKey, public ed25519.PublicKey, simulator Simulator) *Router {
ctx, cancel := context.WithCancel(context.Background())
sw := &Router{
log: log,
context: ctx,
cancel: cancel,
id: id,
simulator: simulator,
send: make(chan types.Frame, 32),
recv: make(chan types.Frame, 32),
}
sw.callbacks = &callbacks{r: sw}
copy(sw.private[:], private)
copy(sw.public[:], public)
sw.log.Println("Switch public key:", hex.EncodeToString(public))
// Prepare the switch ports.
for i := range sw.ports {
sw.ports[i] = &Peer{
r: sw,
port: types.SwitchPortID(i),
}
}
// A Pinecone node implements a few different things: a spanning
// tree, a Chord-like DHT and a source routing pathfinder. Each
// of these.
sw.tree = newSpanningTree(sw, func(parent types.SwitchPortID, coords types.SwitchPorts) {
sw.log.Println("New coordinates:", coords)
})
sw.dht = newDHT(sw)
sw.pathfinder = newPathfinder(sw)
sw.snake = newVirtualSnake(sw)
// Previously the router was in a separate package so we still
// wire up local traffic to a switch port. TODO: We should really
// change this so that peer.go handles local traffic.
pipelocal, pipeswitch := net.Pipe()
if id, err := sw.Connect(pipeswitch, sw.public, "router", -1); err != nil {
panic(err)
} else if id != 0 {
panic("router must be port 0")
}
go sw.reader(pipelocal)
go sw.writer(pipelocal)
go sw.startManhole()
return sw
}
// Close shuts down the node and stops the peering connections.
// The node should not be used after this has been called.
func (r *Router) Close() error {
r.cancel()
for _, port := range r.ports {
if port.started.Load() {
_ = port.stop()
}
}
return nil
}
// PrivateKey returns the ed25519 private key in use by this node.
func (r *Router) PrivateKey() types.PrivateKey {
return r.private
}
// PublicKey returns the ed25519 public key in use by this node.
func (r *Router) PublicKey() types.PublicKey {
return r.public
}
// Coords returns the current spanning tree coordinates of
// this node. The coordinates are effectively a source routing
// path from the root down to the node.
func (r *Router) Coords() types.SwitchPorts {
return r.tree.Coords()
}
// RootPublicKey returns the public key of the node that this
// node believes is the root of the network.
func (r *Router) RootPublicKey() types.PublicKey {
return r.tree.Root().RootPublicKey
}
// IsRoot returns true if this node believes it is the root of
// the network. This will likely return true if the node is
// isolated (e.g. has no peers).
func (r *Router) IsRoot() bool {
return r.tree.Root().RootPublicKey.EqualTo(r.public)
}
// Addr returns a net.Addr instance that addresses this node
// using greedy routing.
func (r *Router) Addr() net.Addr {
public := r.PublicKey()
return &public
}
// Pathfind takes a GreedyAddr (for greedy routing) or a types.PublicKey
// (for snake routing) and performs an active pathfind through the network.
// A SourceAddr will be returned which can be used to send packets using
// source routing instead. Note that this generates protocol traffic, so
// don't call it again unless you are sure the path has changed and the
// remote host is no longer answering. If the pathfind fails then an error
// will be returned instead.
func (r *Router) Pathfind(ctx context.Context, addr net.Addr) (net.Addr, error) {
return r.pathfinder.pathfind(ctx, addr)
}
// DHTSearch initiates a DHT search for the given public key.
// The stopshort flag, if set to true, will return the closest
// node to the public key found in the search. If set to false,
// the search will fail and return an error if the specific node
// is not found.
func (r *Router) DHTSearch(ctx context.Context, pk ed25519.PublicKey, stopshort bool) (types.PublicKey, net.Addr, error) {
var public types.PublicKey
copy(public[:], pk)
return r.dht.search(ctx, public, stopshort)
}
// DHTPredecessor returns the public key of the previous node in
// the DHT snake.
func (r *Router) Predecessor() *types.PublicKey {
r.snake.descendingMutex.RLock()
pr := r.snake.descending
r.snake.descendingMutex.RUnlock()
if pr == nil || time.Since(pr.LastSeen) >= virtualSnakeNeighExpiryPeriod {
return nil
}
pk := pr.PublicKey
return &pk
}
// DHTSuccessor returns the public key of the next node in the
// DHT snake.
func (r *Router) Successor() *types.PublicKey {
r.snake.ascendingMutex.RLock()
su := r.snake.ascending
r.snake.ascendingMutex.RUnlock()
if su == nil || time.Since(su.LastSeen) >= virtualSnakeNeighExpiryPeriod {
return nil
}
pk := su.PublicKey
return &pk
}
// KnownNodes returns a list of all nodes that are known about
// directly. This includes all peers and all ancestor nodes
// between this node and the root node.
func (r *Router) KnownNodes() []types.PublicKey {
known := map[types.PublicKey]struct{}{}
for _, p := range r.activePorts() {
p.mutex.RLock()
known[p.public] = struct{}{}
p.mutex.RUnlock()
}
r.snake.descendingMutex.RLock()
if p := r.snake.descending; p != nil {
known[p.PublicKey] = struct{}{}
}
r.snake.descendingMutex.RUnlock()
r.snake.ascendingMutex.RLock()
if s := r.snake.ascending; s != nil {
known[s.PublicKey] = struct{}{}
}
r.snake.ascendingMutex.RUnlock()
r.snake.tableMutex.RLock()
for k := range r.snake.table {
known[k] = struct{}{}
}
r.snake.tableMutex.RUnlock()
list := make([]types.PublicKey, 0, len(known))
for n := range known {
list = append(list, n)
}
return list
}
func (r *Router) activePorts() peers {
peers := make(peers, 0, PortCount)
for _, p := range r.ports {
switch {
case p.port == 0: // ignore the router
continue
case !p.started.Load() || !p.alive.Load(): // ignore stopped/non-negotiated ports
continue
default:
peers = append(peers, p)
}
}
sort.Sort(peers)
return peers
}
// AuthenticatedConnect initiates a peer connection using
// the given net.Conn connection. The public keys of the
// nodes are exchanged using a handshake. The connection
// will fail if this handshake fails. The port number that
// the node was connected to will be returned in the event
// of a successful connection.
func (r *Router) AuthenticatedConnect(conn net.Conn, zone string, peertype int) (types.SwitchPortID, error) {
select {
case <-time.After(time.Second * 5):
return 0, fmt.Errorf("handshake timed out")
default:
handshake := []byte{
ourVersion,
ourCapabilities,
0, // unused
0, // unused
}
handshake = append(handshake, r.public[:ed25519.PublicKeySize]...)
handshake = append(handshake, ed25519.Sign(r.private[:], handshake)...)
_ = conn.SetDeadline(time.Now().Add(time.Second * 5))
if _, err := conn.Write(handshake); err != nil {
conn.Close()
return 0, err
}
if _, err := io.ReadFull(conn, handshake); err != nil {
conn.Close()
return 0, fmt.Errorf("io.ReadFull: %w", err)
}
_ = conn.SetDeadline(time.Time{})
if theirVersion := handshake[0]; theirVersion != ourVersion {
conn.Close()
return 0, fmt.Errorf("mismatched node version")
}
if theirCapabilities := handshake[1]; theirCapabilities&ourCapabilities != ourCapabilities {
conn.Close()
return 0, fmt.Errorf("mismatched node capabilities")
}
var public types.PublicKey
var signature types.Signature
offset := 4
offset += copy(public[:], handshake[offset:offset+ed25519.PublicKeySize])
copy(signature[:], handshake[offset:offset+ed25519.SignatureSize])
if !ed25519.Verify(public[:], handshake[:offset], signature[:]) {
conn.Close()
return 0, fmt.Errorf("peer sent invalid signature")
}
port, err := r.Connect(conn, public, zone, peertype)
if err != nil {
conn.Close()
return 0, err
}
return port, nil
}
}
// Connect initiates a peer connection using the given
// net.Conn connection, in the event that the public key of
// the node is already known (e.g. in the simulator). The
// port number that the node was connected to will be
// returned in the event of a successful connection.
func (r *Router) Connect(conn net.Conn, public types.PublicKey, zone string, peertype int) (types.SwitchPortID, error) {
if p, ok := r.active.Load(hex.EncodeToString(public[:]) + zone); ok {
if err := r.Disconnect(p.(types.SwitchPortID), nil); err != nil {
return 0, fmt.Errorf("already connected to this node via zone %q", zone)
}
}
for i := types.SwitchPortID(0); i < PortCount; i++ {
if i != 0 && bytes.Equal(r.public[:], public[:]) {
return 0, fmt.Errorf("loopback connection prohibited")
}
if r.ports[i].started.Load() {
continue
}
r.ports[i].mutex.Lock()
r.ports[i].context, r.ports[i].cancel = context.WithCancel(r.context)
r.ports[i].zone = zone
r.ports[i].peertype = peertype
r.ports[i].conn = util.NewBufferedRWC(conn)
r.ports[i].public = public
r.ports[i].protoOut = make(chan *types.Frame, ProtoBufferSize)
r.ports[i].trafficOut = make(chan *types.Frame, TrafficBufferSize)
r.ports[i].advertise = util.NewDispatch()
r.ports[i].statistics.reset()
r.ports[i].mutex.Unlock()
if err := r.ports[i].start(); err != nil {
return 0, fmt.Errorf("port.start: %w", err)
}
if i != 0 {
r.dht.insertNode(r.ports[i])
}
if r.simulator != nil {
r.simulator.ReportNewLink(conn, r.public, public)
}
r.log.Printf("Connected port %d to %s (zone %q)\n", i, conn.RemoteAddr(), zone)
go r.callbacks.onConnected(i, public, peertype)
r.active.Store(hex.EncodeToString(public[:])+zone, i)
return i, nil
}
return 0, fmt.Errorf("no free switch ports")
}
// Disconnect will disconnect whatever is connected to the
// given port number on the Pinecone node. The peering will
// no longer be used and the underlying connection will be
// closed.
func (r *Router) Disconnect(i types.SwitchPortID, err error) error {
if i == 0 {
return fmt.Errorf("cannot disconnect port %d", i)
}
r.active.Delete(hex.EncodeToString(r.ports[i].public[:]) + r.ports[i].zone)
r.ports[i].mutex.Lock()
_ = r.ports[i].stop()
r.ports[i].peertype = 0
r.ports[i].zone = ""
r.ports[i].public = types.PublicKey{}
if len(r.ports[i].protoOut) > 0 {
for range r.ports[i].protoOut {
}
}
if len(r.ports[i].trafficOut) > 0 {
for range r.ports[i].trafficOut {
}
}
r.ports[i].protoOut = nil
r.ports[i].trafficOut = nil
r.ports[i].mutex.Unlock()
if r.ports[i].port != 0 {
r.dht.deleteNode(r.ports[i].public)
}
if r.simulator != nil {
r.simulator.ReportDeadLink(r.public, r.ports[i].public)
}
r.log.Printf("Disconnected port %d: %s\n", i, err)
go r.callbacks.onDisconnected(i, r.ports[i].public, r.ports[i].peertype, err)
r.snake.portWasDisconnected(i)
r.tree.portWasDisconnected(i)
return nil
}
// PeerCount returns the number of nodes that are directly
// connected to this Pinecone node. This will de-duplicate
// peerings to the same node in different zones.
func (r *Router) PeerCount(peertype int) int {
count := 0
ports := r.activePorts()
if peertype < 0 {
return len(ports)
}
for _, p := range ports {
p.mutex.RLock()
if p.peertype == peertype {
count++
}
p.mutex.RUnlock()
}
return count
}
// IsConnected returns true if the node is connected within the
// given zone, or false otherwise.
func (r *Router) IsConnected(key types.PublicKey, zone string) bool {
_, ok := r.active.Load(hex.EncodeToString(key[:]) + zone)
return ok
}
// PeerInfo is a gomobile-friendly type that represents a peer
// connection.
type PeerInfo struct {
Port int
PublicKey string
PeerType int
Zone string
}
// Peers returns a list of PeerInfos that show all of the currently
// connected peers.
func (r *Router) Peers() []PeerInfo {
peers := make([]PeerInfo, 0, PortCount)
for _, p := range r.activePorts() {
p.mutex.RLock()
peers = append(peers, PeerInfo{
Port: int(p.port),
PeerType: p.peertype,
PublicKey: p.public.String(),
Zone: p.zone,
})
p.mutex.RUnlock()
}
return peers
}

375
router/tree.go Normal file
View file

@ -0,0 +1,375 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
import (
"context"
"encoding/hex"
"fmt"
"math"
"sync"
"sync/atomic"
"time"
"github.com/matrix-org/pinecone/types"
"github.com/matrix-org/pinecone/util"
)
// announcementThreshold is the amount of time that must
// pass before the node will accept a root announcement
// again from the same peer.
const announcementThreshold = announcementInterval / 2
// announcementInterval is the frequency at which this
// node will send root announcements to other peers.
const announcementInterval = time.Second * 15
// announcementTimeout is the amount of time that must
// pass without receiving a root announcement before we
// will assume that the peer is dead.
const announcementTimeout = announcementInterval * 3
func (r *Router) handleAnnouncement(peer *Peer, rx *types.Frame) {
defer rx.Done()
old := r.tree.Root()
var new types.SwitchAnnouncement
if _, err := new.UnmarshalBinary(rx.Payload); err != nil {
r.log.Println("Error unmarshalling announcement:", err)
return
}
peer.updateCoords(&new)
peer.alive.Store(true)
if peer.port != 0 {
if new.RootPublicKey.EqualTo(old.RootPublicKey) && new.Sequence < old.Sequence {
// The node has sent a replay of a previous announcement. No
// bueno, drop it.
return
}
}
if err := r.tree.Update(peer, &new); err != nil {
r.log.Println("Error handling announcement:", err)
}
}
// This tries to converge on a minimum spanning tree by optimising
// parent relationships for distance. All other metrics are ignored.
type rootAnnouncementWithTime struct {
*types.SwitchAnnouncement
at time.Time
}
type spanningTree struct {
r *Router //
context context.Context //
advertise util.Dispatch //
advertiseTimer *time.Ticker //
root *rootAnnouncementWithTime // last root announcement
rootMutex sync.RWMutex //
rootReset util.Dispatch //
parent atomic.Value // types.SwitchPortID
coords atomic.Value // types.SwitchPorts
callback func(parent types.SwitchPortID, coords types.SwitchPorts)
}
func newSpanningTree(r *Router, f func(parent types.SwitchPortID, coords types.SwitchPorts)) *spanningTree {
t := &spanningTree{
r: r,
context: r.context,
advertise: util.NewDispatch(),
advertiseTimer: time.NewTicker(announcementInterval),
rootReset: util.NewDispatch(),
callback: f,
}
t.becomeRoot()
t.advertise.Dispatch()
go t.workerForRoot()
go t.workerForAnnouncements()
return t
}
func (t *spanningTree) Coords() types.SwitchPorts {
if coords, ok := t.coords.Load().(types.SwitchPorts); ok {
return coords
}
return types.SwitchPorts{}
}
func (t *spanningTree) Ancestors() ([]types.PublicKey, types.SwitchPortID) {
t.rootMutex.RLock()
defer t.rootMutex.RUnlock()
if t.root == nil || (t.root != nil && time.Since(t.root.at) > announcementTimeout) {
return nil, 0
}
port, ok := t.parent.Load().(types.SwitchPortID)
if !ok || port == 0 {
return nil, 0
}
ancestors := make([]types.PublicKey, len(t.root.Signatures)+1)
if t.root != nil {
ancestors = append(ancestors, t.root.RootPublicKey)
for _, sig := range t.root.Signatures {
ancestors = append(ancestors, sig.PublicKey)
}
}
return ancestors, port
}
func (t *spanningTree) portWasDisconnected(port types.SwitchPortID) {
if t.r.PeerCount(-1) == 0 {
t.becomeRoot()
return
}
if t.parent.Load() == port {
p := t.selectParent()
t.parent.Store(p)
}
}
func (t *spanningTree) becomeRoot() {
t.rootMutex.Lock()
t.root = &rootAnnouncementWithTime{
SwitchAnnouncement: &types.SwitchAnnouncement{
RootPublicKey: t.r.public,
Sequence: types.Varu64(time.Now().UnixNano()),
},
at: time.Now(),
}
t.rootMutex.Unlock()
t.parent.Store(types.SwitchPortID(0))
newCoords := types.SwitchPorts{}
if !t.Coords().EqualTo(newCoords) {
t.coords.Store(types.SwitchPorts{})
t.callback(0, types.SwitchPorts{})
}
t.rootReset.Dispatch()
}
func (t *spanningTree) selectParent() types.SwitchPortID {
bestDist := int64(math.MaxInt64)
var parent types.SwitchPortID
for _, port := range t.r.activePorts() {
ann := port.lastAnnouncement()
if ann == nil || len(ann.Signatures) == 0 {
// The peer either hasn't sent us an announcement yet, or it's
// sent us an invalid announcement with no signatures.
continue
}
if !t.root.RootPublicKey.EqualTo(ann.RootPublicKey) {
// The peer has sent us an announcement, but it's not from the
// root that we're expecting it to be from.
continue
}
if l := int64(len(ann.Signatures)); parent == 0 || l < bestDist {
parent, bestDist = port.port, l
}
}
return parent
}
func (t *spanningTree) workerForAnnouncements() {
advertise := func() {
for _, p := range t.r.ports {
if p.started.Load() {
p.advertise.Dispatch()
}
}
}
for {
select {
case <-t.context.Done():
return
case <-t.advertise:
advertise()
case <-t.advertiseTimer.C:
advertise()
}
}
}
func (t *spanningTree) workerForRoot() {
for {
select {
case <-t.context.Done():
return
case <-time.After(announcementTimeout):
if !t.IsRoot() {
t.r.log.Println("Haven't heard from the root lately")
t.becomeRoot()
}
case <-t.rootReset:
}
}
}
func (t *spanningTree) updateCoordinates() types.SwitchPortID {
// Are we the root? If so then our coords are predetermined
// and we don't have a parent node.
if t.IsRoot() {
newCoords := types.SwitchPorts{}
if !t.Coords().EqualTo(newCoords) {
t.coords.Store(newCoords)
t.callback(0, newCoords)
}
return 0
}
// Otherwise, let's try and work out who are most effective
// parent is. If we get no parent then we're the root.
parent := t.selectParent()
t.parent.Store(parent)
if !t.r.ports[parent].started.Load() || !t.r.ports[parent].alive.Load() {
return 0 // panic("parent shouldn't be nil if we aren't the root node")
}
// Work out what our coordinates are relative to our chosen
// parent.
t.r.ports[parent].mutex.RLock()
defer t.r.ports[parent].mutex.RUnlock()
if ann := t.r.ports[parent].lastAnnouncement(); ann != nil {
if newCoords := ann.Coords(); !t.Coords().EqualTo(newCoords) {
t.coords.Store(newCoords)
t.callback(parent, newCoords)
}
} else {
panic("couldn't get last parent announcement")
}
return parent
}
func (t *spanningTree) IsRoot() bool {
t.rootMutex.RLock()
defer t.rootMutex.RUnlock()
return t.root.RootPublicKey.EqualTo(t.r.public)
}
func (t *spanningTree) Root() *types.SwitchAnnouncement {
t.rootMutex.RLock()
defer t.rootMutex.RUnlock()
if t.root.RootPublicKey.EqualTo(t.r.public) || time.Since(t.root.at) > announcementTimeout {
return &types.SwitchAnnouncement{
RootPublicKey: t.r.public,
Sequence: types.Varu64(time.Now().UnixNano()),
}
}
return &types.SwitchAnnouncement{ // return a copy
RootPublicKey: t.root.RootPublicKey,
Sequence: t.root.Sequence,
Signatures: append([]types.SignatureWithHop{}, t.root.Signatures...),
}
}
func (t *spanningTree) Parent() types.SwitchPortID {
if parent, ok := t.parent.Load().(types.SwitchPortID); ok {
return parent
}
return 0
}
func (t *spanningTree) Remove(p *Peer) {
if t.Parent() == p.port {
t.parent.Store(types.SwitchPortID(0))
}
}
func (t *spanningTree) Update(p *Peer, a *types.SwitchAnnouncement) error {
// Unless the key is really stronger than our current root key,
// throttle how quickly we will act upon root updates.
if last := p.lastAnnouncement(); last != nil && time.Since(last.at) < announcementThreshold {
if a.RootPublicKey.CompareTo(t.Root().RootPublicKey) < 0 {
return fmt.Errorf("rejecting update (too soon)")
}
}
// Check that there are no routing loops in the update.
sigs := make(map[string]struct{})
for _, sig := range a.Signatures {
if sig.Hop == 0 {
return fmt.Errorf("rejecting update (invalid 0 hop)")
}
if p.port != 0 && t.r.public.EqualTo(sig.PublicKey) {
return nil // fmt.Errorf("rejecting update (we signed this already)")
}
pk := hex.EncodeToString(sig.PublicKey[:])
if _, ok := sigs[pk]; ok {
return fmt.Errorf("rejecting update (detected routing loop)")
}
sigs[pk] = struct{}{}
}
// Store the announcement against the peer. This lets us ultimately
// calculate what the coordinates of that peer are later.
p.announcement = &rootAnnouncementWithTime{
SwitchAnnouncement: a,
at: time.Now(),
}
t.rootMutex.RLock()
oldRoot, newRoot := t.root, t.root
t.rootMutex.RUnlock()
// Work out if the announcement came from our selected parent. We will
// only handle subsequent updates from the same root if they came in
// via our chosen parent, otherwise this might cause downstream coords
// to flap.
parent := t.Parent()
isParent := parent == 0 || p.port == parent
// If the advertisement came from the same root then process it.
switch {
case a.RootPublicKey.CompareTo(oldRoot.RootPublicKey) > 0:
// If the advertisement contains a stronger key than the root, or the
// announcement contains the root that we know about, update our stored
// announcement.
newRoot = &rootAnnouncementWithTime{a, time.Now()}
t.rootReset.Dispatch()
case t.r.public.CompareTo(oldRoot.RootPublicKey) >= 0:
// If it turns out after all that our key is stronger than the chosen
// root then we'll become root instead.
t.becomeRoot()
case oldRoot.RootPublicKey.EqualTo(a.RootPublicKey):
// We'll only process the update from the same root if it's actually
// a new update, e.g. the sequence number has increased, and the
// signature count is equal to or shorter than the previous count.
// This stops us from flapping coordinates so much.
if isParent && a.Sequence > oldRoot.Sequence && len(a.Signatures) <= len(oldRoot.Signatures) {
newRoot = &rootAnnouncementWithTime{a, time.Now()}
t.rootReset.Dispatch()
}
}
// If the root has changed then let's do something about it.
if newRoot != oldRoot {
t.rootMutex.Lock()
t.root = newRoot
t.rootMutex.Unlock()
if p.port == t.updateCoordinates() {
t.advertise.Dispatch()
}
}
return nil
}

23
router/version.go Normal file
View file

@ -0,0 +1,23 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package router
const (
// reserved = 1
capabilityVirtualSnake = 2
)
const ourVersion = 0
const ourCapabilities = capabilityVirtualSnake

127
sessions/dial.go Normal file
View file

@ -0,0 +1,127 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package sessions
import (
"context"
"crypto/ed25519"
"crypto/tls"
"encoding/hex"
"fmt"
"net"
"github.com/matrix-org/pinecone/types"
)
// DialContext dials a given public key using the supplied network.
// The network field can be used to specify which routing algorithm to
// use for the session: "ed25519+greedy" for greedy routing or "ed25519+source"
// for source routing - DHT lookups and pathfinds will be performed for these
// networks automatically. Otherwise, the default "ed25519" will use snake
// routing. The address must be the destination public key specified in hex.
// If the context expires then the session will be torn down automatically.
func (q *Sessions) DialContext(ctx context.Context, network, addrstr string) (net.Conn, error) {
host, _, err := net.SplitHostPort(addrstr)
if err != nil {
return nil, fmt.Errorf("net.SplitHostPort: %w", err)
}
pk := make(ed25519.PublicKey, ed25519.PublicKeySize)
pkb, err := hex.DecodeString(host)
if err != nil {
return nil, fmt.Errorf("hex.DecodeString: %w", err)
}
if len(pkb) != ed25519.PublicKeySize {
return nil, fmt.Errorf("host must be length of an ed25519 public key")
}
copy(pk, pkb)
var addr net.Addr
switch network {
case "ed25519+greedy":
_, addr, err = q.r.DHTSearch(ctx, pk, false)
if err != nil {
return nil, fmt.Errorf("q.dht.search: %w", err)
}
case "ed25519+source":
_, coords, err := q.r.DHTSearch(ctx, pk, false)
if err != nil {
return nil, fmt.Errorf("q.dht.search: %w", err)
}
addr, err = q.r.Pathfind(ctx, coords)
if err != nil {
return nil, fmt.Errorf("q.pathfinder.pathfind: %w", err)
}
case "ed25519":
fallthrough
default:
a := &types.PublicKey{}
copy(a[:], pk)
addr = a
}
session, err := q.utpSocket.DialAddrContext(
ctx, addr,
)
if err != nil {
return nil, fmt.Errorf("q.utpSocket.DialContext: %w", err)
}
session = tls.Client(session, &tls.Config{
InsecureSkipVerify: true,
GetClientCertificate: func(info *tls.CertificateRequestInfo) (*tls.Certificate, error) {
return q.tlsCert, nil
},
VerifyConnection: func(state tls.ConnectionState) error {
if c := len(state.PeerCertificates); c != 1 {
return fmt.Errorf("expected exactly one peer certificate but got %d", c)
}
public, ok := state.PeerCertificates[0].PublicKey.(ed25519.PublicKey)
if !ok {
return fmt.Errorf("expected ed25519 public key")
}
if !public.Equal(pk) {
return fmt.Errorf("remote side returned incorrect public key")
}
return nil
},
})
return session, nil
}
// Dial dials a given public key using the supplied network.
// The network field can be used to specify which routing algorithm to
// use for the session: "ed25519+greedy" for greedy routing or "ed25519+source"
// for source routing. DHT lookups and pathfinds will be performed for these
// networks automatically. Otherwise, the default "ed25519" will use snake
// routing. The address must be the destination public key specified in hex.
func (q *Sessions) Dial(network, addr string) (net.Conn, error) {
return q.DialContext(context.Background(), network, addr)
}
// DialTLS is an alias for Dial, as all sessions are TLS-encrypted.
func (q *Sessions) DialTLS(network, addr string) (net.Conn, error) {
return q.DialTLSContext(context.Background(), network, addr)
}
// DialTLSContext is an alias for DialContext, as all sessions are
// TLS-encrypted.
func (q *Sessions) DialTLSContext(ctx context.Context, network, addr string) (net.Conn, error) {
return q.DialContext(ctx, network, addr)
}

58
sessions/http.go Normal file
View file

@ -0,0 +1,58 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package sessions
import (
"net/http"
"time"
)
type HTTP struct {
httpServer *http.Server
httpMux *http.ServeMux
httpTransport *http.Transport
httpClient *http.Client
}
func (q *Sessions) HTTP() *HTTP {
h := &HTTP{
httpServer: &http.Server{
IdleTimeout: time.Second * 30,
},
httpMux: &http.ServeMux{},
httpTransport: &http.Transport{
Dial: q.Dial,
DialTLS: q.DialTLS,
DialContext: q.DialContext,
DialTLSContext: q.DialTLSContext,
},
}
h.httpServer.Handler = h.httpMux
h.httpClient = &http.Client{
Transport: h.httpTransport,
}
go h.httpServer.Serve(q) // nolint:errcheck
return h
}
func (h *HTTP) Mux() *http.ServeMux {
return h.httpMux
}
func (h *HTTP) Client() *http.Client {
return h.httpClient
}

56
sessions/listen.go Normal file
View file

@ -0,0 +1,56 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package sessions
import (
"crypto/tls"
"fmt"
"net"
)
func (q *Sessions) listener() {
q.log.Println("Listening for UTP sessions")
for {
session, err := q.utpSocket.Accept()
if err != nil {
q.log.Println("Failed to accept UTP:", err)
return
}
go func(session net.Conn) {
q.streams <- session
}(session)
}
}
// Accept blocks until a new session request is received. The
// connection returned by this function will be TLS-encrypted.
func (q *Sessions) Accept() (net.Conn, error) {
stream := <-q.streams
if stream == nil {
return nil, fmt.Errorf("listener closed")
}
stream = tls.Server(stream, q.tlsServerCfg)
return stream, nil
}
func (q *Sessions) Addr() net.Addr {
return q.r.Addr()
}
func (q *Sessions) Close() error {
q.cancel()
return nil
}

121
sessions/sessions.go Normal file
View file

@ -0,0 +1,121 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package sessions
import (
"context"
"crypto/ed25519"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/hex"
"encoding/pem"
"log"
"math/big"
"net"
"time"
"github.com/matrix-org/pinecone/router"
"github.com/neilalexander/utp"
)
type Sessions struct {
r *router.Router
log *log.Logger // logger
context context.Context // router context
cancel context.CancelFunc // shut down the router
streams chan net.Conn // accepted connections
tlsCert *tls.Certificate //
tlsServerCfg *tls.Config //
utpSocket *utp.Socket //
}
func NewSessions(log *log.Logger, r *router.Router) *Sessions {
ctx, cancel := context.WithCancel(context.Background())
s, err := utp.NewSocketFromPacketConnNoClose(r)
if err != nil {
panic(err)
}
q := &Sessions{
r: r,
log: log,
context: ctx,
cancel: cancel,
streams: make(chan net.Conn, 16),
utpSocket: s,
}
q.tlsCert = q.generateTLSCertificate()
q.tlsServerCfg = &tls.Config{
Certificates: []tls.Certificate{*q.tlsCert},
ClientAuth: tls.RequireAnyClientCert,
}
go q.listener()
return q
}
func (q *Sessions) Sessions() []ed25519.PublicKey {
var sessions []ed25519.PublicKey
//for _, s := range q.sessions {
//sessions = append(sessions, s.RemoteAddr())
/*
if certs := s.ConnectionState().PeerCertificates; len(certs) > 0 {
sessions = append(sessions, certs[0].PublicKey.(ed25519.PublicKey))
}
*/
//}
return sessions
}
func (q *Sessions) generateTLSCertificate() *tls.Certificate {
private, public := q.r.PrivateKey(), q.r.PublicKey()
id := hex.EncodeToString(public[:])
template := x509.Certificate{
Subject: pkix.Name{
CommonName: id,
},
SerialNumber: big.NewInt(1),
NotAfter: time.Now().Add(time.Hour * 24 * 365),
DNSNames: []string{id},
}
certDER, err := x509.CreateCertificate(
rand.Reader,
&template,
&template,
ed25519.PublicKey(public[:]),
ed25519.PrivateKey(private[:]),
)
if err != nil {
panic(err)
}
privateKey, err := x509.MarshalPKCS8PrivateKey(ed25519.PrivateKey(private[:]))
if err != nil {
panic(err)
}
certPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: certDER})
keyPEM := pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: privateKey})
tlsCert, err := tls.X509KeyPair(certPEM, keyPEM)
if err != nil {
panic(err)
}
return &tlsCert
}

99
types/announcement.go Normal file
View file

@ -0,0 +1,99 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
import (
"crypto/ed25519"
"fmt"
"os"
)
type SwitchAnnouncement struct {
RootPublicKey PublicKey
Sequence Varu64
Signatures []SignatureWithHop
}
func (a SwitchAnnouncement) Sign(privKey ed25519.PrivateKey, forPort SwitchPortID) (*SwitchAnnouncement, error) {
var body [65535]byte
n, err := a.MarshalBinary(body[:])
if err != nil {
return nil, fmt.Errorf("a.MarshalBinary: %w", err)
}
hop := SignatureWithHop{
Hop: Varu64(forPort),
}
copy(hop.PublicKey[:], privKey.Public().(ed25519.PublicKey))
if _, ok := os.LookupEnv("PINECONE_DISABLE_SIGNATURES"); !ok {
copy(hop.Signature[:], ed25519.Sign(privKey, body[:n]))
}
a.Signatures = append(a.Signatures, hop)
return &a, nil
}
func (a *SwitchAnnouncement) UnmarshalBinary(data []byte) (int, error) {
expected := ed25519.PublicKeySize + 1
if size := len(data); size < expected {
return 0, fmt.Errorf("expecting at least %d bytes, got %d bytes", expected, size)
}
copy(a.RootPublicKey[:], data)
remaining := data[ed25519.PublicKeySize:]
if err := a.Sequence.UnmarshalBinary(remaining); err != nil {
return 0, fmt.Errorf("a.Sequence.UnmarshalBinary: %w", err)
}
remaining = remaining[a.Sequence.Length():]
for i := Varu64(0); len(remaining) >= ed25519.PublicKeySize+ed25519.SignatureSize+1; i++ {
var signature SignatureWithHop
n, err := signature.UnmarshalBinary(remaining[:])
if err != nil {
return 0, fmt.Errorf("signature.UnmarshalBinary: %w", err)
}
if _, ok := os.LookupEnv("PINECONE_DISABLE_SIGNATURES"); !ok {
if !ed25519.Verify(signature.PublicKey[:], data[:len(data)-len(remaining)], signature.Signature[:]) {
return 0, fmt.Errorf("signature verification failed for hop %d", signature.Hop)
}
}
a.Signatures = append(a.Signatures, signature)
remaining = remaining[n:]
}
return len(data) - len(remaining), nil
}
func (a *SwitchAnnouncement) MarshalBinary(buffer []byte) (int, error) {
seq, err := a.Sequence.MarshalBinary()
if err != nil {
return 0, fmt.Errorf("a.Sequence.MarshalBinary: %w", err)
}
offset := 0
offset += copy(buffer[offset:], a.RootPublicKey[:]) // a.RootPublicKey
offset += copy(buffer[offset:], seq) // a.Sequence
for _, sig := range a.Signatures {
n, err := sig.MarshalBinary(buffer[offset:])
if err != nil {
return 0, fmt.Errorf("sig.MarshalBinary: %w", err)
}
offset += n
}
return offset, nil
}
func (a *SwitchAnnouncement) Coords() SwitchPorts {
sigs := a.Signatures
coords := make(SwitchPorts, 0, len(sigs))
for _, sig := range sigs {
coords = append(coords, SwitchPortID(sig.Hop))
}
return coords
}

53
types/announcement_test.go Normal file
View file

@ -0,0 +1,53 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
import (
"crypto/ed25519"
"testing"
)
func TestMarshalUnmarshalAnnouncement(t *testing.T) {
pkr, _, _ := ed25519.GenerateKey(nil)
_, sk1, _ := ed25519.GenerateKey(nil)
_, sk2, _ := ed25519.GenerateKey(nil)
_, sk3, _ := ed25519.GenerateKey(nil)
input := &SwitchAnnouncement{
Sequence: 1,
}
copy(input.RootPublicKey[:], pkr)
var err error
input, err = input.Sign(sk1, 1)
if err != nil {
t.Fatal(err)
}
input, err = input.Sign(sk2, 2)
if err != nil {
t.Fatal(err)
}
input, err = input.Sign(sk3, 3)
if err != nil {
t.Fatal(err)
}
var buffer [65535]byte
n, err := input.MarshalBinary(buffer[:])
if err != nil {
t.Fatal(err)
}
var output SwitchAnnouncement
if _, err = output.UnmarshalBinary(buffer[:n]); err != nil {
t.Fatal(err)
}
}

136
types/dht.go Normal file
View file

@ -0,0 +1,136 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
import (
"crypto/ed25519"
"fmt"
"os"
)
type DHTQueryRequest struct {
RequestID [8]byte
PublicKey [ed25519.PublicKeySize]byte
}
func (r *DHTQueryRequest) MarshalBinary(buffer []byte) (int, error) {
if len(buffer) < 8+ed25519.PublicKeySize {
return 0, fmt.Errorf("not enough bytes")
}
offset := 0
offset += copy(buffer[offset:], r.RequestID[:8])
offset += copy(buffer[offset:], r.PublicKey[:ed25519.PublicKeySize])
return offset, nil
}
func (r *DHTQueryRequest) UnmarshalBinary(buffer []byte) (int, error) {
if len(buffer) < ed25519.PublicKeySize {
return 0, fmt.Errorf("not enough bytes")
}
remaining := buffer[:]
remaining = remaining[copy(r.RequestID[:], remaining):]
copy(r.PublicKey[:], remaining)
return len(buffer) - len(remaining), nil
}
type DHTQueryResponse struct {
RequestID [8]byte
Results []DHTNode
PublicKey [ed25519.PublicKeySize]byte
}
func (r *DHTQueryResponse) MarshalBinary(buffer []byte, private ed25519.PrivateKey) (int, error) {
if len(buffer) < 8+1+ed25519.PublicKeySize+ed25519.SignatureSize {
return 0, fmt.Errorf("not enough bytes")
}
offset := 0
offset += copy(buffer[offset:], r.RequestID[:])
offset += copy(buffer[offset:], []byte{byte(len(r.Results))})
for _, result := range r.Results {
n, err := result.MarshalBinary(buffer[offset:])
if err != nil {
return 0, fmt.Errorf("result.MarshalBinary: %w", err)
}
offset += n
}
offset += copy(buffer[offset:], r.PublicKey[:])
if _, ok := os.LookupEnv("PINECONE_DISABLE_SIGNATURES"); !ok {
offset += copy(buffer[offset:], ed25519.Sign(private, buffer[:offset]))
} else {
offset += ed25519.SignatureSize
}
return offset, nil
}
func (r *DHTQueryResponse) UnmarshalBinary(buffer []byte) (int, error) {
if len(buffer) < 8+1 {
return 0, fmt.Errorf("not enough bytes")
}
remaining := buffer[:]
remaining = remaining[copy(r.RequestID[:], remaining):]
responses := int(remaining[0])
remaining = remaining[1:]
for i := 0; i < responses; i++ {
var result DHTNode
n, err := result.UnmarshalBinary(remaining)
if err != nil {
return 0, fmt.Errorf("response.UnmarshalBinary: %w", err)
}
r.Results = append(r.Results, result)
remaining = remaining[n:]
}
remaining = remaining[copy(r.PublicKey[:], remaining):]
boundary := len(buffer) - len(remaining)
if _, ok := os.LookupEnv("PINECONE_DISABLE_SIGNATURES"); !ok {
if !ed25519.Verify(r.PublicKey[:], buffer[:boundary], remaining[:ed25519.SignatureSize]) {
return 0, fmt.Errorf("signature verification failed")
}
}
return len(buffer) - len(remaining), nil
}
type DHTNode struct {
PublicKey PublicKey
Coordinates SwitchPorts
}
func (n *DHTNode) MarshalBinary(buffer []byte) (int, error) {
coords, err := n.Coordinates.MarshalBinary()
if err != nil {
return 0, fmt.Errorf("n.Coordinates.MarshalBinary: %w", err)
}
if len(buffer) < ed25519.PublicKeySize+len(coords) {
return 0, fmt.Errorf("not enough bytes")
}
offset := 0
offset += copy(buffer[offset:], n.PublicKey[:])
offset += copy(buffer[offset:], coords)
return offset, nil
}
func (n *DHTNode) UnmarshalBinary(buffer []byte) (int, error) {
if len(buffer) < ed25519.PublicKeySize+1 {
return 0, fmt.Errorf("not enough bytes")
}
remaining := buffer[:]
remaining = remaining[copy(n.PublicKey[:], remaining[:ed25519.PublicKeySize]):]
l, err := n.Coordinates.UnmarshalBinary(remaining)
if err != nil {
return 0, fmt.Errorf("n.Coordinates.UnmarshalBinary: %w", err)
}
remaining = remaining[l:]
return len(buffer) - len(remaining), nil
}

68
types/dht_test.go Normal file
View file

@ -0,0 +1,68 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
import (
"crypto/ed25519"
"reflect"
"testing"
)
func TestDHTRequest(t *testing.T) {
pk, _, _ := ed25519.GenerateKey(nil)
req := DHTQueryRequest{
RequestID: [8]byte{1, 2, 3, 4, 5, 6, 7, 8},
}
copy(req.PublicKey[:], pk)
var buf [65535]byte
n, err := req.MarshalBinary(buf[:])
if err != nil {
t.Fatal(err)
}
orig := DHTQueryRequest{}
if _, err := orig.UnmarshalBinary(buf[:n]); err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(req, orig) {
t.Fatal("marshalled and unmarshalled structs don't match")
}
}
func TestDHTResponse(t *testing.T) {
pk1, sk1, _ := ed25519.GenerateKey(nil)
pk2, _, _ := ed25519.GenerateKey(nil)
req := DHTQueryResponse{
RequestID: [8]byte{1, 2, 3, 4, 5, 6, 7, 8},
Results: []DHTNode{
{
Coordinates: SwitchPorts{4, 3, 2, 1},
},
},
}
copy(req.PublicKey[:], pk1)
copy(req.Results[0].PublicKey[:], pk2)
var buf [65535]byte
n, err := req.MarshalBinary(buf[:], sk1)
if err != nil {
t.Fatal(err)
}
orig := DHTQueryResponse{}
if _, err := orig.UnmarshalBinary(buf[:n]); err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(req, orig) {
t.Fatal("marshalled and unmarshalled structs don't match")
}
}

42
types/ed25519.go Normal file
View file

@ -0,0 +1,42 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
import (
"bytes"
"crypto/ed25519"
"encoding/hex"
"fmt"
)
type PublicKey [ed25519.PublicKeySize]byte
type PrivateKey [ed25519.PrivateKeySize]byte
type Signature [ed25519.SignatureSize]byte
func (a PublicKey) EqualTo(b PublicKey) bool {
return bytes.Equal(a[:], b[:])
}
func (a PublicKey) CompareTo(b PublicKey) int {
return bytes.Compare(a[:], b[:])
}
func (a PublicKey) String() string {
return fmt.Sprintf("%v", hex.EncodeToString(a[:]))
}
func (a PublicKey) Network() string {
return "ed25519"
}

335
types/frame.go Normal file
View file

@ -0,0 +1,335 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
import (
"bytes"
"crypto/ed25519"
"encoding/binary"
"fmt"
"math"
"sync"
"go.uber.org/atomic"
)
var framePool = &sync.Pool{
New: func() interface{} {
return &Frame{}
},
}
func GetFrame() *Frame {
frame := framePool.Get().(*Frame)
frame.refs.Store(1)
return frame
}
type FrameVersion uint8
type FrameType uint8
const (
TypeSTP FrameType = iota // root announcements, signed before forwarding to all peers
TypeSource // traffic frame, forwarded using source routing
TypeGreedy // traffic frame, forwarded using greedy routing
TypePathfind // protocol frame, sign the update before forwarding it greedily
TypeSwitchUpdate // sent from the switch to the router to update about peers
TypeDHTRequest // protocol frame, forwarded using greedy routing
TypeDHTResponse // protocol frame, forwarded using greedy routing
TypeVirtualSnakeBootstrap // protocol frame, forwarded using SNEK predecessor ordering
TypeVirtualSnakeBootstrapACK // protocol frame, forwarded using greedy routing
TypeVirtualSnakeSetup // protocol frame, forwarded using greedy routing
TypeVirtualSnake // traffic frame, forwarded using SNEK successor ordering
TypeVirtualSnakePathfind // protocol frame, forwarded using SNEK successor ordering
)
const (
Version0 FrameVersion = iota
)
var FrameMagicBytes = []byte{0x70, 0x69, 0x6e, 0x65}
type Frame struct {
refs atomic.Int32
Version FrameVersion
Type FrameType
Destination SwitchPorts
DestinationKey PublicKey
Source SwitchPorts
SourceKey PublicKey
Payload []byte
}
func (f *Frame) Reset() {
f.Version, f.Type = 0, 0
f.Destination = SwitchPorts{}
f.DestinationKey = PublicKey{}
f.Source = SwitchPorts{}
f.SourceKey = PublicKey{}
f.Payload = f.Payload[:0]
}
func (f *Frame) Copy() *Frame {
copy := *f
copy.refs.Store(1)
copy.Payload = append([]byte{}, f.Payload...)
return &copy
}
func (f *Frame) Borrow() *Frame {
f.refs.Inc()
return f
}
func (f *Frame) Done() bool {
refs := f.refs.Dec()
if refs < 0 {
panic("invalid Done call")
}
if refs == 0 {
framePool.Put(f)
return true
}
return false
}
func (f *Frame) MarshalBinary(buffer []byte) (int, error) {
copy(buffer[:4], FrameMagicBytes)
buffer[4], buffer[5] = byte(f.Version), byte(f.Type)
offset := 6
switch f.Type {
case TypeVirtualSnakeBootstrap: // destination = key, source = coords
payloadLen := len(f.Payload)
binary.BigEndian.PutUint16(buffer[offset+0:offset+2], uint16(payloadLen))
src, err := f.Source.MarshalBinary()
if err != nil {
return 0, fmt.Errorf("f.Source.MarshalBinary: %w", err)
}
offset += 2
offset += copy(buffer[offset:], src)
offset += copy(buffer[offset:], f.DestinationKey[:ed25519.PublicKeySize])
if f.Payload != nil {
offset += copy(buffer[offset:], f.Payload[:payloadLen])
}
case TypeVirtualSnakeBootstrapACK:
payloadLen := len(f.Payload)
binary.BigEndian.PutUint16(buffer[offset+0:offset+2], uint16(payloadLen))
dst, err := f.Destination.MarshalBinary()
if err != nil {
return 0, fmt.Errorf("f.Destination.MarshalBinary: %w", err)
}
src, err := f.Source.MarshalBinary()
if err != nil {
return 0, fmt.Errorf("f.Source.MarshalBinary: %w", err)
}
binary.BigEndian.PutUint16(buffer[offset+2:offset+4], uint16(len(dst)))
binary.BigEndian.PutUint16(buffer[offset+4:offset+6], uint16(len(src)))
offset += 6
offset += copy(buffer[offset:], dst)
offset += copy(buffer[offset:], src)
offset += copy(buffer[offset:], f.DestinationKey[:ed25519.PublicKeySize])
offset += copy(buffer[offset:], f.SourceKey[:ed25519.PublicKeySize])
if f.Payload != nil {
offset += copy(buffer[offset:], f.Payload[:payloadLen])
}
case TypeVirtualSnakeSetup: // destination = coords & key, source = key
payloadLen := len(f.Payload)
binary.BigEndian.PutUint16(buffer[offset+0:offset+2], uint16(payloadLen))
dst, err := f.Destination.MarshalBinary()
if err != nil {
return 0, fmt.Errorf("f.Destination.MarshalBinary: %w", err)
}
offset += 2
offset += copy(buffer[offset:], dst)
offset += copy(buffer[offset:], f.SourceKey[:ed25519.PublicKeySize])
offset += copy(buffer[offset:], f.DestinationKey[:ed25519.PublicKeySize])
if f.Payload != nil {
offset += copy(buffer[offset:], f.Payload[:payloadLen])
}
case TypeVirtualSnake, TypeVirtualSnakePathfind: // destination = key, source = key
payloadLen := len(f.Payload)
binary.BigEndian.PutUint16(buffer[offset+0:offset+2], uint16(payloadLen))
offset += 2
offset += copy(buffer[offset:], f.DestinationKey[:ed25519.PublicKeySize])
offset += copy(buffer[offset:], f.SourceKey[:ed25519.PublicKeySize])
if f.Payload != nil {
offset += copy(buffer[offset:], f.Payload[:payloadLen])
}
default: // destination = coords, source = coords
dst, err := f.Destination.MarshalBinary()
if err != nil {
return 0, fmt.Errorf("f.Destination.MarshalBinary: %w", err)
}
src, err := f.Source.MarshalBinary()
if err != nil {
return 0, fmt.Errorf("f.Source.MarshalBinary: %w", err)
}
dstLen, srcLen, payloadLen := len(dst), len(src), len(f.Payload)
if dstLen > math.MaxUint16 || srcLen > math.MaxUint16 || payloadLen > math.MaxUint16 {
return 0, fmt.Errorf("frame contents too large")
}
binary.BigEndian.PutUint16(buffer[offset+0:offset+2], uint16(dstLen))
binary.BigEndian.PutUint16(buffer[offset+2:offset+4], uint16(srcLen))
binary.BigEndian.PutUint16(buffer[offset+4:offset+6], uint16(payloadLen))
offset += 6
offset += copy(buffer[offset:], dst)
offset += copy(buffer[offset:], src)
if f.Payload != nil {
offset += copy(buffer[offset:], f.Payload[:payloadLen])
}
}
return offset, nil
}
func (f *Frame) UnmarshalBinary(data []byte) (int, error) {
f.Reset()
if len(data) < 6 {
return 0, fmt.Errorf("frame is not long enough to include metadata")
}
if !bytes.Equal(data[:4], FrameMagicBytes) {
return 0, fmt.Errorf("frame doesn't contain magic bytes")
}
f.Version, f.Type = FrameVersion(data[4]), FrameType(data[5])
offset := 6
switch f.Type {
case TypeVirtualSnakeBootstrap: // destination = key, source = coords
payloadLen := int(binary.BigEndian.Uint16(data[offset+0 : offset+2]))
srcLen := int(binary.BigEndian.Uint16(data[offset+2 : offset+4]))
if _, err := f.Source.UnmarshalBinary(data[offset+2:]); err != nil {
return 0, fmt.Errorf("f.Source.UnmarshalBinary: %w", err)
}
offset += 4 + srcLen
offset += copy(f.DestinationKey[:], data[offset:])
f.Payload = append([]byte{}, data[offset:offset+payloadLen]...)
return offset, nil
case TypeVirtualSnakeBootstrapACK:
payloadLen := int(binary.BigEndian.Uint16(data[offset+0 : offset+2]))
dstLen := int(binary.BigEndian.Uint16(data[offset+2 : offset+4]))
srcLen := int(binary.BigEndian.Uint16(data[offset+4 : offset+6]))
offset += 6
if _, err := f.Destination.UnmarshalBinary(data[offset:]); err != nil {
return 0, fmt.Errorf("f.Destination.UnmarshalBinary: %w", err)
}
offset += dstLen
if _, err := f.Source.UnmarshalBinary(data[offset:]); err != nil {
return 0, fmt.Errorf("f.Destination.UnmarshalBinary: %w", err)
}
offset += srcLen
offset += copy(f.DestinationKey[:], data[offset:])
offset += copy(f.SourceKey[:], data[offset:])
f.Payload = append([]byte{}, data[offset:offset+payloadLen]...)
return offset, nil
case TypeVirtualSnakeSetup: // destination = coords & key, source = key
payloadLen := int(binary.BigEndian.Uint16(data[offset+0 : offset+2]))
dstLen := int(binary.BigEndian.Uint16(data[offset+2 : offset+4]))
if _, err := f.Destination.UnmarshalBinary(data[offset+2:]); err != nil {
return 0, fmt.Errorf("f.Destination.UnmarshalBinary: %w", err)
}
offset += 4 + dstLen
offset += copy(f.SourceKey[:], data[offset:])
offset += copy(f.DestinationKey[:], data[offset:])
f.Payload = append([]byte{}, data[offset:offset+payloadLen]...)
return offset, nil
case TypeVirtualSnake, TypeVirtualSnakePathfind: // destination = key, source = key
payloadLen := int(binary.BigEndian.Uint16(data[offset+0 : offset+2]))
offset += 2
offset += copy(f.DestinationKey[:], data[offset:])
offset += copy(f.SourceKey[:], data[offset:])
f.Payload = append([]byte{}, data[offset:offset+payloadLen]...)
return offset + payloadLen, nil
default: // destination = coords, source = coords
dstLen := int(binary.BigEndian.Uint16(data[offset+0 : offset+2]))
srcLen := int(binary.BigEndian.Uint16(data[offset+2 : offset+4]))
payloadLen := int(binary.BigEndian.Uint16(data[offset+4 : offset+6]))
offset += 6
if size := offset + dstLen + srcLen + payloadLen; len(data) != int(size) {
return 0, fmt.Errorf("frame expecting %d total bytes, got %d bytes", size, len(data))
}
if _, err := f.Destination.UnmarshalBinary(data[offset : offset+dstLen]); err != nil {
return 0, fmt.Errorf("f.Destination.UnmarshalBinary: %w", err)
}
offset += dstLen
if _, err := f.Source.UnmarshalBinary(data[offset : offset+srcLen]); err != nil {
return 0, fmt.Errorf("f.Source.UnmarshalBinary: %w", err)
}
offset += srcLen
f.Payload = append([]byte{}, data[offset:offset+payloadLen]...)
return offset + payloadLen, nil
}
}
func (f *Frame) UpdateSourceRoutedPath(from SwitchPortID) {
switch f.Type {
case TypeSource:
if len(f.Destination) > 0 {
f.Destination = f.Destination[1:]
}
f.Source = append(SwitchPorts{from}, f.Source...)
case TypeSTP:
f.Destination = SwitchPorts{}
f.Source = SwitchPorts{}
default:
panic("UpdateSourceRoutedPath: incorrect frame type")
}
}
func (t FrameType) String() string {
switch t {
case TypeSTP:
return "TypeSTP"
case TypeSource:
return "TypeSource"
case TypeGreedy:
return "TypeGreedy"
case TypePathfind:
return "TypePathfind"
case TypeSwitchUpdate:
return "TypeSwitchUpdate"
case TypeDHTRequest:
return "TypeDHTRequest"
case TypeDHTResponse:
return "TypeDHTResponse"
case TypeVirtualSnakeBootstrap:
return "TypeVirtualSnakeBootstrap"
case TypeVirtualSnakeBootstrapACK:
return "TypeVirtualSnakeBootstrapACK"
case TypeVirtualSnakeSetup:
return "TypeVirtualSnakeSetup"
case TypeVirtualSnake:
return "TypeVirtualSnake"
case TypeVirtualSnakePathfind:
return "TypeVirtualSnakePathfind"
default:
return "TypeUnknown"
}
}
func (v FrameVersion) String() string {
switch v {
case Version0:
return "Version0"
default:
return "VersionUnknown"
}
}

314
types/frame_test.go Normal file
View file

@ -0,0 +1,314 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
import (
"bytes"
"crypto/ed25519"
"fmt"
"testing"
)
func TestMarshalUnmarshalFrame(t *testing.T) {
input := Frame{
Version: Version0,
Type: TypeGreedy,
Destination: SwitchPorts{1, 2, 3, 4, 5000},
Source: SwitchPorts{4, 3, 2, 1},
Payload: []byte("ABCDEFG"),
}
expected := []byte{
0x70, 0x69, 0x6e, 0x65, // magic bytes
0, // version 0
2, // type greedy
0, 8, // destination len
0, 6, // source len
0, 7, // payload len
0, 6, 1, 2, 3, 4, 167, 8, // destination (2+6 bytes but 5 ports!)
0, 4, 4, 3, 2, 1, // source (2+4 bytes)
65, 66, 67, 68, 69, 70, 71, // payload (7 bytes)
}
buf := make([]byte, 65535)
n, err := input.MarshalBinary(buf)
if err != nil {
t.Fatal(err)
}
if n != len(expected) {
t.Fatalf("wrong marshalled length, got %d, expected %d", n, len(expected))
}
if !bytes.Equal(buf[:n], expected) {
t.Fatalf("wrong marshalled output, got %v", buf[:n])
}
var output Frame
if _, err := output.UnmarshalBinary(buf[:n]); err != nil {
t.Fatal(err)
}
if output.Version != input.Version {
t.Fatal("wrong version")
}
if output.Type != input.Type {
t.Fatal("wrong version")
}
if l := len(output.Destination); l != 5 {
t.Fatalf("wrong destination length (got %d, expected 5)", l)
}
if l := len(output.Source); l != 4 {
t.Fatalf("wrong source length (got %d, expected 4)", l)
}
if l := len(output.Payload); l != 7 {
t.Fatalf("wrong payload length (got %d, expected 7)", l)
}
if !output.Destination.EqualTo(input.Destination) {
t.Fatal("wrong path")
}
if !bytes.Equal(input.Payload, output.Payload) {
t.Fatal("wrong payload")
}
}
func TestMarshalUnmarshalSNEKBootstrapFrame(t *testing.T) {
pk, _, _ := ed25519.GenerateKey(nil)
input := Frame{
Version: Version0,
Type: TypeVirtualSnakeBootstrap,
Source: SwitchPorts{1, 2, 3, 4, 5},
Payload: []byte{9, 9, 9, 9, 9},
}
copy(input.DestinationKey[:], pk)
expected := []byte{
0x70, 0x69, 0x6e, 0x65, // magic bytes
0, // version 0
byte(TypeVirtualSnakeBootstrap), // type greedy
0, 5, // payload length
0, 5, // source length
1, 2, 3, 4, 5, // source coordinates
}
expected = append(expected, pk...)
expected = append(expected, input.Payload...)
buf := make([]byte, 65535)
n, err := input.MarshalBinary(buf)
if err != nil {
t.Fatal(err)
}
if n != len(expected) {
t.Fatalf("wrong marshalled length, got %d, expected %d", n, len(expected))
}
if !bytes.Equal(buf[:n], expected) {
t.Fatalf("wrong marshalled output, got %v", buf[:n])
}
t.Log("Got: ", buf[:n])
t.Log("Want:", expected)
var output Frame
if _, err := output.UnmarshalBinary(buf[:n]); err != nil {
t.Fatal(err)
}
if output.Version != input.Version {
t.Fatal("wrong version")
}
if output.Type != input.Type {
t.Fatal("wrong version")
}
if !output.Destination.EqualTo(input.Destination) {
t.Fatal("wrong path")
}
if !bytes.Equal(input.Payload, output.Payload) {
t.Fatal("wrong payload")
}
}
func TestMarshalUnmarshalSNEKBootstrapACKFrame(t *testing.T) {
pk1, _, _ := ed25519.GenerateKey(nil)
pk2, _, _ := ed25519.GenerateKey(nil)
input := Frame{
Version: Version0,
Type: TypeVirtualSnakeBootstrapACK,
Source: SwitchPorts{1, 2, 3, 4, 5},
Destination: SwitchPorts{5, 4, 3, 2, 1},
Payload: []byte{9, 9, 9, 9, 9},
}
copy(input.DestinationKey[:], pk1)
copy(input.SourceKey[:], pk2)
expected := []byte{
0x70, 0x69, 0x6e, 0x65, // magic bytes
0, // version 0
byte(TypeVirtualSnakeBootstrapACK), // type greedy
0, 5, // payload length
0, 7, // destination length
0, 7, // source length
0, 5, 5, 4, 3, 2, 1, // destination coordinates
0, 5, 1, 2, 3, 4, 5, // source coordinates
}
expected = append(expected, pk1...)
expected = append(expected, pk2...)
expected = append(expected, input.Payload...)
buf := make([]byte, 65535)
n, err := input.MarshalBinary(buf)
if err != nil {
t.Fatal(err)
}
if n != len(expected) {
t.Fatalf("wrong marshalled length, got %d, expected %d", n, len(expected))
}
if !bytes.Equal(buf[:n], expected) {
t.Fatalf("wrong marshalled output, got %v, expected %v", buf[:n], expected)
}
t.Log("Got: ", buf[:n])
t.Log("Want:", expected)
var output Frame
if _, err := output.UnmarshalBinary(buf[:n]); err != nil {
t.Fatal(err)
}
if output.Version != input.Version {
t.Fatal("wrong version")
}
if output.Type != input.Type {
t.Fatal("wrong version")
}
if !output.Destination.EqualTo(input.Destination) {
t.Fatal("wrong path")
}
if !bytes.Equal(input.Payload, output.Payload) {
t.Fatal("wrong payload")
}
}
func TestMarshalUnmarshalSNEKSetupFrame(t *testing.T) {
pk1, _, _ := ed25519.GenerateKey(nil)
pk2, _, _ := ed25519.GenerateKey(nil)
input := Frame{
Version: Version0,
Type: TypeVirtualSnakeSetup,
Destination: SwitchPorts{5, 4, 3, 2, 1},
Payload: []byte{9, 9, 9, 9, 9, 9, 9, 9, 9, 9},
}
copy(input.DestinationKey[:], pk1)
copy(input.SourceKey[:], pk2)
expected := []byte{
0x70, 0x69, 0x6e, 0x65, // magic bytes
0, // version 0
byte(TypeVirtualSnakeSetup), // type greedy
0, 10, // payload length
0, 5, 5, 4, 3, 2, 1, // destination coordinates
}
expected = append(expected, pk2...)
expected = append(expected, pk1...)
expected = append(expected, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9) // payload
buf := make([]byte, 65535)
n, err := input.MarshalBinary(buf)
if err != nil {
t.Fatal(err)
}
if n != len(expected) {
t.Fatalf("wrong marshalled length, \ngot %d, \nexpected %d", n, len(expected))
}
if !bytes.Equal(buf[:n], expected) {
t.Fatalf("wrong marshalled output, \ngot %v, \nexpected %v", buf[:n], expected)
}
t.Log("Got: ", buf[:n])
t.Log("Want:", expected)
var output Frame
if _, err := output.UnmarshalBinary(buf[:n]); err != nil {
t.Fatal(err)
}
if output.Version != input.Version {
t.Fatal("wrong version")
}
if output.Type != input.Type {
t.Fatal("wrong version")
}
if !output.Destination.EqualTo(input.Destination) {
t.Fatal("wrong path")
}
if !bytes.Equal(input.Payload, output.Payload) {
t.Fatal("wrong payload")
}
}
func TestMarshalUnmarshalSNEKFrame(t *testing.T) {
pk1, _, _ := ed25519.GenerateKey(nil)
pk2, _, _ := ed25519.GenerateKey(nil)
input := Frame{
Version: Version0,
Type: TypeVirtualSnake,
Payload: []byte("HELLO!"),
}
copy(input.SourceKey[:], pk1)
copy(input.DestinationKey[:], pk2)
expected := []byte{
0x70, 0x69, 0x6e, 0x65, // magic bytes
0, // version 0
byte(TypeVirtualSnake), // type greedy
0, 6, // payload length
}
expected = append(expected, pk2...)
expected = append(expected, pk1...)
expected = append(expected, input.Payload...)
buf := make([]byte, 65535)
n, err := input.MarshalBinary(buf)
if err != nil {
t.Fatal(err)
}
if n != len(expected) {
t.Fatalf("wrong marshalled length, got %d, expected %d", n, len(expected))
}
if !bytes.Equal(buf[:n], expected) {
fmt.Println("got: ", buf[:n])
fmt.Println("want:", expected)
t.Fatalf("wrong marshalled output")
}
var output Frame
if _, err := output.UnmarshalBinary(buf[:n]); err != nil {
t.Fatal(err)
}
if output.Version != input.Version {
t.Fatal("wrong version")
}
if output.Type != input.Type {
t.Fatal("wrong version")
}
if !output.Destination.EqualTo(input.Destination) {
t.Fatal("wrong path")
}
if !bytes.Equal(input.Payload, output.Payload) {
fmt.Println("want: ", input.Payload)
fmt.Println("got: ", output.Payload)
t.Fatal("wrong payload")
}
}
func TestUpdatePath(t *testing.T) {
frame := Frame{
Version: Version0,
Type: TypeSource,
Destination: SwitchPorts{1, 2, 3, 4},
Source: SwitchPorts{},
}
expectedDst := SwitchPorts{2, 3, 4}
expectedSrc := SwitchPorts{5}
frame.UpdateSourceRoutedPath(5)
if !frame.Destination.EqualTo(expectedDst) {
t.Fatalf("got %v, expected %v", frame.Destination, expectedDst)
}
if !frame.Source.EqualTo(expectedSrc) {
t.Fatalf("got %v, expected %v", frame.Source, expectedSrc)
}
}

115
types/pathfind.go Normal file
View file

@ -0,0 +1,115 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
import (
"crypto/ed25519"
"fmt"
"os"
)
type Pathfind struct {
Boundary uint8
VsetPath bool
Signatures []SignatureWithHop
}
func (p Pathfind) Sign(privKey ed25519.PrivateKey, forPort SwitchPortID) (*Pathfind, error) {
var body [65535]byte
n, err := p.MarshalBinary(body[:])
if err != nil {
return nil, fmt.Errorf("a.MarshalBinary: %w", err)
}
hop := SignatureWithHop{
Hop: Varu64(forPort),
}
copy(hop.PublicKey[:], privKey.Public().(ed25519.PublicKey))
if _, ok := os.LookupEnv("PINECONE_DISABLE_SIGNATURES"); !ok {
copy(hop.Signature[:], ed25519.Sign(privKey, body[1:n]))
}
p.Signatures = append(p.Signatures, hop)
return &p, nil
}
func (p *Pathfind) ReturnPath(reverse bool) SwitchPorts {
signatures := p.Signatures
if len(signatures) < int(p.Boundary) {
return SwitchPorts{}
}
if reverse {
signatures = signatures[p.Boundary:]
} else {
signatures = signatures[:p.Boundary-1]
}
count := len(signatures)
path := make(SwitchPorts, count)
if reverse {
for i, sig := range signatures {
path[count-1-i] = SwitchPortID(sig.Hop)
}
} else {
for i, sig := range signatures {
path[i] = SwitchPortID(sig.Hop)
}
}
return path
}
func (p Pathfind) MarshalBinary(buf []byte) (int, error) {
offset := copy(buf, []byte{byte(p.Boundary)})
if p.VsetPath {
offset += copy(buf[offset:], []byte{1})
} else {
offset += copy(buf[offset:], []byte{0})
}
for _, sig := range p.Signatures {
n, err := sig.MarshalBinary(buf[offset:])
if err != nil {
return 0, fmt.Errorf("a.MarshalBinary: %w", err)
}
if n < SignatureWithHopMinSize {
return 0, fmt.Errorf("not enough signature bytes")
}
offset += n
}
return offset, nil
}
func (p *Pathfind) UnmarshalBinary(b []byte) (int, error) {
if len(b) == 0 {
p.Signatures = []SignatureWithHop{}
return 0, nil
}
p.Boundary = uint8(b[0])
if b[1] != 0 {
p.VsetPath = true
}
remaining := b[2:]
for i := Varu64(0); len(remaining) >= ed25519.PublicKeySize+ed25519.SignatureSize+1; i++ {
var signature SignatureWithHop
n, err := signature.UnmarshalBinary(remaining[:])
if err != nil {
return 0, fmt.Errorf("signature.UnmarshalBinary: %w", err)
}
if _, ok := os.LookupEnv("PINECONE_DISABLE_SIGNATURES"); !ok {
if !ed25519.Verify(signature.PublicKey[:], b[1:len(b)-len(remaining)], signature.Signature[:]) {
return 0, fmt.Errorf("signature verification failed")
}
}
p.Signatures = append(p.Signatures, signature)
remaining = remaining[n:]
}
return len(b) - len(remaining), nil
}

51
types/pathfind_test.go Normal file
View file

@ -0,0 +1,51 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
import (
"crypto/ed25519"
"testing"
)
func TestMarshalUnmarshalPathfind(t *testing.T) {
_, sk1, _ := ed25519.GenerateKey(nil)
_, sk2, _ := ed25519.GenerateKey(nil)
_, sk3, _ := ed25519.GenerateKey(nil)
input := &Pathfind{
Boundary: 1,
}
var err error
input, err = input.Sign(sk1, 1)
if err != nil {
t.Fatal(err)
}
input, err = input.Sign(sk2, 2)
if err != nil {
t.Fatal(err)
}
input, err = input.Sign(sk3, 3)
if err != nil {
t.Fatal(err)
}
var buffer [65535]byte
n, err := input.MarshalBinary(buffer[:])
if err != nil {
t.Fatal(err)
}
var output Pathfind
if _, err = output.UnmarshalBinary(buffer[:n]); err != nil {
t.Fatal(err)
}
}

57
types/signaturehop.go Normal file
View file

@ -0,0 +1,57 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
import (
"crypto/ed25519"
"fmt"
)
type SignatureWithHop struct {
Hop Varu64
PublicKey PublicKey
Signature Signature
}
const SignatureWithHopMinSize = ed25519.PublicKeySize + ed25519.SignatureSize + 1
func (a *SignatureWithHop) UnmarshalBinary(data []byte) (int, error) {
if size := len(data); size < SignatureWithHopMinSize {
return 0, fmt.Errorf("SignatureWithHop expects at least %d bytes, got %d bytes", SignatureWithHopMinSize, size)
}
if err := a.Hop.UnmarshalBinary(data); err != nil {
return 0, fmt.Errorf("a.Hop.UnmarshalBinary: %w", err)
}
offset := a.Hop.Length()
remaining := data[offset:]
remaining = remaining[copy(a.PublicKey[:], remaining):]
remaining = remaining[copy(a.Signature[:], remaining):]
return len(data) - len(remaining), nil
}
func (a *SignatureWithHop) MarshalBinary(data []byte) (int, error) {
hop, err := a.Hop.MarshalBinary()
if err != nil {
return 0, fmt.Errorf("a.Hop.MarshalBinary: %w", err)
}
if len(data) < SignatureWithHopMinSize {
return 0, fmt.Errorf("buffer is not big enough (must be %d bytes)", SignatureWithHopMinSize)
}
offset := 0
offset += copy(data[offset:], hop)
offset += copy(data[offset:offset+ed25519.PublicKeySize], a.PublicKey[:])
offset += copy(data[offset:offset+ed25519.SignatureSize], a.Signature[:])
return offset, nil
}

127
types/switchports.go Normal file
View file

@ -0,0 +1,127 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
import (
"encoding/binary"
"fmt"
"strings"
)
type SwitchPortID Varu64
type SwitchPorts []SwitchPortID
func (s SwitchPorts) Len() int {
return len(s)
}
func (s SwitchPorts) Swap(i, j int) {
s[i], s[j] = s[j], s[i]
}
func (s SwitchPorts) Less(i, j int) bool {
return s[i] < s[j]
}
func (s SwitchPorts) String() string {
ports := make([]string, 0, len(s))
for _, p := range s {
ports = append(ports, fmt.Sprintf("%d", p))
}
return "[" + strings.Join(ports, " ") + "]"
}
func (p SwitchPorts) MarshalBinary() ([]byte, error) {
buf := make([]byte, 2, 2+len(p)*10)
l := 0
for _, a := range p {
b, err := Varu64(a).MarshalBinary()
if err != nil {
return buf, fmt.Errorf("Varu64(a).MarshalBinary: %w", err)
}
buf = append(buf, b...)
l += len(b)
}
binary.BigEndian.PutUint16(buf[:2], uint16(l))
return buf[:l+2], nil
}
func (p *SwitchPorts) UnmarshalBinary(b []byte) (int, error) {
l := int(binary.BigEndian.Uint16(b[:2]))
ports := make(SwitchPorts, 0, 16)
if rl := len(b); rl < 2+l {
return 0, fmt.Errorf("expecting %d bytes but got %d bytes", 2+l, rl)
}
read := 2
b = b[read : 2+l]
for {
if len(b) < 1 {
break
}
var id Varu64
if err := id.UnmarshalBinary(b); err != nil {
return 0, fmt.Errorf("id.UnmarshalBinary: %w", err)
}
ports = append(ports, SwitchPortID(id))
b = b[id.Length():]
read += id.Length()
}
*p = ports
return read, nil
}
func (p SwitchPorts) EqualTo(o SwitchPorts) bool {
if len(p) != len(o) {
return false
}
for i := range p {
if p[i] != o[i] {
return false
}
}
return true
}
func (a SwitchPorts) DistanceTo(b SwitchPorts) int {
// Find out what the common ancestor is between "a" and "b".
ancestor := a[:getCommonPrefix(a, b)]
// Work out what the distance is between the destination and
// the common ancestor, and the peer and the common ancestor.
distA, distB := len(a)-len(ancestor), len(b)-len(ancestor)
// Work out the total distance on the tree between the peer
// and the destination.
return distA + distB
}
func (a *SwitchPorts) Copy() SwitchPorts {
return append(SwitchPorts{}, *a...)
}
func getCommonPrefix(a, b SwitchPorts) int {
c := 0
l := len(a)
if len(b) < l {
l = len(b)
}
for i := 0; i < l; i++ {
if a[i] != b[i] {
break
}
c++
}
return c
}

39
types/switchports_test.go Normal file
View file

@ -0,0 +1,39 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
import (
"bytes"
"testing"
)
func TestSwitchPorts(t *testing.T) {
expected := []byte{0, 5, 1, 2, 3, 159, 32}
input := SwitchPorts{1, 2, 3, 4000}
b, err := input.MarshalBinary()
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(b, expected) {
t.Fatalf("MarshalBinary produced %v, expected %v", b, expected)
}
var output SwitchPorts
if _, err := output.UnmarshalBinary(b); err != nil {
t.Fatal(err)
}
if !input.EqualTo(output) {
t.Fatalf("Expected %v, got %v", input, output)
}
}

50
types/varu64.go Normal file
View file

@ -0,0 +1,50 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
type Varu64 uint64
func (n Varu64) MarshalBinary() ([]byte, error) {
var b [10]byte
i := len(b) - 1
b[i] = byte(n & 0x7f)
for n >>= 7; n != 0; n >>= 7 {
i--
b[i] = byte(n | 0x80)
}
return b[i:], nil
}
func (n *Varu64) UnmarshalBinary(buf []byte) error {
l := 0
*n = Varu64(0)
for _, b := range buf {
*n <<= 7
*n |= Varu64(b & 0x7f)
l++
if b&0x80 == 0 {
break
}
}
return nil
}
func (n Varu64) Length() int {
l := 1
for e := n >> 7; e > 0; e >>= 7 {
l++
}
return l
}

47
types/varu64_test.go Normal file
View file

@ -0,0 +1,47 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package types
import (
"bytes"
"testing"
)
func TestMarshalBinaryVaru64(t *testing.T) {
input := Varu64(12345678)
expected := []byte{133, 241, 194, 78}
bin, err := input.MarshalBinary()
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(bin, expected) {
t.Fatalf("expected %v, got %v", expected, bin)
}
if length := input.Length(); length != len(expected) {
t.Fatalf("expected length %d, got %d", length, len(expected))
}
}
func TestUnmarshalBinaryVaru64(t *testing.T) {
input := []byte{133, 241, 194, 78, 0, 1, 2, 3, 4, 5, 6}
expected := Varu64(12345678)
var num Varu64
if err := num.UnmarshalBinary(input); err != nil {
t.Fatal(err)
}
if num != expected {
t.Fatalf("expected %v, got %v", expected, num)
}
}

54
util/bufferedrwc.go Normal file
View file

@ -0,0 +1,54 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package util
import (
"bufio"
"io"
)
type BufferedRWC struct {
r *bufio.Reader
w *bufio.Writer
io.ReadWriteCloser
}
func NewBufferedRWC(c io.ReadWriteCloser) BufferedRWC {
return BufferedRWC{bufio.NewReader(c), bufio.NewWriter(c), c}
}
func NewBufferedRWCSize(c io.ReadWriteCloser, n int) BufferedRWC {
return BufferedRWC{bufio.NewReaderSize(c, n), bufio.NewWriterSize(c, n), c}
}
func (b BufferedRWC) Peek(n int) ([]byte, error) {
return b.r.Peek(n)
}
func (b BufferedRWC) Read(p []byte) (int, error) {
return b.r.Read(p)
}
func (b BufferedRWC) Write(p []byte) (int, error) {
return b.w.Write(p)
}
func (b BufferedRWC) Flush() error {
return b.w.Flush()
}
func (b BufferedRWC) Discard(n int) (int, error) {
return b.r.Discard(n)
}

28
util/dispatch.go Normal file
View file

@ -0,0 +1,28 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package util
type Dispatch chan struct{}
func NewDispatch() Dispatch {
return make(Dispatch, 1)
}
func (d *Dispatch) Dispatch() {
select {
case *d <- struct{}{}:
default:
}
}

68
util/distance.go Normal file
View file

@ -0,0 +1,68 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package util
import (
"crypto/ed25519"
"github.com/matrix-org/pinecone/types"
)
func LessThan(first, second types.PublicKey) bool {
for i := 0; i < ed25519.PublicKeySize; i++ {
if first[i] < second[i] {
return true
}
if first[i] > second[i] {
return false
}
}
return false
}
// DHTOrdered returns true if the order of A, B and C is
// correct, where A < B < C without wrapping.
func DHTOrdered(a, b, c types.PublicKey) bool {
return LessThan(a, b) && LessThan(b, c)
}
// DHTWrappedOrdered returns true if the ordering of A, B
// and C is correct, where we may wrap around from C to A.
// This gives us the property of the successor always being
// a+1 and the predecessor being a+sizeofkeyspace.
func DHTWrappedOrdered(a, b, c types.PublicKey) bool {
ab, bc, ca := LessThan(a, b), LessThan(b, c), LessThan(c, a)
switch {
case ab && bc:
return true
case bc && ca:
return true
case ca && ab:
return true
}
return false
}
func ReverseOrdering(target types.PublicKey, input []types.PublicKey) func(i, j int) bool {
return func(i, j int) bool {
return DHTWrappedOrdered(input[i], target, input[j])
}
}
func ForwardOrdering(target types.PublicKey, input []types.PublicKey) func(i, j int) bool {
return func(i, j int) bool {
return DHTWrappedOrdered(target, input[i], input[j])
}
}

63
util/distance_test.go Normal file
View file

@ -0,0 +1,63 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package util
import (
"fmt"
"sort"
"strings"
"testing"
"github.com/matrix-org/pinecone/types"
)
func TestDHTWrappedOrdering(t *testing.T) {
target := types.PublicKey{5}
input := []types.PublicKey{
{1}, {2}, {3}, {9}, {7}, {4}, {6}, {8}, {0},
}
reverse := []types.PublicKey{
{4}, {3}, {2}, {1}, {0}, {9}, {8}, {7}, {6},
}
forward := []types.PublicKey{
{6}, {7}, {8}, {9}, {0}, {1}, {2}, {3}, {4},
}
out := func(show []types.PublicKey) string {
var parts []string
for _, i := range show {
parts = append(parts, fmt.Sprintf("%d", i[0]))
}
return strings.Join(parts, ", ")
}
sort.SliceStable(input, ForwardOrdering(target, input))
for i := range input {
if input[i] != forward[i] {
t.Log("Want:", out(forward))
t.Log("Got: ", out(input))
t.Fatalf("Successor ordering incorrect")
}
}
sort.SliceStable(input, ReverseOrdering(target, input))
for i := range input {
if input[i] != reverse[i] {
t.Log("Want:", out(reverse))
t.Log("Got: ", out(input))
t.Fatalf("Predecessor ordering incorrect")
}
}
}

68
util/overlay.go Normal file
View file

@ -0,0 +1,68 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package util
import (
"sort"
"github.com/matrix-org/pinecone/types"
)
type Overlay struct {
target types.PublicKey
ourkey types.PublicKey
keys []types.PublicKey
}
func (o *Overlay) candidates() ([]types.PublicKey, error) {
sort.SliceStable(o.keys, ForwardOrdering(o.ourkey, o.keys))
mustWrap := o.target.CompareTo(o.ourkey) < 0
hasWrapped := !mustWrap
cap, last := len(o.keys), o.keys[0]
for i, k := range o.keys {
if hasWrapped {
if k.CompareTo(o.target) > 0 {
cap = i
break
}
} else {
hasWrapped = k.CompareTo(last) < 0
}
}
o.keys = o.keys[:cap]
nc := len(o.keys)
if nc > 3 {
nc = 3
}
candidates := []types.PublicKey{}
candidates = append(candidates, o.keys[:nc]...)
kc := len(o.keys)
if kc > 10 {
candidates = append(candidates, o.keys[kc/8])
}
if kc > 7 {
candidates = append(candidates, o.keys[kc/4])
}
if kc > 4 {
candidates = append(candidates, o.keys[kc/2])
}
return candidates, nil
}

50
util/overlay_test.go Normal file
View file

@ -0,0 +1,50 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package util
import (
"crypto/ed25519"
"fmt"
"testing"
"github.com/matrix-org/pinecone/types"
)
func TestOverlaySorting(t *testing.T) {
overlay := &Overlay{}
opk, _, _ := ed25519.GenerateKey(nil)
tpk, _, _ := ed25519.GenerateKey(nil)
copy(overlay.ourkey[:], opk)
copy(overlay.target[:], tpk)
fmt.Println("Our key: ", overlay.ourkey)
fmt.Println("Target key:", overlay.target)
for i := 0; i < 32; i++ {
pk, _, _ := ed25519.GenerateKey(nil)
k := types.PublicKey{}
copy(k[:], pk)
overlay.keys = append(overlay.keys[:], k)
}
fmt.Println("Candidates:")
candidates, err := overlay.candidates()
if err != nil {
panic(err)
}
for _, k := range candidates {
fmt.Println("*", k)
}
}

57
util/signedts.go Normal file
View file

@ -0,0 +1,57 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package util
import (
"crypto/ed25519"
"fmt"
"os"
"time"
"github.com/matrix-org/pinecone/types"
)
func SignedTimestamp(private types.PrivateKey) ([]byte, error) {
ts, err := types.Varu64(time.Now().Unix()).MarshalBinary()
if err != nil {
return nil, fmt.Errorf("types.Varu64.MarshalBinary: %w", err)
}
if _, ok := os.LookupEnv("PINECONE_DISABLE_SIGNATURES"); !ok {
ts = append(ts, ed25519.Sign(private[:], ts)...)
} else {
ts = append(ts, make([]byte, ed25519.SignatureSize)...)
}
return ts, nil
}
func VerifySignedTimestamp(public types.PublicKey, payload []byte) bool {
if _, ok := os.LookupEnv("PINECONE_DISABLE_SIGNATURES"); !ok {
if len(payload) < ed25519.SignatureSize+1 {
return false
}
var ts types.Varu64
if err := ts.UnmarshalBinary(payload); err != nil {
return false
}
offset := ts.Length()
if !ed25519.Verify(public[:], payload[:offset], payload[offset:]) {
return false
}
if time.Since(time.Unix(int64(ts), 0)) > time.Minute {
return false
}
}
return true
}

95
util/websocket.go Normal file
View file

@ -0,0 +1,95 @@
// Copyright 2021 The Matrix.org Foundation C.I.C.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package util
import (
"io"
"net"
"time"
"github.com/gorilla/websocket"
)
func WrapWebSocketConn(c *websocket.Conn) *WebSocketConn {
return &WebSocketConn{c: c}
}
type WebSocketConn struct {
r io.Reader
c *websocket.Conn
}
func (c *WebSocketConn) Write(p []byte) (int, error) {
err := c.c.WriteMessage(websocket.BinaryMessage, p)
if err != nil {
return 0, err
}
return len(p), nil
}
func (c *WebSocketConn) Read(p []byte) (int, error) {
for {
if c.r == nil {
// Advance to next message.
var err error
_, c.r, err = c.c.NextReader()
if err != nil {
return 0, err
}
}
n, err := c.r.Read(p)
if err == io.EOF {
// At end of message.
c.r = nil
if n > 0 {
return n, nil
} else {
// No data read, continue to next message.
continue
}
}
return n, err
}
}
func (c *WebSocketConn) Close() error {
return c.c.Close()
}
func (c *WebSocketConn) LocalAddr() net.Addr {
return c.c.LocalAddr()
}
func (c *WebSocketConn) RemoteAddr() net.Addr {
return c.c.RemoteAddr()
}
func (c *WebSocketConn) SetDeadline(t time.Time) error {
if err := c.SetReadDeadline(t); err != nil {
return err
}
if err := c.SetWriteDeadline(t); err != nil {
return err
}
return nil
}
func (c *WebSocketConn) SetReadDeadline(t time.Time) error {
return c.c.SetReadDeadline(t)
}
func (c *WebSocketConn) SetWriteDeadline(t time.Time) error {
return c.c.SetWriteDeadline(t)
}