tutorial.rst revision 792df052
1******** 2Tutorial 3******** 4 5.. highlight:: c 6 7Introduction 8============ 9 10The LSQUIC library provides facilities for operating a QUIC (Google QUIC 11or IETF QUIC) server or client with optional HTTP (or HTTP/3) functionality. 12To do that, it specifies an application programming interface (API) and 13exposes several basic object types to operate upon: 14 15- engine; 16- connection; and 17- stream. 18 19Engine 20------ 21 22An engine manages connections, processes incoming packets, and schedules outgoing packets. It can be instantiated in either server or client mode. If your program needs to have both QUIC client and server functionality, instantiate two engines. (This is what we do in our LiteSpeed ADC server.) 23In addition, HTTP mode can be turned on for gQUIC and HTTP/3 support. 24 25Connection 26---------- 27 28A connection carries one or more streams, ensures reliable data delivery, and handles the protocol details. 29In client mode, a connection is created using a function call, which we will cover later in the tutorial. 30In server mode, by the time the user code gets a hold of the connection object, the handshake has already been completed successfully. This is not the case in client mode. 31 32Stream 33------ 34 35A connection can have several streams in parallel and many streams during its lifetime. 36Streams do not exist by themselves; they belong to a connection. Streams are bidirectional and usually correspond to a request/response exchange - depending on the application protocol. 37Application data is transmitted over streams. 38 39HTTP Mode 40--------- 41 42The HTTP support is included directly into LSQUIC. The library hides the interaction between the HTTP application layer and the QUIC transport layer and presents a simple, unified way of sending and receiving HTTP messages. (By "unified way," we mean between Google QUIC and HTTP/3). Behind the scenes, the library will compress and decompress HTTP headers, add and remove HTTP/3 stream framing, and operate the necessary control streams. 43 44In the following sections, we will describe how to: 45 46- initialize the library; 47- configure and instantiate an engine object; 48- send and receive packets; and 49- work with connections and streams. 50 51Include Files 52------------- 53 54In your source files, you need to include a single header, "lsquic.h". 55It pulls in an auxiliary file "lsquic_types.h". 56 57:: 58 59 #include "lsquic.h" 60 61Library Initialization 62====================== 63 64Before the first engine object is instantiated, the library must be 65initialized using :func:`lsquic_global_init()`: 66 67:: 68 69 if (0 != lsquic_global_init(LSQUIC_GLOBAL_CLIENT|LSQUIC_GLOBAL_SERVER)) 70 { 71 exit(EXIT_FAILURE); 72 } 73 /* OK, do something useful */ 74 75This will initialize the crypto library, gQUIC server certificate cache, and, depending on the platform, monotonic timers. 76If you plan to instantiate engines only in a single mode, client or server, 77you can omit the appropriate flag. 78 79After all engines have been destroyed and the LSQUIC library is no longer 80going to be used, the global initialization can be undone: 81 82:: 83 84 lsquic_global_cleanup(); 85 exit(EXIT_SUCCESS); 86 87Engine Instantiation 88==================== 89 90Engine instantiation is performed by :func:`lsquic_engine_new()`: 91 92:: 93 94 /* Create an engine in server mode with HTTP behavior: */ 95 lsquic_engine_t *engine 96 = lsquic_engine_new(LSENG_SERVER|LSENG_HTTP, &engine_api); 97 98The engine mode is selected by using the :macro:`LSENG_SERVER` flag. 99If present, the engine will be in server mode; if not, the engine will 100be in client mode. If you need both server and client functionality 101in your program, instantiate two engines (or as many as you like). 102 103Using the :macro:`LSENG_HTTP` flag enables the HTTP behavior: The library 104hides the interaction between the HTTP application layer and the QUIC 105transport layer and presents a simple, unified (between Google QUIC and 106HTTP/3) way of sending and receiving HTTP messages. Behind the scenes, 107the library will compress and uncompress HTTP headers, add and remove 108HTTP/3 stream framing, and operate the necessary control streams. 109 110Engine Configuration 111-------------------- 112 113The second argument to :func:`lsquic_engine_new()` is a pointer to 114a struct of type :type:`lsquic_engine_api`. This structure lists 115several user-specified function pointers that the engine is to use 116to perform various functions. Mandatory among these are: 117 118- function to set packets out, :member:`lsquic_engine_api.ea_packets_out`; 119- functions linked to connection and stream events, 120 :member:`lsquic_engine_api.ea_stream_if`; 121- function to look up certificate to use, :member:`lsquic_engine_api.ea_lookup_cert` (in server mode); and 122- function to fetch SSL context, :member:`lsquic_engine_api.ea_get_ssl_ctx` (in server mode). 123 124The minimal structure for a client will look like this: 125 126:: 127 128 lsquic_engine_api engine_api = { 129 .ea_packets_out = send_packets_out, 130 .ea_packets_out_ctx = (void *) sockfd, /* For example */ 131 .ea_stream_if = &stream_callbacks, 132 .ea_stream_if_ctx = &some_context, 133 }; 134 135Engine Settings 136--------------- 137 138Engine settings can be changed by specifying 139:member:`lsquic_engine_api.ea_settings`. There are **many** parameters 140to tweak: supported QUIC versions, amount of memory dedicated to connections 141and streams, various timeout values, and so on. See 142:ref:`apiref-engine-settings` for full details. If ``ea_settings`` is set 143to ``NULL``, the engine will use the defaults, which should be OK. 144 145 146Receiving Packets 147================= 148 149UDP datagrams are passed to the engine using the :func:`lsquic_engine_packet_in()` function. This is the only way to do so. 150A pointer to the UDP payload is passed along with the size of the payload. 151Local and peer socket addresses are passed in as well. 152The void "peer ctx" pointer is associated with the peer address. It gets passed to the function that sends outgoing packets and to a few other callbacks. In a standard setup, this is most likely the socket file descriptor, but it could be pointing to something else. 153The ECN value is in the range of 0 through 3, as in RFC 3168. 154 155:: 156 157 /* 0: processed by real connection 158 * 1: handled 159 * -1: error: invalid arguments, malloc failure 160 */ 161 int 162 lsquic_engine_packet_in (lsquic_engine_t *, 163 const unsigned char *udp_payload, size_t sz, 164 const struct sockaddr *sa_local, 165 const struct sockaddr *sa_peer, 166 void *peer_ctx, int ecn); 167 168Why specify local address 169------------------------- 170 171The local address is necessary because it becomes the source address of the outgoing packets. This is important in a multihomed configuration, when packets arriving at a socket can have different destination addresses. Changes in local and peer addresses are also used to detect changes in paths, such as path migration during the classic "parking lot" scenario or NAT rebinding. When path change is detected, QUIC connection performs special steps to validate the new path. 172 173Sending Packets 174=============== 175 176The :member:`lsquic_engine_api.ea_packets_out` is the function that gets 177called when an engine instance has packets to send. It could look like 178this: 179 180:: 181 182 /* Return number of packets sent or -1 on error */ 183 static int 184 send_packets_out (void *ctx, const struct lsquic_out_spec *specs, 185 unsigned n_specs) 186 { 187 struct msghdr msg; 188 int sockfd; 189 unsigned n; 190 191 memset(&msg, 0, sizeof(msg)); 192 sockfd = (int) (uintptr_t) ctx; 193 194 for (n = 0; n < n_specs; ++n) 195 { 196 msg.msg_name = (void *) specs[n].dest_sa; 197 msg.msg_namelen = sizeof(struct sockaddr_in); 198 msg.msg_iov = specs[n].iov; 199 msg.msg_iovlen = specs[n].iovlen; 200 if (sendmsg(sockfd, &msg, 0) < 0) 201 break; 202 } 203 204 return (int) n; 205 } 206 207Note that the version above is very simple: it does not use local 208address and ECN value specified in :type:`lsquic_out_spec`. 209These can be set using ancillary data in a platform-dependent way. 210 211When an error occurs 212-------------------- 213 214When an error occurs, the value of ``errno`` is examined: 215 216- ``EAGAIN`` (or ``EWOULDBLOCK``) means that the packets could not be sent and to retry later. It is up to the caller to call :func:`lsquic_engine_send_unsent_packets()` when sending can resume. 217- ``EMSGSIZE`` means that a packet was too large. This occurs when lsquic send MTU probes. In that case, the engine will retry sending without the offending packet immediately. 218- Any other error causes the connection whose packet could not be sent to be terminated. 219 220Outgoing Packet Specification 221----------------------------- 222 223:: 224 225 struct lsquic_out_spec 226 { 227 struct iovec *iov; 228 size_t iovlen; 229 const struct sockaddr *local_sa; 230 const struct sockaddr *dest_sa; 231 void *peer_ctx; 232 int ecn; /* 0 - 3; see RFC 3168 */ 233 }; 234 235 236Each packet specification in the array given to the "packets out" function looks like this. In addition to the packet payload, specified via an iovec, the specification contains local and remote addresses, the peer context associated with the connection (which is just a file descriptor in tut.c), and ECN. 237The reason for using iovec in the specification is that a UDP datagram may contain several QUIC packets. QUIC packets with long headers, which are used during QUIC handshake, can be coalesced and lsquic tries to do that to reduce the number of datagrams needed to be sent. On the incoming side, :func:`lsquic_engine_packet_in()` takes care of splitting incoming UDP datagrams into individual packets. 238 239When to process connections 240=========================== 241 242Now that we covered how to initialize the library, instantiate an engine, and send and receive packets, it is time to see how to make the engine tick. "LSQUIC" has the concept of "tick," which is a way to describe a connection doing something productive. Other verbs could have been "kick," "prod," "poke," and so on, but we settled on "tick." 243 244There are several ways for a connection to do something productive. When a connection can do any of these things, it is "tickable:" 245 246- There are incoming packets to process 247- A user wants to read from a stream and there is data that can be read 248- A user wants to write to a stream and the stream is writeable 249- A stream has buffered packets generated when a user has written to stream outside of the regular callback mechanism. (This is allowed as an optimization: sometimes data becomes available and it's faster to just write to stream than to buffer it in the user code and wait for the "on write" callback.) 250- Internal QUIC protocol or LSQUIC maintenance actions need to be taken, such as sending out a control frame or recycling a stream. 251 252:: 253 254 /* Returns true if there are connections to be processed, in 255 * which case `diff' is set to microseconds from current time. 256 */ 257 int 258 lsquic_engine_earliest_adv_tick (lsquic_engine_t *, int *diff); 259 260There is a single function, 261:func:`lsquic_engine_earliest_adv_tick()`, that can tell the user whether and when there is at least one connection managed by an engine that needs to be ticked. "Adv" in the name of the function stands for "advisory," meaning that you do not have to process connections at that exact moment; it is simply recommended. If there is a connection to be ticked, the function will return a true value and ``diff`` will be set to a relative time to when the connection is to be ticked. This value may be negative, which means that the best time to tick the connection has passed. 262The engine keeps all connections in several data structures. It tracks each connection's timers and knows when it needs to fire. 263 264Example with libev 265------------------ 266 267:: 268 269 void 270 process_conns (struct tut *tut) 271 { 272 ev_tstamp timeout; 273 int diff; 274 ev_timer_stop(); 275 lsquic_engine_process_conns(engine); 276 if (lsquic_engine_earliest_adv_tick(engine, &diff) { 277 if (diff > 0) 278 timeout = (ev_tstamp) diff / 1000000; /* To seconds */ 279 else 280 timeout = 0.; 281 ev_timer_init(timeout) 282 ev_timer_start(); 283 } 284 } 285 286Here is a simple example that uses the libev library. First, we stop the timer and process connections. Then, we query the engine to tell us when the next advisory tick time is. Based on that, we calculate the timeout to reinitialize the timer with and start the timer. 287If ``diff`` is negative, we set timeout to zero. 288When the timer expires (not shown here), it simply calls this ``process_conns()`` again. 289 290Note that one could ignore the advisory tick time and simply process connections every few milliseconds and it will still work. This, however, will result in worse performance. 291 292Processing Connections 293---------------------- 294 295Recap: 296To process connections, call :func:`lsquic_engine_process_conns()`. 297This will call necessary callbacks to read from and write to streams 298and send packets out. Call `lsquic_engine_process_conns()` when advised 299by `lsquic_engine_earliest_adv_tick()`. 300 301Do not call `lsquic_engine_process_conns()` from inside callbacks, for 302this function is not reentrant. 303 304Another function that sends packets is 305:func:`lsquic_engine_send_unsent_packets()`. Call it if there was a 306previous failure to send out all packets 307 308Required Engine Callbacks 309========================= 310 311Now we continue to initialize our engine instance. We have covered the callback to send out packets. This is one of the required engine callbacks. 312Other required engine callbacks are a set of stream and connection callbacks that get called on various events in then connections and stream lifecycles and a callback to get the default TLS context. 313 314:: 315 316 struct lsquic_engine_api engine_api = { 317 /* --- 8< --- snip --- 8< --- */ 318 .ea_stream_if = &stream_callbacks, 319 .ea_stream_if_ctx = &some_context, 320 .ea_get_ssl_ctx = get_ssl_ctx, /* Server only */ 321 }; 322 323 324Optional Callbacks 325------------------ 326 327Here we mention some optional callbacks. While they are not covered by 328this tutorial, it is good to know that they are available. 329 330- Looking up certificate and TLS context by SNI. 331- Callbacks to control memory allocation for outgoing packets. These are useful when sending packets using a custom library. For example, when all packets must be in contiguous memory. 332- Callbacks to observe connection ID lifecycle. These are useful in multi-process applications. 333- Callbacks that provide access to a shared-memory hash. This is also used in multi-process applications. 334- HTTP header set processing. These callbacks may be used in HTTP mode for HTTP/3 and Google QUIC. 335 336Please refer to :ref:`apiref-engine-settings` for details. 337 338Stream and connection callbacks 339=============================== 340 341Stream and connection callbacks are the way that the library communicates with user code. Some of these callbacks are mandatory; others are optional. 342They are all collected in :type:`lsquic_stream_if` ("if" here stands 343for "interface"). 344The mandatory callbacks include calls when connections and streams are created and destroyed and callbacks when streams can be read from or written to. 345The optional callbacks are used to observe some events in the connection lifecycle, such as being informed when handshake has succeeded (or failed) or when a goaway signal is received from peer. 346 347:: 348 349 struct lsquic_stream_if 350 { 351 /* Mandatory callbacks: */ 352 lsquic_conn_ctx_t *(*on_new_conn)(void *stream_if_ctx, 353 lsquic_conn_t *c); 354 void (*on_conn_closed)(lsquic_conn_t *c); 355 lsquic_stream_ctx_t * 356 (*on_new_stream)(void *stream_if_ctx, lsquic_stream_t *s); 357 void (*on_read) (lsquic_stream_t *s, lsquic_stream_ctx_t *h); 358 void (*on_write) (lsquic_stream_t *s, lsquic_stream_ctx_t *h); 359 void (*on_close) (lsquic_stream_t *s, lsquic_stream_ctx_t *h); 360 361 /* Optional callbacks: */ 362 void (*on_goaway_received)(lsquic_conn_t *c); 363 void (*on_hsk_done)(lsquic_conn_t *c, enum lsquic_hsk_status s); 364 void (*on_new_token)(lsquic_conn_t *c, const unsigned char *token, 365 void (*on_sess_resume_info)(lsquic_conn_t *c, const unsigned char *, size_t); 366 }; 367 368On new connection 369----------------- 370 371When a connection object is created, the "on new connection" callback is called. In server mode, the handshake is already known to have succeeded; in client mode, the connection object is created before the handshake is attempted. The client can tell when handshake succeeds or fails by relying on the optional "handshake is done" callback or the "on connection close" callback. 372 373:: 374 375 /* Return pointer to per-connection context. OK to return NULL. */ 376 static lsquic_conn_ctx_t * 377 my_on_new_conn (void *ea_stream_if_ctx, lsquic_conn_t *conn) 378 { 379 struct some_context *ctx = ea_stream_if_ctx; 380 struct my_conn_ctx *my_ctx = my_ctx_new(ctx); 381 if (ctx->is_client) 382 /* Need a stream to send request */ 383 lsquic_conn_make_stream(conn); 384 return (void *) my_ctx; 385 } 386 387In the made-up example above, a new per-connection context is allocated and returned. This context is then associated with the connection and can be retrieved using a dedicated function. Note that it is OK to return a ``NULL`` pointer. 388Note that in client mode, this is a good place to request that the connection make a new stream by calling :func:`lsquic_conn_make_stream()`. The connection will create a new stream when handshake succeeds. 389 390On new stream 391------------- 392 393QUIC allows either endpoint to create streams and send and receive data on them. There are unidirectional and bidirectional streams. Thus, there are four stream types. In our tutorial, however, we use the familiar paradigm of the client sending requests to the server using bidirectional stream. 394 395On the server, new streams are created when client requests arrive. On the client, streams are created when possible after the user code has requested stream creation by calling :func:`lsquic_conn_make_stream()`. 396 397:: 398 399 /* Return pointer to per-connection context. OK to return NULL. */ 400 static lsquic_stream_ctx_t * 401 my_on_new_stream (void *ea_stream_if_ctx, lsquic_stream_t *stream) { 402 struct some_context *ctx = ea_stream_if_ctx; 403 /* Associate some data with this stream: */ 404 struct my_stream_ctx *stream_ctx 405 = my_stream_ctx_new(ea_stream_if_ctx); 406 stream_ctx->stream = stream; 407 if (ctx->is_client) 408 lsquic_stream_wantwrite(stream, 1); 409 return (void *) stream_ctx; 410 } 411 412In a pattern similar to the "on new connection" callback, a per-stream context can be created at this time. The function returns this context and other stream callbacks - "on read," "on write," and "on close" - will be passed a pointer to it. As before, it is OK to return ``NULL``. 413You can register an interest in reading from or writing to the stream by using a "want read" or "want write" function. Alternatively, you can simply read or write; be prepared that this may fail and you have to try again in the "regular way." We talk about that next. 414 415On read 416------- 417 418When the "on read" callback is called, there is data to be read from stream, end-of-stream has been reached, or there is an error. 419 420:: 421 422 static void 423 my_on_read (lsquic_stream_t *stream, lsquic_stream_ctx_t *h) { 424 struct my_stream_ctx *my_stream_ctx = (void *) h; 425 unsigned char buf[BUFSZ]; 426 427 ssize_t nr = lsquic_stream_read(stream, buf, sizeof(buf)); 428 /* Do something with the data.... */ 429 if (nr == 0) /* EOF */ { 430 lsquic_stream_shutdown(stream, 0); 431 lsquic_stream_wantwrite(stream, 1); /* Want to reply */ 432 } 433 } 434 435To read the data or to collect the error, call :func:`lsquic_stream_read`. If a negative value is returned, examine ``errno``. If it is not ``EWOULDBLOCK``, then an error has occurred, and you should close the stream. Here, an error means an application error, such as peer resetting the stream. A protocol error or an internal library error (such as memory allocation failure) lead to the connection being closed outright. 436To reiterate, the "on read" callback is called only when the user has registered interest in reading from the stream. 437 438On write 439-------- 440 441The "on write" callback is called when the stream can be written to. At this point, you should be able to write at least a byte to the stream. 442As with the "on read" callback, for this callback to be called, the user must have registered interest in writing to stream using :func:`lsquic_stream_wantwrite()`. 443 444 445:: 446 447 static void 448 my_on_write (lsquic_stream_t *stream, lsquic_stream_ctx_t *h) { 449 struct my_stream_ctx *my_stream_ctx = (void *) h; 450 ssize_t nw = lsquic_stream_write(stream, 451 my_stream_ctx->resp, my_stream_ctx->resp_sz); 452 if (nw == my_stream_ctx->resp_sz) 453 lsquic_stream_close(stream); 454 } 455 456By default, "on read" and "on write" callbacks will be called in a loop as long as there is data to read or the stream can be written to. If you are done reading from or writing to stream, you should either shut down the appropriate end, close the stream, or unregister your interest. The library implements a circuit breaker to stop would-be infinite loops when no reading or writing progress is made. Both loop dispatch and the circuit breaker are configurable (see :member:`lsquic_engine_settings.es_progress_check` and :member:`lsquic_engine_settings.es_rw_once`). 457 458On stream close 459--------------- 460 461When reading and writing ends of the stream have been closed, the "on close" callback is called. After this function returns, pointers to the stream become invalid. (The library destroys the stream object when it deems proper.) 462This is a good place to perform necessary cleanup. 463 464:: 465 466 static void 467 my_on_close (lsquic_stream_t *stream, lsquic_stream_ctx_t *h) { 468 lsquic_conn_t *conn = lsquic_stream_conn(stream); 469 struct my_conn_ctx *my_ctx = lsquic_conn_get_ctx(conn); 470 if (!has_more_reqs_to_send(my_ctx)) /* For example */ 471 lsquic_conn_close(conn); 472 free(h); 473 } 474 475In the made-up example above, we free the per-stream context allocated in the "on new stream" callback and we may close the connection. 476 477On connection close 478------------------- 479 480When either :func:`lsquic_conn_close()` has been called; or the peer has closed the connection; or an error has occurred, the "on connection close" callback is called. At this point, it is time to free the per-connection context, if any. 481 482:: 483 484 static void 485 my_on_conn_closed (lsquic_conn_t *conn) { 486 struct my_conn_ctx *my_ctx = lsquic_conn_get_ctx(conn); 487 struct some_context *ctx = my_ctx->some_context; 488 489 --ctx->n_conns; 490 if (0 == ctx->n_conn && (ctx->flags & CLOSING)) 491 exit_event_loop(ctx); 492 493 free(my_ctx); 494 } 495 496In the example above, you see the call to :func:`lsquic_conn_get_ctx()`. This returns the pointer returned by the "on new connection" callback. 497 498Using Streams 499============= 500 501To reduce buffering, most of the time bytes written to stream are written into packets directly. Bytes are buffered in the stream until a full packet can be created. Alternatively, one could flush the data by calling :func:`lsquic_stream_flush`. 502It is impossible to write more data than the congestion window. This prevents excessive buffering inside the library. 503Inside the "on read" and "on write" callbacks, reading and writing should succeed. The exception is error collection inside the "on read" callback. 504Outside of the callbacks, be ready to handle errors. For reading, it is -1 with ``EWOULDBLOCK`` errno. For writing, it is the return value of 0. 505 506More stream functions 507--------------------- 508 509Here are a few more useful stream functions. 510 511:: 512 513 /* Flush any buffered data. This triggers packetizing even a single 514 * byte into a separate frame. 515 */ 516 int 517 lsquic_stream_flush (lsquic_stream_t *); 518 519 /* Possible values for how are 0, 1, and 2. See shutdown(2). */ 520 int 521 lsquic_stream_shutdown (lsquic_stream_t *, int how); 522 523 int 524 lsquic_stream_close (lsquic_stream_t *); 525 526As mentioned before, calling :func:`lsquic_stream_flush()` will cause the stream to packetize the buffered data. Note that it may not happen immediately, as there may be higher-priority writes pending or there may not be sufficient congestion window to do so. Calling "flush" only schedules writing to packets. 527 528:func:`lsquic_stream_shutdown()` and :func:`lsquic_stream_close()` mimic the interface of the "shutdown" and "close" socket functions. After both read and write ends of a stream are closed, the "on stream close" callback will soon be called. 529 530Stream return values 531-------------------- 532 533The stream read and write functions are modeled on the standard UNIX read and write functions, including the use of the ``errno``. The most important of these error codes are ``EWOULDBLOCK`` and ``ECONNRESET`` because you may encounter these even if you structure your code correctly. Other errors typically occur when the user code does something unexpected. 534 535Return value of 0 is different for reads and writes. For reads, it means that EOF has been reached and you need to stop reading from the stream. For writes, it means that you should try writing later. 536 537If writing to stream returns an error, it may mean an internal error. If the error is not recoverable, the library will abort the connection; if it is recoverable (the only recoverable error is failure to allocate memory), attempting to write later may succeed. 538 539Scatter/gather stream functions 540------------------------------- 541 542There is the scatter/gather way to read from and write to stream and the interface is similar to the usual "readv" and "writev" functions. All return values and error codes are the same as in the stream read and write functions we have just discussed. Those are actually just wrappers around the scatter/gather versions. 543 544:: 545 546 ssize_t 547 lsquic_stream_readv (lsquic_stream_t *, const struct iovec *, 548 int iovcnt); 549 ssize_t 550 lsquic_stream_writev (lsquic_stream_t *, const struct iovec *, 551 int count); 552 553Read using a callback 554--------------------- 555 556The scatter/gather functions themselves are also wrappers. LSQUIC provides stream functions that skip intermediate buffering. They are used for zero-copy stream processing. 557 558:: 559 560 ssize_t 561 lsquic_stream_readf (lsquic_stream_t *, 562 size_t (*readf)(void *ctx, const unsigned char *, size_t len, int fin), 563 void *ctx); 564 565 566The second argument to :func:`lsquic_stream_readf()` is a callback that 567returns the number of bytes processed. The callback is passed: 568 569- Pointer to user-supplied context; 570- Pointer to the data; 571- Data size (can be zero); and 572- Indicator whether the FIN follows the data. 573 574If callback returns 0 or value smaller than `len`, reading stops. 575 576Read with callback: Example 1 577----------------------------- 578 579Here is the first example of reading from stream using a callback. Now the process of reading from stream 580is split into two functions. 581 582:: 583 584 static void 585 tut_client_on_read_v1 (lsquic_stream_t *stream, lsquic_stream_ctx_t *h) 586 { 587 struct tut *tut = (struct tut *) h; 588 size_t nread = lsquic_stream_readf(stream, tut_client_readf_v1, NULL); 589 if (nread == 0) 590 { 591 LOG("read to end-of-stream: close and read from stdin again"); 592 lsquic_stream_shutdown(stream, 0); 593 ev_io_start(tut->tut_loop, &tut->tut_u.c.stdin_w); 594 } 595 /* ... */ 596 } 597 598Here, we see the :func:`lsquic_stream_readf()` call. The return value is the same as the other read functions. 599Because in this example there is no extra information to pass to the callback (we simply print data to stdout), 600the third argument is NULL. 601 602:: 603 604 static size_t 605 tut_client_readf_v1 (void *ctx, const unsigned char *data, 606 size_t len, int fin) 607 { 608 if (len) 609 { 610 fwrite(data, 1, len, stdout); 611 fflush(stdout); 612 } 613 return len; 614 } 615 616Here is the callback itself. You can see it is very simple. If there is data to be processed, 617it is printed to stdout. 618 619Note that the data size (``len`` above) can be anything. It is not limited by UDP datagram size. This is because when incoming STREAM frames pass some fragmentation threshold, LSQUIC begins to copy incoming STREAM data to a data structure that is impervious to stream fragmentation attacks. Thus, it is possible for the callback to pass a pointer to data that is over 3KB in size. The implementation may change, so again, no guarantees. 620When the fourth argument, ``fin``, is true, this indicates that the incoming data ends after ``len`` bytes have been read. 621 622Read with callback: Example 2: Use FIN 623-------------------------------------- 624 625The FIN indicator passed to the callback gives us yet another way to detect end-of-stream. 626The previous version checked the return value of :func:`lsquic_stream_readf()` to check for EOS. 627Instead, we can use ``fin`` in the callback. 628 629The second zero-copy read example is a little more efficient as it saves us 630an extra call to ``tut_client_on_read_v2``. 631Here, we package pointers to the tut struct and stream into a special struct and pass it to 632``lsquic_stream_readf()``. 633 634:: 635 636 struct client_read_v2_ctx { struct tut *tut; lsquic_stream_t *stream; }; 637 638 static void 639 tut_client_on_read_v2 (lsquic_stream_t *stream, 640 lsquic_stream_ctx_t *h) 641 { 642 struct tut *tut = (struct tut *) h; 643 struct client_read_v2_ctx v2ctx = { tut, stream, }; 644 ssize_t nread = lsquic_stream_readf(stream, tut_client_readf_v2, 645 &v2ctx); 646 if (nread < 0) 647 /* ERROR */ 648 } 649 650Now the callback becomes more complicated, as we moved the logic to stop reading from stream into it. We need pointer to both stream and user context when "fin" is true. In that case, we call :func:`lsquic_stream_shutdown()` and begin reading from stdin again to grab the next line of input. 651 652:: 653 654 static size_t 655 tut_client_readf_v2 (void *ctx, const unsigned char *data, 656 size_t len, int fin) 657 { 658 struct client_read_v2_ctx *v2ctx = ctx; 659 if (len) 660 fwrite(data, 1, len, stdout); 661 if (fin) 662 { 663 fflush(stdout); 664 LOG("read to end-of-stream: close and read from stdin again"); 665 lsquic_stream_shutdown(v2ctx->stream, 0); 666 ev_io_start(v2ctx->tut->tut_loop, &v2ctx->tut->tut_u.c.stdin_w); 667 } 668 return len; 669 } 670 671Writing to stream: Example 1 672---------------------------- 673 674Now let's consider writing to stream. 675 676:: 677 678 static void 679 tut_server_on_write_v0 (lsquic_stream_t *stream, lsquic_stream_ctx_t *h) 680 { 681 struct tut_server_stream_ctx *const tssc = (void *) h; 682 ssize_t nw = lsquic_stream_write(stream, 683 tssc->tssc_buf + tssc->tssc_off, tssc->tssc_sz - tssc->tssc_off); 684 if (nw > 0) 685 { 686 tssc->tssc_off += nw; 687 if (tssc->tssc_off == tssc->tssc_sz) 688 lsquic_stream_close(stream); 689 /* ... */ 690 } 691 692Here, we call :func:`lsquic_stream_write()` directly. If writing succeeds and we reached the 693end of the buffer we wanted to write, we close the stream. 694 695Write using callbacks 696--------------------- 697 698To write using a callback, we need to use :func:`lsquic_stream_writef()`. 699 700:: 701 702 struct lsquic_reader { 703 /* Return number of bytes written to buf */ 704 size_t (*lsqr_read) (void *lsqr_ctx, void *buf, size_t count); 705 /* Return number of bytes remaining in the reader. */ 706 size_t (*lsqr_size) (void *lsqr_ctx); 707 void *lsqr_ctx; 708 }; 709 710 /* Return umber of bytes written or -1 on error. */ 711 ssize_t 712 lsquic_stream_writef (lsquic_stream_t *, struct lsquic_reader *); 713 714We must specify not only the function that will perform the copy, but also the function that will return the number of bytes remaining. This is useful in situations where the size of the data source may change. For example, an underlying file may change size. 715The :member:`lsquic_reader.lsqr_read` callback will be called in a loop until stream can write no more or until :member:`lsquic_reader.lsqr_size` returns zero. 716The return value of ``lsquic_stream_writef`` is the same as :func:`lsquic_stream_write()` and :func:`lsquic_stream_writev()`, which are just wrappers around the "writef" version. 717 718Writing to stream: Example 2 719---------------------------- 720 721Here is the second version of the "on write" callback. It uses :func:`lsquic_stream_writef()`. 722 723:: 724 725 static void 726 tut_server_on_write_v1 (lsquic_stream_t *stream, lsquic_stream_ctx_t *h) 727 { 728 struct tut_server_stream_ctx *const tssc = (void *) h; 729 struct lsquic_reader reader = { tssc_read, tssc_size, tssc, }; 730 ssize_t nw = lsquic_stream_writef(stream, &reader); 731 if (nw > 0 && tssc->tssc_off == tssc->tssc_sz) 732 lsquic_stream_close(stream); 733 /* ... */ 734 } 735 736 737The reader struct is initialized with pointers to read and size functions and this struct is passed 738to the "writef" function. 739 740:: 741 742 static size_t 743 tssc_size (void *ctx) 744 { 745 struct tut_server_stream_ctx *tssc = ctx; 746 return tssc->tssc_sz - tssc->tssc_off; 747 } 748 749 750The size callback simply returns the number of bytes left. 751 752:: 753 754 static size_t 755 tssc_read (void *ctx, void *buf, size_t count) 756 { 757 struct tut_server_stream_ctx *tssc = ctx; 758 759 if (count > tssc->tssc_sz - tssc->tssc_off) 760 count = tssc->tssc_sz - tssc->tssc_off; 761 memcpy(buf, tssc->tssc_buf + tssc->tssc_off, count); 762 tssc->tssc_off += count; 763 return count; 764 } 765 766 767The read callback (so called because you *read* data from the source) writes no more than ``count`` bytes 768to memory location pointed by ``buf`` and returns the number of bytes copied. 769In our case, ``count`` is never larger than the number of bytes still left to write. 770This is because the caller - the LSQUIC library - gets the value of ``count`` from the ``lsqr_size()`` callback. When reading from a file descriptor, on the other hand, this can very well happen that you don't have as much data to write as you thought you had. 771 772Client: making connection 773========================= 774 775We now switch our attention to making a QUIC connection. The function :func:`lsquic_engine_connect()` does that. This function has twelve arguments. (These arguments have accreted over time.) 776 777:: 778 779 lsquic_conn_t * 780 lsquic_engine_connect (lsquic_engine_t *, 781 enum lsquic_version, /* Set to N_LSQVER for default */ 782 const struct sockaddr *local_sa, 783 const struct sockaddr *peer_sa, 784 void *peer_ctx, 785 lsquic_conn_ctx_t *conn_ctx, 786 const char *hostname, /* Used for SNI */ 787 unsigned short base_plpmtu, /* 0 means default */ 788 const unsigned char *sess_resume, size_t sess_resume_len, 789 const unsigned char *token, size_t token_sz); 790 791- The first argument is the pointer to the engine instance. 792- The second argument is the QUIC version to use. 793- The third and fourth arguments specify local and destination addresses, respectively. 794- The fifth argument is the so-called "peer context." 795- The sixth argument is the connection context. This is used if you need to pass a pointer to the "on new connection" callback. This context is overwritten by the return value of the "on new connection" callback. 796- The argument "hostname," which is the seventh argument, is used for SNI. This argument is optional, just as the rest of the arguments that follow. 797- The eighth argument is the initial maximum size of the UDP payload. This will be the base PLPMTU if DPLPMTUD is enabled. Specifying zero, or default, is the safe way to go: lsquic will pick a good starting value. 798- The next two arguments allow one to specify a session resumption information to establish a connection faster. In the case of IETF QUIC, this is the TLS Session Ticket. To get this ticket, specify the :member:`lsquic_stream_if.on_sess_resume_info` callback. 799- The last pair of arguments is for specifying a token to try to prevent a potential stateless retry from the server. The token is learned in a previous session. See the optional callback :member:`lsquic_stream_if.on_new_token`. 800 801:: 802 803 tut.tut_u.c.conn = lsquic_engine_connect( 804 tut.tut_engine, N_LSQVER, 805 (struct sockaddr *) &tut.tut_local_sas, &addr.sa, 806 (void *) (uintptr_t) tut.tut_sock_fd, /* Peer ctx */ 807 NULL, NULL, 0, NULL, 0, NULL, 0); 808 if (!tut.tut_u.c.conn) 809 { 810 LOG("cannot create connection"); 811 exit(EXIT_FAILURE); 812 } 813 tut_process_conns(&tut); 814 815Here is an example from a tutorial program. The connect call is a lot less intimidating in real life, as half the arguments are set to zero. 816We pass a pointer to the engine instance, N_LSQVER to let the engine pick the version to use and the two socket addresses. 817The peer context is simply the socket file descriptor cast to a pointer. 818This is what is passed to the "send packets out" callback. 819 820Specifying QUIC version 821======================= 822 823QUIC versions in LSQUIC are gathered in an enum, :type:`lsquic_version`, and have an arbitrary value. 824 825:: 826 827 enum lsquic_version { 828 LSQVER_043, LSQVER_046, LSQVER_050, /* Google QUIC */ 829 LSQVER_ID27, LSQVER_ID28, LSQVER_ID29, /* IETF QUIC */ 830 /* ...some special entries skipped */ 831 N_LSQVER /* <====================== Special value */ 832 }; 833 834The special value "N_LSQVER" is used to let the engine pick the QUIC version. 835It picks the latest non-experimental version, so in this case it picks ID-29. 836(Experimental from the point of view of the library.) 837 838Because version enum values are small -- and that is by design -- a list of 839versions can be passed around as bitmasks. 840 841:: 842 843 /* This allows list of versions to be specified as bitmask: */ 844 es_versions = (1 << LSQVER_ID28) | (1 << LSQVER_ID29); 845 846This is done, for example, when 847specifying list of versions to enable in engine settings using :member:`lsquic_engine_api.ea_versions`. 848There are a couple of more places in the API where this technique is used. 849 850Server callbacks 851================ 852 853The server requires SSL callbacks to be present. The basic required callback is :member:`lsquic_engine_api.ea_get_ssl_ctx`. It is used to get a pointer to an initialized ``SSL_CTX``. 854 855:: 856 857 typedef struct ssl_ctx_st * (*lsquic_lookup_cert_f)( 858 void *lsquic_cert_lookup_ctx, const struct sockaddr *local, 859 const char *sni); 860 861 struct lsquic_engine_api { 862 lsquic_lookup_cert_f ea_lookup_cert; 863 void *ea_cert_lu_ctx; 864 struct ssl_ctx_st * (*ea_get_ssl_ctx)(void *peer_ctx); 865 /* (Other members of the struct are not shown) */ 866 }; 867 868In case SNI is used, LSQUIC will call :member:`lsquic_engine_api.ea_lookup_cert`. 869For example, SNI is required in HTTP/3. 870In `our web server`_, each virtual host has its own SSL context. Note that besides the SNI string, the callback is also given the local socket address. This makes it possible to implement a flexible lookup mechanism. 871 872Engine settings 873=============== 874 875Besides the engine API struct passed to the engine constructor, there is also an engine settings struct, :type:`lsquic_engine_settings`. :member:`lsquic_engine_api.ea_settings` in the engine API struct 876can be pointed to a custom settings struct. By default, this pointer is ``NULL``. 877In that case, the engine uses default settings. 878 879There are many settings, controlling everything from flow control windows to the number of times an "on read" callback can be called in a loop before it is deemed an infinite loop and the circuit breaker is tripped. To make changing default settings values easier, the library provides functions to initialize the settings struct to defaults and then to check these values for sanity. 880 881Settings helper functions 882------------------------- 883 884:: 885 886 /* Initialize `settings' to default values */ 887 void 888 lsquic_engine_init_settings (struct lsquic_engine_settings *, 889 /* Bitmask of LSENG_SERVER and LSENG_HTTP */ 890 unsigned lsquic_engine_flags); 891 892 /* Check settings for errors, return 0 on success, -1 on failure. */ 893 int 894 lsquic_engine_check_settings (const struct lsquic_engine_settings *, 895 unsigned lsquic_engine_flags, 896 /* Optional, can be NULL: */ 897 char *err_buf, size_t err_buf_sz); 898 899The first function is :func:`lsquic_engine_init_settings()`, which does just that. 900The second argument is a bitmask to specify whether the engine is in server mode 901and whether HTTP mode is turned on. These should be the same flags as those 902passed to the engine constructor. 903 904Once you have initialized the settings struct in this manner, change the setting 905or settings you want and then call :func:`lsquic_engine_check_settings()`. The 906first two arguments are the same as in the initializer. The third and fourth 907argument are used to pass a pointer to a buffer into which a human-readable error 908string can be placed. 909 910The checker function does only the basic sanity checks. If you really set out 911to misconfigure LSQUIC, you can. On the bright side, each setting is clearly 912documented (see :ref:`apiref-engine-settings`). Most settings are standalone; 913when there is interplay between them, it is also documented. 914Test before deploying! 915 916Settings example 917---------------- 918 919The example is adapted from a tutorial program. Here, command-line options 920are processed and appropriate options is set. The first time the ``-o`` 921flag is encountered, the settings struct is initialized. Then the argument 922is parsed to see which setting to alter. 923 924:: 925 926 while (/* getopt */) 927 { 928 case 'o': /* For example: -o version=h3-27 -o cc_algo=2 */ 929 if (!settings_initialized) { 930 lsquic_engine_init_settings(&settings, 931 cert_file || key_file ? LSENG_SERVER : 0); 932 settings_initialized = 1; 933 } 934 /* ... */ 935 else if (0 == strncmp(optarg, "cc_algo=", val - optarg)) 936 settings.es_cc_algo = atoi(val); 937 /* ... */ 938 } 939 940 /* Check settings */ 941 if (0 != lsquic_engine_check_settings(&settings, 942 tut.tut_flags & TUT_SERVER ? LSENG_SERVER : 0, 943 errbuf, sizeof(errbuf))) 944 { 945 LOG("invalid settings: %s", errbuf); 946 exit(EXIT_FAILURE); 947 } 948 949 /* ... */ 950 eapi.ea_settings = &settings; 951 952After option processing is completed, the settings are checked. The error 953buffer is used to log a configuration error. 954 955Finally, the settings struct is pointed to by the engine API struct before 956the engine constructor is called. 957 958Logging 959======= 960 961LSQUIC provides a simple logging interface using a single callback function. 962By default, no messages are logged. This can be changed by calling :func:`lsquic_logger_init()`. 963This will set a library-wide logger callback function. 964 965:: 966 967 void lsquic_logger_init(const struct lsquic_logger_if *, 968 void *logger_ctx, enum lsquic_logger_timestamp_style); 969 970 struct lsquic_logger_if { 971 int (*log_buf)(void *logger_ctx, const char *buf, size_t len); 972 }; 973 974 enum lsquic_logger_timestamp_style { LLTS_NONE, LLTS_HHMMSSMS, 975 LLTS_YYYYMMDD_HHMMSSMS, LLTS_CHROMELIKE, LLTS_HHMMSSUS, 976 LLTS_YYYYMMDD_HHMMSSUS, N_LLTS }; 977 978You can instruct the library to generate a timestamp and include it as part of the message. 979Several timestamp formats are available. Some display microseconds, some do not; some 980display the date, some do not. One of the most useful formats is "chromelike," 981which matches the somewhat weird timestamp format used by Chromium. This makes it easy to 982compare the two logs side by side. 983 984There are eight log levels in LSQUIC: debug, info, notice, warning, error, alert, emerg, 985and crit. 986These correspond to the usual log levels. (For example, see ``syslog(3)``). Of these, only five are used: debug, info, notice, warning, and error. Usually, warning and error messages are printed when there is a bug in the library or something very unusual has occurred. Memory allocation failures might elicit a warning as well, to give the operator a heads up. 987 988LSQUIC possesses about 40 logging modules. Each module usually corresponds to a single piece 989of functionality in the library. The exception is the "event" module, which logs events of note in many modules. 990There are two functions to manipulate which log messages will be generated. 991 992:: 993 994 /* Set log level for all modules */ 995 int 996 lsquic_set_log_level (const char *log_level); 997 998 /* Set log level per module "event=debug" */ 999 int 1000 lsquic_logger_lopt (const char *optarg); 1001 1002The first is :func:`lsquic_set_log_level()`. It sets the same log level for each module. 1003The second is :func:`lsquic_logger_lopt()`. This function takes a comma-separated list of name-value pairs. For example, "event=debug." 1004 1005Logging Example 1006--------------- 1007 1008The following example is adapted from a tutorial program. In the program, log messages 1009are written to a file handle. By default, this is the standard error. One can change 1010that by using the "-f" command-line option and specify the log file. 1011 1012:: 1013 1014 static int 1015 tut_log_buf (void *ctx, const char *buf, size_t len) { 1016 FILE *out = ctx; 1017 fwrite(buf, 1, len, out); 1018 fflush(out); 1019 return 0; 1020 } 1021 static const struct lsquic_logger_if logger_if = { tut_log_buf, }; 1022 1023 lsquic_logger_init(&logger_if, s_log_fh, LLTS_HHMMSSUS); 1024 1025 1026``tut_log_buf()`` returns 0, but the truth is that the return value is ignored. 1027There is just nothing for the library to do when the user-supplied log function fails! 1028 1029:: 1030 1031 case 'l': /* e.g. -l event=debug,cubic=info */ 1032 if (0 != lsquic_logger_lopt(optarg)) { 1033 fprintf(stderr, "error processing -l option\n"); 1034 exit(EXIT_FAILURE); 1035 } 1036 break; 1037 case 'L': /* e.g. -L debug */ 1038 if (0 != lsquic_set_log_level(optarg)) { 1039 fprintf(stderr, "error processing -L option\n"); 1040 exit(EXIT_FAILURE); 1041 } 1042 break; 1043 1044Here you can see how we use ``-l`` and ``-L`` command-line options to call one of 1045the two log level functions. These functions can fail if the incorrect log level 1046or module name is passed. Both log level and module name are treated in case-insensitive manner. 1047 1048Sample log messages 1049------------------- 1050 1051When log messages are turned on, you may see something like this in your log file (timestamps and 1052log levels are elided for brevity): 1053 1054.. code-block:: text 1055 1056 [QUIC:B508E8AA234E0421] event: generated STREAM frame: stream 0, offset: 0, size: 3, fin: 1 1057 [QUIC:B508E8AA234E0421-0] stream: flushed to or past required offset 3 1058 [QUIC:B508E8AA234E0421] event: sent packet 13, type Short, crypto: forw-secure, size 32, frame types: STREAM, ecn: 0, spin: 0; kp: 0, path: 0, flags: 9470472 1059 [QUIC:B508E8AA234E0421] event: packet in: 15, type: Short, size: 44; ecn: 0, spin: 0; path: 0 1060 [QUIC:B508E8AA234E0421] rechist: received 15 1061 [QUIC:B508E8AA234E0421] event: ACK frame in: [13-9] 1062 [QUIC:B508E8AA234E0421] conn: about to process QUIC_FRAME_STREAM frame 1063 [QUIC:B508E8AA234E0421] event: STREAM frame in: stream 0; offset 0; size 3; fin: 1 1064 [QUIC:B508E8AA234E0421-0] stream: received stream frame, offset 0x0, len 3; fin: 1 1065 [QUIC:B508E8AA234E0421-0] di: FIN set at 3 1066 1067Here we see the connection ID, ``B508E8AA234E0421``, and logging for modules "event", "stream", "rechist" 1068(that stands for "receive history"), "conn", and "di" (the "data in" module). When the connection ID is 1069followed by a dash and that number, the number is the stream ID. Note that stream ID is logged not just 1070for the stream, but for some other modules as well. 1071 1072Key logging and Wireshark 1073========================= 1074 1075`Wireshark`_ supports IETF QUIC. The developers have been very good at keeping up with latest versions. 1076You will need version 3.3 of Wireshark to support Internet-Draft 29. Support for HTTP/3 is in progress. 1077 1078LSQUIC supports exporting TLS secrets. For that, you need to specify a set of function pointers via 1079:member:`lsquic_engine_api.ea_keylog_if`. 1080 1081:: 1082 1083 /* Secrets are logged per connection. Interface to open file (handle), 1084 * log lines, and close file. 1085 */ 1086 struct lsquic_keylog_if { 1087 void * (*kli_open) (void *keylog_ctx, lsquic_conn_t *); 1088 void (*kli_log_line) (void *handle, const char *line); 1089 void (*kli_close) (void *handle); 1090 }; 1091 1092 struct lsquic_engine_api { 1093 /* --- 8< --- snip --- 8< --- */ 1094 const struct lsquic_keylog_if *ea_keylog_if; 1095 void *ea_keylog_ctx; 1096 }; 1097 1098There are three functions: one to open a file, one to write a line into the file, and one to close the file. The lines are not interpreted. 1099In the engine API struct, there are two members to set: one is the pointer to the struct with the function pointers, and the other is the context passed to "kli_open" function. 1100 1101Key logging example 1102------------------- 1103 1104:: 1105 1106 static void * 1107 keylog_open (void *ctx, lsquic_conn_t *conn) 1108 { 1109 const lsquic_cid_t *cid; 1110 FILE *fh; 1111 int sz; 1112 unsigned i; 1113 char id_str[MAX_CID_LEN * 2 + 1]; 1114 char path[PATH_MAX]; 1115 static const char b2c[16] = "0123456789ABCDEF"; 1116 1117 cid = lsquic_conn_id(conn); 1118 for (i = 0; i < cid->len; ++i) 1119 { 1120 id_str[i * 2 + 0] = b2c[ cid->idbuf[i] >> 4 ]; 1121 id_str[i * 2 + 1] = b2c[ cid->idbuf[i] & 0xF ]; 1122 } 1123 id_str[i * 2] = '\0'; 1124 sz = snprintf(path, sizeof(path), "/secret_dir/%s.keys", id_str); 1125 if ((size_t) sz >= sizeof(path)) 1126 { 1127 LOG("WARN: %s: file too long", __func__); 1128 return NULL; 1129 } 1130 fh = fopen(path, "wb"); 1131 if (!fh) 1132 LOG("WARN: could not open %s for writing: %s", path, strerror(errno)); 1133 return fh; 1134 } 1135 1136 static void 1137 keylog_log_line (void *handle, const char *line) 1138 { 1139 fputs(line, handle); 1140 fputs("\n", handle); 1141 fflush(handle); 1142 } 1143 1144 static void 1145 keylog_close (void *handle) 1146 { 1147 fclose(handle); 1148 } 1149 1150The function to open the file is passed the connection object. It can be used to generate a filename 1151based on the connection ID. 1152We see that the line logger simply writes the passed C string to the filehandle and appends a newline. 1153 1154Wireshark screenshot 1155-------------------- 1156 1157After jumping through those hoops, our reward is a decoded QUIC trace in Wireshark! 1158 1159.. image:: wireshark-screenshot.png 1160 1161Here, we highlighted the STREAM frame payload. 1162Other frames in view are ACK and TIMESTAMP frames. 1163In the top panel with the packet list, you can see that frames are listed after the packet number. 1164Another interesting item is the DCID. This stands for "Destination Connection ID," and you can 1165see that there are two different values there. This is because the two peers of the QUIC connection 1166place different connection IDs in the packets! 1167 1168Connection IDs 1169============== 1170 1171A QUIC connection has two sets of connection IDs: source connection IDs and destination connection IDs. The source connection IDs set is what the peer uses to place in QUIC packets; the destination connection IDs is what this endpoint uses to include in the packets it sends to the peer. One's source CIDs is the other's destination CIDs and vice versa. 1172What interesting is that either side of the QUIC connection may change the DCID. Use CIDs with care. 1173 1174:: 1175 1176 #define MAX_CID_LEN 20 1177 1178 typedef struct lsquic_cid 1179 { 1180 uint_fast8_t len; 1181 union { 1182 uint8_t buf[MAX_CID_LEN]; 1183 uint64_t id; 1184 } u_cid; 1185 #define idbuf u_cid.buf 1186 } lsquic_cid_t; 1187 1188 #define LSQUIC_CIDS_EQ(a, b) ((a)->len == 8 ? \ 1189 (b)->len == 8 && (a)->u_cid.id == (b)->u_cid.id : \ 1190 (a)->len == (b)->len && 0 == memcmp((a)->idbuf, (b)->idbuf, (a)->len)) 1191 1192The LSQUIC representation of a CID is the struct above. The CID can be up to 20 bytes in length. 1193By default, LSQUIC uses 8-byte CIDs to speed up comparisons. 1194 1195Get this-and-that API 1196===================== 1197 1198Here are a few functions to get different LSQUIC objects from other objects. 1199 1200:: 1201 1202 const lsquic_cid_t * 1203 lsquic_conn_id (const lsquic_conn_t *); 1204 1205 lsquic_conn_t * 1206 lsquic_stream_conn (const lsquic_stream_t *); 1207 1208 lsquic_engine_t * 1209 lsquic_conn_get_engine (lsquic_conn_t *); 1210 1211 int lsquic_conn_get_sockaddr (lsquic_conn_t *, 1212 const struct sockaddr **local, const struct sockaddr **peer); 1213 1214The CID returned by :func:`lsquic_conn_id()` is that used for logging: server and client should return the same CID. As noted earlier, you should not rely on this value to identify a connection! 1215You can get a pointer to the connection from a stream and a pointer to the engine from a connection. 1216Calling :func:`lsquic_conn_get_sockaddr()` will point ``local`` and ``peer`` to the socket addressess of the current path. QUIC supports multiple paths during migration, but access to those paths has not been exposed via an API yet. This may change when or if QUIC adds true multipath support. 1217 1218.. _`our web server`: https://www.litespeedtech.com/products 1219.. _`Wireshark`: https://www.wireshark.org/ 1220