deployment.rst (5934B)
1 Deployment 2 ========== 3 4 .. currentmodule:: websockets 5 6 When you deploy your websockets server to production, at a high level, your 7 architecture will almost certainly look like the following diagram: 8 9 .. image:: deployment.svg 10 11 The basic unit for scaling a websockets server is "one server process". Each 12 blue box in the diagram represents one server process. 13 14 There's more variation in routing. While the routing layer is shown as one big 15 box, it is likely to involve several subsystems. 16 17 When you design a deployment, your should consider two questions: 18 19 1. How will I run the appropriate number of server processes? 20 2. How will I route incoming connections to these processes? 21 22 These questions are strongly related. There's a wide range of acceptable 23 answers, depending on your goals and your constraints. 24 25 You can find a few concrete examples in the :ref:`deployment how-to guides 26 <deployment-howto>`. 27 28 Running server processes 29 ------------------------ 30 31 How many processes do I need? 32 ............................. 33 34 Typically, one server process will manage a few hundreds or thousands 35 connections, depending on the frequency of messages and the amount of work 36 they require. 37 38 CPU and memory usage increase with the number of connections to the server. 39 40 Often CPU is the limiting factor. If a server process goes to 100% CPU, then 41 you reached the limit. How much headroom you want to keep is up to you. 42 43 Once you know how many connections a server process can manage and how many 44 connections you need to handle, you can calculate how many processes to run. 45 46 You can also automate this calculation by configuring an autoscaler to keep 47 CPU usage or connection count within acceptable limits. 48 49 Don't scale with threads. Threads doesn't make sense for a server built with 50 :mod:`asyncio`. 51 52 How do I run processes? 53 ....................... 54 55 Most solutions for running multiple instances of a server process fall into 56 one of these three buckets: 57 58 1. Running N processes on a platform: 59 60 * a Kubernetes Deployment 61 62 * its equivalent on a Platform as a Service provider 63 64 2. Running N servers: 65 66 * an AWS Auto Scaling group, a GCP Managed instance group, etc. 67 68 * a fixed set of long-lived servers 69 70 3. Running N processes on a server: 71 72 * preferably via a process manager or supervisor 73 74 Option 1 is easiest of you have access to such a platform. 75 76 Option 2 almost always combines with option 3. 77 78 How do I start a process? 79 ......................... 80 81 Run a Python program that invokes :func:`~server.serve`. That's it. 82 83 Don't run an ASGI server such as Uvicorn, Hypercorn, or Daphne. They're 84 alternatives to websockets, not complements. 85 86 Don't run a WSGI server such as Gunicorn, Waitress, or mod_wsgi. They aren't 87 designed to run WebSocket applications. 88 89 Applications servers handle network connections and expose a Python API. You 90 don't need one because websockets handles network connections directly. 91 92 How do I stop a process? 93 ........................ 94 95 Process managers send the SIGTERM signal to terminate processes. Catch this 96 signal and exit the server to ensure a graceful shutdown. 97 98 Here's an example: 99 100 .. literalinclude:: ../../example/faq/shutdown_server.py 101 :emphasize-lines: 12-15,18 102 103 When exiting the context manager, :func:`~server.serve` closes all connections 104 with code 1001 (going away). As a consequence: 105 106 * If the connection handler is awaiting 107 :meth:`~server.WebSocketServerProtocol.recv`, it receives a 108 :exc:`~exceptions.ConnectionClosedOK` exception. It can catch the exception 109 and clean up before exiting. 110 111 * Otherwise, it should be waiting on 112 :meth:`~server.WebSocketServerProtocol.wait_closed`, so it can receive the 113 :exc:`~exceptions.ConnectionClosedOK` exception and exit. 114 115 This example is easily adapted to handle other signals. 116 117 If you override the default signal handler for SIGINT, which raises 118 :exc:`KeyboardInterrupt`, be aware that you won't be able to interrupt a 119 program with Ctrl-C anymore when it's stuck in a loop. 120 121 Routing connections 122 ------------------- 123 124 What does routing involve? 125 .......................... 126 127 Since the routing layer is directly exposed to the Internet, it should provide 128 appropriate protection against threats ranging from Internet background noise 129 to targeted attacks. 130 131 You should always secure WebSocket connections with TLS. Since the routing 132 layer carries the public domain name, it should terminate TLS connections. 133 134 Finally, it must route connections to the server processes, balancing new 135 connections across them. 136 137 How do I route connections? 138 ........................... 139 140 Here are typical solutions for load balancing, matched to ways of running 141 processes: 142 143 1. If you're running on a platform, it comes with a routing layer: 144 145 * a Kubernetes Ingress and Service 146 147 * a service mesh: Istio, Consul, Linkerd, etc. 148 149 * the routing mesh of a Platform as a Service 150 151 2. If you're running N servers, you may load balance with: 152 153 * a cloud load balancer: AWS Elastic Load Balancing, GCP Cloud Load 154 Balancing, etc. 155 156 * A software load balancer: HAProxy, NGINX, etc. 157 158 3. If you're running N processes on a server, you may load balance with: 159 160 * A software load balancer: HAProxy, NGINX, etc. 161 162 * The operating system — all processes listen on the same port 163 164 You may trust the load balancer to handle encryption and to provide security. 165 You may add another layer in front of the load balancer for these purposes. 166 167 There are many possibilities. Don't add layers that you don't need, though. 168 169 How do I implement a health check? 170 .................................. 171 172 Load balancers need a way to check whether server processes are up and running 173 to avoid routing connections to a non-functional backend. 174 175 websockets provide minimal support for responding to HTTP requests with the 176 :meth:`~server.WebSocketServerProtocol.process_request` hook. 177 178 Here's an example: 179 180 .. literalinclude:: ../../example/faq/health_check_server.py 181 :emphasize-lines: 7-9,18