File Based Dynamic Configuration of Routes in Envoy Proxy
| This article appeared first on the Computas Blog. |
As developers are increasingly involved in the configuration of edge proxies, like Envoy, it is important that these appliances provide a developer experience that supports the modern developers’ workflow. With the dynamic service discovery capabilities, Envoy makes this possible. All aspects of the proxy except a static bootstrap configuration can be dynamically configured. This means that the configuration of an Envoy proxy can be updated without cold or hot restart. This is in many cases the preferred and easiest method of handling frequent configuration changes.
The configuration can either be provided by a configuration server, or by files. There are some open source and commercial implementations of such servers. If desired, projects can implement their own, based on available libraries, implementations and examples (envoyproxy / go-control-plane, envoyproxy / java-control-plane). If the requirements do not necessitate a server based solution, a simple file based approach might be sufficient. A file with the new configuration is created, and in a file system move operation the current file is overwritten with the new configuration. Using inotify to monitor the file for changes, Envoy will load the new configuration, and apply it if it is valid. If it is not valid, the previous state will remain active, and a corresponding message will be written to the log.
The whole family of discovery services is designated as xDS. This post shows how the route discovery service (RDS) can be implemented using files as configuration source instead of servers.
The code accompanying this post is available on GitHub. It is based on the front proxy example code from the Envoy proxy repository, documented in the envoy docs.
In the Envoy configuration file, find the field route_config in the
envoy.http_connection_manager filter that you would like to replace by a dynamic
configuration:
- name: envoy.http_connection_manager
config:
codec_type: auto
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: backend
domains:
- "*"
routes:
- match:
prefix: "/service/1"
route:
cluster: service1
- match:
prefix: "/service/2"
route:
cluster: service2
http_filters:
- name: envoy.router
config: {}Replace this field with a field called
rds
that points to the file containing the corresponding configuration:
- name: envoy.http_connection_manager
config:
codec_type: auto
stat_prefix: ingress_http
rds:
route_config_name: local_route
config_source:
path: /etc/rds/rds.yaml
http_filters:
- name: envoy.router
config: {}The contents of the file rds.yaml is
version_info: "0"
resources:
- "@type": type.googleapis.com/envoy.api.v2.RouteConfiguration
name: local_route
virtual_hosts:
- name: backend
domains:
- "*"
routes:
- match:
prefix: "/service/1"
route:
cluster: service1
- match:
prefix: "/service/2"
route:
cluster: service2Now, if the configuration needs to be changed, a file with a new configuration
can be created, say rds-new.yaml, and then moved over the existing file:
mv rds-new.yaml rds.yamlEnvoy will pick up the change, validate and activate the new configuration.