Just as title says.
Background Description
While wandering around the community, I found a user confused about "how to use multilingual plugins in APISIX Ingress environment". I happen to be a user of go-plugin-runner and have a little knowledge of the APISIX Ingress project, so this document was born.
Proposal Description
Based on version 0.3 of the go-plugin-runner plugin and version 1.4.0 of APISIX Ingress, this article goes through building the cluster, building the image, customizing the helm chart package, and finally, deploying the resources. It is guaranteed that the final result can be derived in full based on this documentation.
go-plugin-runner: 0.3APISIX Ingress: 1.4.0kind: kind v0.12.0 go1.17.8 linux/amd64kubectl version: Client Version: v1.23.5/Server Version: v1.23.4golang: go1.18 linux/amd64
Begin
Build a cluster environment
Select kind
to build a local cluster environment. The command is as follows:
cat <<EOF | kind create cluster --config=-kind: ClusterapiVersion: kind.x-k8s.io/v1alpha4nodes:- role: control-planekubeadmConfigPatches:- |kind: InitConfigurationnodeRegistration:kubeletExtraArgs:node-labels: "ingress-ready=true"extraPortMappings:- containerPort: 80hostPort: 80protocol: TCP- containerPort: 443hostPort: 443protocol: TCPEOF
Build the go-plugin-runner executable
If you have finished writing the plugin, you can start compiling the executable to run with APISIX.
This article recommends two packaging build options.
- put the packaging process into the Dockerfile and finish the compilation process when you build the docker image later.
- You can also follow the scheme used in this document, building the executable first and then copying the packaged executable to the image.
How you choose the option should depend on your local hardware considerations. The reason for selecting the second option here is that I want to rely on my powerful local hardware to increase the building speed and speed up the process.
Go to the go-plugin-runner directory
Choose a folder address /home/chever/api7/cloud_native/tasks/plugin-runner
and place our apisix-go-plugin-runner
project in this folder.
After successful placement, the file tree is shown below:
chever@cloud-native-01:~/api7/cloud_native/tasks/plugin-runner$ tree -L 1.└── apisix-go-plugin-runner1 directory, 0 files
Then you need to go to the apisix-go-plugin-runner/cmd/go-runner/plugins
directory and write the plugins you need in that directory. This article will use the default plugin say
for demonstration purposes.
chever@cloud-native-01:~/api7/cloud_native/tasks/plugin-runner/apisix-go-plugin-runner$ tree cmdcmd└── go-runner├── main.go├── main_test.go├── plugins│ ├── fault_injection.go│ ├── fault_injection_test.go│ ├── limit_req.go│ ├── limit_req_test.go│ ├── say.go│ └── say_test.go└── version.go2 directories, 10 files
After writing the plugins, start compiling the executable formally, and note here that you should build static executables, not dynamic ones.
The package compile command is as follows.
CGO_ENABLED=0 go build -a -ldflags '-extldflags "-static"' .
This successfully packages a statically compiled go-runner
executable.
In the apisix-go-plugin-runner/cmd/go-runner/
directory, you can see that the current file tree looks like this:
chever@cloud-native-01:~/api7/cloud_native/tasks/plugin-runner/apisix-go-plugin-runner/cmd/go-runner$ tree -L 1.├── go-runner├── main.go├── main_test.go├── plugins└── version.go1 directory, 4 files
Please remember the path apisix-go-plugin-runner/cmd/go-runner/go-runner
, we will use it later.
Build Docker Image
The image is built here in preparation for installing APISIX later using helm
.
Write Dockerfile
Return to the path /home/chever/api7/cloud_native/tasks/plugin-runner
and create a Dockerfile in that directory, a demonstration of which is given here.
ARG ENABLE_PROXY=false# Build Apache APISIXFROM api7/apisix-base:1.19.9.1.5ADD ./apisix-go-plugin-runner /usr/local/apisix-go-plugin-runnerARG APISIX_VERSION=2.13.1LABEL apisix_version="${APISIX_VERSION}"ARG ENABLE_PROXYRUN set -x \&& (test "${ENABLE_PROXY}" != "true" || /bin/sed -i 's,http://dl-cdn.alpinelinux.org,https://mirrors.aliyun.com,g' /etc/apk/repositories) \&& apk add --no-cache --virtual .builddeps \build-base \automake \autoconf \make \libtool \pkgconfig \cmake \unzip \curl \openssl \git \openldap-dev \&& luarocks install https://github.com/apache/apisix/raw/master/rockspec/apisix-${APISIX_VERSION}-0.rockspec --tree=/usr/local/apisix/deps PCRE_DIR=/usr/local/openresty/pcre \&& cp -v /usr/local/apisix/deps/lib/luarocks/rocks-5.1/apisix/${APISIX_VERSION}-0/bin/apisix /usr/bin/ \&& (function ver_lt { [ "$1" = "$2" ] && return 1 || [ "$1" = "`echo -e "$1\n$2" | sort -V | head -n1`" ]; }; if [ "$APISIX_VERSION" = "master" ] || ver_lt 2.2.0 $APISIX_VERSION; then echo 'use shell ';else bin='#! /usr/local/openresty/luajit/bin/luajit\npackage.path = "/usr/local/apisix/?.lua;" .. package.path'; sed -i "1s@.*@$bin@" /usr/bin/apisix ; fi;) \&& mv /usr/local/apisix/deps/share/lua/5.1/apisix /usr/local/apisix \&& apk del .builddeps \&& apk add --no-cache \bash \curl \libstdc++ \openldap \tzdata \# forward request and error logs to docker log collector&& ln -sf /dev/stdout /usr/local/apisix/logs/access.log \&& ln -sf /dev/stderr /usr/local/apisix/logs/error.logWORKDIR /usr/local/apisixENV PATH=$PATH:/usr/local/openresty/luajit/bin:/usr/local/openresty/nginx/sbin:/usr/local/openresty/binEXPOSE 9080 9443CMD ["sh", "-c", "/usr/bin/apisix init && /usr/bin/apisix init_etcd && /usr/local/openresty/bin/openresty -p /usr/local/apisix -g 'daemon off;'"]STOPSIGNAL SIGQUIT
这份 Dockerfile 配置文档,来源于这个链接。我做的唯一修改如下:
ARG ENABLE_PROXY=false# Build Apache APISIXFROM api7/apisix-base:1.19.9.1.5ADD ./apisix-go-plugin-runner /usr/local/apisix-go-plugin-runnerARG APISIX_VERSION=2.13.1LABEL apisix_version="${APISIX_VERSION}"ARG ENABLE_PROXY
Package all the /apisix-go-plugin-runner
files in the /home/chever/api7/cloud_native/tasks/plugin-runner
directory into a Docker image. Note down the location of the executable apisix-go-plugin-runner/cmd/go-runner/go-runner
and the location of the /usr/local/apisix-go-plugin-runner
directory in the Dockerfile above to get the final location of the executable in the Docker image is located as follows.
/usr/local/apisix-go-plugin-runner/cmd/go-runner/go-runner
Please make a note of this address. We will use it in the rest of the configuration.
Begin to build Docker Image
Start building a Docker image based on the Dockerfile. The command is executed in the /home/chever/api7/cloud_native/tasks/plugin-runner
directory. The command is as follows:
docker build -t apisix/forrunner:0.1 .
Command Explanation: Build an image with the name apisix/forrunner
and mark it as version 0.1.
Load the image to the cluster environment
kind load docker-image apisix/forrunner:0.1
Load the image into the kind cluster environment to pull the custom local image for installation during the helm installation.
Install APISIX Ingress
Customize the helm chart
This section focuses on modifying the values.yaml
file in the official helm package so that it can install locally packaged images and run the go-plugin-runner
executable properly.
Fetch Official helm chart
First, fetch the latest apisix helm chart package with the following command:
helm fetch apisix/apisix
The file tree is as follows:
chever@cloud-native-01:~/api7/cloud_native/tasks/plugin-runner$ tree -L 1.├── apisix-0.9.1.tgz└── apisix-go-plugin-runner1 directory, 1 file
Unzip
Unzip the apisix-0.9.1.tgz
file and prepare to rewrite the configuration. The unzip command is as follows.
tar zxvf apisix-0.9.1.tgz
The file tree is as follows:
chever@cloud-native-01:~/api7/cloud_native/tasks/plugin-runner$ tree -L 1.├── apisix├── apisix-0.9.1.tgz└── apisix-go-plugin-runner2 directories, 1 file
Change values.yaml
Go to the apisix
folder and modify the values.yaml
file. The two changes are as follows:
image:repository: apisix/forrunnerpullPolicy: IfNotPresent# Overrides the image tag whose default is the chart appVersion.tag: 0.1
The first change sets the image for the helm installation to be a locally packaged image of your own.
extPlugin:enabled: truecmd: ["/usr/local/apisix-go-plugin-runner/cmd/go-runner/go-runner", "run"]
The second change sets the position of go-runner in the container after running the container.
Compress the modified helm chart
Once configured, compress the apisix
file. The compression command is as follows:
tar zcvf apisix.tgz apisix/
The compressed file is obtained, at which point the file tree is as follows:
chever@cloud-native-01:~/api7/cloud_native/tasks/plugin-runner$ tree -L 1.├── apisix├── apisix-0.9.1.tgz├── apisix-go-plugin-runner└── apisix.tgz2 directories, 2 files
Execute the helm install command
Create namespace
Before installation, create namespaces with the following command:
kubectl create ns ingress-apisix
Then install APISIX using helm with the following command:
helm install apisix ./apisix.tgz --set gateway.type=NodePort --set ingress-controller.enabled=true --namespace ingress-apisix --set ingress-controller.config.apisix.serviceNamespace=ingress-apisix
Create httpbin service and ApisixRoute resources
Create an httpbin backend resource to run with the deployed ApisixRoute resource to test that the functionality is working correctly.
Create httpbin service
Create an httpbin service with the following command:
kubectl run httpbin --image kennethreitz/httpbin --port 80
Expose the port with the following command:
kubectl expose pod httpbin --port 80
Create ApisixRoute Resource
Create the go-plugin-runner-route.yaml
file to enable the ApisixRoute resource, with the following configuration file:
apiVersion: apisix.apache.org/v2beta3kind: ApisixRoutemetadata:name: plugin-runner-demospec:http:- name: rule1match:hosts:- local.httpbin.orgpaths:- /getbackends:- serviceName: httpbinservicePort: 80plugins:- name: ext-plugin-pre-reqenable: trueconfig:conf:- name: "say"value: "{\"body\": \"hello\"}"
The create resource command is as follows:
kubectl apply -f go-plugin-runner-route.yaml
Test
The command is as follows to test if the plugin written in Golang is working correctly:
kubectl exec -it -n ${namespace of Apache APISIX} ${Pod name of Apache APISIX} -- curl http://127.0.0.1:9080/get -H 'Host: local.httpbin.org'
Here I derived from the kubectl get pods --all-namespaces
command that the ${namespace of Apache APISIX}
and ${Pod name of Apache APISIX}
parameters here are ingress-apisix
and apisix- 55d476c64-s5lzw
, execute the command as follows:
kubectl exec -it -n ingress-apisix apisix-55d476c64-s5lzw -- curl http://127.0.0.1:9080/get -H 'Host: local.httpbin.org'
The expected response obtained is:
chever@cloud-native-01:~/api7/cloud_native/tasks/plugin-runner$ kubectl exec -it -n ingress-apisix apisix-55d476c64-s5lzw -- curl http://127.0.0.1:9080/get -H 'Host: local.httpbin.org'Defaulted container "apisix" out of: apisix, wait-etcd (init)hello