What is Signway?

Signway is a server that proxies authentic pre-signed requests to the specified destination, adding the appropriate authentication headers if necessary.

What does it do?

It was initially designed for working with APIs that stream their response, like OpenAI's ChatGPT API.

The problem starts when the backend that queries OpenAI's API wants to re-stream the response back to the frontend, sending the data progressively chunk by chunk as it comes. This can get tricky or be even be impossible depending on the backend's stack.

Instead of re-streaming the response from backend to frontend, why not let the frontend do the request to OpenAI itself? without Signway, the answer is simple:

because OpenAI's key would be leaked.

Signway proposes the following solution:

In the backend, instead of querying OpenAI's API, create a pre-signed URL with a short expiration time.
Send the pre-signed URL back to the frontend.
Let the frontend do the request itself to OpenAI using that pre-signed URL before it expires.
The request will pass through Signway, who will verify that the signature is authentic and has not expired, and if successful, it will proxy the request to OpenAI's API, adding the authentication header if necessary.

With this, the frontend can query OpenAI's API passing through Signway, and it will be almost like a direct request to OpenAI.

What can be used for?

Not only OpenAI's API, but almost any API. Signway is a fast gateway written in Rust, designed specifically for high throughput, so it can be used for leveraging any heavy IO task.

Using Docker

Signway is meant to be used with Docker, so if you want to launch Signway locally for testing it, make sure you have it installed in your machine https://docs.docker.com/engine/install.

Running Signway

The image is publicly available, so you can run Signway with:

docker run gabotechs/signway "<my-id>" "<my-secret>"

<my-id>: you can choose pretty much whatever you want, as it is meant to be public, so there is no need for extra security while storing this one.

<my-secret>: you want to choose a secure string. Think of this as a password, you do not want other people to guess it. If you are using Signway in production, make sure to store this secret securely, as you will need it for creating URL signatures.

Adding headers to the proxy-ed request

Imagine that you want to use Signway with OpenAI's API.

You do not want your users to see your OpenAI's API key, but if they are the ones making the request through Signway... then who sets the Authorization: Bearer <openai-token> header?

You can configure Signway to add that header automatically for you:

docker run gabotechs/signway <my-id> <my-secret> \
  --header "Authorization: Bearer <openai-token>"

Whenever anyone does a request through Signway, and the signature of that request is authentic, an additional Authorization: Bearer <openai-token> header will be added to the request.

Using Signway with OpenAI's API

This example will use Signway's Python SDK for creating signed URLs, and will use Signway's docker image for proxying the verified signed requests to OpenAI's api, so you will need:

Python +3.9 or greater installed in you system
Docker installed in your system
An OpenAI's Api key

Choose an `id` and a `secret` for configuring Signway

export SW_ID="app-id"
export SW_SECRET="super-secure-string"

You don't need to necessarily use this values, but if you want to just copy-paste those that's fine.

Don't forget to also export your OpenAI api key, this should be a valid OpenAI key for this example to work.

export OPENAI_TOKEN="your valid openai api key goes here"

Launching the Signway server

Open a terminal and launch the Signway server:

docker run -p 3000:3000 gabotechs/signway $SW_ID $SW_SECRET \
 --header "Authorization: Bearer $OPENAI_TOKEN"

This Signway server will only accept requests correctly signed using $SW_ID and $SW_SECRET.

Creating a signed URL using the Python SDK

Create a new virtual environment and install the Python SDK:

python3 -m venv venv
source venv/bin/activate
pip install signway-sdk

Remember to also export here the same id and secret from before:

export SW_ID="app-id"
export SW_SECRET="super-secure-string"

Create a new Python file called sign.py and paste this content:

# sign.py
from signway_sdk import sign_url
import os

print(sign_url(
    id=os.environ['SW_ID'],
    secret=os.environ['SW_SECRET'],
    host="http://localhost:3000",
    proxy_url="https://api.openai.com/v1/completions",
    expiry=10,
    method="POST"
))

Executing this script within the venv will output a URL that looks like this:

$ python sign.py

http://localhost:3000/?X-Sw-Algorithm=SW1-HMAC-SHA256&X-Sw-Credential=app-id%2F20230613&X-Sw-Date=20230613T162311Z&X-Sw-Expires=300&X-Sw-Proxy=https%3A%2F%2Fapi.openai.com%2Fv1%2Fchat%2Fcompletions&X-Sw-SignedHeaders=&X-Sw-Body=false&X-Sw-Signature=ebf9dcd8fb2f298af7744a0dbbc96b10d21b38f6e85292f1e06605873088f6e5

Note that the URL points to your Signway server running in the localhost, but it has the X-Sw-Proxy query parameter set to https://api.openai.com/v1/completions. This tells signway where should the request be proxy-ed.

Querying Open AI

Now, try to make a request with curl as if you wanted to query directly OpenAI's API but passing through Signway:

curl $(python sign.py) \
-H "Content-Type: application/json" \
-d '{"model": "text-davinci-003", "prompt": "Say this is a test"}'

If the OPENAI_TOKEN set while launching Signway is valid, you should have received an actual response from OpenAI, and you didn't need to provide any token in the curl request. Signway added it for you.

Let the URL expire

Try this now, store the signed URL in an env variable:

export SIGNED_URL=$(python sign.py)

We configured the script to sign URLs with an expiration date of 10 seconds, so wait 10 seconds and do the request again:

curl $SIGNED_URL \
-H "Content-Type: application/json" \
-d '{"model": "text-davinci-003", "prompt": "Say this is a test"}'

If you are quick copy-pasting this commands in a terminal, you may have seen the request succeed, but after the configured 10s have passed, the request will be rejected.

Sign more things

Right now, the Content-Type header and the body are not signed, so consumer of the signed URL are allowed to do whatever they want with those things.

You can be more restrictive, and also sign both, so that users consuming the signed URL are forced to pass the headers and the body that you want.

For that, edit the Python script:

# sign.py
from signway_sdk import sign_url
import os

print(sign_url(
    id=os.environ['SW_ID'],
    secret=os.environ['SW_SECRET'],
    host="http://localhost:3000",
    proxy_url="https://api.openai.com/v1/completions",
    expiry=10,
    method="POST",
    headers={"Content-Type": "application/json"},
    body='{"model": "text-davinci-003", "prompt": "Say this is a test"}'
))

Now, try to make again the request with curl:

curl $(python sign.py) \
-H "Content-Type: application/json" \
-d '{"model": "text-davinci-003", "prompt": "Say this is a test"}'

That should work, as the provided header and body are the same that were declared in the signature, but what if the body changes for example?

curl $(python sign.py) \
-H "Content-Type: application/json" \
-d '{"model": "text-davinci-003", "prompt": "Say this is NOT a test"}'

You will get rejected by Signway, as the body now is contributing to the URL's signature.

The Signway Book