Good API design is intentional. You are building something that needs to be easy to use, maintainable, scalable, and robust.

This is the problem that OpenAPI helps solve. Instead of hacking an API together, OpenAPI lets you design within a specification and generate code from there. While OpenAPI definitely improves API design, documentation, and collaboration, it has a couple of challenges:

• It has a steep learning curve. For developers new to OpenAPI, there can be a learning curve to understand the specification's syntax, structure, and best practices. This may require an initial investment of time and effort.
• It’s verbose. OpenAPI specifications can become quite large and verbose, especially for complex APIs. This can make it difficult to manage, maintain, and understand the API design.

These issues lead to more errors in the specifications, which lead to more errors in the eventual endpoints. They also make OpenAPI specs difficult to grok for new developers, and difficult to reuse for even senior developers. Again, both of these issues lead to more errors.

Making API design easier, both in terms of learning and writing is the motivation behind Microsoft’s TypeSpec. TypeSpec is "a language for describing cloud service APIs and generating other API description languages, client and service code, documentation, and other assets."

Basically, it is TypeScript for APIs. Like TypeScript, TypeSpec allows you to explicitly define types for variables, function parameters, and return values, which helps catch type-related errors during compile time (you can compile your TypeSpec to OpenAPI, or REST, or gRPC, or graphQL). It also lets you take advantage of Improved tooling and editor support for better code intelligence, better code organization with interfaces, namespaces, and modules, and better error handling with defined error messages. It brings a more dev-focused approach to API design.

Let’s take a look at how TypeSpec works and some of the main challenges in API design it addresses.

The challenges with designing APIs​

The fact that TypeSpec comes from Microsoft is not a coincidence. Across Azure, the teams are dealing with hundreds of endpoints for internal and external use, for dozens of services. They are constantly designing new APIs. Having a system that can be configurable and understandable across the organization is critical.

The usual way to deal with this is through guidelines and best practices. As the scale and complexity of APIs grow, it becomes increasingly important to establish and enforce guidelines and best practices for API design. This can include aspects such as naming conventions, error handling, versioning, and security.

But these are just guidelines. Ensuring that developers across different teams adhere to these guidelines is a huge challenge. Specs such as OpenAPI help with this, but they don’t encourage or enforce:

• Reuse: As the number of API endpoints and resources increases, it can be challenging to identify and reuse common patterns, data structures, and functionalities. Ensuring that APIs follow consistent patterns and avoid duplicating functionality is crucial for maintainability, ease of use, and performance.
• Modularity: APIs at scale should be designed to be modular, allowing components to be developed, tested, and maintained independently. This helps to avoid tight coupling between components and facilitates a more manageable codebase. Striking the right balance between modularity and simplicity can be challenging.
• Consistency: Ensuring consistency across a large set of APIs is essential for usability and maintainability. This includes consistent naming conventions, data formats, and behaviors. As the number of API endpoints and developers working on them increases, maintaining consistency can become more difficult.

OpenAPI also doesn’t support multiple protocols. Designing APIs at scale may require support for multiple protocols, such as REST, GraphQL, gRPC, or WebSocket. There isn’t a paradigm for accommodating these different protocols while maintaining a consistent design and user experience can be challenging.

TypeSpec is a protocol-agnostic language for building API descriptions at scale. It applies learnings from programming languages to API design. It has:

• A familiar syntax, being based on TypeScript
• Tooling for intellisence and code completion
• Templating for reuse
• Extensibility to support any protocol.

Let’s work through an example to show some of these at work.

Using TypeSpec​

Let’s build an OpenAPI specification via the TypeSpec language. We’ll build a toy API, emit an OpenAPI specification, and then show how you can start to modularize your API design for better consistency and easier reuse.

As TypeSpec is based on TypeScript, the install and initialization will look familiar to anyone that knows JS. First, we’ll use NPM to install our compiler:

npm install -g @typespec/compiler

Being a Microsoft product, you can also get nice extensions for Code and VScode:

tsp code install
tsp vs install

Once the compiler is installed, we want to initialize the first project:

tsp init

This will prompt you with a few questions. As you build your own projects, you should choose the setup that fits your needs. Here, we’re going to pick the Generic Rest API template, a project name, and the @typespec/openapi3 library.

Then you can install the dependencies:

tsp install

This will install some node packages (this is node and npm under the skin) and create three new files:

• package.json. A package manifest defining your typespec project as a node package.
• tspconfig.yaml. Your TypeSpec project configuration file to configure emitters, emitter options, compiler options, etc.
• main.tsp. The TypeSpec entrypoint

With that done you can start creating your TypeSpec. If we look at main.tsp now we’ll see just the top import:

import "@typespec/rest";
import "@typespec/openapi3";

We’ll use the HTTP service example from the CADL playground to work through (CADL was the internal project name for this at Microsoft. If you are looking for more information on TypeSpec, be sure to search for CADL as well).

After the imports, the first line sets up the generic namespace we are using (TypeSpec.Http).

Then we use the @service decorator to specify the service definition. We use this to set the title of the service and the version. We then set the namespace for our specific service. Namespaces will let us group related types together for better organization and to prevent name conflicts:

//...

using TypeSpec.Http;
@service({
  title: "Widget Service",
  version: "1.0.0",
})
namespace DemoService;

When we’re setting up our namespace, we can add a couple of other decorators:

• @server - the endpoint for the service
• @doc - a human-readable definition of the service

With that done, we can define our first model. We do that with the model keyword followed by the name we want for that model.

We are now at the Types part of TypeSpec. In each model we can set the type expected for each parameter for that model:

• id is expecting a string
• Weight in expecting and integer
• Color is expecting a string with two possible values, red or blue.

There is also a @path decorator before the id, meaning that this property is a path parameter:

//...

model Widget {
  @path id: string;
  weight: int32;
  color: "red" | "blue";
}

When we’ve defined our models we can define our interface.

First we’ll define our @route for this particular endpoint. Then a @tag that we can use with OpenAPI or Swagger to group our paths.

Finally, we’ll define an interface object containing all our operations. Each operation is defined using a decorator, then has the parameters and the return type. So, for instance:

@get list(): Widget[] | Error;

Here, we’ll set this up for a regular REST operations:

// ...

@route("/widgets")
@tag("Widgets")
interface Widgets {
  @get list(): Widget[] | Error;
  @get read(@path id: string): Widget | Error;
  @post create(...Widget): Widget | Error;
  @patch update(...Widget): Widget | Error;
  @delete delete(@path id: string): void | Error;
}

Finally, we want to define that Error that might be returned. This is a simple error, but you should use Problem Details.

// ...

@error
model Error {
  code: int32;
  message: string;
}

With the service, model, interface, and error defined we can compile into an OpenAPI specification:

tsp compile .

This will produce a new directory, tsp-output, and ‘emit’ an openapi.yaml file defining your API.

openapi: 3.0.0
info:
  title: Widget Service
  version: 1.0.0
tags:
  - name: Widgets
paths:
  /widgets:
    get:
      tags:
        - Widgets
      operationId: Widgets_list
      parameters: []
      responses:
        "200":
          description: The request has succeeded.
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: "#/components/schemas/Widget"
                x-typespec-name: Widget[]
        default:
          description: An unexpected error response.
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
    post:
      tags:
        - Widgets

Going through the entire spec, we can see how some of the elements in the TypeSpec have compiled to OpenAPI. Here’s our Widget model, with the properties and requirements:

Widget:
  type: object
  properties:
    id:
      type: string
    weight:
      type: integer
      format: int32
    color:
      type: string
      enum:
        - red
        - blue
      x-typespec-name: red | blue
  required:
    - id
    - weight
    - color

Here’s is our error model:

Error:
  type: object
  properties:
    code:
      type: integer
      format: int32
    message:
      type: string
  required:
    - code
    - message

What if we want to add another endpoint? We can do the obvious thing, which would be to copy the Widgets model and interface and just change the names. But TypeSpec gives us another options for this–Templates

We’ll create a new file called library.tsp and add our interface and error into this file:

import "@typespec/http";

using TypeSpec.Http;

@error
model Error {
 code: int32;
 message: string;
}

interface ResourceInterface<T> {
  @get list(): T[] | Error;
  @get read(@path id: string): T | Error;
  @post create(...T): T | Error;
  @patch update(...T): T | Error;
  @delete delete(@path id: string): void | Error;
}

We create a generic ResourceInterface that takes in TypeScript parameters (T). We can then use these in our main.tsp. We can import our library and then use our template ResourceInterface to create our model-specific interfaces:

import "@typespec/http";
import "./library.tsp";

using TypeSpec.Http;
@service({
 title: "Widget Service",
 version: "1.0.0",
})
namespace DemoService;

model Widget {
 @path
 id: string;
 weight: int32;
 color: "red" | "blue";
}

model Gadget {
 @path id: string;
 height: float32;
 width: float32;
 color: "green" | "yellow";
}

@route("/widgets")
@tag("Widgets")
interface Widgets extends ResourceInterface<Widget> {
}

@route("/gadgets")
@tag("Gadgets")
interface Gadgets extends ResourceInterface<Gadget> {
}

These ~30 lines can then be compiled to produce a 250-line OpenAPI specification.

More robustness from less work​

From even the toy example above, you can start to see the possibilities with TypeSpec, especially if you are working on APIs at scale or trying to define standards across teams. Leaders in an organization can literally set the interfaces, errors, conventions, and models in a repo for thousands of team members.

For example, If you have a specific way you want errors to be returned, it’s a few lines and then available to everybody, enforced through the compilation step. If someone tries to add their own Error model in main.tsp when they are also importing one from a library, they’ll get an error at compile time.

Here, we’ve only described a REST API and an OpenAPI specification. But emitters can be produced for any protocol, and you can extend TypeSpec exactly the way needed for your team. You can tailor it for your environment. We are doing exactly this at Zuplo.

You can check out more about TypeSpec on GitHub and play around with the language on the playground. Microsoft has written more about it here and here.

HTTP/1.1 403 Forbidden

We've all seen it. Whether in a browser or in Postman or in a terminal.

And we all know the drill. You start checking your credentials and your permissions. You maybe try a GET instead of a POST, and, of course, you try exactly what you tried the first time around in case you've magicked it fixed somehow.

You have to do all of this because just telling you 'Access Forbidden' isn't that helpful. You can see you don't have access, but 'Forbidden' how?

Some developers add more explicative error responses. But these are ad-hoc, with the format entirely down to that API engineering team. This leads to inconsistency and, most importantly, makes it harder for clients to understand and process the errors. APIs are supposed to be machine-readable interfaces. This lack of standardization makes it difficult for API consumers to programmatically handle different types of errors, potentially leading to more fragile and less interoperable systems.

Step in RFC 7807, Problem Details for HTTP APIs. This is a new IETF standard that helps API developers follow a simple pattern for responding to 'problem' requests. Instead of every developer having to reinvent the wheel, they can use this standard to define the details in each request.

Let's go through the details of the standard and how to implement it easily for your APIs.

The Art of Telling Bad News​

Problem Details are a machine-readable object for expressing errors in HTTP APIs. It helps provide a standardized structure for error responses, making it easier for clients to understand and process the errors.

The main components of a Problem Details object are:

• type (string, URI): A URI that identifies the specific error type. This helps clients understand the error and potentially find more information or documentation about it. Ideally, this URI should be stable and not change over time.
• title (string): A short, human-readable summary of the problem. This should be a brief description that concisely conveys the error. The title should not change for a given "type" URI.
• status (integer, optional): The HTTP status code generated by the origin server for this occurrence of the problem. This helps clients understand the nature of the error and how it relates to the HTTP protocol.
• detail (string, optional): A more detailed, human-readable explanation of the problem. This can include specific information about the error and what might have caused it. The "detail" field is intended to provide context and suggestions to clients on how they might address the problem.
• instance (string, URI, optional): A URI that identifies the specific occurrence of the problem. This can help clients and servers correlate and track individual instances of errors.

Let's say you have a credit-based SaaS tool. Users pay for credits upfront and use them up over time. When a user doesn't have enough credits to access the tool, you might return a '403 Forbidden' status:

HTTP/1.1 403 Forbidden

But on its own, this falls foul of the problems above. Neither human nor machine knows what the underlying problem is. With Problem Details, we can add a JSON object to the response with the components above:

HTTP/1.1 403 Forbidden
Content-Type: application/problem+json
Content-Language: en

{
  "type": "https://example.com/probs/out-of-credit",
  "title": "You do not have enough credit.",
  "detail": "Your current balance is 30, but that costs 50.",
  "instance": "/account/12345/msgs/abc",
  "balance": 30,
  "accounts": ["/account/12345", "/account/67890"]
}

Here we've used:

• type set to https://example.com/probs/out-of-credit. The client can dereference this URI to provide more general information about this problem.
• title set to You do not have enough credit. This gives a user a quick understanding of the specific problem.
• detail set to Your current balance is 30, but that costs 50. This goes into more detail about the balance of the account here. Immediately, a user will understand why they are forbidden to access this page and what is needed to resolve the issue
• instance set to /account/12345/msgs/abc. In this case, this will resolve to a log message about the specific instance of the problem. This means a client/user can use this when referencing the issue with the provider.

This set is extensible. Besides the standard fields, Problem Details objects can include custom properties that provide more information about the error. These additional properties can be application-specific and may contain any relevant data that helps clients understand or resolve the error. Here we've added:

• balance set to 30. This tells the client the current balance of the accounts, which can be passed on to a payments portal or to the user to increase their balance.
• accounts set to ["/account/12345","/account/67890"], telling the client the specific accounts related to this problem. Like instance (and type if needed), these are relative URIs. The client can link through to these accounts.

Here's another example from the RFC:

HTTP/1.1 400 Bad Request
Content-Type: application/problem+json
Content-Language: en

{
  "type": "https://example.net/validation-error",
  "title": "Your request parameters didn't validate.",
  "invalid-params": [
    {
      "name": "age",
      "reason": "must be a positive integer"
    },
    {
      "name": "color",
      "reason": "must be 'green', 'red' or 'blue'"
    }
  ]
}

This is a good example for two reasons:

1. It only uses the two required fields of the main members, type and title.
2. The invalid-params property gives detailed information about what's needed for this API. This gives the client or user an instant understanding of their error and what's needed to fix.

Setting up problem details in your own APIs​

Let's go through a couple of quick examples of how these can be added to APIs. As the responses are basic JSON, there is nothing special you have to do to set up Problem Details. The main preparation is enumerating your errors so you can set up types (and URIs) and titles for each, alongside any further properties (such as logs) or parameter responses such as in the 400 example above.

To set up a Problem Detail response in TypeScript, you can set up a ProblemDetails interface and then pass it in the body of the status response. Here's an example using express:

import express, { Request, Response } from "express";

// Define the Problem Details type
interface ProblemDetails {
  type: string;
  title: string;
  status: number;
  detail: string;
  instance?: string;
  balance?: number;
  accounts?: string[];
}

const app = express();
const port = 3000;

app.get("/resource", (req: Request, res: Response) => {
  // Simulate an insufficient credit check
  const hasSufficientCredit = false;

  if (!hasSufficientCredit) {
    const errorDetails: ProblemDetails = {
      type: "https://example.com/probs/out-of-credit",
      title: "You do not have enough credit.",
      status: 403,
      detail: "Your current balance is 30, but that costs 50.",
      instance: "/account/12345/msgs/abc",
      balance: 30,
      accounts: ["/account/12345", "/account/67890"],
    };

    res.status(403).json(errorDetails);
  } else {
    res.json({ message: "Access granted to the resource!" });
  }
});

app.listen(port, () => {
  console.log(`Server running at http://localhost:${port}`);
});

Same goes for Python. Here we'll use pydantic to set up the ProblemDetails class with types and server it with FastAPI. The response is sent as a dict to the client:

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from typing import List

app = FastAPI()

# Define the Problem Details model
class ProblemDetails(BaseModel):
    type: str
    title: str
    status: int
    detail: str
    instance: str = None
    balance: int = None
    accounts: List[str] = None

@app.get("/resource")
async def get_resource():
    has_sufficient_credit = False

    if not has_sufficient_credit:
        error_details = ProblemDetails(
            type="https://example.com/probs/out-of-credit",
            title="You do not have enough credit.",
            status=403,
            detail="Your current balance is 30, but that costs 50.",
            instance="/account/12345/msgs/abc",
            balance=30,
            accounts=["/account/12345", "/account/67890"]
        )

        raise HTTPException(status_code=403, detail=error_details.dict())

    return {"message": "Access granted to the resource!"}

Easing the Pain of API Errors​

Josh talked with Erik Wilde, one of the authors of RFC 7807 a few weeks ago about the impetus and implementation of the standard. Check it out here:

We recently made problem details the default format for errors on all Zuplo Gateways (overridable of course) to help our customers more easily provide better API errors. Once you start to play with these standards, you can start to see the opportunity available through Problem Details for much more robust APIs. The 'sad path' is one many users will walk, so giving them the guidance needed to find the happy path again is a great developer experience. Problem Details do just that, making your API simply more helpful and better to work with than your competitors.

We're working on adding support for Firebase auth to Zuplo (as we did with Supabase) and in doing so came across their JWT signing approach which is one of the more unusual. They sign their tokens with certs and they publish these certs at a well-known URL: https://www.googleapis.com/service_accounts/v1/metadata/x509/securetoken@system.gserviceaccount.com.

It took us a while to work out how to validate these JWT tokens using our preferred library: Jose.

Hopefully this will save you some time if you're trying to do the same thing. Note that we cache the certs in memory since they do not change often (hopefully!).

let publicKeys: any;

const getPublicKeys = async () => {
  if (publicKeys) {
    return publicKeys;
  }
  const res = await fetch(
    `https://www.googleapis.com/service_accounts/v1/metadata/x509/securetoken@system.gserviceaccount.com`
  );
  publicKeys = await res.json();
  return publicKeys;
};

// This goes
// inside your auth function or middleware

const authHeader = request.headers.get("authorization");
const token = authHeader.substring("bearer ".length);

const firebaseProjectId = "your-project-id";

const verifyFirebaseJwt = async (firebaseJwt) => {
  const publicKeys = await getPublicKeys();
  const decodedToken = await jwtVerify(
    firebaseJwt,
    async (header, _alg) => {
      const x509Cert = publicKeys[header.kid];
      const publicKey = await importX509(x509Cert, "RS256");
      return publicKey;
    },
    {
      issuer: `https://securetoken.google.com/${firebaseProjectId}`,
      audience: firebaseProjectId,
      algorithms: ["RS256"],
    }
  );
  return decodedToken.payload;
};

try {
  // This will throw an error if the token is invalid
  const tokenData = await verifyFirebaseJwt(token);
  console.log(`We got a valid token`, tokenData);
} catch (err) {
  console.error(`Invalid Token`, err);
}

Hope you find that useful. Of course, you could always sign up for a free Zuplo account at portal.zuplo.com and get Firebase JWT auth as easy as configuring a policy 👏 👏 👏

If you have used React in the last few years you can't have avoided the useEffect react hook. It's the Batman to your Robin, the Han Solo to your Chewbacca, of the React world.

If you're unfamiliar, the useEffect hook in React is used to handle side-effects in functional components. It allows developers to perform operations such as fetching data from APIs, subscribing to events, and setting up timers. The hook runs after every render or change to dependent property, and provides a way to manage stateful logic in a relatively declarative way.

And therein lies the terror - a dependent property. It's all too easy to create a loop in React, where your useEffect causes a chain of changes that updates the dependent property which triggers your useEffect call, which causes a chain of changes that updates the dependent property which triggers your useEffect call, which causes a chain of changes that updates teh dep... you get the idea.

Here's a simple example

import React, { useState, useEffect } from "react";

function ExampleComponent() {
  const [data, setData] = useState([]);

  useEffect(() => {
    fetch("/api/data")
      .then((response) => response.json())
      .then((result) => setData(result))
      .catch((error) => console.log(error));
  }, [data]);

  return (
    <div>
      <ul>
        {data.map((item) => (
          <li key={item.id}>{item.name}</li>
        ))}
      </ul>
    </div>
  );
}

export default ExampleComponent;

Note that the useEffect call declares a dependency on data but also updates that variable with setData. In most cases, these bugs are not so easy to spot and the loop is hidden by many different calls and a cascade of changes making it much harder to debug.

And notice, in this example we are invoking an API at /api/data. We just made a distributed DDoS machine. If you're lucky, this condition will always happen like in the sample above and you'll notice it immediately. If, as happened to me recently, the loop only happens when the app is in a very specific state nested in a bunch of if statements, it might make it to production. At that point, it only takes a small percentage of your user base to experience this bug for you to bring down a full DDoS on your API.

The reality is that you rarely need rate-limiting to protect from your enemies. It's nearly always your partners or in-house development team that will accidentally take your backend down. Unfortunately the nature of React and useEffect makes this more likely.

The good news is it's easy to protect yourself. You can help prevent performance degradation, reduce costs, and improve the overall user experience of your application by implementing a rate-limiting on your API. You can use an open source package like express-rate-limit or use a SaaS service like Zuplo which is fully-managed, multi-cloud and distributed (over 200 data centers worldwide) - see how to get started with rate-limiting using zuplo - you can start for free.

UPDATE See this post for a detailed walkthrough Shipping a ChatGPT plugin in record time

OpenAI recently announced Chat Plugins for Chat GPT.

To make a plugin you just need an API with an accompanying OpenAPI definition that the Chat GPT. The plugin engine is especially impressive - don't take my word for it; here's Mitchell Hashimoto, Founder of HashiCorp:

See tweet here

If you already have an API and are excited to make it into a Chat GPT plugin there's a few things you'll need to do

• Authentication - support auth with an API Key or OAuth2 client-id and secret.
• OpenAPI - offer an OpenAPI definition so GPT can understand your API
• Rate-limiting - add rate-limiting to your API (with retry-after) so OpenAI don't over-consume your resources

Fortunately, Zuplo is here to help. Zuplo is an API Gateway that natively supports OpenAPI (you can generate it using our route designer or import your own). What's more we make it easy to support API key authentication and rate-limiting, making it the fastest and easiest way for you to surface your plugin to OpenAI.

Get started for free, sign-up at https://portal.zuplo.com.

In light of some recent news about Docker deleting organizations and the containers that are registered with those organizations I figured I would share how we manage our Docker Containers. Zuplo uses a simple Github Action that runs on a cron schedule that mirrors containers we depend on. We initially built this because we experienced some downtime with Docker Hub that caused interruptions to our deployments. The other reason we mirror images is to keep them close to where we use them - in our case that means GCP Artifact Registry.

The Github Action is fairly simple (see below). This has worked well for us and has removed our dependency on Docker Hub for day to day deployments.

name: Mirror Docker Images
on:
  workflow_dispatch:
  schedule:
    - cron: "0 1 * * *"

jobs:
  release:
    name: Mirror Images
    runs-on: ubuntu-latest

    permissions:
      contents: "read"
      id-token: "write"

    env:
      PROJECT_ID: my-project
      REPO_NAME: docker-registry

    strategy:
      fail-fast: false
      matrix:
        image:
          - "ubuntu:latest"
          - "node:18-alpine3.16"

    steps:
      - uses: actions/checkout@v3

      # Uses workload federation: https://github.com/google-github-actions/auth#setting-up-workload-identity-federation
      - id: "auth-gcp"
        name: "Authenticate to Google Cloud"
        uses: "google-github-actions/auth@v1"
        with:
          token_format: "access_token"
          workload_identity_provider: "your-provider"
          service_account: "your-service-account"
          access_token_lifetime: "300s"

      - name: Set up Cloud SDK
        uses: google-github-actions/setup-gcloud@v1.1.0
        with:
          project_id: ${{ env.PROJECT_ID }}

      - name: Authenticate Docker
        run: gcloud auth configure-docker us-docker.pkg.dev

      - name: Pull Image from Docker Hub
        run: docker pull ${{ matrix.image }}

      - name: Tag Image
        run: docker tag ${{ matrix.image }} us-docker.pkg.dev/${{ env.PROJECT_ID }}/${{ env.REPO_NAME }}/${{ matrix.image }}

      - name: Push Image
        run: docker push us-docker.pkg.dev/${{ env.PROJECT_ID }}/${{ env.REPO_NAME }}/${{ matrix.image }}

Welcome to OpenAPI week at Zuplo!

Today we're announcing our official support for OpenAPI. Unlike other gateways, we're not simply adding an import feature for OpenAPI but are now OpenAPI native, with the format being the core of how route configuration is specified in the gateway.

Any valid OpenAPI document is a valid API Gateway configuration for Zuplo, additional properties and configuration is added via x-zuplo vendor extensions.

As a result, Zuplo now offers the most seamless workflow for API design-first teams and users of OpenAPI. For those that don't, not to worry, you'll be using OpenAPI in Zuplo but won't even notice.

Check out the demo video

Read more about this in our OpenAPI docs.

To celebrate, we're hosting an OpenAPI week with an interviews with special guests from the OpenAPI community. Sign up for the premiere of each chat below:

Darrel Miller

Editor of the OpenAPI specification and API architect at Microsoft.

On Tuesday, 3/7, we discuss the Future of OpenAPI (and some history)

Watch Darrel Miller's session here

Phil Sturgeon

Staff Author and co-host of APIs you won't hate

On Wednesday 3/8, we discuss living with OpenAPI in the real world

Watch Phil Sturgeon's session here

Kevin Swiber

Marketing Chair, OpenAPI

On Thursday, 3/9 we talk about the spec wars and how OpenAPI plays a role in the API lifecycle

Watch Kevin Swiber's session here

Erik Wilde

Author, RFC 7807

On Friday 3/10 we'll look at the new(ish) Problem Details for HTTP APIs specification with one of its authors

Watch Erik Wilde's session here

These conversations will be premiering on our YouTube channel. Subscribe for notifications, follow us on Twitter or our Discord to get notified when these great conversations drop.

OpenAI is all the rage now and developers are rushing to leverage this technology in their apps and SaaS products. But such is the demand for this stuff, you might need to think about how you protect yourself from abuse - here's a post from a colleague I saw on LinkedIn recently:

You can use a Zuplo gateway to store your API keys and enforce a number of layers of protection to discourage abuse of your OpenAI API keys.

How it works​

Zuplo allows you to easily perform authentication translation, that is, change the authentication method your clients use. For example you might require your clients to use

• JWT tokens
• API Keys issued to each of your customers (different to your OpenAI key, so you can identify each individual customer)
• Anonymously in a web browser — but ensure the call is coming from the right origin, enforce CORS and rate-limit by IP etc.

Setting up Zuplo to send the API Key​

This is a CURL command to call the OpenAI API directly, note the need for an API KEY

curl https://api.openai.com/v1/completions \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer YOUR_API_KEY_HERE' \
  -d '{
  "model": "babbage",
  "prompt": "Say this is a test",
  "max_tokens": 7,
  "temperature": 0
}'

To get started we'll create a simple Zuplo gateway that removes the need to specify the API key.

Create a new project and add a route:

• Summary: My OpenAI Completions
• Path: /v1/my-completions
• Method: POST
• CORS: No CORS
• Handler: URL Rewrite - https://api.openai.com/v1/completions

Next, we need to add a policy that will set the authorization header when calling OpenAI. Open the Policies section and click Add Policy.

Choose the Add or Set Request Headers policy. Set the policy configuration as follows

{
  "export": "SetHeadersInboundPolicy",
  "module": "$import(@zuplo/runtime)",
  "options": {
    "headers": [
      {
        "name": "authorization",
        "value": "Bearer $env(OPEN_AI_API_KEY)",
        "overwrite": true
      }
    ]
  }
}

Note that we will read the API Key from a secure secret stored as an environment variable - go setup your OPEN_AI_API_KEY env var.

Save your changes, and we're ready.

Take the above curl command and remove the authorization header and change the URL to your project URL:

curl https://open-ai-main-298dc8d.d2.zuplo.dev/v1/my-completions \
  -H 'Content-Type: application/json' \
  -d '{
  "model": "babbage",
  "prompt": "Say this is a test",
  "max_tokens": 7,
  "temperature": 0
}'

Look no API key 👏 - but your request should work fine as Zuplo will add the key on the way out.

Securing Zuplo​

You now have several choices to secure Zuplo.

1. Require your users to login (with a service like Auth0) and then use an Auth0 JWT with Zuplo.
2. Issue API Keys to all your users using Zuplo's API Key Service.
3. Host anonymously but add additional safe guards, including requiring a specific Origin and strict CORS using custom CORS policies.

Make sure to add rate limiting - based on user or maybe IP (for anonymous use case).

Event Streaming (data-only server-sent events)​

OpenAI supports event streaming, this is easy to get working with Zuplo and works out of the box. You can try this by adding a stream: true property to your POST to OpenAI:

curl https://open-ai-main-298dc8d.d2.zuplo.dev/v1/my-completions \
  -H 'Content-Type: application/json' \
  -d '{
  "model": "babbage",
  "prompt": "Say this is a test",
  "max_tokens": 7,
  "temperature": 0,
  "stream": true
}'

However, what if you want to support EventSource in the browser? That is easy to accomplish with Zuplo also by taking the incoming GET request created by EventSource and translating it into a POST request, with the appropriate headers and body inside Zuplo.

Create a new route:

• Summary: My OpenAI Completions for Browser Event Source
• Path: /v1/my-browser-completions
• Method: GET
• CORS: Anything Goes
• Handler: URL Rewrite - https://api.openai.com/v1/completions

Add the following policies

• Reuse your Set Header policy that sets the authorization key above.
• Add a Change Method policy to update the request to be a POST
• Add another Set Header policy to set the content-type header to application/json
• Finally, add a Set Body policy with the following configuration.

{
  "export": "SetBodyInboundPolicy",
  "module": "$import(@zuplo/runtime)",
  "options": {
    "body": "{  \"model\": \"babbage\",  \"prompt\": \"Say this is a test\", \"max_tokens\": 7, \"temperature\": 0, \"stream\": true }"
  }
}

You can now use an EventSource in a browser and call Zuplo as follows

const evtSource = new EventSource(
  "open-ai-main-298dc8d.d2.zuplo.dev/v1/my-browser-completions"
);

evtSource.onmessage = (evt) => {
  if (evt.data === "[DONE]") {
    console.log("end of event stream...");
    return;
  }
  console.log(JSON.parse(evt.data));
};

You could also make the POST body dynamic, based on a querystring in the EventSource - you would then read the querystring values in a custom policy and set the body based on values in the querystring (you would no longer need the Set Body policy in this case).

The custom code (inbound policy) might look like this

import { ZuploContext, ZuploRequest } from "@zuplo/runtime";

export default async function (
  request: ZuploRequest,
  context: ZuploContext,
  options: never,
  policyName: string
) {
  const prompt = request.query.prompt;

  // perform any appropriate validation on `prompt` here

  const data = {
    model: "babbage",
    prompt, // pass the query value in here
    max_tokens: 7,
    temperature: 0,
  };

  return new ZuploRequest(request, {
    body: JSON.stringify(data),
  });
}

We just published a new video showing how you can add smart routing, behind a single common API for multiple backends, in 1 page of TypeScript. Metadata is loaded from an external service (in this case, Xata but you could use Supabase, Mongo etc).

Here's the code used in the demonstration:

import {
  ZuploContext,
  ZuploRequest,
  environment,
  ZoneCache,
} from "@zuplo/runtime";

interface RouteInfo {
  customerId: string;
  primaryUrl: string;
  secondaryUrl?: string;
}

const CACHE_KEY = "ROUTE_RECORDS";
const CACHE_NAME = "ROUTE_INFO";

async function loadRouteInfoFromApi(context: ZuploContext) {
  const cache = new ZoneCache(CACHE_NAME, context);

  const records = await cache.get(CACHE_KEY);

  if (!records) {
    const options = {
      method: "POST",
      headers: {
        Authorization: `Bearer ${environment.XATA_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: '{"page":{"size":15}}',
    };

    const response = await fetch(
      "https://YOUR-XATA-URL.xata.sh/db/test:main/tables/routing/query",
      options
    );

    const data = await response.json();
    cache.put(CACHE_KEY, data.records, 300); // 5 minutes

    context.log.info("RouteInfo loaded from API");
    return data.records;
  }

  context.log.info("RouteInfo loaded from Cache");
  return records;
}

export default async function (request: ZuploRequest, context: ZuploContext) {
  const customerId = request.user.data.customerId;

  const routing = await loadRouteInfoFromApi(context);

  const routeInfo = routing.find((r) => r.customerId === customerId);

  if (!routeInfo) {
    return new Response(`No route found for customer '${customerId}'`, {
      status: 404,
    });
  }

  const response = await fetch(routeInfo.primaryUrl);
  if (response.status !== 200 && routeInfo.secondaryUrl) {
    context.log.info(
      `First request failed, trying secondary (${response.status})`
    );
    const response2 = await fetch(routeInfo.secondaryUrl);
    return response2;
  }

  return response;
}

Got questions or feedback? Join us on Discord.

Your supabase backend is often exposed to the public, anybody can sign in create an account and work with data. This can be a problem, if you get a malicious or clumsy user that is hitting your service too hard. That's where you need rate-limits, a way of making sure a single user doesn't starve others of resources (or cost you too much $).

With Zuplo, you can add user-based rate-limiting to a supabase backend in a couple of minutes. There is a video tutorial version of this guide here: YouTube: Per-user rate limit your supabase backend.

Best of all, the only code you'll need to change in your client is the URL of the supabase service (because traffic will now go via Zuplo).

Here are the steps

1/ Create a new project in Zuplo (get a free account at portal.zuplo.com).

2/ Add a route to your new project. Set the following properties

• path: /(.*) - this is wildcard route that will match all paths
• methods: all - select all methods in the dropdown
• CORS: anything goes - this is easiest, but you can set stricter policies
• URL Rewrite: <https://your-supabase-domain>${pathname} - make sure to add your supabase URL, e.g. https://rxodffgalrhwpvjugcio.supabase.co${pathname}

3/ Add a policy to the request pipeline - choose the supabase-jwt-auth policy. Remove the required claims from the JSON template.

{
  "export": "SupabaseJwtInboundPolicy",
  "module": "$import(@zuplo/runtime)",
  "options": {
    "secret": "$env(SUPABASE_JWT_SECRET)",
    "allowUnauthenticatedRequests": false
  }
}

4/ Create an environment variable called SUPABASE_JWT_SECRET (this is in Settings > Environment Variables). Paste in the JWT Secret from supabase (available in Settings > API).

5/ Add a rate-limiting policy at the end of the request pipeline. Configure it to be a user mode rate limit, suggest 2 requests per minute for demo purposes.

{
  "export": "RateLimitInboundPolicy",
  "module": "$import(@zuplo/runtime)",
  "options": {
    "rateLimitBy": "user",
    "requestsAllowed": 2,
    "timeWindowMinutes": 1
  }
}

6/ Get the URL for your gateway by going to Getting Started tab and copying the gateway URL. Replace the Supabase URL in your client and boom 💥!

You now have a rate-limit protected supabase backend. Stay tuned for a subsequent tutorial where we'll show how to ensure folks have to come via Zuplo to call your Supabase backend.

Got questions or feedback, join us on Discord.