Skip to main content

4 posts tagged with "WebAssembly"

View All Tags

GraphQL Server in the Browser using WebAssembly

· 5 min read
Shadaj Laddad
Co-founder @ Exograph

On the heels of our last feature release, we are excited to announce a new feature: Exograph Playground in the browser! Thanks to the magic of WebAssembly, you can now run Exograph servers entirely in your browser, so you can try Exograph without even installing it. That's right, we run a Tree Sitter parser, typechecker, GraphQL runtime, and Postgres all in your browser.

See it in action

Head over to the Exograph Playground to try it out. It shows a few pre-made models and populates data for you to explore. It also shows sample queries to help you get started.

A few models need authentication, so you can click on the "Key" icon in the middle center of the GraphiQL component to simulate the login action.

When you open the playground, the top-left portion shows the model. The top-right portion shows tabs to give insight into Exograph's inner workings. The bottom portion shows the GraphiQL interface to run queries and mutations.


You can also create your model by replacing the existing model in the playground. The playground supports sharing your playground project as a gist.

How it works

The Exograph Playground runs entirely in the browser. Besides the initial loading of static assets (like the WebAssembly binary and JavaScript code), you don't need to be connected to the internet. This is possible because we compiled Exograph, written in Rust, to WebAssembly.

Playground Architecture

Builder

The builder plays a role equivalent to the exo build command. It reads the source code, parses and typechecks it, and produces an intermediate representation (equivalent to the exo_ir file). The builder also includes elements equivalent to the exo schema command to compute the SQL schema and migrations.

On every change to the source code, the builder validates the model, reports any errors, and produces an updated intermediate representation. You can see the errors in the "Problems" tab.

The builder also produces the initial SQL schema and migrations for the model as you change it. The playground will automatically apply migrations as needed. You can see the schema in the "Schema" tab.

Runtime

The runtime is equivalent to the exo-server command. It processes the intermediate representation the builder produced and serves the GraphQL API. When you run a query, it sends it to the runtime, which computes the SQL query, runs against the database, and returns the results.

The runtime also sends logs to the playground, which you can see in the "Traces" tab. This aids in understanding what Exograph is doing under the hood, including the SQL queries it executes.

Postgres

The playground uses pglite, which is Postgres compiled to WebAssembly. Currently, we store Postgres data in memory, so you will lose the data when you refresh the page. We plan to add support for saving the data to local storage.

GraphiQL

The GraphiQL is a standard GraphQL query interface. You can run queries and mutations against the Exograph server. The playground populates the initial query for you to get started.

Sharing Playground Project as a Gist

The playground supports sharing the content as a gist. You can load such a gist using the gist query parameter. For example, to load the gist with ID abcd, you can use the URL https://exograph.dev/playground?gist=abcd.

You can create a gist to share your model, the seed data, and the initial query populated in GraphiQL. The files in gist follow the same layout as the project directory in Exograph, except you use :: as the directory separator.

  • src::index.exo: The model
  • tests::init.gql:The seed data. See Initializing seed data for more details.
  • playground::query.graphql: The initial query in GraphiQL
  • README.md: The README to show in the playground

We will keep improving the sharing experience in the future.

What's next

This is just the beginning to make Exograph easier to explore. Here are a few planned features to make the playground even better (your feedback is welcome!):

  • Support JavaScript Modules: In non-browser environments, Exograph supports Deno Modules. Deno cannot be compiled to WebAssembly, so we cannot run it in the browser. However, browsers already have a JavaScript runtime 🙂, which we will support in the playground.
  • Persistent Data: We plan to add support for saving the data to local storage so you can continue working on your data model across sessions.
  • Improved Sharing: We will add a simple way to create gists for your playground content and share it with others.

Try it out and let us know what you think. If you develop a cool model, publish it as a gist and share it with us on Twitter or Discord. We would appreciate a star on GitHub!

Share:

Exograph at the Edge with Cloudflare Workers

· 5 min read
Ramnivas Laddad
Co-founder @ Exograph

We are excited to announce that Exograph can now run as a Cloudflare Worker! This new capability allows deploying Exograph servers at the edge, closer to your users, and with lower latency.

Cloudflare Workers is a good choice for deploying APIs due to the following characteristics:

  • They scale automatically to handle changing traffic patterns, including scaling down to zero.
  • They have an excellent cold start time (in milliseconds).
  • They get deployed in Cloudflare's global network, thus the workers can be placed optimally for better latency and performance.
  • They have generous free tier limits that can be sufficient for many applications.

With Cloudflare as a deployment option, a question remains: How do we develop backends? Typical backend development can be complex, time-consuming, and expensive, requiring specialized teams to ensure secure and efficient execution. This is where Exograph shines. With Exograph, developers:

  • Focus only on defining the domain model and authorization rules.
  • Get inferred APIs (currently GraphQL, with REST and RPC coming soon) that execute securely and efficiently.
  • Use the provided tools to develop locally, deploy to the cloud, migrate database schemas, etc.
  • Use telemetry to monitor production usage.

Combine Cloudflare Workers with Exograph, and you get cost-effective development and deployment.

In this blog, we will show you how to deploy Exograph backends on Cloudflare Workers and how to use Hyperdrive to reduce latency.

A taste of Exograph on Cloudflare Workers

Exograph provides a CLI command to create a WebAssembly distribution suitable for Cloudflare. It also creates starter configuration files to develop locally and deploy to the cloud.

exo deploy cf-worker

The command will provide instructions for setting up the database connection. You can create a new database or use an existing one and add its URL as the EXO_POSTGRES_URL secret. Cloudflare Workers also integrate with databases such as Neon to add this secret through Cloudflare's dashboard.

To run the worker locally, you can use the following command:

npx wrangler dev
...
Using vars defined in .dev.vars
Your worker has access to the following bindings:
- Vars:
- EXO_POSTGRES_URL: "(hidden)"
- EXO_JWT_SECRET: "(hidden)"
⎔ Starting local server...
[wrangler:inf] Ready on http://localhost:8787

And when you are ready to deploy to the cloud, run:

npx wrangler deploy
...
Uploaded todo (2.19 sec)
Published tod (0.20 sec)
https://todo.<domain>.workers.dev
...

Please see the Exograph documentation for more details.

Using Hyperdrive to reduce latency

Let's measure the latency of the request with a query to fetch all todos. Here, we have deployed the worker that connects to a Postgres database managed by Neon.

oha -c 1 -n 10 -m POST -d '{ "query": "{todos { id }}"}' <worker-url>
Slowest: 0.5357 secs
Fastest: 0.2436 secs
Average: 0.2872 secs

The mean response time of 287ms is good but not stellar. The main reason for increased latency is that the worker has to open a new connection to the Postgres database for every request. If connection establishment time is the problem, connection pooling is a solution. For Cloudflare Worker, connection pooling comes in the form of Hyperdrive.

Behind the scenes

To extract latency benefits, we dealt with a few challenges. You can read more about them in our previous blog posts on "Latency at the Edge with Rust/WebAssembly and Postgres": Part 1 and Part 2.

To use this connection pooling option, you create a Hyperdrive using either the npx wrangler hyperdrive create command or the Cloudflare Worker's dashboard. Then add the following to your wrangler.toml:

EXO_HYPERDRIVE_BINDING = "<binding-name>"

[[hyperdrive]]
binding = "<binding-name>"
id = "..."

The worker will now use Hyperdrive to manage the database connections, significantly reducing the latency of the requests. Let's measure the latency again:

oha -c 1 -n 10 -m POST -d '{ "query": "{todos { id }}"}' <worker-url>

Slowest: 0.3588 secs
Fastest: 0.0879 secs
Average: 0.0967 secs

Much better! We have reduced the mean response time to 97ms, significantly faster than the previous 287ms.

How does it work?

Cloudflare Worker is, at its core, a V8 runtime capable of running JavaScript code. V8 also supports JavaScript loading and executing WebAssembly modules. To make Exograph run on Cloudflare Workers, we compiled Exograph to WebAssembly. Currently, Rust has the best tooling to target WebAssembly. Here, our decision to implement Exograph in Rust paid off!

How it works

As for the developers using Exograph, we ship a WebAssembly binary distribution for exo-server, which provides bindings to the Cloudflare Workers runtime and implements a few optimizations that we will see in the next section. It also creates JavaScript scaffolding to interact with the WebAssembly binary.

Roadmap

Our current Cloudflare worker support is a preview. We are planning on adding more features and improvements in the upcoming releases. Here is a high-level roadmap:

  • Improved performance: While the performance in the current release is already pretty good, especially with Hyperdrive, Exograph's ahead-of-time compilation offers more opportunities to improve it, and we will explore them.
  • JS Integration: Exograph embeds Deno as the JavaScript engine, but that won't work in Cloudflare Workers. However, Cloudflare worker's primary runtime is JavaScript (WebAssembly is a guest), so we will support integrating Exograph with the host system's JavaScript runtime.
  • Trusted documents: The current release doesn't yet support trusted documents, but we are working on it.

What's Next?

Exograph's WebAssembly target is a significant milestone in our journey to bring new possibilities to the Exograph ecosystem. But this is just the beginning. The next blog post will showcase another exciting feature due to this new capability. Stay tuned!

We are eager to know how you plan to use Exograph in Cloudflare workers. You can reach us on Twitter or Discord with your feedback. We would appreciate a star on GitHub!

Share:

Latency at the edge with Rust/WebAssembly and Postgres: Part 2

· 9 min read
Ramnivas Laddad
Co-founder @ Exograph

In the previous post, we implemented a simple Cloudflare Worker in Rust/WebAssembly connecting to a Neon Postgres database and measured end-to-end latency. Without any pooling, we got a mean response time of 345ms.

The two issues we suspected for the high latency were:

Establishing connection to the database: The worker creates a new connection for each request. Given that a secure connection, it takes 7+ round trips. Not surprisingly, latency is high.

Executing the query: The query method in our code causes the Rust Postgres driver to make two round trips: to prepare the statement and to bind/execute the query. It also sends a one-way message to close the prepared statement.

In this part, we will deal with connection establishment time by introducing a pool. We will fork the driver to deal with multiple round trips (which incidentally also helps with connection pooling). We will also learn a few things about Postgres's query protocol.

Source code

The source code with all the examples explored in this post is available on GitHub. With it, you can perform measurements and experiments on your own.

Introducing connection pooling

If the problem is establishing a connection, the solution could be a pool. This way, we can reuse the existing connections instead of creating a new one for each request.

Application-level pooling

Could we use a pooling crate such as deadpool?

While that would be a good option in a typical Rust environment (and Exograph uses it), it is not an option in the Cloudflare Worker environment. A worker is considered stateless and should not maintain any state between requests. Since a pool is a stateful object (holding the connections), it can't be used in a worker. If you try to use it, you will get the following runtime error on every other request:

Error: Cannot perform I/O on behalf of a different request. I/O objects (such as streams, request/response bodies, and others) created in the context of one request handler cannot be accessed from a different request's handler. This is a limitation of Cloudflare Workers which allows us to improve overall performance. (I/O type: WritableStreamSink)

When the client makes the first request, the worker creates a pool and successfully executes the query. For the second request, the worker tries to reuse the pool, but it fails due to the error above, leading to the eviction of the worker by the Cloudflare runtime. For the third request, a fresh worker creates another pool, and the cycle continues.

The error is clear: we cannot use application-level pooling in this environment.

External pooling

Since application-level pooling won't work in this environment, could we try an external pool? Cloudflare provides Hyperdrive for connection pooling (and more, such as query caching). Let's try that.

#[event(fetch)]
async fn main(_req: Request, env: Env, _ctx: Context) -> Result<Response> {
let hyperdrive = env.hyperdrive("todo-db-hyperdrive")?;

let config = hyperdrive
.connection_string()
.parse::<tokio_postgres::Config>()
.map_err(|e| worker::Error::RustError(format!("Failed to parse configuration: {:?}", e)))?;

let host = hyperdrive.host();
let port = hyperdrive.port();

// Same as before
}

Besides how we get the host and port, the rest of the code (to connect to the database and execute the query) remains the same as the one in part 1.

You will need to create a Hyperdrive instance using the following command (replace the connection string with your own):

npx wrangler hyperdrive create todo-db-hyperdrive --caching-disabled --connection-string "postgres://..."

We disable query caching since that will cause skipping most database calls. Due to the empty cache, the first request will hit the database. For subsequent requests (which execute the same SQL query in our setup), Hyperdrive will likely serve them from its cache. We are interested in measuring the latency to include database calls. With caching turned on, the comparison to the baseline would be apples-to-oranges.

note

For the real-world scenario, you may enable caching to balance database load and freshness of data.

Next, you will need to put the Hyperdrive information in wrangler.toml:

[[hyperdrive]]
binding = "todo-db-hyperdrive"
id = "<your-hyperdrive-id>"

Let's test this worker.

curl <worker-url>
INTERNAL SERVER ERROR

Hmm... that failed.

Fast moving ground

This is due to an issue with the current Hyperdrive implementation. The support for prepared statements is still new and (currently) works only with caching enabled. I have made the Cloudflare team aware of it. I think this will be fixed soon 🤞.

As things change, I will add updates here.

What's going on? Postgres has two kinds of query protocols:

  1. Simple query protocol: With this protocol, you must supply the SQL as a string and include any parameter values in the query (for example, SELECT * FROM todos WHERE id = 1). The driver makes one round trip to execute such a query.
  2. Extended query protocol: With this protocol, you may have the SQL query with placeholders for parameters (for example, SELECT * FROM todos WHERE id = $1), and its execution requires a preparation step. We will go into detail in the next section.

Let's explore both protocols.

Hyperdrive with the simple query protocol

To explore the simple query protocol, we will use the simple_query method. Since it doesn't allow specifying parameters, we inline them.

#[event(fetch)]
async fn main(_req: Request, env: Env, _ctx: Context) -> Result<Response> {

/// Hyperdrive setup as before

let rows = client
.simple_query("SELECT id, title, completed FROM todos WHERE completed = true")
.await
.map_err(|e| worker::Error::RustError(format!("Failed to query: {:?}", e)))?;

...
}

Does it work and how does this perform?

oha -c 1 -n 100 <worker-url>
Slowest: 0.2871 secs
Fastest: 0.0476 secs
Average: 0.0633 secs

That's more like it! The mean response time is 63ms, a significant improvement over the previous 345ms.

Since the simple query protocol needed only one round trip, Hyperdrive was able to use an existing connection without too much additional logic, so it worked without any issues and performed well.

But... the simple query protocol forces us to use string interpolation to inline the parameters in the query, which is a big no-no in the world of databases due to the risk of SQL injection attacks. So let's not do that!

Hyperdrive with the extended query protocol

Let's go back to the extended query protocol and figure out why Hyperdrive might be struggling with it. As it happens, all external pooling services deal with the same issue; for example, only recently did pgBouncer start to support it.

When using the extended query protocol through query, the driver executes the following steps:

  1. Prepare: Sends a prepare request. This contains a name for the statement (for example, "s1") and a query with the placeholders for parameters to be provided later (for example, $1, $2 etc.). The server sends back the expected parameter types.
  2. Bind/execute: Sends the name of the prepared statement and the parameters serialized in the format appropriate for the types. The server looks up the prepared statement by name and executes it with the provided parameters. It sends back the rows.
  3. Close: Closes the prepared statement to free up the resources on the server. In tokio-postgres, this is a fire-and-forget operation (doesn't wait for a response).

When you add in a connection pool, the driver must invoke the "bind/execute" and "close" with the same connection it used for "prepare". This requires some bookkeeping and is a source of complexity.

What if we combine all three steps into a single network package? This is what Exograph's fork of tokio-postgres (fork, PR) does. The client must specify the parameter values and their types (we no longer perform a round trip to discover parameter types). This way, the driver can serialize the parameters in the correct format in the same network package.

#[event(fetch)]
async fn main(_req: Request, env: Env, _ctx: Context) -> Result<Response> {

/// Hyperdrive setup as before

let rows: Vec<tokio_postgres::Row> = client
.query_with_param_types(
"SELECT id, title, completed FROM todos where completed <> $1",
&[(&true, Type::BOOL)],
)
.await
.map_err(|e| worker::Error::RustError(format!("query_with_param_types: {:?}", e)))?;

...
}

How does this perform?

oha -c 1 -n 100 <worker-url>
Slowest: 0.2883 secs
Fastest: 0.0466 secs
Average: 0.0620 secs

Nice! The mean response time is now 62ms, which matches the simple query protocol (63ms).

Summary

Let's summarize the mean response times for the various configurations:

Method ➡️querysimple_queryquery_with_param_types
Pooling ⬇️Timing in milliseconds
None345312312
Hyperdrivesee above6362

With connection pooling through Hyperdrive, we have brought the mean response time by a factor of 5.5 (from 345ms to 62ms)!

Round trip cost

The 33ms improvement between query (345ms) and query_with_param_types (312ms) is likely to be due to saving the extra round trip for the "prepare" step but needs further investigation.

The source code is available on GitHub, so you can check this yourself. If you find any improvements or issues, please let me know.

So what should you use? Assuming that the issue with Hyperdrive and query method has been fixed:

  • If you don't want to use Hyperdrive, use the query_with_type_params method with the forked driver. It does the job in one roundtrip and gives you the best performance without any risk of SQL injection attacks.
  • If you want to use Hyperdrive:
    • If you frequently make the same queries, the query method will likely do better. Hyperdrive may cache the "prepare" part of the step, making subsequent queries faster.
    • If you make a variety of queries, you can use the query_with_param_types method. Since you won't execute the same query frequently, Hyperdrive's prepare statement caching is unlikely to help. Instead, this method's fewer round trips will be beneficial.

Watch Exograph's blog for more explorations and insights as we ship its Cloudflare Worker support. You can reach us on Twitter or Discord. We would appreciate a star on GitHub!

Share:

Latency at the Edge with Rust/WebAssembly and Postgres: Part 1

· 6 min read
Ramnivas Laddad
Co-founder @ Exograph

We have been working on enabling Exograph on WebAssembly. Since we have implemented Exograph using Rust, it was natural to target WebAssembly. You can soon build secure, flexible, and efficient GraphQL backends using Exograph and run them at the edge.

During our journey towards WebAssembly support, we learned a few things to improve the latency of Rust-based programs targeting WebAssembly in Cloudflare Workers connecting to Postgres. This two-part series shares those learnings. In this first post, we will set up a simple Cloudflare Worker connecting to a Postgres database and get baseline latency measurements. In the next post, we will explore various ways to improve it.

Even though we experimented in the context of Exograph, the learnings should apply to anyone using WebAssembly in Cloudflare Workers (or other platforms that support WebAssembly) to connect to Postgres.

Second Part

Read Part 2 that improves latency by a factor of 6!

Rust Cloudflare Workers

Cloudflare Workers is a serverless platform that allows you to run code at the edge. The V8 engine forms the underpinning of the Cloudflare Worker platform. Since V8 supports JavaScript, it is the primary language for writing Cloudflare Workers. However, JavaScript running in V8 can load WebAssembly modules. Therefore, you can write some parts of a worker in other languages, such as Rust, compile it to WebAssembly, and load that from JavaScript.

Cloudflare Worker's Rust tooling enables writing workers entirely in Rust. Behind the scenes, the tooling compiles the Rust code to WebAssembly and loads it in a JavaScript host. The Rust code you write must be able to compile to wasm32-unknown-unknown target. Consequently, it must follow the restrictions of WebAssembly. For example, it cannot access the filesystem or network directly. Instead, it must rely on the host-provided capabilities. Cloudflare provides such capabilities through the worker-rs crate. This crate, in turn, uses wasm-bindgen to export a few JavaScript functions to the Rust code. For example, it allows opening network sockets. We will use this capability later to integrate Postgres.

Here is a minimal Cloudflare Worker in Rust:

use worker::*;

#[event(fetch)]
async fn main(_req: Request, _env: Env, _ctx: Context) -> Result<Response> {
Ok(Response::ok("Hello, Cloudflare!")?)
}

To deploy, you can use the npx wrangler deploy command, which compiles the Rust code to WebAssembly, generates the necessary JavaScript code, and deploys it to the Cloudflare network.

Before moving on, let's measure the latency of this worker. We will use Ohayou, an HTTP load generator written in Rust. We measure latency using a single concurrent client (-c 1) and one hundred requests (-n 100).

oha -c 1 -n 100 <worker-url>
...
Slowest: 0.2806 secs
Fastest: 0.0127 secs
Average: 0.0214 secs
...

It takes an average of 21ms to respond to a request. This is a good baseline to compare when we add Postgres to the mix.

Focusing on latency

We will focus on measuring the lower bound for latency of the roundtrip for a request to the worker who queries a Postgres database before responding. Here is our setup:

  • Use a Neon Postgres database with the following table and no rows to focus on network latency (and not database processing time).

    CREATE TABLE todos (
    id SERIAL PRIMARY KEY,
    title TEXT NOT NULL,
    completed BOOLEAN NOT NULL
    );
  • Implement a Cloudflare Worker that responds to GET by fetching all completed todos from the table and returning them as a JSON response (of course, since there is no data, the response will be an empty array, but the use of a predicate will allow us to explore some practical considerations where the queries will have a few parameters).

  • Place the worker, database, and client in the same region. While, we can't control the worker placement, Cloudflare will place the worker close to either the client or the database (which we've put in the same region).

All right, let's get started!

Connecting to Postgres

Let's implement a simple worker that fetches all completed todos from the Neon Postgres database. We will use the tokio-postgres crate to connect to the database.

#[event(fetch)]
async fn main(_req: Request, env: Env, _ctx: Context) -> Result<Response> {
let config = tokio_postgres::config::Config::from_str(&env.secret("DATABASE_URL")?.to_string())
.map_err(|e| worker::Error::RustError(format!("Failed to parse configuration: {:?}", e)))?;

let host = match &config.get_hosts()[0] {
Host::Tcp(host) => host,
_ => {
return Err(worker::Error::RustError("Could not parse host".to_string()));
}
};
let port = config.get_ports()[0];

let socket = Socket::builder()
.secure_transport(SecureTransport::StartTls)
.connect(host, port)?;

let (client, connection) = config
.connect_raw(socket, PassthroughTls)
.await
.map_err(|e| worker::Error::RustError(format!("Failed to connect: {:?}", e)))?;

wasm_bindgen_futures::spawn_local(async move {
if let Err(error) = connection.await {
console_log!("connection error: {:?}", error);
}
});

let rows: Vec<tokio_postgres::Row> = client
.query(
"SELECT id, title, completed FROM todos WHERE completed = $1",
&[&true],
)
.await
.map_err(|e| worker::Error::RustError(format!("Failed to query: {:?}", e)))?;

Ok(Response::ok(format!("{:?}", rows))?)
}

There are several notable things (especially if you are new to WebAssembly):

  • In a non-WebAssembly platform, you would get the client and connection directly using the database URL, which opens a socket to the database. For example, you would have done something like this:

    let (client, connection) = config.connect(tls).await?;

    However, that won't work in a WebAssembly environment since there is no way to connect to a server (or, for that matter, any other resources such as filesystem). This is the core characteristic of WebAssembly: it is a sandboxed environment that cannot access resources unless explicitly provided (thought functions exported to the WebAssembly module). Therefore, we use Socket::builder().connect() to create a socket (which, in turn, uses TCP Socket API provided by Cloudflare runtime). Then, we use config.connect_raw() to lay the Postgres protocol over that socket.

  • We would have marked the main function with, for example, #[tokio::main] to bring in an async executor. However, here too, WebAssembly is different. Instead, we must rely on the host to provide the async runtime. In our case, Cloudflare worker provides a runtime (which uses JavaScript's event loop).

  • In a typical Rust program, we would have used tokio::spawn to spawn a task. However, in WebAssembly, we use wasm_bindgen_futures::spawn_local, which runs in the context of JavaScript's event loop.

We will deploy it using npx wrangler deploy. You will need to create a database and add the DATABASE_URL secret to the worker.

You can test the worker using curl:

curl https://<worker-url>

And measure the latency:

oha -c 1 -n 100 <worker-url>
...
Slowest: 0.8975 secs
Fastest: 0.2795 secs
Average: 0.3441 secs

So, our worker takes an average of 345ms to respond to a request. Depending on the use case, this can be between okay-ish and unacceptable. But why is it so slow?

We are dealing with two issues here:

  1. Establishing connection to the database: The worker creates a new connection for each request. Given that a secure connection, it takes 7+ round trips. Not surprisingly, latency is high.
  2. Executing the query: The query method in our code causes the Rust Postgres driver to make two round trips: to prepare the statement and to bind/execute the query. It also sends a one-way message to close the prepared statement.

How can we improve? We will address that in the next post by exploring connection pooling and possible changes to the driver. Stay tuned!

Share: