Console

·

2 min read

Collect Cloudflare Workers Logs to Serverless Kafka

Noah Fischer

DevRel @Upstash

Kafka Setup
Project Setup
The Code
Test and Deploy

In this tutorial, we will show how to collect logs from Cloudflare Workers to Serverless Kafka.

Kafka Setup

Create a Kafka cluster and topic using Upstash Console or Upstash CLI. Copy the Webhook URL to be used in the next steps.

Project Setup

We will use Wrangler 2 for deployment, so please install (or upgrade) Wrangler 2.

Create a folder for your project and run wrangler init. Select js:

➜  quickstart-cloudflare-workers wrangler init
 ⛅️ wrangler 2.0.7
-------------------
Using npm as package manager.
✨ Created wrangler.toml
Would you like to use git to manage this Worker? (y/n)
✨ Initialized git repository
No package.json found. Would you like to create one? (y/n)
✨ Created package.json
Would you like to use TypeScript? (y/n)
Would you like to create a Worker at src/index.js? (y/n)
✨ Created src/index.js

The Code

Update src/index.js as below:

export default {
  async fetch(request, env, context) {
    context.waitUntil(postLog("some log from cf workers!"))
    return new Response("Hello World!");
  },
};

function postLog(data) {
  return fetch("REPLACE_UPSTASH_WEBHOOK_URL", {
    method: "POST",
    body: data,
  });
}

Copy/paste the webhook URL from Upstash Console. Webhook URL simply pushes everything posted to the Kafka. You can also use Upstash Kafka SDK or directly the REST API.

Test and Deploy

You can test the function locally with wrangler dev.

Deploy your function to Cloudflare with wrangler publish

The endpoint of the function will be printed. You can check if logs are collected in Kafka by copying the curl expression from the console:

curl https://definite-goldfish-14184-us1-rest-kafka.upstash.io/consume/GROUP_NAME/GROUP_INSTANCE_NAME/mytopic -H "Kafka-Auto-Offset-Reset: earliest" -u \
  REPLACE_HERE


© 2023 Upstash, Inc. Based in California.

* Redis is a trademark of Redis Ltd. Any rights therein are reserved to Redis Ltd. Any use by Upstash is for referential purposes only and does not indicate any sponsorship, endorsement or affiliation between Redis and Upstash.

** Cloudflare, the Cloudflare logo, and Cloudflare Workers are trademarks and/or registered trademarks of Cloudflare, Inc. in the United States and other jurisdictions.