Federated search with Server Sent Events
October 16, 2023 · 5 min read

This is hopefully the last time I write about service workers, but in my defense I did spend quite some time going down that rabbit hole and ended up learning a lot.

For this post to make sense to you, I'm assuming you have the following knowledge:

SSEs can be used for a variety of uses, but we are particularly interested in one. They are a fun way to keep a request open as long as it takes for you to gather all your data and stream it in small chunks. Think of it this way - if you do implement federated search - search APIs for each of your service will have different response times. Even if you make all your service calls parallelly, it will still be as slow as your slowest service.

Let me help you visualize. Assume you have 3 services - Service A, Service B, Service C. Each service has a search API at the /search endpoint which lets you pass a query and get an array as a response. The average response time for each service's search API is:

Service A - 50ms
Service B - 200ms
Service C - 100ms

Now, your federated search API calls all 3 services (parallelly), stitches their response together and responds back to the user. Assuming the time taken for the federated search API to stich them and transform the response is negligible, the average response time of it should be around ~200ms, almost same as that of the slowest service. If you're incredibly conscious about making users wait and want a fast search API, you should stream your response. This means Service A (with 50ms response) will respond first and the results will be shown to user and other services will respond back asynchronously and results for them will show up eventually.

This can be achieved with any backend service/framework/language, but I'm going to explain it with a service worker.

SWs can act as a proxy for all of your API calls, so if you want to, I don't suggest you do, but if you really really want to, you can "create" an API in your service worker without even touching your backend code.

SWs have a fetch event that will allow you to intercept all your client requests and respond back or pass-through. This is incredibly useful for caching your API requests on the browser, like using urql for caching your GraphQL queries. But in this scenario, we will intercept a fictitious "Search API" that has no backend implementation and use it as our federated search API.

self.addEventListener('fetch', fetchHandler)

const fetchHandler = (request: Request) => {
  const { pathname } = new URL(request.url)
  if (pathname === '/search') {
    // federated search handler comes in here
  }

  // All other requests will simply pass through as
  // you do not have a handler for it.
}

You have control of the /search path 🎉. We will now be setting up a streaming request.

A Server Sent Event request works pretty much like a normal fetch request with 2 major changes - You won't use fetch, but will instead use something called an EventSource to make a request on the client side. And on the server side where you would normally respond with a text or JSON, you use a ReadableStream.

To get started, you need a TransformStream that gives you two things - a writeable to write your data to and a readable to respond to the client with.

// A TextEncoder converts the data into string format.
// We need the data to be a string like `data: {actual data}`
const encoder = new TextEncoder()
const { readable, writable } = new TransformStream({
  transform: (chunk, controller) => {
    controller.enqueue(encoder.encode(`data: ${JSON.stringify(chunk)}\r\n\r\n`))
  }
})

Now, we can call our individual search APIs and keep pushing it using the writable.

Promise.all([
  () => {
    // call service 1
    writer.write(data)
  },
  () => {
    // call service 2
    writer.write(data)
  },
  ...
])
// 👆🏻 this particular section calls the services paralelly and writes to the steam.
// They are completely async and there's no guarantee which Promise will resolve first.

All that's left to do is respond with the readable to be used by the client. Two headers are important to let the browser know that the response is a stream and the connection needs to be kept open until done. This is achieved using Content-Type: text/event-stream and Connection: keep-alive HTTP headers. A valid response would look like this:

return new Response(readable, {
  headers: {
    'Content-Type': 'text/event-stream',
    'Cache-Control': 'no-cache', // let the browser know that the response cannot be cached
    'Connection': 'keep-alive'
  }
})

Are we done? Not yet. You can't simply make a fetch call and expect this to work, instead we use EventSource.

const source = new EventSource('/search?query=')
source.onmessage = ({ data }) => {
  console.log(JSON.parse(data)) // your data
}

The onmessage callback will be fired whenever the server pushes data and this can be appended to a global state variable that will cause the UI to re-render and show the results.