Search Speed Slashed: How We Cut Response Times by Nearly a Second

One of the most consistent pieces of feedback we’ve received during our open beta is that search needs to be faster. We heard you, and we’ve been hard at work on a major overhaul to address it.

I’m thrilled to announce that we’ve successfully rolled out a significant architectural change that has reduced the average UX search response time for first results from 2.5 seconds to just 1.5 seconds—a nearly 40% improvement.

This was one of the most challenging post-launch refactors to date, but the results have made it well worth the effort.

From Post/Poll to Real-Time: The Switch to SSE

The core of this improvement comes from switching our data delivery pattern from a traditional POST/POLL system to Server-Sent Events (SSE). [18, 19] Here’s why that made such a huge difference:

1. Just-in-Time Delivery: Our old polling system wasn’t always timed perfectly. Many polls would execute later than necessary, leaving the user waiting. With SSE, the server pushes data the instant it’s ready, eliminating that lag. [19]
2. Irrelevant Request Latency: Each poll request had its own latency. By holding a single connection open, SSE makes that repeated request/response overhead disappear. [18]
3. Reduced Client Strain: The constant polling was jamming up the UI client with a queue of requests. SSE simplifies this to a single, persistent connection, making the front-end more responsive.

As a side benefit, this new architecture has also allowed us to break up our data responses into more, smaller components. This means different parts of the search results page can appear even earlier, as soon as they are processed, further enhancing the perception of speed.

It’s Not Just SSE

I should mention that we also did some data centralization work during this refactor. While the move to SSE was the primary driver, these centralization efforts also contributed to the overall performance gains.

The Journey to a Faster Search Isn’t Over

This is a major milestone, and I’m incredibly proud of the result. For our beta testers, speed has been a significant impediment to enjoying the product, and my hope is that this update makes a tangible difference in your day-to-day use.

However, the work is far from done. While the delivery pipeline is now significantly faster, the next major hurdle is to improve the speed of the central search pipeline itself. That will be the biggest win yet, and it’s our next major focus.

Thank you to everyone who has provided feedback. Your insights are what guide these improvements. The product remains in beta as we continue to address your valuable suggestions.

Please, keep the feedback coming!

— Ryan