Waypoint

One Sunday I wanted something hard to build, so I picked a search engine.

A search engine is a few parts: an analyzer that turns text into tokens, an inverted index mapping each term to the documents that contain it, and a scorer, usually BM25, that ranks them. I started with the indexer and went down the rabbit hole: papers, source code, a lot of questions to Claude.

Every road led to Apache Lucene, the open-source index under Elasticsearch and Solr, tuned for over twenty years. Beating it outright was never on the table. The question became: is there one axis where it can be beaten?

I read Lucene’s source and ranked the options.

• Index sizeBit-packed blocks and compressed tries. Not a chance.
• Warm queriesBlock-max pruning and SIMD decoding, its most-tuned code. Unlikely.
• Build speed74% of a build is tokenising, so it’s really a tokenizer contest.
• Cold startLucene does a lot of work before its first answer. The one worth trying.

Before a single query, Lucene finds the latest commit, then for every segment validates each file’s header and checksum and builds a reader per field. That buys deletes, updates and live indexing, paid on every open whether you use them or not.

So Waypoint drops them. The index is built once and frozen into one file. Opening it is one memory map and a 128-byte header read; everything else is read lazily when a query first touches it.

• 3.3×faster to first result on a stock JVM
• 12.3×on the engine’s own work, JVM start-up subtracted
• 1,000classes loaded for one query, against 2,512 for Lucene

On a cold JVM, time is classes loaded. So the read path has no lambdas, streams, regex or pattern-matching switch: each compiles to invokedynamic and drags in the method-handle machinery on first use. A test forks a JVM, counts classes for one query, and fails the build over budget.

• 98.0%of 20,266 scores bit-identical to Lucene’s
• 2.3e-7the largest difference in the rest

Fast doesn’t count if the ranking drifts, so Lucene is the test oracle. Matching it meant copying its arithmetic, not just BM25: document lengths quantised to a byte, a 256-entry reciprocal cache, the same rewrite that keeps scores monotonic in floats.

The last 2% can’t be fixed. Float addition isn’t associative, and the two engines add a query’s clauses in a different order.

Before writing any code, I promised to withdraw the headline if Waypoint wasn’t 3× faster with both engines fully tuned.

It landed at 2×. The 3× claim is withdrawn.

Part of the reason: a bare JVM loads 440 classes and spends about 90 ms before main. Both engines pay that floor, so it drags every ratio towards 1. A 12× gap in engine work shows up as 3× in what you wait for.

• AND queriesLucene, up to 2.9×
• Common termsLucene, 2.2×
• Index sizeLucene, 1.21× smaller
• Rare termsWaypoint, 1.4×
• Reopening an indexWaypoint, 12.5×

For almost every production workload, Lucene is the right answer. Waypoint wins one shape: a small, frozen index queried from processes that start, answer once, and exit.

Fast is what you leave out.

• Flags are tradesLucene’s SIMD flag is up to 1.8× faster warm, but loads 207 more classes and is slower cold. A benchmark that doesn’t say which side it ran on is overstating something.
• Sound reasoning, wrong answerAbout 40% of Waypoint’s classes came from the foreign-memory API. Swapping it for MappedByteBuffer saved 26 classes and no time. Probe before you refactor.
• Report the floorWall clock without the JVM’s own start-up next to it credits the engine with the runtime’s cost.