I think those fee ranges are actually accurate, and the problem is that it's not always possible to know which block a transaction belongs to from its fee rate alone, because transaction selection also depends on size (and to some extent randomness).

Consider an extreme example:

Imagine the mempool consists only of many thousands of huge 80,000vb transactions paying exactly 50 sats/vb, plus thousands of small 100vb transactions each paying 1 sat/vb.

Each projected block can fit 12 of the large, high-rate transactions, leaving 1,000,000 - (12 x 80,000) = 40,000 vbytes free.

That free space can only be filled by 400 of the small 100vb, 1 s/vb transactions.

So the first projected block's feeRange looks like [1, 1, 1, 50]. But so does the second, third, fourth block etc, and so their feeRanges all overlap.

That means if we're given only the fee rate of a random transaction, there's no way to tell which block it "belongs" to.

I'm not sure what the best solution is.

We do have the actual positions of transactions in each projected block available in the backend, so perhaps we should just send that info over the websocket when you subscribe to an unconfirmed transaction?

Ok. So that is also the cause of this thing when the fee range doesn't look sorted by fee and you get a sudden high fee data point like here?

I just don't seem to understand why it's not sorted by sats/vB / effective fee and we see 16 - 17 and 16- 17 in 2 blocks instead of 17-17 and 16-16. I guess a lower fee rate transaction that is bigger, can be more profitable than a few smaller high fee txs?

We do have the actual positions of transactions in each projected block available in the backend, so perhaps we should just send that info over the websocket when you subscribe to an unconfirmed transaction?

Never really worked in that code but I would have thought it was already the case. Are you saying the backend and the frontend have different code to compute the position in the mempool?

Ok. So that is also the cause of this thing when the fee range doesn't look sorted by fee and you get a sudden high fee data point like here?

I think the weird fee gradients are caused by something more complicated, but yes this probably contributes to it.

I just don't seem to understand why it's not sorted by sats/vB / effective fee and we see 16 - 17 and 16- 17 in 2 blocks instead of 17-17 and 16-16. I guess a lower fee rate transaction that is bigger, can be more profitable than a few smaller high fee txs?

More that a few smaller low fee transactions are more profitable than nothing.

Like in my example earlier, miners will try to fit as many large, high fee rate transactions as possible. But when they run out of space in a block for those large transactions, they'll fill up the rest with smaller lower rate transactions rather than leave any empty space.

When that happens, the lowest fee rate in the first projected block can be less than the highest fee rate in the second projected block etc.

Never really worked in that code but I would have thought it was already the case. Are you saying the backend and the frontend have different code to compute the position in the mempool?

Well, the backend doesn't explicitly compute mempool positions, but the result of the advanced gbt algorithm is an ordered list of transactions for each of the first 8 mempool blocks.

We don't send those full lists of transactions to the frontend (it would be a lot of data), so instead we estimate mempool positions there by searching for the first mempool block feeRange interval that spans the transaction's effective fee rate:

When we used the old getSimonTemplate algorithm the two methods were ~equivalent because transactions in projected blocks were always strictly sorted by effective fee rate, but that's no longer true with advanced GBT.