feat: http caching

[flakey5]

Implements bare-bones opt-in http caching as per rfc9111. Bare-bones in this case means what's required by the spec and a few extra bits, with more coming in future prs.

Opening as a draft since there's still some more work to be done (mostly tests, but a bit more functionality-wise)

No request cache directives are supported at this time, this will come later.

Response caching directives supported:

• public
• private
• s-maxage
• max-age
• Expires header
• no-cache
• no-store
• stale-while-revalidate

This relates to...

Closes #3231
Closes #2760
Closes #2256
Closes #1146

Changes

Features

• Opt-in http caching
• In-memory default cache store

Bug Fixes

n/a

Breaking Changes and Deprecations

n/a

Status

• I have read and agreed to the Developer's Certificate of Origin
• Tested
• Benchmarked (optional)
• Documented
• Review ready
• In review
• Merge ready

[ronag]

Would it maybe be better to use jsdoc types rather than separate type definition files? @mcollina

[metcoder95]

I wouldn't add support just yet for Promises, maybe keep it sync for now and see if needed later

[flakey5]

cc @mcollina wdyt?

[mcollina]

We really should support promises from the get-go, we should consider adding a fs based one using promises too.

[metcoder95]

Was having double-thoughts on this. Most of the APIs are callback based, shall we do something similar?

e.g. retry allows async ops by passing a callback that will trigger (or cancel) the next retry

[mcollina]

Would it maybe be better to use jsdoc types rather than separate type definition files? @mcollina

I think using type definitions are in line with the rest of the codebase, and therefore the correct implementation.

If we want to migrate to a jsdoc-generated world, let's discuss in another issue!

[metcoder95]

We should also invalidate cache upon successful status code for non-safe methods (e.g. PUT, POST, PATCH)

https://www.rfc-editor.org/rfc/rfc9111.html#name-invalidating-stored-respons

[flakey5]

Trying to think on a good way to be able to tell the cache store to purge all of the values for a certain host, wdyt of just something like store.deleteAllFromHost('localhost')?

[metcoder95]

Yeah, that could work; I imagine will require to change the way we store the data?

[flakey5]

Hmm I think this also means we need to always run this interceptor for unsafe methods, and the methods property in the options can just detail what safe methods the user wants to cache. Wdyt?

[ronag]

@flakey5 please make sure to add me and @IsakT as co-authors (assuming you took some of our code in nxt-undici).

[Uzlopak]

headers could be null or undefined

[Uzlopak]

@ronag
Should we use fastTimers?

[Uzlopak]

horrible! why??

[Uzlopak]

I cheated by using vscode to make the review of the files in lib by adding // @ts-code at the top of the js file.

[Uzlopak]

@flakey5

Please revert the fastTimer changes for now.

[Uzlopak]

AHA! In case of the interceptor, you could instead of calling this.#handler.onError you could call super.onError ?!

[metcoder95]

this would always be set

[flakey5]

The types allow it to be undefined however?

undici/types/dispatcher.d.ts

Line 220 in 0ad6007

onError?(err: Error): void;

[metcoder95]

Yeah, not sure why's that but I haven't find that scenario at all

[metcoder95]

the function would always be there as marks the kick-off for processing the response; shouldn't be possible to find a non onHeaders (the whole process will be wrong if not provided). Same to handler.onError

[ronag]

I think we should change the cache store interface to:

interace CacheStore {
   createWriteStream(key, props)
   createReadStrream(key)
}

and move the max entry size into the cache store. That way we can store any size of request without necessarily buffering through memory.

@mcollina wdyt?

Something like:

class CacheHandler extends DecoratorHandler {
  onHeaders(statusCode, rawHeaders, resume, statusMessage, headers) {
    this.#value = this.#store
      .createWriteStream({ statusCode, rawHeaders, statusMessage, key: this.#key, headers })
      .on('drain', resume)
      .on('error', () => {
        // TODO
      })
    return this.#handler.onHeaders(statusCode, rawHeaders, resume, statusMessage, headers)
  }

  onData(chunk) {
    return this.#handler.onData(chunk) && this.#value.write(chunk)
  }

  onComplete(rawTrailers) {
    this.#value.rawTrailers = rawTrailers
    this.#value.end()
    return this.#handler.onComplete(rawTrailers)
  }

  onError(err) {
    this.#value.destroy(err)
    return this.#handler.onError(err
  }
}

[flakey5]

I think we should change the cache store interface to:

I'd be in favor of this - I think this would also work a lot better than what I was thinking for reusing responses too (https://www.rfc-editor.org/rfc/rfc9111.html#section-4-7).

If a cache store was writing to some async non-memory store this should also work great still

[flakey5]

and move the max entry size into the cache store

The max entry size is already in the cache store though? Or unless you were talking about where we check the response size in the CacheHandler

[flakey5]

this is kinda wrong will fix