Primeiro commit do projeto Angular
This commit is contained in:
+16
@@ -0,0 +1,16 @@
|
||||
ISC License
|
||||
|
||||
Copyright (c) npm, Inc.
|
||||
|
||||
Permission to use, copy, modify, and/or distribute this software for
|
||||
any purpose with or without fee is hereby granted, provided that the
|
||||
above copyright notice and this permission notice appear in all copies.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS" AND THE COPYRIGHT HOLDER DISCLAIMS
|
||||
ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED
|
||||
WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE
|
||||
COPYRIGHT HOLDER BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR
|
||||
CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS
|
||||
OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
|
||||
OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE
|
||||
USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||
+715
@@ -0,0 +1,715 @@
|
||||
# cacache [](https://npm.im/cacache) [](https://npm.im/cacache) [](https://travis-ci.org/npm/cacache) [](https://ci.appveyor.com/project/npm/cacache) [](https://coveralls.io/github/npm/cacache?branch=latest)
|
||||
|
||||
[`cacache`](https://github.com/npm/cacache) is a Node.js library for managing
|
||||
local key and content address caches. It's really fast, really good at
|
||||
concurrency, and it will never give you corrupted data, even if cache files
|
||||
get corrupted or manipulated.
|
||||
|
||||
On systems that support user and group settings on files, cacache will
|
||||
match the `uid` and `gid` values to the folder where the cache lives, even
|
||||
when running as `root`.
|
||||
|
||||
It was written to be used as [npm](https://npm.im)'s local cache, but can
|
||||
just as easily be used on its own.
|
||||
|
||||
## Install
|
||||
|
||||
`$ npm install --save cacache`
|
||||
|
||||
## Table of Contents
|
||||
|
||||
* [Example](#example)
|
||||
* [Features](#features)
|
||||
* [Contributing](#contributing)
|
||||
* [API](#api)
|
||||
* [Using localized APIs](#localized-api)
|
||||
* Reading
|
||||
* [`ls`](#ls)
|
||||
* [`ls.stream`](#ls-stream)
|
||||
* [`get`](#get-data)
|
||||
* [`get.stream`](#get-stream)
|
||||
* [`get.info`](#get-info)
|
||||
* [`get.hasContent`](#get-hasContent)
|
||||
* Writing
|
||||
* [`put`](#put-data)
|
||||
* [`put.stream`](#put-stream)
|
||||
* [`rm.all`](#rm-all)
|
||||
* [`rm.entry`](#rm-entry)
|
||||
* [`rm.content`](#rm-content)
|
||||
* [`index.compact`](#index-compact)
|
||||
* [`index.insert`](#index-insert)
|
||||
* Utilities
|
||||
* [`clearMemoized`](#clear-memoized)
|
||||
* [`tmp.mkdir`](#tmp-mkdir)
|
||||
* [`tmp.withTmp`](#with-tmp)
|
||||
* Integrity
|
||||
* [Subresource Integrity](#integrity)
|
||||
* [`verify`](#verify)
|
||||
* [`verify.lastRun`](#verify-last-run)
|
||||
|
||||
### Example
|
||||
|
||||
```javascript
|
||||
const cacache = require('cacache')
|
||||
const fs = require('fs')
|
||||
|
||||
const cachePath = '/tmp/my-toy-cache'
|
||||
const key = 'my-unique-key-1234'
|
||||
|
||||
// Cache it! Use `cachePath` as the root of the content cache
|
||||
cacache.put(cachePath, key, '10293801983029384').then(integrity => {
|
||||
console.log(`Saved content to ${cachePath}.`)
|
||||
})
|
||||
|
||||
const destination = '/tmp/mytar.tgz'
|
||||
|
||||
// Copy the contents out of the cache and into their destination!
|
||||
// But this time, use stream instead!
|
||||
cacache.get.stream(
|
||||
cachePath, key
|
||||
).pipe(
|
||||
fs.createWriteStream(destination)
|
||||
).on('finish', () => {
|
||||
console.log('done extracting!')
|
||||
})
|
||||
|
||||
// The same thing, but skip the key index.
|
||||
cacache.get.byDigest(cachePath, integrityHash).then(data => {
|
||||
fs.writeFile(destination, data, err => {
|
||||
console.log('tarball data fetched based on its sha512sum and written out!')
|
||||
})
|
||||
})
|
||||
```
|
||||
|
||||
### Features
|
||||
|
||||
* Extraction by key or by content address (shasum, etc)
|
||||
* [Subresource Integrity](#integrity) web standard support
|
||||
* Multi-hash support - safely host sha1, sha512, etc, in a single cache
|
||||
* Automatic content deduplication
|
||||
* Fault tolerance (immune to corruption, partial writes, process races, etc)
|
||||
* Consistency guarantees on read and write (full data verification)
|
||||
* Lockless, high-concurrency cache access
|
||||
* Streaming support
|
||||
* Promise support
|
||||
* Fast -- sub-millisecond reads and writes including verification
|
||||
* Arbitrary metadata storage
|
||||
* Garbage collection and additional offline verification
|
||||
* Thorough test coverage
|
||||
* There's probably a bloom filter in there somewhere. Those are cool, right? 🤔
|
||||
|
||||
### Contributing
|
||||
|
||||
The cacache team enthusiastically welcomes contributions and project participation! There's a bunch of things you can do if you want to contribute! Please don't hesitate to jump in if you'd like to, or even ask us questions if something isn't clear.
|
||||
|
||||
All participants and maintainers in this project are expected to follow [Code of Conduct](CODE_OF_CONDUCT.md), and just generally be excellent to each other.
|
||||
|
||||
Please refer to the [Changelog](CHANGELOG.md) for project history details, too.
|
||||
|
||||
Happy hacking!
|
||||
|
||||
### API
|
||||
|
||||
#### <a name="ls"></a> `> cacache.ls(cache) -> Promise<Object>`
|
||||
|
||||
Lists info for all entries currently in the cache as a single large object. Each
|
||||
entry in the object will be keyed by the unique index key, with corresponding
|
||||
[`get.info`](#get-info) objects as the values.
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
cacache.ls(cachePath).then(console.log)
|
||||
// Output
|
||||
{
|
||||
'my-thing': {
|
||||
key: 'my-thing',
|
||||
integrity: 'sha512-BaSe64/EnCoDED+HAsh=='
|
||||
path: '.testcache/content/deadbeef', // joined with `cachePath`
|
||||
time: 12345698490,
|
||||
size: 4023948,
|
||||
metadata: {
|
||||
name: 'blah',
|
||||
version: '1.2.3',
|
||||
description: 'this was once a package but now it is my-thing'
|
||||
}
|
||||
},
|
||||
'other-thing': {
|
||||
key: 'other-thing',
|
||||
integrity: 'sha1-ANothER+hasH=',
|
||||
path: '.testcache/content/bada55',
|
||||
time: 11992309289,
|
||||
size: 111112
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### <a name="ls-stream"></a> `> cacache.ls.stream(cache) -> Readable`
|
||||
|
||||
Lists info for all entries currently in the cache as a single large object.
|
||||
|
||||
This works just like [`ls`](#ls), except [`get.info`](#get-info) entries are
|
||||
returned as `'data'` events on the returned stream.
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
cacache.ls.stream(cachePath).on('data', console.log)
|
||||
// Output
|
||||
{
|
||||
key: 'my-thing',
|
||||
integrity: 'sha512-BaSe64HaSh',
|
||||
path: '.testcache/content/deadbeef', // joined with `cachePath`
|
||||
time: 12345698490,
|
||||
size: 13423,
|
||||
metadata: {
|
||||
name: 'blah',
|
||||
version: '1.2.3',
|
||||
description: 'this was once a package but now it is my-thing'
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
key: 'other-thing',
|
||||
integrity: 'whirlpool-WoWSoMuchSupport',
|
||||
path: '.testcache/content/bada55',
|
||||
time: 11992309289,
|
||||
size: 498023984029
|
||||
}
|
||||
|
||||
{
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
#### <a name="get-data"></a> `> cacache.get(cache, key, [opts]) -> Promise({data, metadata, integrity})`
|
||||
|
||||
Returns an object with the cached data, digest, and metadata identified by
|
||||
`key`. The `data` property of this object will be a `Buffer` instance that
|
||||
presumably holds some data that means something to you. I'm sure you know what
|
||||
to do with it! cacache just won't care.
|
||||
|
||||
`integrity` is a [Subresource
|
||||
Integrity](#integrity)
|
||||
string. That is, a string that can be used to verify `data`, which looks like
|
||||
`<hash-algorithm>-<base64-integrity-hash>`.
|
||||
|
||||
If there is no content identified by `key`, or if the locally-stored data does
|
||||
not pass the validity checksum, the promise will be rejected.
|
||||
|
||||
A sub-function, `get.byDigest` may be used for identical behavior, except lookup
|
||||
will happen by integrity hash, bypassing the index entirely. This version of the
|
||||
function *only* returns `data` itself, without any wrapper.
|
||||
|
||||
See: [options](#get-options)
|
||||
|
||||
##### Note
|
||||
|
||||
This function loads the entire cache entry into memory before returning it. If
|
||||
you're dealing with Very Large data, consider using [`get.stream`](#get-stream)
|
||||
instead.
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
// Look up by key
|
||||
cache.get(cachePath, 'my-thing').then(console.log)
|
||||
// Output:
|
||||
{
|
||||
metadata: {
|
||||
thingName: 'my'
|
||||
},
|
||||
integrity: 'sha512-BaSe64HaSh',
|
||||
data: Buffer#<deadbeef>,
|
||||
size: 9320
|
||||
}
|
||||
|
||||
// Look up by digest
|
||||
cache.get.byDigest(cachePath, 'sha512-BaSe64HaSh').then(console.log)
|
||||
// Output:
|
||||
Buffer#<deadbeef>
|
||||
```
|
||||
|
||||
#### <a name="get-stream"></a> `> cacache.get.stream(cache, key, [opts]) -> Readable`
|
||||
|
||||
Returns a [Readable Stream](https://nodejs.org/api/stream.html#stream_readable_streams) of the cached data identified by `key`.
|
||||
|
||||
If there is no content identified by `key`, or if the locally-stored data does
|
||||
not pass the validity checksum, an error will be emitted.
|
||||
|
||||
`metadata` and `integrity` events will be emitted before the stream closes, if
|
||||
you need to collect that extra data about the cached entry.
|
||||
|
||||
A sub-function, `get.stream.byDigest` may be used for identical behavior,
|
||||
except lookup will happen by integrity hash, bypassing the index entirely. This
|
||||
version does not emit the `metadata` and `integrity` events at all.
|
||||
|
||||
See: [options](#get-options)
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
// Look up by key
|
||||
cache.get.stream(
|
||||
cachePath, 'my-thing'
|
||||
).on('metadata', metadata => {
|
||||
console.log('metadata:', metadata)
|
||||
}).on('integrity', integrity => {
|
||||
console.log('integrity:', integrity)
|
||||
}).pipe(
|
||||
fs.createWriteStream('./x.tgz')
|
||||
)
|
||||
// Outputs:
|
||||
metadata: { ... }
|
||||
integrity: 'sha512-SoMeDIGest+64=='
|
||||
|
||||
// Look up by digest
|
||||
cache.get.stream.byDigest(
|
||||
cachePath, 'sha512-SoMeDIGest+64=='
|
||||
).pipe(
|
||||
fs.createWriteStream('./x.tgz')
|
||||
)
|
||||
```
|
||||
|
||||
#### <a name="get-info"></a> `> cacache.get.info(cache, key) -> Promise`
|
||||
|
||||
Looks up `key` in the cache index, returning information about the entry if
|
||||
one exists.
|
||||
|
||||
##### Fields
|
||||
|
||||
* `key` - Key the entry was looked up under. Matches the `key` argument.
|
||||
* `integrity` - [Subresource Integrity hash](#integrity) for the content this entry refers to.
|
||||
* `path` - Filesystem path where content is stored, joined with `cache` argument.
|
||||
* `time` - Timestamp the entry was first added on.
|
||||
* `metadata` - User-assigned metadata associated with the entry/content.
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
cacache.get.info(cachePath, 'my-thing').then(console.log)
|
||||
|
||||
// Output
|
||||
{
|
||||
key: 'my-thing',
|
||||
integrity: 'sha256-MUSTVERIFY+ALL/THINGS=='
|
||||
path: '.testcache/content/deadbeef',
|
||||
time: 12345698490,
|
||||
size: 849234,
|
||||
metadata: {
|
||||
name: 'blah',
|
||||
version: '1.2.3',
|
||||
description: 'this was once a package but now it is my-thing'
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### <a name="get-hasContent"></a> `> cacache.get.hasContent(cache, integrity) -> Promise`
|
||||
|
||||
Looks up a [Subresource Integrity hash](#integrity) in the cache. If content
|
||||
exists for this `integrity`, it will return an object, with the specific single integrity hash
|
||||
that was found in `sri` key, and the size of the found content as `size`. If no content exists for this integrity, it will return `false`.
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
cacache.get.hasContent(cachePath, 'sha256-MUSTVERIFY+ALL/THINGS==').then(console.log)
|
||||
|
||||
// Output
|
||||
{
|
||||
sri: {
|
||||
source: 'sha256-MUSTVERIFY+ALL/THINGS==',
|
||||
algorithm: 'sha256',
|
||||
digest: 'MUSTVERIFY+ALL/THINGS==',
|
||||
options: []
|
||||
},
|
||||
size: 9001
|
||||
}
|
||||
|
||||
cacache.get.hasContent(cachePath, 'sha521-NOT+IN/CACHE==').then(console.log)
|
||||
|
||||
// Output
|
||||
false
|
||||
```
|
||||
|
||||
##### <a name="get-options"></a> Options
|
||||
|
||||
##### `opts.integrity`
|
||||
If present, the pre-calculated digest for the inserted content. If this option
|
||||
is provided and does not match the post-insertion digest, insertion will fail
|
||||
with an `EINTEGRITY` error.
|
||||
|
||||
##### `opts.memoize`
|
||||
|
||||
Default: null
|
||||
|
||||
If explicitly truthy, cacache will read from memory and memoize data on bulk read. If `false`, cacache will read from disk data. Reader functions by default read from in-memory cache.
|
||||
|
||||
##### `opts.size`
|
||||
If provided, the data stream will be verified to check that enough data was
|
||||
passed through. If there's more or less data than expected, insertion will fail
|
||||
with an `EBADSIZE` error.
|
||||
|
||||
|
||||
#### <a name="put-data"></a> `> cacache.put(cache, key, data, [opts]) -> Promise`
|
||||
|
||||
Inserts data passed to it into the cache. The returned Promise resolves with a
|
||||
digest (generated according to [`opts.algorithms`](#optsalgorithms)) after the
|
||||
cache entry has been successfully written.
|
||||
|
||||
See: [options](#put-options)
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
fetch(
|
||||
'https://registry.npmjs.org/cacache/-/cacache-1.0.0.tgz'
|
||||
).then(data => {
|
||||
return cacache.put(cachePath, 'registry.npmjs.org|cacache@1.0.0', data)
|
||||
}).then(integrity => {
|
||||
console.log('integrity hash is', integrity)
|
||||
})
|
||||
```
|
||||
|
||||
#### <a name="put-stream"></a> `> cacache.put.stream(cache, key, [opts]) -> Writable`
|
||||
|
||||
Returns a [Writable
|
||||
Stream](https://nodejs.org/api/stream.html#stream_writable_streams) that inserts
|
||||
data written to it into the cache. Emits an `integrity` event with the digest of
|
||||
written contents when it succeeds.
|
||||
|
||||
See: [options](#put-options)
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
request.get(
|
||||
'https://registry.npmjs.org/cacache/-/cacache-1.0.0.tgz'
|
||||
).pipe(
|
||||
cacache.put.stream(
|
||||
cachePath, 'registry.npmjs.org|cacache@1.0.0'
|
||||
).on('integrity', d => console.log(`integrity digest is ${d}`))
|
||||
)
|
||||
```
|
||||
|
||||
##### <a name="put-options"></a> Options
|
||||
|
||||
##### `opts.metadata`
|
||||
|
||||
Arbitrary metadata to be attached to the inserted key.
|
||||
|
||||
##### `opts.size`
|
||||
|
||||
If provided, the data stream will be verified to check that enough data was
|
||||
passed through. If there's more or less data than expected, insertion will fail
|
||||
with an `EBADSIZE` error.
|
||||
|
||||
##### `opts.integrity`
|
||||
|
||||
If present, the pre-calculated digest for the inserted content. If this option
|
||||
is provided and does not match the post-insertion digest, insertion will fail
|
||||
with an `EINTEGRITY` error.
|
||||
|
||||
`algorithms` has no effect if this option is present.
|
||||
|
||||
##### `opts.integrityEmitter`
|
||||
|
||||
*Streaming only* If present, uses the provided event emitter as a source of
|
||||
truth for both integrity and size. This allows use cases where integrity is
|
||||
already being calculated outside of cacache to reuse that data instead of
|
||||
calculating it a second time.
|
||||
|
||||
The emitter must emit both the `'integrity'` and `'size'` events.
|
||||
|
||||
NOTE: If this option is provided, you must verify that you receive the correct
|
||||
integrity value yourself and emit an `'error'` event if there is a mismatch.
|
||||
[ssri Integrity Streams](https://github.com/npm/ssri#integrity-stream) do this for you when given an expected integrity.
|
||||
|
||||
##### `opts.algorithms`
|
||||
|
||||
Default: ['sha512']
|
||||
|
||||
Hashing algorithms to use when calculating the [subresource integrity
|
||||
digest](#integrity)
|
||||
for inserted data. Can use any algorithm listed in `crypto.getHashes()` or
|
||||
`'omakase'`/`'お任せします'` to pick a random hash algorithm on each insertion. You
|
||||
may also use any anagram of `'modnar'` to use this feature.
|
||||
|
||||
Currently only supports one algorithm at a time (i.e., an array length of
|
||||
exactly `1`). Has no effect if `opts.integrity` is present.
|
||||
|
||||
##### `opts.memoize`
|
||||
|
||||
Default: null
|
||||
|
||||
If provided, cacache will memoize the given cache insertion in memory, bypassing
|
||||
any filesystem checks for that key or digest in future cache fetches. Nothing
|
||||
will be written to the in-memory cache unless this option is explicitly truthy.
|
||||
|
||||
If `opts.memoize` is an object or a `Map`-like (that is, an object with `get`
|
||||
and `set` methods), it will be written to instead of the global memoization
|
||||
cache.
|
||||
|
||||
Reading from disk data can be forced by explicitly passing `memoize: false` to
|
||||
the reader functions, but their default will be to read from memory.
|
||||
|
||||
##### `opts.tmpPrefix`
|
||||
Default: null
|
||||
|
||||
Prefix to append on the temporary directory name inside the cache's tmp dir.
|
||||
|
||||
#### <a name="rm-all"></a> `> cacache.rm.all(cache) -> Promise`
|
||||
|
||||
Clears the entire cache. Mainly by blowing away the cache directory itself.
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
cacache.rm.all(cachePath).then(() => {
|
||||
console.log('THE APOCALYPSE IS UPON US 😱')
|
||||
})
|
||||
```
|
||||
|
||||
#### <a name="rm-entry"></a> `> cacache.rm.entry(cache, key, [opts]) -> Promise`
|
||||
|
||||
Alias: `cacache.rm`
|
||||
|
||||
Removes the index entry for `key`. Content will still be accessible if
|
||||
requested directly by content address ([`get.stream.byDigest`](#get-stream)).
|
||||
|
||||
By default, this appends a new entry to the index with an integrity of `null`.
|
||||
If `opts.removeFully` is set to `true` then the index file itself will be
|
||||
physically deleted rather than appending a `null`.
|
||||
|
||||
To remove the content itself (which might still be used by other entries), use
|
||||
[`rm.content`](#rm-content). Or, to safely vacuum any unused content, use
|
||||
[`verify`](#verify).
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
cacache.rm.entry(cachePath, 'my-thing').then(() => {
|
||||
console.log('I did not like it anyway')
|
||||
})
|
||||
```
|
||||
|
||||
#### <a name="rm-content"></a> `> cacache.rm.content(cache, integrity) -> Promise`
|
||||
|
||||
Removes the content identified by `integrity`. Any index entries referring to it
|
||||
will not be usable again until the content is re-added to the cache with an
|
||||
identical digest.
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
cacache.rm.content(cachePath, 'sha512-SoMeDIGest/IN+BaSE64==').then(() => {
|
||||
console.log('data for my-thing is gone!')
|
||||
})
|
||||
```
|
||||
|
||||
#### <a name="index-compact"></a> `> cacache.index.compact(cache, key, matchFn, [opts]) -> Promise`
|
||||
|
||||
Uses `matchFn`, which must be a synchronous function that accepts two entries
|
||||
and returns a boolean indicating whether or not the two entries match, to
|
||||
deduplicate all entries in the cache for the given `key`.
|
||||
|
||||
If `opts.validateEntry` is provided, it will be called as a function with the
|
||||
only parameter being a single index entry. The function must return a Boolean,
|
||||
if it returns `true` the entry is considered valid and will be kept in the index,
|
||||
if it returns `false` the entry will be removed from the index.
|
||||
|
||||
If `opts.validateEntry` is not provided, however, every entry in the index will
|
||||
be deduplicated and kept until the first `null` integrity is reached, removing
|
||||
all entries that were written before the `null`.
|
||||
|
||||
The deduplicated list of entries is both written to the index, replacing the
|
||||
existing content, and returned in the Promise.
|
||||
|
||||
#### <a name="index-insert"></a> `> cacache.index.insert(cache, key, integrity, opts) -> Promise`
|
||||
|
||||
Writes an index entry to the cache for the given `key` without writing content.
|
||||
|
||||
It is assumed if you are using this method, you have already stored the content
|
||||
some other way and you only wish to add a new index to that content. The `metadata`
|
||||
and `size` properties are read from `opts` and used as part of the index entry.
|
||||
|
||||
Returns a Promise resolving to the newly added entry.
|
||||
|
||||
#### <a name="clear-memoized"></a> `> cacache.clearMemoized()`
|
||||
|
||||
Completely resets the in-memory entry cache.
|
||||
|
||||
#### <a name="tmp-mkdir"></a> `> tmp.mkdir(cache, opts) -> Promise<Path>`
|
||||
|
||||
Returns a unique temporary directory inside the cache's `tmp` dir. This
|
||||
directory will use the same safe user assignment that all the other stuff use.
|
||||
|
||||
Once the directory is made, it's the user's responsibility that all files
|
||||
within are given the appropriate `gid`/`uid` ownership settings to match
|
||||
the rest of the cache. If not, you can ask cacache to do it for you by
|
||||
calling [`tmp.fix()`](#tmp-fix), which will fix all tmp directory
|
||||
permissions.
|
||||
|
||||
If you want automatic cleanup of this directory, use
|
||||
[`tmp.withTmp()`](#with-tpm)
|
||||
|
||||
See: [options](#tmp-options)
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
cacache.tmp.mkdir(cache).then(dir => {
|
||||
fs.writeFile(path.join(dir, 'blablabla'), Buffer#<1234>, ...)
|
||||
})
|
||||
```
|
||||
|
||||
#### <a name="tmp-fix"></a> `> tmp.fix(cache) -> Promise`
|
||||
|
||||
Sets the `uid` and `gid` properties on all files and folders within the tmp
|
||||
folder to match the rest of the cache.
|
||||
|
||||
Use this after manually writing files into [`tmp.mkdir`](#tmp-mkdir) or
|
||||
[`tmp.withTmp`](#with-tmp).
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
cacache.tmp.mkdir(cache).then(dir => {
|
||||
writeFile(path.join(dir, 'file'), someData).then(() => {
|
||||
// make sure we didn't just put a root-owned file in the cache
|
||||
cacache.tmp.fix().then(() => {
|
||||
// all uids and gids match now
|
||||
})
|
||||
})
|
||||
})
|
||||
```
|
||||
|
||||
#### <a name="with-tmp"></a> `> tmp.withTmp(cache, opts, cb) -> Promise`
|
||||
|
||||
Creates a temporary directory with [`tmp.mkdir()`](#tmp-mkdir) and calls `cb`
|
||||
with it. The created temporary directory will be removed when the return value
|
||||
of `cb()` resolves, the tmp directory will be automatically deleted once that
|
||||
promise completes.
|
||||
|
||||
The same caveats apply when it comes to managing permissions for the tmp dir's
|
||||
contents.
|
||||
|
||||
See: [options](#tmp-options)
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
cacache.tmp.withTmp(cache, dir => {
|
||||
return fs.writeFile(path.join(dir, 'blablabla'), 'blabla contents', { encoding: 'utf8' })
|
||||
}).then(() => {
|
||||
// `dir` no longer exists
|
||||
})
|
||||
```
|
||||
|
||||
##### <a name="tmp-options"></a> Options
|
||||
|
||||
##### `opts.tmpPrefix`
|
||||
Default: null
|
||||
|
||||
Prefix to append on the temporary directory name inside the cache's tmp dir.
|
||||
|
||||
#### <a name="integrity"></a> Subresource Integrity Digests
|
||||
|
||||
For content verification and addressing, cacache uses strings following the
|
||||
[Subresource
|
||||
Integrity spec](https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity).
|
||||
That is, any time cacache expects an `integrity` argument or option, it
|
||||
should be in the format `<hashAlgorithm>-<base64-hash>`.
|
||||
|
||||
One deviation from the current spec is that cacache will support any hash
|
||||
algorithms supported by the underlying Node.js process. You can use
|
||||
`crypto.getHashes()` to see which ones you can use.
|
||||
|
||||
##### Generating Digests Yourself
|
||||
|
||||
If you have an existing content shasum, they are generally formatted as a
|
||||
hexadecimal string (that is, a sha1 would look like:
|
||||
`5f5513f8822fdbe5145af33b64d8d970dcf95c6e`). In order to be compatible with
|
||||
cacache, you'll need to convert this to an equivalent subresource integrity
|
||||
string. For this example, the corresponding hash would be:
|
||||
`sha1-X1UT+IIv2+UUWvM7ZNjZcNz5XG4=`.
|
||||
|
||||
If you want to generate an integrity string yourself for existing data, you can
|
||||
use something like this:
|
||||
|
||||
```javascript
|
||||
const crypto = require('crypto')
|
||||
const hashAlgorithm = 'sha512'
|
||||
const data = 'foobarbaz'
|
||||
|
||||
const integrity = (
|
||||
hashAlgorithm +
|
||||
'-' +
|
||||
crypto.createHash(hashAlgorithm).update(data).digest('base64')
|
||||
)
|
||||
```
|
||||
|
||||
You can also use [`ssri`](https://npm.im/ssri) to have a richer set of functionality
|
||||
around SRI strings, including generation, parsing, and translating from existing
|
||||
hex-formatted strings.
|
||||
|
||||
#### <a name="verify"></a> `> cacache.verify(cache, opts) -> Promise`
|
||||
|
||||
Checks out and fixes up your cache:
|
||||
|
||||
* Cleans up corrupted or invalid index entries.
|
||||
* Custom entry filtering options.
|
||||
* Garbage collects any content entries not referenced by the index.
|
||||
* Checks integrity for all content entries and removes invalid content.
|
||||
* Fixes cache ownership.
|
||||
* Removes the `tmp` directory in the cache and all its contents.
|
||||
|
||||
When it's done, it'll return an object with various stats about the verification
|
||||
process, including amount of storage reclaimed, number of valid entries, number
|
||||
of entries removed, etc.
|
||||
|
||||
##### <a name="verify-options"></a> Options
|
||||
|
||||
##### `opts.concurrency`
|
||||
|
||||
Default: 20
|
||||
|
||||
Number of concurrently read files in the filesystem while doing clean up.
|
||||
|
||||
##### `opts.filter`
|
||||
Receives a formatted entry. Return false to remove it.
|
||||
Note: might be called more than once on the same entry.
|
||||
|
||||
##### `opts.log`
|
||||
Custom logger function:
|
||||
```
|
||||
log: { silly () {} }
|
||||
log.silly('verify', 'verifying cache at', cache)
|
||||
```
|
||||
|
||||
##### Example
|
||||
|
||||
```sh
|
||||
echo somegarbage >> $CACHEPATH/content/deadbeef
|
||||
```
|
||||
|
||||
```javascript
|
||||
cacache.verify(cachePath).then(stats => {
|
||||
// deadbeef collected, because of invalid checksum.
|
||||
console.log('cache is much nicer now! stats:', stats)
|
||||
})
|
||||
```
|
||||
|
||||
#### <a name="verify-last-run"></a> `> cacache.verify.lastRun(cache) -> Promise`
|
||||
|
||||
Returns a `Date` representing the last time `cacache.verify` was run on `cache`.
|
||||
|
||||
##### Example
|
||||
|
||||
```javascript
|
||||
cacache.verify(cachePath).then(() => {
|
||||
cacache.verify.lastRun(cachePath).then(lastTime => {
|
||||
console.log('cacache.verify was last called on' + lastTime)
|
||||
})
|
||||
})
|
||||
```
|
||||
+29
@@ -0,0 +1,29 @@
|
||||
'use strict'
|
||||
|
||||
const contentVer = require('../../package.json')['cache-version'].content
|
||||
const hashToSegments = require('../util/hash-to-segments')
|
||||
const path = require('path')
|
||||
const ssri = require('ssri')
|
||||
|
||||
// Current format of content file path:
|
||||
//
|
||||
// sha512-BaSE64Hex= ->
|
||||
// ~/.my-cache/content-v2/sha512/ba/da/55deadbeefc0ffee
|
||||
//
|
||||
module.exports = contentPath
|
||||
|
||||
function contentPath (cache, integrity) {
|
||||
const sri = ssri.parse(integrity, { single: true })
|
||||
// contentPath is the *strongest* algo given
|
||||
return path.join(
|
||||
contentDir(cache),
|
||||
sri.algorithm,
|
||||
...hashToSegments(sri.hexDigest())
|
||||
)
|
||||
}
|
||||
|
||||
module.exports.contentDir = contentDir
|
||||
|
||||
function contentDir (cache) {
|
||||
return path.join(cache, `content-v${contentVer}`)
|
||||
}
|
||||
+165
@@ -0,0 +1,165 @@
|
||||
'use strict'
|
||||
|
||||
const fs = require('fs/promises')
|
||||
const fsm = require('fs-minipass')
|
||||
const ssri = require('ssri')
|
||||
const contentPath = require('./path')
|
||||
const Pipeline = require('minipass-pipeline')
|
||||
|
||||
module.exports = read
|
||||
|
||||
const MAX_SINGLE_READ_SIZE = 64 * 1024 * 1024
|
||||
async function read (cache, integrity, opts = {}) {
|
||||
const { size } = opts
|
||||
const { stat, cpath, sri } = await withContentSri(cache, integrity, async (cpath, sri) => {
|
||||
// get size
|
||||
const stat = size ? { size } : await fs.stat(cpath)
|
||||
return { stat, cpath, sri }
|
||||
})
|
||||
|
||||
if (stat.size > MAX_SINGLE_READ_SIZE) {
|
||||
return readPipeline(cpath, stat.size, sri, new Pipeline()).concat()
|
||||
}
|
||||
|
||||
const data = await fs.readFile(cpath, { encoding: null })
|
||||
|
||||
if (stat.size !== data.length) {
|
||||
throw sizeError(stat.size, data.length)
|
||||
}
|
||||
|
||||
if (!ssri.checkData(data, sri)) {
|
||||
throw integrityError(sri, cpath)
|
||||
}
|
||||
|
||||
return data
|
||||
}
|
||||
|
||||
const readPipeline = (cpath, size, sri, stream) => {
|
||||
stream.push(
|
||||
new fsm.ReadStream(cpath, {
|
||||
size,
|
||||
readSize: MAX_SINGLE_READ_SIZE,
|
||||
}),
|
||||
ssri.integrityStream({
|
||||
integrity: sri,
|
||||
size,
|
||||
})
|
||||
)
|
||||
return stream
|
||||
}
|
||||
|
||||
module.exports.stream = readStream
|
||||
module.exports.readStream = readStream
|
||||
|
||||
function readStream (cache, integrity, opts = {}) {
|
||||
const { size } = opts
|
||||
const stream = new Pipeline()
|
||||
// Set all this up to run on the stream and then just return the stream
|
||||
Promise.resolve().then(async () => {
|
||||
const { stat, cpath, sri } = await withContentSri(cache, integrity, async (cpath, sri) => {
|
||||
// get size
|
||||
const stat = size ? { size } : await fs.stat(cpath)
|
||||
return { stat, cpath, sri }
|
||||
})
|
||||
|
||||
return readPipeline(cpath, stat.size, sri, stream)
|
||||
}).catch(err => stream.emit('error', err))
|
||||
|
||||
return stream
|
||||
}
|
||||
|
||||
module.exports.copy = copy
|
||||
|
||||
function copy (cache, integrity, dest) {
|
||||
return withContentSri(cache, integrity, (cpath) => {
|
||||
return fs.copyFile(cpath, dest)
|
||||
})
|
||||
}
|
||||
|
||||
module.exports.hasContent = hasContent
|
||||
|
||||
async function hasContent (cache, integrity) {
|
||||
if (!integrity) {
|
||||
return false
|
||||
}
|
||||
|
||||
try {
|
||||
return await withContentSri(cache, integrity, async (cpath, sri) => {
|
||||
const stat = await fs.stat(cpath)
|
||||
return { size: stat.size, sri, stat }
|
||||
})
|
||||
} catch (err) {
|
||||
if (err.code === 'ENOENT') {
|
||||
return false
|
||||
}
|
||||
|
||||
if (err.code === 'EPERM') {
|
||||
/* istanbul ignore else */
|
||||
if (process.platform !== 'win32') {
|
||||
throw err
|
||||
} else {
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function withContentSri (cache, integrity, fn) {
|
||||
const sri = ssri.parse(integrity)
|
||||
// If `integrity` has multiple entries, pick the first digest
|
||||
// with available local data.
|
||||
const algo = sri.pickAlgorithm()
|
||||
const digests = sri[algo]
|
||||
|
||||
if (digests.length <= 1) {
|
||||
const cpath = contentPath(cache, digests[0])
|
||||
return fn(cpath, digests[0])
|
||||
} else {
|
||||
// Can't use race here because a generic error can happen before
|
||||
// a ENOENT error, and can happen before a valid result
|
||||
const results = await Promise.all(digests.map(async (meta) => {
|
||||
try {
|
||||
return await withContentSri(cache, meta, fn)
|
||||
} catch (err) {
|
||||
if (err.code === 'ENOENT') {
|
||||
return Object.assign(
|
||||
new Error('No matching content found for ' + sri.toString()),
|
||||
{ code: 'ENOENT' }
|
||||
)
|
||||
}
|
||||
return err
|
||||
}
|
||||
}))
|
||||
// Return the first non error if it is found
|
||||
const result = results.find((r) => !(r instanceof Error))
|
||||
if (result) {
|
||||
return result
|
||||
}
|
||||
|
||||
// Throw the No matching content found error
|
||||
const enoentError = results.find((r) => r.code === 'ENOENT')
|
||||
if (enoentError) {
|
||||
throw enoentError
|
||||
}
|
||||
|
||||
// Throw generic error
|
||||
throw results.find((r) => r instanceof Error)
|
||||
}
|
||||
}
|
||||
|
||||
function sizeError (expected, found) {
|
||||
/* eslint-disable-next-line max-len */
|
||||
const err = new Error(`Bad data size: expected inserted data to be ${expected} bytes, but got ${found} instead`)
|
||||
err.expected = expected
|
||||
err.found = found
|
||||
err.code = 'EBADSIZE'
|
||||
return err
|
||||
}
|
||||
|
||||
function integrityError (sri, path) {
|
||||
const err = new Error(`Integrity verification failed for ${sri} (${path})`)
|
||||
err.code = 'EINTEGRITY'
|
||||
err.sri = sri
|
||||
err.path = path
|
||||
return err
|
||||
}
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
'use strict'
|
||||
|
||||
const fs = require('fs/promises')
|
||||
const contentPath = require('./path')
|
||||
const { hasContent } = require('./read')
|
||||
|
||||
module.exports = rm
|
||||
|
||||
async function rm (cache, integrity) {
|
||||
const content = await hasContent(cache, integrity)
|
||||
// ~pretty~ sure we can't end up with a content lacking sri, but be safe
|
||||
if (content && content.sri) {
|
||||
await fs.rm(contentPath(cache, content.sri), { recursive: true, force: true })
|
||||
return true
|
||||
} else {
|
||||
return false
|
||||
}
|
||||
}
|
||||
+206
@@ -0,0 +1,206 @@
|
||||
'use strict'
|
||||
|
||||
const events = require('events')
|
||||
|
||||
const contentPath = require('./path')
|
||||
const fs = require('fs/promises')
|
||||
const { moveFile } = require('@npmcli/fs')
|
||||
const { Minipass } = require('minipass')
|
||||
const Pipeline = require('minipass-pipeline')
|
||||
const Flush = require('minipass-flush')
|
||||
const path = require('path')
|
||||
const ssri = require('ssri')
|
||||
const uniqueFilename = require('unique-filename')
|
||||
const fsm = require('fs-minipass')
|
||||
|
||||
module.exports = write
|
||||
|
||||
// Cache of move operations in process so we don't duplicate
|
||||
const moveOperations = new Map()
|
||||
|
||||
async function write (cache, data, opts = {}) {
|
||||
const { algorithms, size, integrity } = opts
|
||||
|
||||
if (typeof size === 'number' && data.length !== size) {
|
||||
throw sizeError(size, data.length)
|
||||
}
|
||||
|
||||
const sri = ssri.fromData(data, algorithms ? { algorithms } : {})
|
||||
if (integrity && !ssri.checkData(data, integrity, opts)) {
|
||||
throw checksumError(integrity, sri)
|
||||
}
|
||||
|
||||
for (const algo in sri) {
|
||||
const tmp = await makeTmp(cache, opts)
|
||||
const hash = sri[algo].toString()
|
||||
try {
|
||||
await fs.writeFile(tmp.target, data, { flag: 'wx' })
|
||||
await moveToDestination(tmp, cache, hash, opts)
|
||||
} finally {
|
||||
if (!tmp.moved) {
|
||||
await fs.rm(tmp.target, { recursive: true, force: true })
|
||||
}
|
||||
}
|
||||
}
|
||||
return { integrity: sri, size: data.length }
|
||||
}
|
||||
|
||||
module.exports.stream = writeStream
|
||||
|
||||
// writes proxied to the 'inputStream' that is passed to the Promise
|
||||
// 'end' is deferred until content is handled.
|
||||
class CacacheWriteStream extends Flush {
|
||||
constructor (cache, opts) {
|
||||
super()
|
||||
this.opts = opts
|
||||
this.cache = cache
|
||||
this.inputStream = new Minipass()
|
||||
this.inputStream.on('error', er => this.emit('error', er))
|
||||
this.inputStream.on('drain', () => this.emit('drain'))
|
||||
this.handleContentP = null
|
||||
}
|
||||
|
||||
write (chunk, encoding, cb) {
|
||||
if (!this.handleContentP) {
|
||||
this.handleContentP = handleContent(
|
||||
this.inputStream,
|
||||
this.cache,
|
||||
this.opts
|
||||
)
|
||||
this.handleContentP.catch(error => this.emit('error', error))
|
||||
}
|
||||
return this.inputStream.write(chunk, encoding, cb)
|
||||
}
|
||||
|
||||
flush (cb) {
|
||||
this.inputStream.end(() => {
|
||||
if (!this.handleContentP) {
|
||||
const e = new Error('Cache input stream was empty')
|
||||
e.code = 'ENODATA'
|
||||
// empty streams are probably emitting end right away.
|
||||
// defer this one tick by rejecting a promise on it.
|
||||
return Promise.reject(e).catch(cb)
|
||||
}
|
||||
// eslint-disable-next-line promise/catch-or-return
|
||||
this.handleContentP.then(
|
||||
(res) => {
|
||||
res.integrity && this.emit('integrity', res.integrity)
|
||||
// eslint-disable-next-line promise/always-return
|
||||
res.size !== null && this.emit('size', res.size)
|
||||
cb()
|
||||
},
|
||||
(er) => cb(er)
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
function writeStream (cache, opts = {}) {
|
||||
return new CacacheWriteStream(cache, opts)
|
||||
}
|
||||
|
||||
async function handleContent (inputStream, cache, opts) {
|
||||
const tmp = await makeTmp(cache, opts)
|
||||
try {
|
||||
const res = await pipeToTmp(inputStream, cache, tmp.target, opts)
|
||||
await moveToDestination(
|
||||
tmp,
|
||||
cache,
|
||||
res.integrity,
|
||||
opts
|
||||
)
|
||||
return res
|
||||
} finally {
|
||||
if (!tmp.moved) {
|
||||
await fs.rm(tmp.target, { recursive: true, force: true })
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function pipeToTmp (inputStream, cache, tmpTarget, opts) {
|
||||
const outStream = new fsm.WriteStream(tmpTarget, {
|
||||
flags: 'wx',
|
||||
})
|
||||
|
||||
if (opts.integrityEmitter) {
|
||||
// we need to create these all simultaneously since they can fire in any order
|
||||
const [integrity, size] = await Promise.all([
|
||||
events.once(opts.integrityEmitter, 'integrity').then(res => res[0]),
|
||||
events.once(opts.integrityEmitter, 'size').then(res => res[0]),
|
||||
new Pipeline(inputStream, outStream).promise(),
|
||||
])
|
||||
return { integrity, size }
|
||||
}
|
||||
|
||||
let integrity
|
||||
let size
|
||||
const hashStream = ssri.integrityStream({
|
||||
integrity: opts.integrity,
|
||||
algorithms: opts.algorithms,
|
||||
size: opts.size,
|
||||
})
|
||||
hashStream.on('integrity', i => {
|
||||
integrity = i
|
||||
})
|
||||
hashStream.on('size', s => {
|
||||
size = s
|
||||
})
|
||||
|
||||
const pipeline = new Pipeline(inputStream, hashStream, outStream)
|
||||
await pipeline.promise()
|
||||
return { integrity, size }
|
||||
}
|
||||
|
||||
async function makeTmp (cache, opts) {
|
||||
const tmpTarget = uniqueFilename(path.join(cache, 'tmp'), opts.tmpPrefix)
|
||||
await fs.mkdir(path.dirname(tmpTarget), { recursive: true })
|
||||
return {
|
||||
target: tmpTarget,
|
||||
moved: false,
|
||||
}
|
||||
}
|
||||
|
||||
async function moveToDestination (tmp, cache, sri) {
|
||||
const destination = contentPath(cache, sri)
|
||||
const destDir = path.dirname(destination)
|
||||
if (moveOperations.has(destination)) {
|
||||
return moveOperations.get(destination)
|
||||
}
|
||||
moveOperations.set(
|
||||
destination,
|
||||
fs.mkdir(destDir, { recursive: true })
|
||||
.then(async () => {
|
||||
await moveFile(tmp.target, destination, { overwrite: false })
|
||||
tmp.moved = true
|
||||
return tmp.moved
|
||||
})
|
||||
.catch(err => {
|
||||
if (!err.message.startsWith('The destination file exists')) {
|
||||
throw Object.assign(err, { code: 'EEXIST' })
|
||||
}
|
||||
}).finally(() => {
|
||||
moveOperations.delete(destination)
|
||||
})
|
||||
|
||||
)
|
||||
return moveOperations.get(destination)
|
||||
}
|
||||
|
||||
function sizeError (expected, found) {
|
||||
/* eslint-disable-next-line max-len */
|
||||
const err = new Error(`Bad data size: expected inserted data to be ${expected} bytes, but got ${found} instead`)
|
||||
err.expected = expected
|
||||
err.found = found
|
||||
err.code = 'EBADSIZE'
|
||||
return err
|
||||
}
|
||||
|
||||
function checksumError (expected, found) {
|
||||
const err = new Error(`Integrity check failed:
|
||||
Wanted: ${expected}
|
||||
Found: ${found}`)
|
||||
err.code = 'EINTEGRITY'
|
||||
err.expected = expected
|
||||
err.found = found
|
||||
return err
|
||||
}
|
||||
+336
@@ -0,0 +1,336 @@
|
||||
'use strict'
|
||||
|
||||
const crypto = require('crypto')
|
||||
const {
|
||||
appendFile,
|
||||
mkdir,
|
||||
readFile,
|
||||
readdir,
|
||||
rm,
|
||||
writeFile,
|
||||
} = require('fs/promises')
|
||||
const { Minipass } = require('minipass')
|
||||
const path = require('path')
|
||||
const ssri = require('ssri')
|
||||
const uniqueFilename = require('unique-filename')
|
||||
|
||||
const contentPath = require('./content/path')
|
||||
const hashToSegments = require('./util/hash-to-segments')
|
||||
const indexV = require('../package.json')['cache-version'].index
|
||||
const { moveFile } = require('@npmcli/fs')
|
||||
|
||||
const lsStreamConcurrency = 5
|
||||
|
||||
module.exports.NotFoundError = class NotFoundError extends Error {
|
||||
constructor (cache, key) {
|
||||
super(`No cache entry for ${key} found in ${cache}`)
|
||||
this.code = 'ENOENT'
|
||||
this.cache = cache
|
||||
this.key = key
|
||||
}
|
||||
}
|
||||
|
||||
module.exports.compact = compact
|
||||
|
||||
async function compact (cache, key, matchFn, opts = {}) {
|
||||
const bucket = bucketPath(cache, key)
|
||||
const entries = await bucketEntries(bucket)
|
||||
const newEntries = []
|
||||
// we loop backwards because the bottom-most result is the newest
|
||||
// since we add new entries with appendFile
|
||||
for (let i = entries.length - 1; i >= 0; --i) {
|
||||
const entry = entries[i]
|
||||
// a null integrity could mean either a delete was appended
|
||||
// or the user has simply stored an index that does not map
|
||||
// to any content. we determine if the user wants to keep the
|
||||
// null integrity based on the validateEntry function passed in options.
|
||||
// if the integrity is null and no validateEntry is provided, we break
|
||||
// as we consider the null integrity to be a deletion of everything
|
||||
// that came before it.
|
||||
if (entry.integrity === null && !opts.validateEntry) {
|
||||
break
|
||||
}
|
||||
|
||||
// if this entry is valid, and it is either the first entry or
|
||||
// the newEntries array doesn't already include an entry that
|
||||
// matches this one based on the provided matchFn, then we add
|
||||
// it to the beginning of our list
|
||||
if ((!opts.validateEntry || opts.validateEntry(entry) === true) &&
|
||||
(newEntries.length === 0 ||
|
||||
!newEntries.find((oldEntry) => matchFn(oldEntry, entry)))) {
|
||||
newEntries.unshift(entry)
|
||||
}
|
||||
}
|
||||
|
||||
const newIndex = '\n' + newEntries.map((entry) => {
|
||||
const stringified = JSON.stringify(entry)
|
||||
const hash = hashEntry(stringified)
|
||||
return `${hash}\t${stringified}`
|
||||
}).join('\n')
|
||||
|
||||
const setup = async () => {
|
||||
const target = uniqueFilename(path.join(cache, 'tmp'), opts.tmpPrefix)
|
||||
await mkdir(path.dirname(target), { recursive: true })
|
||||
return {
|
||||
target,
|
||||
moved: false,
|
||||
}
|
||||
}
|
||||
|
||||
const teardown = async (tmp) => {
|
||||
if (!tmp.moved) {
|
||||
return rm(tmp.target, { recursive: true, force: true })
|
||||
}
|
||||
}
|
||||
|
||||
const write = async (tmp) => {
|
||||
await writeFile(tmp.target, newIndex, { flag: 'wx' })
|
||||
await mkdir(path.dirname(bucket), { recursive: true })
|
||||
// we use @npmcli/move-file directly here because we
|
||||
// want to overwrite the existing file
|
||||
await moveFile(tmp.target, bucket)
|
||||
tmp.moved = true
|
||||
}
|
||||
|
||||
// write the file atomically
|
||||
const tmp = await setup()
|
||||
try {
|
||||
await write(tmp)
|
||||
} finally {
|
||||
await teardown(tmp)
|
||||
}
|
||||
|
||||
// we reverse the list we generated such that the newest
|
||||
// entries come first in order to make looping through them easier
|
||||
// the true passed to formatEntry tells it to keep null
|
||||
// integrity values, if they made it this far it's because
|
||||
// validateEntry returned true, and as such we should return it
|
||||
return newEntries.reverse().map((entry) => formatEntry(cache, entry, true))
|
||||
}
|
||||
|
||||
module.exports.insert = insert
|
||||
|
||||
async function insert (cache, key, integrity, opts = {}) {
|
||||
const { metadata, size, time } = opts
|
||||
const bucket = bucketPath(cache, key)
|
||||
const entry = {
|
||||
key,
|
||||
integrity: integrity && ssri.stringify(integrity),
|
||||
time: time || Date.now(),
|
||||
size,
|
||||
metadata,
|
||||
}
|
||||
try {
|
||||
await mkdir(path.dirname(bucket), { recursive: true })
|
||||
const stringified = JSON.stringify(entry)
|
||||
// NOTE - Cleverness ahoy!
|
||||
//
|
||||
// This works because it's tremendously unlikely for an entry to corrupt
|
||||
// another while still preserving the string length of the JSON in
|
||||
// question. So, we just slap the length in there and verify it on read.
|
||||
//
|
||||
// Thanks to @isaacs for the whiteboarding session that ended up with
|
||||
// this.
|
||||
await appendFile(bucket, `\n${hashEntry(stringified)}\t${stringified}`)
|
||||
} catch (err) {
|
||||
if (err.code === 'ENOENT') {
|
||||
return undefined
|
||||
}
|
||||
|
||||
throw err
|
||||
}
|
||||
return formatEntry(cache, entry)
|
||||
}
|
||||
|
||||
module.exports.find = find
|
||||
|
||||
async function find (cache, key) {
|
||||
const bucket = bucketPath(cache, key)
|
||||
try {
|
||||
const entries = await bucketEntries(bucket)
|
||||
return entries.reduce((latest, next) => {
|
||||
if (next && next.key === key) {
|
||||
return formatEntry(cache, next)
|
||||
} else {
|
||||
return latest
|
||||
}
|
||||
}, null)
|
||||
} catch (err) {
|
||||
if (err.code === 'ENOENT') {
|
||||
return null
|
||||
} else {
|
||||
throw err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
module.exports.delete = del
|
||||
|
||||
function del (cache, key, opts = {}) {
|
||||
if (!opts.removeFully) {
|
||||
return insert(cache, key, null, opts)
|
||||
}
|
||||
|
||||
const bucket = bucketPath(cache, key)
|
||||
return rm(bucket, { recursive: true, force: true })
|
||||
}
|
||||
|
||||
module.exports.lsStream = lsStream
|
||||
|
||||
function lsStream (cache) {
|
||||
const indexDir = bucketDir(cache)
|
||||
const stream = new Minipass({ objectMode: true })
|
||||
|
||||
// Set all this up to run on the stream and then just return the stream
|
||||
Promise.resolve().then(async () => {
|
||||
const { default: pMap } = await import('p-map')
|
||||
const buckets = await readdirOrEmpty(indexDir)
|
||||
await pMap(buckets, async (bucket) => {
|
||||
const bucketPath = path.join(indexDir, bucket)
|
||||
const subbuckets = await readdirOrEmpty(bucketPath)
|
||||
await pMap(subbuckets, async (subbucket) => {
|
||||
const subbucketPath = path.join(bucketPath, subbucket)
|
||||
|
||||
// "/cachename/<bucket 0xFF>/<bucket 0xFF>./*"
|
||||
const subbucketEntries = await readdirOrEmpty(subbucketPath)
|
||||
await pMap(subbucketEntries, async (entry) => {
|
||||
const entryPath = path.join(subbucketPath, entry)
|
||||
try {
|
||||
const entries = await bucketEntries(entryPath)
|
||||
// using a Map here prevents duplicate keys from showing up
|
||||
// twice, I guess?
|
||||
const reduced = entries.reduce((acc, entry) => {
|
||||
acc.set(entry.key, entry)
|
||||
return acc
|
||||
}, new Map())
|
||||
// reduced is a map of key => entry
|
||||
for (const entry of reduced.values()) {
|
||||
const formatted = formatEntry(cache, entry)
|
||||
if (formatted) {
|
||||
stream.write(formatted)
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
if (err.code === 'ENOENT') {
|
||||
return undefined
|
||||
}
|
||||
throw err
|
||||
}
|
||||
},
|
||||
{ concurrency: lsStreamConcurrency })
|
||||
},
|
||||
{ concurrency: lsStreamConcurrency })
|
||||
},
|
||||
{ concurrency: lsStreamConcurrency })
|
||||
stream.end()
|
||||
return stream
|
||||
}).catch(err => stream.emit('error', err))
|
||||
|
||||
return stream
|
||||
}
|
||||
|
||||
module.exports.ls = ls
|
||||
|
||||
async function ls (cache) {
|
||||
const entries = await lsStream(cache).collect()
|
||||
return entries.reduce((acc, xs) => {
|
||||
acc[xs.key] = xs
|
||||
return acc
|
||||
}, {})
|
||||
}
|
||||
|
||||
module.exports.bucketEntries = bucketEntries
|
||||
|
||||
async function bucketEntries (bucket, filter) {
|
||||
const data = await readFile(bucket, 'utf8')
|
||||
return _bucketEntries(data, filter)
|
||||
}
|
||||
|
||||
function _bucketEntries (data) {
|
||||
const entries = []
|
||||
data.split('\n').forEach((entry) => {
|
||||
if (!entry) {
|
||||
return
|
||||
}
|
||||
|
||||
const pieces = entry.split('\t')
|
||||
if (!pieces[1] || hashEntry(pieces[1]) !== pieces[0]) {
|
||||
// Hash is no good! Corruption or malice? Doesn't matter!
|
||||
// EJECT EJECT
|
||||
return
|
||||
}
|
||||
let obj
|
||||
try {
|
||||
obj = JSON.parse(pieces[1])
|
||||
} catch (_) {
|
||||
// eslint-ignore-next-line no-empty-block
|
||||
}
|
||||
// coverage disabled here, no need to test with an entry that parses to something falsey
|
||||
// istanbul ignore else
|
||||
if (obj) {
|
||||
entries.push(obj)
|
||||
}
|
||||
})
|
||||
return entries
|
||||
}
|
||||
|
||||
module.exports.bucketDir = bucketDir
|
||||
|
||||
function bucketDir (cache) {
|
||||
return path.join(cache, `index-v${indexV}`)
|
||||
}
|
||||
|
||||
module.exports.bucketPath = bucketPath
|
||||
|
||||
function bucketPath (cache, key) {
|
||||
const hashed = hashKey(key)
|
||||
return path.join.apply(
|
||||
path,
|
||||
[bucketDir(cache)].concat(hashToSegments(hashed))
|
||||
)
|
||||
}
|
||||
|
||||
module.exports.hashKey = hashKey
|
||||
|
||||
function hashKey (key) {
|
||||
return hash(key, 'sha256')
|
||||
}
|
||||
|
||||
module.exports.hashEntry = hashEntry
|
||||
|
||||
function hashEntry (str) {
|
||||
return hash(str, 'sha1')
|
||||
}
|
||||
|
||||
function hash (str, digest) {
|
||||
return crypto
|
||||
.createHash(digest)
|
||||
.update(str)
|
||||
.digest('hex')
|
||||
}
|
||||
|
||||
function formatEntry (cache, entry, keepAll) {
|
||||
// Treat null digests as deletions. They'll shadow any previous entries.
|
||||
if (!entry.integrity && !keepAll) {
|
||||
return null
|
||||
}
|
||||
|
||||
return {
|
||||
key: entry.key,
|
||||
integrity: entry.integrity,
|
||||
path: entry.integrity ? contentPath(cache, entry.integrity) : undefined,
|
||||
size: entry.size,
|
||||
time: entry.time,
|
||||
metadata: entry.metadata,
|
||||
}
|
||||
}
|
||||
|
||||
function readdirOrEmpty (dir) {
|
||||
return readdir(dir).catch((err) => {
|
||||
if (err.code === 'ENOENT' || err.code === 'ENOTDIR') {
|
||||
return []
|
||||
}
|
||||
|
||||
throw err
|
||||
})
|
||||
}
|
||||
+170
@@ -0,0 +1,170 @@
|
||||
'use strict'
|
||||
|
||||
const Collect = require('minipass-collect')
|
||||
const { Minipass } = require('minipass')
|
||||
const Pipeline = require('minipass-pipeline')
|
||||
|
||||
const index = require('./entry-index')
|
||||
const memo = require('./memoization')
|
||||
const read = require('./content/read')
|
||||
|
||||
async function getData (cache, key, opts = {}) {
|
||||
const { integrity, memoize, size } = opts
|
||||
const memoized = memo.get(cache, key, opts)
|
||||
if (memoized && memoize !== false) {
|
||||
return {
|
||||
metadata: memoized.entry.metadata,
|
||||
data: memoized.data,
|
||||
integrity: memoized.entry.integrity,
|
||||
size: memoized.entry.size,
|
||||
}
|
||||
}
|
||||
|
||||
const entry = await index.find(cache, key, opts)
|
||||
if (!entry) {
|
||||
throw new index.NotFoundError(cache, key)
|
||||
}
|
||||
const data = await read(cache, entry.integrity, { integrity, size })
|
||||
if (memoize) {
|
||||
memo.put(cache, entry, data, opts)
|
||||
}
|
||||
|
||||
return {
|
||||
data,
|
||||
metadata: entry.metadata,
|
||||
size: entry.size,
|
||||
integrity: entry.integrity,
|
||||
}
|
||||
}
|
||||
module.exports = getData
|
||||
|
||||
async function getDataByDigest (cache, key, opts = {}) {
|
||||
const { integrity, memoize, size } = opts
|
||||
const memoized = memo.get.byDigest(cache, key, opts)
|
||||
if (memoized && memoize !== false) {
|
||||
return memoized
|
||||
}
|
||||
|
||||
const res = await read(cache, key, { integrity, size })
|
||||
if (memoize) {
|
||||
memo.put.byDigest(cache, key, res, opts)
|
||||
}
|
||||
return res
|
||||
}
|
||||
module.exports.byDigest = getDataByDigest
|
||||
|
||||
const getMemoizedStream = (memoized) => {
|
||||
const stream = new Minipass()
|
||||
stream.on('newListener', function (ev, cb) {
|
||||
ev === 'metadata' && cb(memoized.entry.metadata)
|
||||
ev === 'integrity' && cb(memoized.entry.integrity)
|
||||
ev === 'size' && cb(memoized.entry.size)
|
||||
})
|
||||
stream.end(memoized.data)
|
||||
return stream
|
||||
}
|
||||
|
||||
function getStream (cache, key, opts = {}) {
|
||||
const { memoize, size } = opts
|
||||
const memoized = memo.get(cache, key, opts)
|
||||
if (memoized && memoize !== false) {
|
||||
return getMemoizedStream(memoized)
|
||||
}
|
||||
|
||||
const stream = new Pipeline()
|
||||
// Set all this up to run on the stream and then just return the stream
|
||||
Promise.resolve().then(async () => {
|
||||
const entry = await index.find(cache, key)
|
||||
if (!entry) {
|
||||
throw new index.NotFoundError(cache, key)
|
||||
}
|
||||
|
||||
stream.emit('metadata', entry.metadata)
|
||||
stream.emit('integrity', entry.integrity)
|
||||
stream.emit('size', entry.size)
|
||||
stream.on('newListener', function (ev, cb) {
|
||||
ev === 'metadata' && cb(entry.metadata)
|
||||
ev === 'integrity' && cb(entry.integrity)
|
||||
ev === 'size' && cb(entry.size)
|
||||
})
|
||||
|
||||
const src = read.readStream(
|
||||
cache,
|
||||
entry.integrity,
|
||||
{ ...opts, size: typeof size !== 'number' ? entry.size : size }
|
||||
)
|
||||
|
||||
if (memoize) {
|
||||
const memoStream = new Collect.PassThrough()
|
||||
memoStream.on('collect', data => memo.put(cache, entry, data, opts))
|
||||
stream.unshift(memoStream)
|
||||
}
|
||||
stream.unshift(src)
|
||||
return stream
|
||||
}).catch((err) => stream.emit('error', err))
|
||||
|
||||
return stream
|
||||
}
|
||||
|
||||
module.exports.stream = getStream
|
||||
|
||||
function getStreamDigest (cache, integrity, opts = {}) {
|
||||
const { memoize } = opts
|
||||
const memoized = memo.get.byDigest(cache, integrity, opts)
|
||||
if (memoized && memoize !== false) {
|
||||
const stream = new Minipass()
|
||||
stream.end(memoized)
|
||||
return stream
|
||||
} else {
|
||||
const stream = read.readStream(cache, integrity, opts)
|
||||
if (!memoize) {
|
||||
return stream
|
||||
}
|
||||
|
||||
const memoStream = new Collect.PassThrough()
|
||||
memoStream.on('collect', data => memo.put.byDigest(
|
||||
cache,
|
||||
integrity,
|
||||
data,
|
||||
opts
|
||||
))
|
||||
return new Pipeline(stream, memoStream)
|
||||
}
|
||||
}
|
||||
|
||||
module.exports.stream.byDigest = getStreamDigest
|
||||
|
||||
function info (cache, key, opts = {}) {
|
||||
const { memoize } = opts
|
||||
const memoized = memo.get(cache, key, opts)
|
||||
if (memoized && memoize !== false) {
|
||||
return Promise.resolve(memoized.entry)
|
||||
} else {
|
||||
return index.find(cache, key)
|
||||
}
|
||||
}
|
||||
module.exports.info = info
|
||||
|
||||
async function copy (cache, key, dest, opts = {}) {
|
||||
const entry = await index.find(cache, key, opts)
|
||||
if (!entry) {
|
||||
throw new index.NotFoundError(cache, key)
|
||||
}
|
||||
await read.copy(cache, entry.integrity, dest, opts)
|
||||
return {
|
||||
metadata: entry.metadata,
|
||||
size: entry.size,
|
||||
integrity: entry.integrity,
|
||||
}
|
||||
}
|
||||
|
||||
module.exports.copy = copy
|
||||
|
||||
async function copyByDigest (cache, key, dest, opts = {}) {
|
||||
await read.copy(cache, key, dest, opts)
|
||||
return key
|
||||
}
|
||||
|
||||
module.exports.copy.byDigest = copyByDigest
|
||||
|
||||
module.exports.hasContent = read.hasContent
|
||||
+42
@@ -0,0 +1,42 @@
|
||||
'use strict'
|
||||
|
||||
const get = require('./get.js')
|
||||
const put = require('./put.js')
|
||||
const rm = require('./rm.js')
|
||||
const verify = require('./verify.js')
|
||||
const { clearMemoized } = require('./memoization.js')
|
||||
const tmp = require('./util/tmp.js')
|
||||
const index = require('./entry-index.js')
|
||||
|
||||
module.exports.index = {}
|
||||
module.exports.index.compact = index.compact
|
||||
module.exports.index.insert = index.insert
|
||||
|
||||
module.exports.ls = index.ls
|
||||
module.exports.ls.stream = index.lsStream
|
||||
|
||||
module.exports.get = get
|
||||
module.exports.get.byDigest = get.byDigest
|
||||
module.exports.get.stream = get.stream
|
||||
module.exports.get.stream.byDigest = get.stream.byDigest
|
||||
module.exports.get.copy = get.copy
|
||||
module.exports.get.copy.byDigest = get.copy.byDigest
|
||||
module.exports.get.info = get.info
|
||||
module.exports.get.hasContent = get.hasContent
|
||||
|
||||
module.exports.put = put
|
||||
module.exports.put.stream = put.stream
|
||||
|
||||
module.exports.rm = rm.entry
|
||||
module.exports.rm.all = rm.all
|
||||
module.exports.rm.entry = module.exports.rm
|
||||
module.exports.rm.content = rm.content
|
||||
|
||||
module.exports.clearMemoized = clearMemoized
|
||||
|
||||
module.exports.tmp = {}
|
||||
module.exports.tmp.mkdir = tmp.mkdir
|
||||
module.exports.tmp.withTmp = tmp.withTmp
|
||||
|
||||
module.exports.verify = verify
|
||||
module.exports.verify.lastRun = verify.lastRun
|
||||
+72
@@ -0,0 +1,72 @@
|
||||
'use strict'
|
||||
|
||||
const { LRUCache } = require('lru-cache')
|
||||
|
||||
const MEMOIZED = new LRUCache({
|
||||
max: 500,
|
||||
maxSize: 50 * 1024 * 1024, // 50MB
|
||||
ttl: 3 * 60 * 1000, // 3 minutes
|
||||
sizeCalculation: (entry, key) => key.startsWith('key:') ? entry.data.length : entry.length,
|
||||
})
|
||||
|
||||
module.exports.clearMemoized = clearMemoized
|
||||
|
||||
function clearMemoized () {
|
||||
const old = {}
|
||||
MEMOIZED.forEach((v, k) => {
|
||||
old[k] = v
|
||||
})
|
||||
MEMOIZED.clear()
|
||||
return old
|
||||
}
|
||||
|
||||
module.exports.put = put
|
||||
|
||||
function put (cache, entry, data, opts) {
|
||||
pickMem(opts).set(`key:${cache}:${entry.key}`, { entry, data })
|
||||
putDigest(cache, entry.integrity, data, opts)
|
||||
}
|
||||
|
||||
module.exports.put.byDigest = putDigest
|
||||
|
||||
function putDigest (cache, integrity, data, opts) {
|
||||
pickMem(opts).set(`digest:${cache}:${integrity}`, data)
|
||||
}
|
||||
|
||||
module.exports.get = get
|
||||
|
||||
function get (cache, key, opts) {
|
||||
return pickMem(opts).get(`key:${cache}:${key}`)
|
||||
}
|
||||
|
||||
module.exports.get.byDigest = getDigest
|
||||
|
||||
function getDigest (cache, integrity, opts) {
|
||||
return pickMem(opts).get(`digest:${cache}:${integrity}`)
|
||||
}
|
||||
|
||||
class ObjProxy {
|
||||
constructor (obj) {
|
||||
this.obj = obj
|
||||
}
|
||||
|
||||
get (key) {
|
||||
return this.obj[key]
|
||||
}
|
||||
|
||||
set (key, val) {
|
||||
this.obj[key] = val
|
||||
}
|
||||
}
|
||||
|
||||
function pickMem (opts) {
|
||||
if (!opts || !opts.memoize) {
|
||||
return MEMOIZED
|
||||
} else if (opts.memoize.get && opts.memoize.set) {
|
||||
return opts.memoize
|
||||
} else if (typeof opts.memoize === 'object') {
|
||||
return new ObjProxy(opts.memoize)
|
||||
} else {
|
||||
return MEMOIZED
|
||||
}
|
||||
}
|
||||
+80
@@ -0,0 +1,80 @@
|
||||
'use strict'
|
||||
|
||||
const index = require('./entry-index')
|
||||
const memo = require('./memoization')
|
||||
const write = require('./content/write')
|
||||
const Flush = require('minipass-flush')
|
||||
const { PassThrough } = require('minipass-collect')
|
||||
const Pipeline = require('minipass-pipeline')
|
||||
|
||||
const putOpts = (opts) => ({
|
||||
algorithms: ['sha512'],
|
||||
...opts,
|
||||
})
|
||||
|
||||
module.exports = putData
|
||||
|
||||
async function putData (cache, key, data, opts = {}) {
|
||||
const { memoize } = opts
|
||||
opts = putOpts(opts)
|
||||
const res = await write(cache, data, opts)
|
||||
const entry = await index.insert(cache, key, res.integrity, { ...opts, size: res.size })
|
||||
if (memoize) {
|
||||
memo.put(cache, entry, data, opts)
|
||||
}
|
||||
|
||||
return res.integrity
|
||||
}
|
||||
|
||||
module.exports.stream = putStream
|
||||
|
||||
function putStream (cache, key, opts = {}) {
|
||||
const { memoize } = opts
|
||||
opts = putOpts(opts)
|
||||
let integrity
|
||||
let size
|
||||
let error
|
||||
|
||||
let memoData
|
||||
const pipeline = new Pipeline()
|
||||
// first item in the pipeline is the memoizer, because we need
|
||||
// that to end first and get the collected data.
|
||||
if (memoize) {
|
||||
const memoizer = new PassThrough().on('collect', data => {
|
||||
memoData = data
|
||||
})
|
||||
pipeline.push(memoizer)
|
||||
}
|
||||
|
||||
// contentStream is a write-only, not a passthrough
|
||||
// no data comes out of it.
|
||||
const contentStream = write.stream(cache, opts)
|
||||
.on('integrity', (int) => {
|
||||
integrity = int
|
||||
})
|
||||
.on('size', (s) => {
|
||||
size = s
|
||||
})
|
||||
.on('error', (err) => {
|
||||
error = err
|
||||
})
|
||||
|
||||
pipeline.push(contentStream)
|
||||
|
||||
// last but not least, we write the index and emit hash and size,
|
||||
// and memoize if we're doing that
|
||||
pipeline.push(new Flush({
|
||||
async flush () {
|
||||
if (!error) {
|
||||
const entry = await index.insert(cache, key, integrity, { ...opts, size })
|
||||
if (memoize && memoData) {
|
||||
memo.put(cache, entry, memoData, opts)
|
||||
}
|
||||
pipeline.emit('integrity', integrity)
|
||||
pipeline.emit('size', size)
|
||||
}
|
||||
},
|
||||
}))
|
||||
|
||||
return pipeline
|
||||
}
|
||||
+31
@@ -0,0 +1,31 @@
|
||||
'use strict'
|
||||
|
||||
const { rm } = require('fs/promises')
|
||||
const glob = require('./util/glob.js')
|
||||
const index = require('./entry-index')
|
||||
const memo = require('./memoization')
|
||||
const path = require('path')
|
||||
const rmContent = require('./content/rm')
|
||||
|
||||
module.exports = entry
|
||||
module.exports.entry = entry
|
||||
|
||||
function entry (cache, key, opts) {
|
||||
memo.clearMemoized()
|
||||
return index.delete(cache, key, opts)
|
||||
}
|
||||
|
||||
module.exports.content = content
|
||||
|
||||
function content (cache, integrity) {
|
||||
memo.clearMemoized()
|
||||
return rmContent(cache, integrity)
|
||||
}
|
||||
|
||||
module.exports.all = all
|
||||
|
||||
async function all (cache) {
|
||||
memo.clearMemoized()
|
||||
const paths = await glob(path.join(cache, '*(content-*|index-*)'), { silent: true, nosort: true })
|
||||
return Promise.all(paths.map((p) => rm(p, { recursive: true, force: true })))
|
||||
}
|
||||
+7
@@ -0,0 +1,7 @@
|
||||
'use strict'
|
||||
|
||||
const { glob } = require('glob')
|
||||
const path = require('path')
|
||||
|
||||
const globify = (pattern) => pattern.split(path.win32.sep).join(path.posix.sep)
|
||||
module.exports = (path, options) => glob(globify(path), options)
|
||||
+7
@@ -0,0 +1,7 @@
|
||||
'use strict'
|
||||
|
||||
module.exports = hashToSegments
|
||||
|
||||
function hashToSegments (hash) {
|
||||
return [hash.slice(0, 2), hash.slice(2, 4), hash.slice(4)]
|
||||
}
|
||||
+26
@@ -0,0 +1,26 @@
|
||||
'use strict'
|
||||
|
||||
const { withTempDir } = require('@npmcli/fs')
|
||||
const fs = require('fs/promises')
|
||||
const path = require('path')
|
||||
|
||||
module.exports.mkdir = mktmpdir
|
||||
|
||||
async function mktmpdir (cache, opts = {}) {
|
||||
const { tmpPrefix } = opts
|
||||
const tmpDir = path.join(cache, 'tmp')
|
||||
await fs.mkdir(tmpDir, { recursive: true, owner: 'inherit' })
|
||||
// do not use path.join(), it drops the trailing / if tmpPrefix is unset
|
||||
const target = `${tmpDir}${path.sep}${tmpPrefix || ''}`
|
||||
return fs.mkdtemp(target, { owner: 'inherit' })
|
||||
}
|
||||
|
||||
module.exports.withTmp = withTmp
|
||||
|
||||
function withTmp (cache, opts, cb) {
|
||||
if (!cb) {
|
||||
cb = opts
|
||||
opts = {}
|
||||
}
|
||||
return withTempDir(path.join(cache, 'tmp'), cb, opts)
|
||||
}
|
||||
+258
@@ -0,0 +1,258 @@
|
||||
'use strict'
|
||||
|
||||
const {
|
||||
mkdir,
|
||||
readFile,
|
||||
rm,
|
||||
stat,
|
||||
truncate,
|
||||
writeFile,
|
||||
} = require('fs/promises')
|
||||
const contentPath = require('./content/path')
|
||||
const fsm = require('fs-minipass')
|
||||
const glob = require('./util/glob.js')
|
||||
const index = require('./entry-index')
|
||||
const path = require('path')
|
||||
const ssri = require('ssri')
|
||||
|
||||
const hasOwnProperty = (obj, key) =>
|
||||
Object.prototype.hasOwnProperty.call(obj, key)
|
||||
|
||||
const verifyOpts = (opts) => ({
|
||||
concurrency: 20,
|
||||
log: { silly () {} },
|
||||
...opts,
|
||||
})
|
||||
|
||||
module.exports = verify
|
||||
|
||||
async function verify (cache, opts) {
|
||||
opts = verifyOpts(opts)
|
||||
opts.log.silly('verify', 'verifying cache at', cache)
|
||||
|
||||
const steps = [
|
||||
markStartTime,
|
||||
fixPerms,
|
||||
garbageCollect,
|
||||
rebuildIndex,
|
||||
cleanTmp,
|
||||
writeVerifile,
|
||||
markEndTime,
|
||||
]
|
||||
|
||||
const stats = {}
|
||||
for (const step of steps) {
|
||||
const label = step.name
|
||||
const start = new Date()
|
||||
const s = await step(cache, opts)
|
||||
if (s) {
|
||||
Object.keys(s).forEach((k) => {
|
||||
stats[k] = s[k]
|
||||
})
|
||||
}
|
||||
const end = new Date()
|
||||
if (!stats.runTime) {
|
||||
stats.runTime = {}
|
||||
}
|
||||
stats.runTime[label] = end - start
|
||||
}
|
||||
stats.runTime.total = stats.endTime - stats.startTime
|
||||
opts.log.silly(
|
||||
'verify',
|
||||
'verification finished for',
|
||||
cache,
|
||||
'in',
|
||||
`${stats.runTime.total}ms`
|
||||
)
|
||||
return stats
|
||||
}
|
||||
|
||||
async function markStartTime () {
|
||||
return { startTime: new Date() }
|
||||
}
|
||||
|
||||
async function markEndTime () {
|
||||
return { endTime: new Date() }
|
||||
}
|
||||
|
||||
async function fixPerms (cache, opts) {
|
||||
opts.log.silly('verify', 'fixing cache permissions')
|
||||
await mkdir(cache, { recursive: true })
|
||||
return null
|
||||
}
|
||||
|
||||
// Implements a naive mark-and-sweep tracing garbage collector.
|
||||
//
|
||||
// The algorithm is basically as follows:
|
||||
// 1. Read (and filter) all index entries ("pointers")
|
||||
// 2. Mark each integrity value as "live"
|
||||
// 3. Read entire filesystem tree in `content-vX/` dir
|
||||
// 4. If content is live, verify its checksum and delete it if it fails
|
||||
// 5. If content is not marked as live, rm it.
|
||||
//
|
||||
async function garbageCollect (cache, opts) {
|
||||
opts.log.silly('verify', 'garbage collecting content')
|
||||
const { default: pMap } = await import('p-map')
|
||||
const indexStream = index.lsStream(cache)
|
||||
const liveContent = new Set()
|
||||
indexStream.on('data', (entry) => {
|
||||
if (opts.filter && !opts.filter(entry)) {
|
||||
return
|
||||
}
|
||||
|
||||
// integrity is stringified, re-parse it so we can get each hash
|
||||
const integrity = ssri.parse(entry.integrity)
|
||||
for (const algo in integrity) {
|
||||
liveContent.add(integrity[algo].toString())
|
||||
}
|
||||
})
|
||||
await new Promise((resolve, reject) => {
|
||||
indexStream.on('end', resolve).on('error', reject)
|
||||
})
|
||||
const contentDir = contentPath.contentDir(cache)
|
||||
const files = await glob(path.join(contentDir, '**'), {
|
||||
follow: false,
|
||||
nodir: true,
|
||||
nosort: true,
|
||||
})
|
||||
const stats = {
|
||||
verifiedContent: 0,
|
||||
reclaimedCount: 0,
|
||||
reclaimedSize: 0,
|
||||
badContentCount: 0,
|
||||
keptSize: 0,
|
||||
}
|
||||
await pMap(
|
||||
files,
|
||||
async (f) => {
|
||||
const split = f.split(/[/\\]/)
|
||||
const digest = split.slice(split.length - 3).join('')
|
||||
const algo = split[split.length - 4]
|
||||
const integrity = ssri.fromHex(digest, algo)
|
||||
if (liveContent.has(integrity.toString())) {
|
||||
const info = await verifyContent(f, integrity)
|
||||
if (!info.valid) {
|
||||
stats.reclaimedCount++
|
||||
stats.badContentCount++
|
||||
stats.reclaimedSize += info.size
|
||||
} else {
|
||||
stats.verifiedContent++
|
||||
stats.keptSize += info.size
|
||||
}
|
||||
} else {
|
||||
// No entries refer to this content. We can delete.
|
||||
stats.reclaimedCount++
|
||||
const s = await stat(f)
|
||||
await rm(f, { recursive: true, force: true })
|
||||
stats.reclaimedSize += s.size
|
||||
}
|
||||
return stats
|
||||
},
|
||||
{ concurrency: opts.concurrency }
|
||||
)
|
||||
return stats
|
||||
}
|
||||
|
||||
async function verifyContent (filepath, sri) {
|
||||
const contentInfo = {}
|
||||
try {
|
||||
const { size } = await stat(filepath)
|
||||
contentInfo.size = size
|
||||
contentInfo.valid = true
|
||||
await ssri.checkStream(new fsm.ReadStream(filepath), sri)
|
||||
} catch (err) {
|
||||
if (err.code === 'ENOENT') {
|
||||
return { size: 0, valid: false }
|
||||
}
|
||||
if (err.code !== 'EINTEGRITY') {
|
||||
throw err
|
||||
}
|
||||
|
||||
await rm(filepath, { recursive: true, force: true })
|
||||
contentInfo.valid = false
|
||||
}
|
||||
return contentInfo
|
||||
}
|
||||
|
||||
async function rebuildIndex (cache, opts) {
|
||||
opts.log.silly('verify', 'rebuilding index')
|
||||
const { default: pMap } = await import('p-map')
|
||||
const entries = await index.ls(cache)
|
||||
const stats = {
|
||||
missingContent: 0,
|
||||
rejectedEntries: 0,
|
||||
totalEntries: 0,
|
||||
}
|
||||
const buckets = {}
|
||||
for (const k in entries) {
|
||||
/* istanbul ignore else */
|
||||
if (hasOwnProperty(entries, k)) {
|
||||
const hashed = index.hashKey(k)
|
||||
const entry = entries[k]
|
||||
const excluded = opts.filter && !opts.filter(entry)
|
||||
excluded && stats.rejectedEntries++
|
||||
if (buckets[hashed] && !excluded) {
|
||||
buckets[hashed].push(entry)
|
||||
} else if (buckets[hashed] && excluded) {
|
||||
// skip
|
||||
} else if (excluded) {
|
||||
buckets[hashed] = []
|
||||
buckets[hashed]._path = index.bucketPath(cache, k)
|
||||
} else {
|
||||
buckets[hashed] = [entry]
|
||||
buckets[hashed]._path = index.bucketPath(cache, k)
|
||||
}
|
||||
}
|
||||
}
|
||||
await pMap(
|
||||
Object.keys(buckets),
|
||||
(key) => {
|
||||
return rebuildBucket(cache, buckets[key], stats, opts)
|
||||
},
|
||||
{ concurrency: opts.concurrency }
|
||||
)
|
||||
return stats
|
||||
}
|
||||
|
||||
async function rebuildBucket (cache, bucket, stats) {
|
||||
await truncate(bucket._path)
|
||||
// This needs to be serialized because cacache explicitly
|
||||
// lets very racy bucket conflicts clobber each other.
|
||||
for (const entry of bucket) {
|
||||
const content = contentPath(cache, entry.integrity)
|
||||
try {
|
||||
await stat(content)
|
||||
await index.insert(cache, entry.key, entry.integrity, {
|
||||
metadata: entry.metadata,
|
||||
size: entry.size,
|
||||
time: entry.time,
|
||||
})
|
||||
stats.totalEntries++
|
||||
} catch (err) {
|
||||
if (err.code === 'ENOENT') {
|
||||
stats.rejectedEntries++
|
||||
stats.missingContent++
|
||||
} else {
|
||||
throw err
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function cleanTmp (cache, opts) {
|
||||
opts.log.silly('verify', 'cleaning tmp directory')
|
||||
return rm(path.join(cache, 'tmp'), { recursive: true, force: true })
|
||||
}
|
||||
|
||||
async function writeVerifile (cache, opts) {
|
||||
const verifile = path.join(cache, '_lastverified')
|
||||
opts.log.silly('verify', 'writing verifile to ' + verifile)
|
||||
return writeFile(verifile, `${Date.now()}`)
|
||||
}
|
||||
|
||||
module.exports.lastRun = lastRun
|
||||
|
||||
async function lastRun (cache) {
|
||||
const data = await readFile(path.join(cache, '_lastverified'), { encoding: 'utf8' })
|
||||
return new Date(+data)
|
||||
}
|
||||
+55
@@ -0,0 +1,55 @@
|
||||
# Blue Oak Model License
|
||||
|
||||
Version 1.0.0
|
||||
|
||||
## Purpose
|
||||
|
||||
This license gives everyone as much permission to work with
|
||||
this software as possible, while protecting contributors
|
||||
from liability.
|
||||
|
||||
## Acceptance
|
||||
|
||||
In order to receive this license, you must agree to its
|
||||
rules. The rules of this license are both obligations
|
||||
under that agreement and conditions to your license.
|
||||
You must not do anything with this software that triggers
|
||||
a rule that you cannot or will not follow.
|
||||
|
||||
## Copyright
|
||||
|
||||
Each contributor licenses you to do everything with this
|
||||
software that would otherwise infringe that contributor's
|
||||
copyright in it.
|
||||
|
||||
## Notices
|
||||
|
||||
You must ensure that everyone who gets a copy of
|
||||
any part of this software from you, with or without
|
||||
changes, also gets the text of this license or a link to
|
||||
<https://blueoakcouncil.org/license/1.0.0>.
|
||||
|
||||
## Excuse
|
||||
|
||||
If anyone notifies you in writing that you have not
|
||||
complied with [Notices](#notices), you can keep your
|
||||
license by taking all practical steps to comply within 30
|
||||
days after the notice. If you do not do so, your license
|
||||
ends immediately.
|
||||
|
||||
## Patent
|
||||
|
||||
Each contributor licenses you to do everything with this
|
||||
software that would otherwise infringe any patent claims
|
||||
they can license or become able to license.
|
||||
|
||||
## Reliability
|
||||
|
||||
No contributor can revoke this license.
|
||||
|
||||
## No Liability
|
||||
|
||||
***As far as the law allows, this software comes as is,
|
||||
without any warranty or condition, and no contributor
|
||||
will be liable to anyone for any damages related to this
|
||||
software or this license, under any kind of legal claim.***
|
||||
+383
@@ -0,0 +1,383 @@
|
||||
# lru-cache
|
||||
|
||||
A cache object that deletes the least-recently-used items.
|
||||
|
||||
Specify a max number of the most recently used items that you
|
||||
want to keep, and this cache will keep that many of the most
|
||||
recently accessed items.
|
||||
|
||||
This is not primarily a TTL cache, and does not make strong TTL
|
||||
guarantees. There is no preemptive pruning of expired items by
|
||||
default, but you _may_ set a TTL on the cache or on a single
|
||||
`set`. If you do so, it will treat expired items as missing, and
|
||||
delete them when fetched. If you are more interested in TTL
|
||||
caching than LRU caching, check out
|
||||
[@isaacs/ttlcache](http://npm.im/@isaacs/ttlcache).
|
||||
|
||||
As of version 7, this is one of the most performant LRU
|
||||
implementations available in JavaScript, and supports a wide
|
||||
diversity of use cases. However, note that using some of the
|
||||
features will necessarily impact performance, by causing the
|
||||
cache to have to do more work. See the "Performance" section
|
||||
below.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
npm install lru-cache --save
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```js
|
||||
// hybrid module, either works
|
||||
import { LRUCache } from 'lru-cache'
|
||||
// or:
|
||||
const { LRUCache } = require('lru-cache')
|
||||
// or in minified form for web browsers:
|
||||
import { LRUCache } from 'http://unpkg.com/lru-cache@9/dist/mjs/index.min.mjs'
|
||||
|
||||
// At least one of 'max', 'ttl', or 'maxSize' is required, to prevent
|
||||
// unsafe unbounded storage.
|
||||
//
|
||||
// In most cases, it's best to specify a max for performance, so all
|
||||
// the required memory allocation is done up-front.
|
||||
//
|
||||
// All the other options are optional, see the sections below for
|
||||
// documentation on what each one does. Most of them can be
|
||||
// overridden for specific items in get()/set()
|
||||
const options = {
|
||||
max: 500,
|
||||
|
||||
// for use with tracking overall storage size
|
||||
maxSize: 5000,
|
||||
sizeCalculation: (value, key) => {
|
||||
return 1
|
||||
},
|
||||
|
||||
// for use when you need to clean up something when objects
|
||||
// are evicted from the cache
|
||||
dispose: (value, key, reason) => {
|
||||
freeFromMemoryOrWhatever(value)
|
||||
},
|
||||
|
||||
// for use when you need to know that an item is being inserted
|
||||
// note that this does NOT allow you to prevent the insertion,
|
||||
// it just allows you to know about it.
|
||||
onInsert: (value, key, reason) => {
|
||||
logInsertionOrWhatever(key, value)
|
||||
},
|
||||
|
||||
// how long to live in ms
|
||||
ttl: 1000 * 60 * 5,
|
||||
|
||||
// return stale items before removing from cache?
|
||||
allowStale: false,
|
||||
|
||||
updateAgeOnGet: false,
|
||||
updateAgeOnHas: false,
|
||||
|
||||
// async method to use for cache.fetch(), for
|
||||
// stale-while-revalidate type of behavior
|
||||
fetchMethod: async (key, staleValue, { options, signal, context }) => {},
|
||||
}
|
||||
|
||||
const cache = new LRUCache(options)
|
||||
|
||||
cache.set('key', 'value')
|
||||
cache.get('key') // "value"
|
||||
|
||||
// non-string keys ARE fully supported
|
||||
// but note that it must be THE SAME object, not
|
||||
// just a JSON-equivalent object.
|
||||
var someObject = { a: 1 }
|
||||
cache.set(someObject, 'a value')
|
||||
// Object keys are not toString()-ed
|
||||
cache.set('[object Object]', 'a different value')
|
||||
assert.equal(cache.get(someObject), 'a value')
|
||||
// A similar object with same keys/values won't work,
|
||||
// because it's a different object identity
|
||||
assert.equal(cache.get({ a: 1 }), undefined)
|
||||
|
||||
cache.clear() // empty the cache
|
||||
```
|
||||
|
||||
If you put more stuff in the cache, then less recently used items
|
||||
will fall out. That's what an LRU cache is.
|
||||
|
||||
For full description of the API and all options, please see [the
|
||||
LRUCache typedocs](https://isaacs.github.io/node-lru-cache/)
|
||||
|
||||
## Storage Bounds Safety
|
||||
|
||||
This implementation aims to be as flexible as possible, within
|
||||
the limits of safe memory consumption and optimal performance.
|
||||
|
||||
At initial object creation, storage is allocated for `max` items.
|
||||
If `max` is set to zero, then some performance is lost, and item
|
||||
count is unbounded. Either `maxSize` or `ttl` _must_ be set if
|
||||
`max` is not specified.
|
||||
|
||||
If `maxSize` is set, then this creates a safe limit on the
|
||||
maximum storage consumed, but without the performance benefits of
|
||||
pre-allocation. When `maxSize` is set, every item _must_ provide
|
||||
a size, either via the `sizeCalculation` method provided to the
|
||||
constructor, or via a `size` or `sizeCalculation` option provided
|
||||
to `cache.set()`. The size of every item _must_ be a positive
|
||||
integer.
|
||||
|
||||
If neither `max` nor `maxSize` are set, then `ttl` tracking must
|
||||
be enabled. Note that, even when tracking item `ttl`, items are
|
||||
_not_ preemptively deleted when they become stale, unless
|
||||
`ttlAutopurge` is enabled. Instead, they are only purged the
|
||||
next time the key is requested. Thus, if `ttlAutopurge`, `max`,
|
||||
and `maxSize` are all not set, then the cache will potentially
|
||||
grow unbounded.
|
||||
|
||||
In this case, a warning is printed to standard error. Future
|
||||
versions may require the use of `ttlAutopurge` if `max` and
|
||||
`maxSize` are not specified.
|
||||
|
||||
If you truly wish to use a cache that is bound _only_ by TTL
|
||||
expiration, consider using a `Map` object, and calling
|
||||
`setTimeout` to delete entries when they expire. It will perform
|
||||
much better than an LRU cache.
|
||||
|
||||
Here is an implementation you may use, under the same
|
||||
[license](./LICENSE) as this package:
|
||||
|
||||
```js
|
||||
// a storage-unbounded ttl cache that is not an lru-cache
|
||||
const cache = {
|
||||
data: new Map(),
|
||||
timers: new Map(),
|
||||
set: (k, v, ttl) => {
|
||||
if (cache.timers.has(k)) {
|
||||
clearTimeout(cache.timers.get(k))
|
||||
}
|
||||
cache.timers.set(
|
||||
k,
|
||||
setTimeout(() => cache.delete(k), ttl),
|
||||
)
|
||||
cache.data.set(k, v)
|
||||
},
|
||||
get: k => cache.data.get(k),
|
||||
has: k => cache.data.has(k),
|
||||
delete: k => {
|
||||
if (cache.timers.has(k)) {
|
||||
clearTimeout(cache.timers.get(k))
|
||||
}
|
||||
cache.timers.delete(k)
|
||||
return cache.data.delete(k)
|
||||
},
|
||||
clear: () => {
|
||||
cache.data.clear()
|
||||
for (const v of cache.timers.values()) {
|
||||
clearTimeout(v)
|
||||
}
|
||||
cache.timers.clear()
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
If that isn't to your liking, check out
|
||||
[@isaacs/ttlcache](http://npm.im/@isaacs/ttlcache).
|
||||
|
||||
## Storing Undefined Values
|
||||
|
||||
This cache never stores undefined values, as `undefined` is used
|
||||
internally in a few places to indicate that a key is not in the
|
||||
cache.
|
||||
|
||||
You may call `cache.set(key, undefined)`, but this is just
|
||||
an alias for `cache.delete(key)`. Note that this has the effect
|
||||
that `cache.has(key)` will return _false_ after setting it to
|
||||
undefined.
|
||||
|
||||
```js
|
||||
cache.set(myKey, undefined)
|
||||
cache.has(myKey) // false!
|
||||
```
|
||||
|
||||
If you need to track `undefined` values, and still note that the
|
||||
key is in the cache, an easy workaround is to use a sigil object
|
||||
of your own.
|
||||
|
||||
```js
|
||||
import { LRUCache } from 'lru-cache'
|
||||
const undefinedValue = Symbol('undefined')
|
||||
const cache = new LRUCache(...)
|
||||
const mySet = (key, value) =>
|
||||
cache.set(key, value === undefined ? undefinedValue : value)
|
||||
const myGet = (key, value) => {
|
||||
const v = cache.get(key)
|
||||
return v === undefinedValue ? undefined : v
|
||||
}
|
||||
```
|
||||
|
||||
## Performance
|
||||
|
||||
As of January 2022, version 7 of this library is one of the most
|
||||
performant LRU cache implementations in JavaScript.
|
||||
|
||||
Benchmarks can be extremely difficult to get right. In
|
||||
particular, the performance of set/get/delete operations on
|
||||
objects will vary _wildly_ depending on the type of key used. V8
|
||||
is highly optimized for objects with keys that are short strings,
|
||||
especially integer numeric strings. Thus any benchmark which
|
||||
tests _solely_ using numbers as keys will tend to find that an
|
||||
object-based approach performs the best.
|
||||
|
||||
Note that coercing _anything_ to strings to use as object keys is
|
||||
unsafe, unless you can be 100% certain that no other type of
|
||||
value will be used. For example:
|
||||
|
||||
```js
|
||||
const myCache = {}
|
||||
const set = (k, v) => (myCache[k] = v)
|
||||
const get = k => myCache[k]
|
||||
|
||||
set({}, 'please hang onto this for me')
|
||||
set('[object Object]', 'oopsie')
|
||||
```
|
||||
|
||||
Also beware of "Just So" stories regarding performance. Garbage
|
||||
collection of large (especially: deep) object graphs can be
|
||||
incredibly costly, with several "tipping points" where it
|
||||
increases exponentially. As a result, putting that off until
|
||||
later can make it much worse, and less predictable. If a library
|
||||
performs well, but only in a scenario where the object graph is
|
||||
kept shallow, then that won't help you if you are using large
|
||||
objects as keys.
|
||||
|
||||
In general, when attempting to use a library to improve
|
||||
performance (such as a cache like this one), it's best to choose
|
||||
an option that will perform well in the sorts of scenarios where
|
||||
you'll actually use it.
|
||||
|
||||
This library is optimized for repeated gets and minimizing
|
||||
eviction time, since that is the expected need of a LRU. Set
|
||||
operations are somewhat slower on average than a few other
|
||||
options, in part because of that optimization. It is assumed
|
||||
that you'll be caching some costly operation, ideally as rarely
|
||||
as possible, so optimizing set over get would be unwise.
|
||||
|
||||
If performance matters to you:
|
||||
|
||||
1. If it's at all possible to use small integer values as keys,
|
||||
and you can guarantee that no other types of values will be
|
||||
used as keys, then do that, and use a cache such as
|
||||
[lru-fast](https://npmjs.com/package/lru-fast), or
|
||||
[mnemonist's
|
||||
LRUCache](https://yomguithereal.github.io/mnemonist/lru-cache)
|
||||
which uses an Object as its data store.
|
||||
|
||||
2. Failing that, if at all possible, use short non-numeric
|
||||
strings (ie, less than 256 characters) as your keys, and use
|
||||
[mnemonist's
|
||||
LRUCache](https://yomguithereal.github.io/mnemonist/lru-cache).
|
||||
|
||||
3. If the types of your keys will be anything else, especially
|
||||
long strings, strings that look like floats, objects, or some
|
||||
mix of types, or if you aren't sure, then this library will
|
||||
work well for you.
|
||||
|
||||
If you do not need the features that this library provides
|
||||
(like asynchronous fetching, a variety of TTL staleness
|
||||
options, and so on), then [mnemonist's
|
||||
LRUMap](https://yomguithereal.github.io/mnemonist/lru-map) is
|
||||
a very good option, and just slightly faster than this module
|
||||
(since it does considerably less).
|
||||
|
||||
4. Do not use a `dispose` function, size tracking, or especially
|
||||
ttl behavior, unless absolutely needed. These features are
|
||||
convenient, and necessary in some use cases, and every attempt
|
||||
has been made to make the performance impact minimal, but it
|
||||
isn't nothing.
|
||||
|
||||
## Testing
|
||||
|
||||
When writing tests that involve TTL-related functionality, note
|
||||
that this module creates an internal reference to the global
|
||||
`performance` or `Date` objects at import time. If you import it
|
||||
statically at the top level, those references cannot be mocked or
|
||||
overridden in your test environment.
|
||||
|
||||
To avoid this, dynamically import the package within your tests
|
||||
so that the references are captured after your mocks are applied.
|
||||
For example:
|
||||
|
||||
```ts
|
||||
// ❌ Not recommended
|
||||
import { LRUCache } from 'lru-cache'
|
||||
// mocking timers, e.g. jest.useFakeTimers()
|
||||
|
||||
// ✅ Recommended for TTL tests
|
||||
// mocking timers, e.g. jest.useFakeTimers()
|
||||
const { LRUCache } = await import('lru-cache')
|
||||
```
|
||||
|
||||
This ensures that your mocked timers or time sources are
|
||||
respected when testing TTL behavior.
|
||||
|
||||
Additionally, you can pass in a `perf` option when creating your
|
||||
LRUCache instance. This option accepts any object with a `now`
|
||||
method that returns a number.
|
||||
|
||||
For example, this would be a very bare-bones time-mocking system
|
||||
you could use in your tests, without any particular test
|
||||
framework:
|
||||
|
||||
```ts
|
||||
import { LRUCache } from 'lru-cache'
|
||||
|
||||
let myClockTime = 0
|
||||
|
||||
const cache = new LRUCache<string>({
|
||||
max: 10,
|
||||
ttl: 1000,
|
||||
perf: {
|
||||
now: () => myClockTime,
|
||||
},
|
||||
})
|
||||
|
||||
// run tests, updating myClockTime as needed
|
||||
```
|
||||
|
||||
## Breaking Changes in Version 7
|
||||
|
||||
This library changed to a different algorithm and internal data
|
||||
structure in version 7, yielding significantly better
|
||||
performance, albeit with some subtle changes as a result.
|
||||
|
||||
If you were relying on the internals of LRUCache in version 6 or
|
||||
before, it probably will not work in version 7 and above.
|
||||
|
||||
## Breaking Changes in Version 8
|
||||
|
||||
- The `fetchContext` option was renamed to `context`, and may no
|
||||
longer be set on the cache instance itself.
|
||||
- Rewritten in TypeScript, so pretty much all the types moved
|
||||
around a lot.
|
||||
- The AbortController/AbortSignal polyfill was removed. For this
|
||||
reason, **Node version 16.14.0 or higher is now required**.
|
||||
- Internal properties were moved to actual private class
|
||||
properties.
|
||||
- Keys and values must not be `null` or `undefined`.
|
||||
- Minified export available at `'lru-cache/min'`, for both CJS
|
||||
and MJS builds.
|
||||
|
||||
## Breaking Changes in Version 9
|
||||
|
||||
- Named export only, no default export.
|
||||
- AbortController polyfill returned, albeit with a warning when
|
||||
used.
|
||||
|
||||
## Breaking Changes in Version 10
|
||||
|
||||
- `cache.fetch()` return type is now `Promise<V | undefined>`
|
||||
instead of `Promise<V | void>`. This is an irrelevant change
|
||||
practically speaking, but can require changes for TypeScript
|
||||
users.
|
||||
|
||||
For more info, see the [change log](CHANGELOG.md).
|
||||
+1323
File diff suppressed because it is too large
Load Diff
+1
File diff suppressed because one or more lines are too long
+1589
File diff suppressed because it is too large
Load Diff
+1
File diff suppressed because one or more lines are too long
+2
File diff suppressed because one or more lines are too long
+7
File diff suppressed because one or more lines are too long
+3
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"type": "commonjs"
|
||||
}
|
||||
+1323
File diff suppressed because it is too large
Load Diff
+1
File diff suppressed because one or more lines are too long
+1585
File diff suppressed because it is too large
Load Diff
+1
File diff suppressed because one or more lines are too long
+2
File diff suppressed because one or more lines are too long
+7
File diff suppressed because one or more lines are too long
+3
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"type": "module"
|
||||
}
|
||||
+101
@@ -0,0 +1,101 @@
|
||||
{
|
||||
"name": "lru-cache",
|
||||
"description": "A cache object that deletes the least-recently-used items.",
|
||||
"version": "11.2.6",
|
||||
"author": "Isaac Z. Schlueter <i@izs.me>",
|
||||
"keywords": [
|
||||
"mru",
|
||||
"lru",
|
||||
"cache"
|
||||
],
|
||||
"sideEffects": false,
|
||||
"scripts": {
|
||||
"build": "npm run prepare",
|
||||
"prepare": "tshy && bash fixup.sh",
|
||||
"pretest": "npm run prepare",
|
||||
"presnap": "npm run prepare",
|
||||
"test": "tap",
|
||||
"snap": "tap",
|
||||
"preversion": "npm test",
|
||||
"postversion": "npm publish",
|
||||
"prepublishOnly": "git push origin --follow-tags",
|
||||
"format": "prettier --write .",
|
||||
"typedoc": "typedoc --tsconfig ./.tshy/esm.json ./src/*.ts",
|
||||
"benchmark-results-typedoc": "bash scripts/benchmark-results-typedoc.sh",
|
||||
"prebenchmark": "npm run prepare",
|
||||
"benchmark": "make -C benchmark",
|
||||
"preprofile": "npm run prepare",
|
||||
"profile": "make -C benchmark profile"
|
||||
},
|
||||
"main": "./dist/commonjs/index.min.js",
|
||||
"types": "./dist/commonjs/index.d.ts",
|
||||
"tshy": {
|
||||
"exports": {
|
||||
"./raw": "./src/index.ts",
|
||||
".": {
|
||||
"import": {
|
||||
"types": "./dist/esm/index.d.ts",
|
||||
"default": "./dist/esm/index.min.js"
|
||||
},
|
||||
"require": {
|
||||
"types": "./dist/commonjs/index.d.ts",
|
||||
"default": "./dist/commonjs/index.min.js"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+ssh://git@github.com/isaacs/node-lru-cache.git"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^24.3.0",
|
||||
"benchmark": "^2.1.4",
|
||||
"esbuild": "^0.25.9",
|
||||
"marked": "^4.2.12",
|
||||
"mkdirp": "^3.0.1",
|
||||
"prettier": "^3.6.2",
|
||||
"tap": "^21.1.0",
|
||||
"tshy": "^3.0.2",
|
||||
"typedoc": "^0.28.12"
|
||||
},
|
||||
"license": "BlueOak-1.0.0",
|
||||
"files": [
|
||||
"dist"
|
||||
],
|
||||
"engines": {
|
||||
"node": "20 || >=22"
|
||||
},
|
||||
"tap": {
|
||||
"node-arg": [
|
||||
"--expose-gc"
|
||||
],
|
||||
"plugin": [
|
||||
"@tapjs/clock"
|
||||
]
|
||||
},
|
||||
"exports": {
|
||||
"./raw": {
|
||||
"import": {
|
||||
"types": "./dist/esm/index.d.ts",
|
||||
"default": "./dist/esm/index.js"
|
||||
},
|
||||
"require": {
|
||||
"types": "./dist/commonjs/index.d.ts",
|
||||
"default": "./dist/commonjs/index.js"
|
||||
}
|
||||
},
|
||||
".": {
|
||||
"import": {
|
||||
"types": "./dist/esm/index.d.ts",
|
||||
"default": "./dist/esm/index.min.js"
|
||||
},
|
||||
"require": {
|
||||
"types": "./dist/commonjs/index.d.ts",
|
||||
"default": "./dist/commonjs/index.min.js"
|
||||
}
|
||||
}
|
||||
},
|
||||
"type": "module",
|
||||
"module": "./dist/esm/index.min.js"
|
||||
}
|
||||
+82
@@ -0,0 +1,82 @@
|
||||
{
|
||||
"name": "cacache",
|
||||
"version": "20.0.3",
|
||||
"cache-version": {
|
||||
"content": "2",
|
||||
"index": "5"
|
||||
},
|
||||
"description": "Fast, fault-tolerant, cross-platform, disk-based, data-agnostic, content-addressable cache.",
|
||||
"main": "lib/index.js",
|
||||
"files": [
|
||||
"bin/",
|
||||
"lib/"
|
||||
],
|
||||
"scripts": {
|
||||
"test": "tap",
|
||||
"snap": "tap",
|
||||
"coverage": "tap",
|
||||
"test-docker": "docker run -it --rm --name pacotest -v \"$PWD\":/tmp -w /tmp node:latest npm test",
|
||||
"lint": "npm run eslint",
|
||||
"npmclilint": "npmcli-lint",
|
||||
"lintfix": "npm run eslint -- --fix",
|
||||
"postsnap": "npm run lintfix --",
|
||||
"postlint": "template-oss-check",
|
||||
"posttest": "npm run lint",
|
||||
"template-oss-apply": "template-oss-apply --force",
|
||||
"eslint": "eslint \"**/*.{js,cjs,ts,mjs,jsx,tsx}\""
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://github.com/npm/cacache.git"
|
||||
},
|
||||
"keywords": [
|
||||
"cache",
|
||||
"caching",
|
||||
"content-addressable",
|
||||
"sri",
|
||||
"sri hash",
|
||||
"subresource integrity",
|
||||
"cache",
|
||||
"storage",
|
||||
"store",
|
||||
"file store",
|
||||
"filesystem",
|
||||
"disk cache",
|
||||
"disk storage"
|
||||
],
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"@npmcli/fs": "^5.0.0",
|
||||
"fs-minipass": "^3.0.0",
|
||||
"glob": "^13.0.0",
|
||||
"lru-cache": "^11.1.0",
|
||||
"minipass": "^7.0.3",
|
||||
"minipass-collect": "^2.0.1",
|
||||
"minipass-flush": "^1.0.5",
|
||||
"minipass-pipeline": "^1.2.4",
|
||||
"p-map": "^7.0.2",
|
||||
"ssri": "^13.0.0",
|
||||
"unique-filename": "^5.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@npmcli/eslint-config": "^6.0.1",
|
||||
"@npmcli/template-oss": "4.28.0",
|
||||
"tap": "^16.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": "^20.17.0 || >=22.9.0"
|
||||
},
|
||||
"templateOSS": {
|
||||
"//@npmcli/template-oss": "This file is partially managed by @npmcli/template-oss. Edits may be overwritten.",
|
||||
"windowsCI": false,
|
||||
"version": "4.28.0",
|
||||
"publish": "true"
|
||||
},
|
||||
"author": "GitHub Inc.",
|
||||
"tap": {
|
||||
"nyc-arg": [
|
||||
"--exclude",
|
||||
"tap-snapshots/**"
|
||||
]
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user