Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V7 #97

Merged
merged 36 commits into from
Jan 14, 2025
Merged

V7 #97

Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
b8d8235
v7 wip
mafintosh Nov 5, 2024
0e13e24
missing getter on CoreTracker
chm-diederichs Nov 8, 2024
a188a6b
forward user preload to core
chm-diederichs Nov 8, 2024
9cc3217
add back sessions and namespaces
mafintosh Nov 8, 2024
36d34f5
missing encryption key
chm-diederichs Nov 8, 2024
fc1661f
must wait for root in sessions
mafintosh Nov 8, 2024
babd303
or null
mafintosh Nov 8, 2024
ba38ae9
decode keys
mafintosh Nov 11, 2024
643c93a
add some compat fields
mafintosh Nov 11, 2024
90df779
wait should default to true
chm-diederichs Nov 11, 2024
71cf3e1
root core store closes all cores
chm-diederichs Nov 11, 2024
5973fca
switch to rocksdb branch
mafintosh Nov 12, 2024
ea2ff8f
update package.json
chm-diederichs Nov 14, 2024
e01bb35
primaryKey can be optionally passed in
chm-diederichs Nov 15, 2024
d7f4d41
global cache
mafintosh Nov 27, 2024
c03852b
support setNamespace for mutating the ns if you know what you are doing
mafintosh Nov 27, 2024
3f615ea
Fix namespace before ready (#98)
chm-diederichs Nov 28, 2024
1ccf2ff
add week ref test
mafintosh Nov 28, 2024
6e6c052
kill bootstrap and move to simple from key (#100)
mafintosh Dec 3, 2024
fce5023
test for preready sessions
mafintosh Dec 3, 2024
d47b41e
missing close
mafintosh Dec 3, 2024
4f9e317
Session manager (#101)
mafintosh Dec 3, 2024
a80ff95
limit preload usage (#102)
mafintosh Dec 4, 2024
0f3995f
Support passing encryption in preload (#103)
chm-diederichs Dec 13, 2024
1567ca0
pre tag with 7
mafintosh Dec 31, 2024
5dc5e16
cores.get supports buffers
mafintosh Jan 6, 2025
a11281a
Fix get (#104)
HDegroote Jan 6, 2025
b6a35be
we fixed the wrong typo
mafintosh Jan 6, 2025
5f7544d
add watch/unwatch for monitoring core flows
mafintosh Jan 6, 2025
aa90de5
pin to specific hash tmp
mafintosh Jan 9, 2025
c4b989c
v7 bump to hypercore 11 (#105)
chm-diederichs Jan 14, 2025
55d3744
tweak core setup to use alias and fix open
mafintosh Jan 14, 2025
e9bc5db
test aliases
mafintosh Jan 14, 2025
ac7ff60
docs
mafintosh Jan 14, 2025
bcf076d
also decode dkey since public
mafintosh Jan 14, 2025
052e7d8
basic replication test
mafintosh Jan 14, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
v7 wip
mafintosh committed Nov 5, 2024
commit b8d8235f77e35f0c3349c2d0fcc3f585832e8570
21 changes: 0 additions & 21 deletions LICENSE

This file was deleted.

104 changes: 6 additions & 98 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,109 +1,17 @@
# Corestore
# corestore2

### [See the full API docs at docs.holepunch.to](https://docs.holepunch.to/helpers/corestore)
WIP - nothing to see here

Corestore is a Hypercore factory that makes it easier to manage large collections of named Hypercores.

Corestore provides:
1. __Key Derivation__ - All writable Hypercore keys are derived from a single master key and a user-provided name.
2. __Session Handling__ - If a single Hypercore is loaded multiple times through the `get` method, the underlying resources will only be opened once (using Hypercore 10's new session feature). Once all sessions are closed, the resources will be released.
3. __Storage Management__ - Hypercores can be stored in any random-access-storage instance, where they will be keyed by their discovery keys.
4. __Namespacing__ - You can share a single Corestore instance between multiple applications or components without worrying about naming collisions by creating "namespaces" (e.g. `corestore.namespace('my-app').get({ name: 'main' })`)

### Installation
`npm install corestore`

### Usage
A corestore instance can be constructed with a random-access-storage module, a function that returns a random-access-storage module given a path, or a string. If a string is specified, it will be assumed to be a path to a local storage directory:
```js
const Corestore = require('corestore')

const store = new Corestore('./my-storage')
const core1 = store.get({ name: 'core-1' })
const core2 = store.get({ name: 'core-2' })
```

### API
#### `const store = new Corestore(storage, opts = {})`
Create a new Corestore instance.

`storage` can be either a random-access-storage module, a string, or a function that takes a path and returns an random-access-storage instance.

Opts include:
```js
{
inflightRange: null // Advanced option. Forwarded to the Hypercores created by corestore.get(...)
}
npm install corestore2
```

#### `const core = store.get(key | { name: 'a-name', exclusive, ...hypercoreOpts})`
Loads a Hypercore, either by name (if the `name` option is provided), or from the provided key (if the first argument is a Buffer or String with hex/z32 key, or if the `key` options is set).

If that Hypercore has previously been loaded, subsequent calls to `get` will return a new Hypercore session on the existing core.

If you set the `exclusive` option and you are opening a writable session it will wait for all other exclusive writable to close before
opening the Hypercore effectively meaning any op on the core will wait until its exclusive.

All other options besides `name` and `key` and `exclusive` will be forwarded to the Hypercore constructor.

#### `const stream = store.replicate(optsOrStream)`
Creates a replication stream that's capable of replicating all Hypercores that are managed by the Corestore, assuming the remote peer has the correct capabilities.

`opts` will be forwarded to Hypercore's `replicate` function.

Corestore replicates in an "all-to-all" fashion, meaning that when replication begins, it will attempt to replicate every Hypercore that's currently loaded and in memory. These attempts will fail if the remote side doesn't have a Hypercore's capability -- Corestore replication does not exchange Hypercore keys.

If the remote side dynamically adds a new Hypercore to the replication stream, Corestore will load and replicate that core if possible.

Using [Hyperswarm](https://github.com/holepunchto/hyperswarm) you can easily replicate corestores
## Usage

``` js
const swarm = new Hyperswarm()

// join the relevant topic
swarm.join(...)

// simply pass the connection stream to corestore
swarm.on('connection', (connection) => store.replicate(connection))
```

#### `const store = store.namespace(name)`
Create a new namespaced Corestore. Namespacing is useful if you're going to be sharing a single Corestore instance between many applications or components, as it prevents name collisions.

Namespaces can be chained:
```js
const ns1 = store.namespace('a')
const ns2 = ns1.namespace('b')
const core1 = ns1.get({ name: 'main' }) // These will load different Hypercores
const core2 = ns2.get({ name: 'main' })
```

#### `const storeB = storeA.session(opts)`
Create a new Corestore that shares resources with the original, like cache, cores, replication streams, and storage, while optionally resetting the namespace, overriding `primaryKey`.
Useful when an application wants to accept an optional Corestore, but needs to maintain a predictable key derivation.

`opts` are the same as the constructor options:

```js
{
primaryKey, // Overrides the primaryKey for this session
namespace, // If set to null it will reset to the DEFAULT_NAMESPACE
detach: true, // By disabling this, closing the session will also close the store that created the session
inflightRange: null // Advanced option. Forwarded to the Hypercores created by `session.get(...)
}
const corestore2 = require('corestore2')
```

#### `await store.close()`
Fully close this Corestore instance.

#### `store.on('core-open', core)`
Emitted when the first session for a core is opened.
## License

*Note: This core may close at any time, so treat it as a weak reference*

#### `store.on('core-close', core)`
Emitted when the last session for a core is closed.

### License
MIT

21 changes: 21 additions & 0 deletions example.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import Corestore from './index.js'

const store = new Corestore('./store.db')

const core = store.get({ name: 'yo' })
// await core.close()

const store2 = new Corestore('./store2.db')

await core.ready()
await core.append('yo')

const clone = store2.get({ key: core.key })

const stream = store.replicate(true)
const stream2 = store2.replicate(false)

stream.pipe(stream2).pipe(stream)

await clone.ready()
console.log(core, clone)
Loading