tldraw/packages/store
David Sheldrick f15a8797f0
Independent instance state persistence (#1493)
This PR 

- Removes UserDocumentRecordType
  - moving isSnapMode to user preferences
  - moving isGridMode and isPenMode to InstanceRecordType
  - deleting the other properties which are no longer needed.
 
- Creates a separate pipeline for persisting instance state.
  
Previously the instance state records were stored alongside the document
state records, and in order to load the state for a particular instance
(in our case, a particular tab) you needed to pass the 'instanceId'
prop. This prop ended up totally pervading the public API and people ran
into all kinds of issues with it, e.g. using the same instance id in
multiple editor instances.

There was also an issue whereby it was hard for us to clean up old
instance state so the idb table ended up bloating over time.
  
This PR makes it so that rather than passing an instanceId, you load the
instance state yourself while creating the store. It provides tools to
make that easy.

- Undoes the assumption that we might have more than one instance's
state in the store.

- Like `document`, `instance` now has a singleton id
`instance:instance`.
- Page state ids and camera ids are no longer random, but rather derive
from the page they belong to. This is like having a foreign primary key
in SQL databases. It's something i'd love to support fully as part of
the RecordType/Store api.

Tests to do

- [x] Test Migrations
- [x] Test Store.listen filtering
- [x] Make type sets in Store public and readonly
- [x] Test RecordType.createId
- [x] Test Instance state snapshot loading/exporting
- [x] Manual test File I/O
- [x] Manual test Vscode extension with multiple tabs
- [x] Audit usages of store.query
- [x] Audit usages of changed types: InstanceRecordType, 'instance',
InstancePageStateRecordType, 'instance_page_state', 'user_document',
'camera', CameraRecordType, InstancePresenceRecordType,
'instance_presence'
- [x] Test user preferences
- [x] Manual test isSnapMode and isGridMode and isPenMode
- [ ] Test indexedDb functions
- [x] Add instanceId stuff back


### Change Type

- [x] `major` — Breaking Change


### Test Plan

1. Add a step-by-step description of how to test your PR here.
2.

- [ ] Unit Tests
- [ ] Webdriver tests

### Release Notes

- Add a brief release note for your PR here.
2023-06-05 14:11:07 +00:00
..
src Independent instance state persistence (#1493) 2023-06-05 14:11:07 +00:00
api-extractor.json Rename tlstore to store (#1507) 2023-06-03 08:59:04 +00:00
api-report.md Independent instance state persistence (#1493) 2023-06-05 14:11:07 +00:00
CHANGELOG.md Rename tlstore to store (#1507) 2023-06-03 08:59:04 +00:00
package.json Rename tlstore to store (#1507) 2023-06-03 08:59:04 +00:00
README.md Rename tlstore to store (#1507) 2023-06-03 08:59:04 +00:00
tsconfig.json Rename tlstore to store (#1507) 2023-06-03 08:59:04 +00:00

@tldraw/tlstore

tlstore is a library for creating and managing data.

In this library, a "record" is an object that is stored under a typed id.

tlstore is used by tldraw to store its data.

It is designed to be used with tlstate (@tldraw/tlstate).

Usage

First create types for your records.

interface Book extends BaseRecord<'book'> {
	title: string
	author: ID<Author>
	numPages: number
}

interface Author extends BaseRecord<'author'> {
	name: string
	isPseudonym: boolean
}

Then create your RecordType instances.

const Book = createRecordType<Book>('book')

const Author = createRecordType<Author>('author').withDefaultProperties(() => ({
	isPseudonym: false,
}))

Then create your RecordStore instance.

const store = new RecordStore<Book | Author>()

Then you can create records, add them to the store, update, and remove them.

const tolkeinId = Author.createCustomId('tolkein')

store.put([
	Author.create({
		id: jrrTolkeinId,
		name: 'J.R.R Tolkein',
	}),
])

store.update(tolkeinId, (author) => ({
	...author,
	name: 'DJJ Tolkz',
	isPseudonym: true,
}))

store.remove(tolkeinId)

API

RecordStore

The RecordStore class is the main class of the library.

const store = new RecordStore()

put(records: R[]): void

Add some records to the store. It's an error if they already exist.

const record = Author.create({
	name: 'J.R.R Tolkein',
	id: Author.createCustomId('tolkein'),
})

store.put([record])

update(id: ID<R>, updater: (record: R) => R): void

Update a record. To update multiple records at once, use the update method of the TypedRecordStore class.

const id = Author.createCustomId('tolkein')

store.update(id, (r) => ({ ...r, name: 'Jimmy Tolks' }))

remove(ids: ID<R>[]): void

Remove some records from the store via their ids.

const id = Author.createCustomId('tolkein')

store.remove([id])

get(id: ID<R>): R

Get the value of a store record by its id.

const id = Author.createCustomId('tolkein')

const result = store.get(id)

allRecords(): R[]

Get an array of all values in the store.

const results = store.allRecords()

clear(): void

Remove all records from the store.

store.clear()

has(id: ID<R>): boolean

Get whether the record store has an record stored under the given id.

const id = Author.createCustomId('tolkein')

const result = store.has(id)

serialize(filter?: (record: R) => boolean): RecordStoreSnapshot<R>

Opposite of deserialize. Creates a JSON payload from the record store.

const serialized = store.serialize()
const serialized = store.serialize((record) => record.name === 'J.R.R Tolkein')

deserialize(snapshot: RecordStoreSnapshot<R>): void

Opposite of serialize. Replace the store's current records with records as defined by a simple JSON structure into the stores.

const serialized = { ... }

store.deserialize(serialized)

listen(listener: ((entry: HistoryEntry) => void): () => void

Add a new listener to the store The store will call the function each time the history changes. Returns a function to remove the listener.

store.listen((entry) => doSomethingWith(entry))

mergeRemoteChanges(fn: () => void): void

Merge changes from a remote source without triggering listeners.

store.mergeRemoteChanges(() => {
	store.put(recordsFromRemoteSource)
})

createDerivationCache(name: string, derive: ((record: R) => R | undefined)): DerivationCache<R>

Create a new derivation cache.

const derivationCache = createDerivationCache('popular_authors', (record) => {
	return record.popularity > 62 ? record : undefined
})

RecordType

The RecordType class is used to define the structure of a record.

const recordType = new RecordType('author', () => ({ living: true }))

RecordType instances are most often created with createRecordType.

create(properties: Pick<R, RequiredProperties> & Omit<Partial<R>, RequiredProperties>): R

Create a new record of this type.

const record = recordType.create({ name: 'J.R.R Tolkein' })

clone(record: R): R

Clone a record of this type.

const record = recordType.create({ name: 'J.R.R Tolkein' })

const clone = recordType.clone(record)

createId(): ID<R>

Create an Id for a record of this type.

const id = recordType.createId()

createCustomId(id: string): ID<R>

Create a custom Id for a record of this type.

const id = recordType.createCustomId('tolkein')

isInstance

Check if a value is an instance of this record type.

const record = recordType.create({ name: 'J.R.R Tolkein' })

const result1 = recordType.isInstance(record) // true
const result2 = recordType.isInstance(someOtherRecord) // false

isId

Check if a value is an id for a record of this type.

const id = recordType.createCustomId('tolkein')

const result1 = recordType.isId(id) // true
const result2 = recordType.isId(someOtherId) // false

withDefaultProperties

Create a new record type with default properties.

const youngLivingAuthor = new RecordType('author', () => ({ age: 28, living: true }))

const oldDeadAuthor = recordType.withDefaultProperties({ age: 93, living: false })

RecordStoreQueries

TODO

Helpers

executeQuery

TODO

DerivationCache

The DerivationCache class is used to create a cache of derived records.

const derivationCache = new DerivationCache('popular_authors', (record) => {
	return record.popularity > 62 ? record : undefined
})

createRecordType

A helper used to create a new RecordType instance with no default properties.

const recordType = createRecordType('author'))

assertIdType

A helper used to assert that a value is an id for a record of a given type.

const id = recordType.createCustomId('tolkein')

assertIdType(id, recordType)

Types

ID

A type used to represent a record's id.

const id: ID<Author> = Author.createCustomId('tolkein')

BaseRecord

A BaseRecord is a record that has an id and a type. It is the base type for all records.

type AuthorRecord extends BaseRecord<"author"> {
  name: string
  age: number
  living: boolean
}

AllRecords

A helper to get the type of all records in a record store.

type AllAuthorRecords = AllRecords<RecordStore<Author>>

RecordsDiff

A diff describing the changes to a record.

CollectionDiff

A diff describing the changes to a collection.

License

The source code in this repository (as well as our 2.0+ distributions and releases) are currently licensed under Apache-2.0. These licenses are subject to change in our upcoming 2.0 release. If you are planning to use tldraw in a commercial product, please reach out at hello@tldraw.com.