This adds the HTMLRewriter-based bookmark unfurler to the demo server.
It moves the unfurler into worker-shared, and adds some better shared
error handling across our workers.
I removed the fallback bookmark fetcher where we try and fetch websites
locally. This will almost never work, as it requires sites to set public
CORS.
### Change type
- [x] `other`
This PR adds a `GET /api/unfurl?url=blahblah` endpoint to our worker.
I tried out the existing cheerio implementation but it added 300kb to
our worker bundle in the end, due to transitive dependencies.
So I implemented the same logic with cloudflare's sanctioned streaming
HTML parser `HTMLRewriter` and it seems to work fine.
I also made the vscode extension do its fetching locally (from the node
process so it's not bound by security policies), retaining the cheerio
version for that. At the same time I fixed a bug in the RPC layer that
was preventing unfurled metadata from loading correctly.
In a few months we can retire the bookmark-extractor app by just
deleting it in the vercel dashboard.
### Change Type
<!-- ❗ Please select a 'Type' label ❗️ -->
- [ ] `feature` — New feature
- [x] `improvement` — Product improvement
- [ ] `api` — API change
- [ ] `bugfix` — Bug fix
- [ ] `other` — Changes that don't affect SDK users, e.g. internal or
.com changes
### Test Plan
1. Add a step-by-step description of how to test your PR here.
2.
- [ ] Unit Tests
- [ ] End to end tests
### Release Notes
- Do link unfurling on the same subdomain as all our other api
endpoints.
Previously, we had the `ae-forgotten-export` rule from api-extractor
disabled. This rule makes sure that everything that's referred to in the
public API is actually exported. There are more details on the rule
[here](https://api-extractor.com/pages/messages/ae-forgotten-export/),
but not exporting public API entires is bad because they're hard to
document and can't be typed/called from consumer code. For us, the big
effect is that they don't appear in our docs at all.
This diff re-enables that rule. Now, if you introduce something new to
the public API but don't export it, your build will fail.
### Change Type
- [x] `docs` — Changes to the documentation, examples, or templates.
- [x] `improvement` — Improving existing features
Typescript's type aliases (`type X = thing`) can refer to basically
anything, which makes it hard to write an automatic document formatter
for them. Interfaces on the other hand are only object, so they play
much nicer with docs. Currently, object-flavoured type aliases don't
really get expanded at all on our docs site, which means we have a bunch
of docs content that's not shown on the site.
This diff introduces a lint rule that forces `interface X {foo: bar}`s
instead of `type X = {foo: bar}` where possible, as it results in a much
better documentation experience:
Before:
<img width="437" alt="Screenshot 2024-05-22 at 15 24 13"
src="https://github.com/tldraw/tldraw/assets/1489520/32606fd1-6832-4a1e-aa5f-f0534d160c92">
After:
<img width="431" alt="Screenshot 2024-05-22 at 15 33 01"
src="https://github.com/tldraw/tldraw/assets/1489520/4e0d59ee-c38e-4056-b9fd-6a7f15d28f0f">
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `docs` — Changes to the documentation, examples, or templates.
- [x] `improvement` — Improving existing features
Describe what your pull request does. If appropriate, add GIFs or images
showing the before and after.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `galaxy brain` — Architectural changes
### Test Plan
1. Add a step-by-step description of how to test your PR here.
2.
- [ ] Unit Tests
- [ ] End to end tests
### Release Notes
#### BREAKING CHANGES
- The `Migrations` type is now called `LegacyMigrations`.
- The serialized schema format (e.g. returned by
`StoreSchema.serialize()` and `Store.getSnapshot()`) has changed. You
don't need to do anything about it unless you were reading data directly
from the schema for some reason. In which case it'd be best to avoid
that in the future! We have no plans to change the schema format again
(this time was traumatic enough) but you never know.
- `compareRecordVersions` and the `RecordVersion` type have both
disappeared. There is no replacement. These were public by mistake
anyway, so hopefully nobody had been using it.
- `compareSchemas` is a bit less useful now. Our migrations system has
become a little fuzzy to allow for simpler UX when adding/removing
custom extensions and 3rd party dependencies, and as a result we can no
longer compare serialized schemas in any rigorous manner. You can rely
on this function to return `0` if the schemas are the same. Otherwise it
will return `-1` if the schema on the right _seems_ to be newer than the
schema on the left, but it cannot guarantee that in situations where
migration sequences have been removed over time (e.g. if you remove one
of the builtin tldraw shapes).
Generally speaking, the best way to check schema compatibility now is to
call `store.schema.getMigrationsSince(persistedSchema)`. This will throw
an error if there is no upgrade path from the `persistedSchema` to the
current version.
- `defineMigrations` has been deprecated and will be removed in a future
release. For upgrade instructions see
https://tldraw.dev/docs/persistence#Updating-legacy-shape-migrations-defineMigrations
- `migrate` has been removed. Nobody should have been using this but if
you were you'll need to find an alternative. For migrating tldraw data,
you should stick to using `schema.migrateStoreSnapshot` and, if you are
building a nuanced sync engine that supports some amount of backwards
compatibility, also feel free to use `schema.migratePersistedRecord`.
- the `Migration` type has changed. If you need the old one for some
reason it has been renamed to `LegacyMigration`. It will be removed in a
future release.
- the `Migrations` type has been renamed to `LegacyMigrations` and will
be removed in a future release.
- the `SerializedSchema` type has been augmented. If you need the old
version specifically you can use `SerializedSchemaV1`
---------
Co-authored-by: Steve Ruiz <steveruizok@gmail.com>
This PR adds a validation mode whereby previous known-to-be-valid values
can be used to speed up the validation process itself. At the same time
it enables us to do fine-grained equality checking on records much more
quickly than by using something like lodash isEqual, and using that we
can prevent triggering effects for record updates that don't actually
alter any values in the store.
Here's some preliminary perf testing of average time spent in
`store.put()` during some common interactions
| task | before (ms) | after (ms) |
| ---- | ---- | ---- |
| drawing lines | 0.0403 | 0.0214 |
| drawing boxes | 0.0408 | 0.0348 |
| translating lines | 0.0352 | 0.0042 |
| translating boxes | 0.0051 | 0.0032 |
| rotating lines | 0.0312 | 0.0065 |
| rotating boxes | 0.0053 | 0.0035 |
| brush selecting boxes | 0.0200 | 0.0232 |
| traversal with shapes | 0.0130 | 0.0108 |
| traversal without shapes | 0.0201 | 0.0173 |
**traversal** means moving the camera and pointer around the canvas
#### Discussion
At the scale of hundredths of a millisecond these .put operations are so
fast that even if they became literally instantaneous the change would
not be human perceptible. That said, there is an overall marked
improvement here. Especially for dealing with draw shapes.
These figures are also mostly in line with expectations, aside from a
couple of things:
- I don't understand why the `brush selecting boxes` task got slower
after the change.
- I don't understand why the `traversal` tasks are slower than the
`translating boxes` task, both before and after. I would expect that
.putting shape records would be much slower than .putting pointer/camera
records (since the latter have fewer and simpler properties)
### Change Type
- [x] `patch` — Bug fix
### Test Plan
1. Add a step-by-step description of how to test your PR here.
2.
- [ ] Unit Tests
- [ ] End to end tests
### Release Notes
- Add a brief release note for your PR here.
Currently, we type our fractional index keys as `string` and don't have
any validation for them. I'm touching some of this code for my work on
line handles and wanted to change that:
- fractional indexes are now `IndexKey`s, not `string`s. `IndexKey`s
have a brand property so can't be used interchangeably with strings
(like our IDs)
- There's a new `T.indexKey` validator which we can use in our
validations to make sure we don't end up with nonsense keys.
This PR is part of a series - please don't merge it until the things
before it have landed!
1. #2827 (you are here)
2. #2831
3. #2793
4. #2841
5. #2845
### Change Type
- [x] `patch` — Bug fix
### Test Plan
1. Mostly relying on unit & end to end tests here - no user facing
changes.
- [x] Unit Tests
Adds validation for urls we use for our shapes and assets. This PR
includes a migration so we should check that existing rooms still load
correctly. There might be some that won't, but that means that they had
invalid url set.
### Change Type
- [x] `patch` — Bug fix
- [ ] `minor` — New feature
- [ ] `major` — Breaking change
- [ ] `dependencies` — Changes to package dependencies[^1]
- [ ] `documentation` — Changes to the documentation only[^2]
- [ ] `tests` — Changes to any test code only[^2]
- [ ] `internal` — Any other changes that don't affect the published
package[^2]
- [ ] I don't know
[^1]: publishes a `patch` release, for devDependencies use `internal`
[^2]: will not publish a new version
### Test Plan
1. Existing rooms should still load correctly (there should be no
validation errors).
2. Adding new images and videos should also work (test both local and
multiplayer rooms as they handle assets differently).
- [x] Unit Tests
- [ ] End to end tests
### Release Notes
- Add validation to urls.
---------
Co-authored-by: alex <alex@dytry.ch>
This PR moves code between our packages so that:
- @tldraw/editor is a “core” library with the engine and canvas but no
shapes, tools, or other things
- @tldraw/tldraw contains everything particular to the experience we’ve
built for tldraw
At first look, this might seem like a step away from customization and
configuration, however I believe it greatly increases the configuration
potential of the @tldraw/editor while also providing a more accurate
reflection of what configuration options actually exist for
@tldraw/tldraw.
## Library changes
@tldraw/editor re-exports its dependencies and @tldraw/tldraw re-exports
@tldraw/editor.
- users of @tldraw/editor WITHOUT @tldraw/tldraw should almost always
only import things from @tldraw/editor.
- users of @tldraw/tldraw should almost always only import things from
@tldraw/tldraw.
- @tldraw/polyfills is merged into @tldraw/editor
- @tldraw/indices is merged into @tldraw/editor
- @tldraw/primitives is merged mostly into @tldraw/editor, partially
into @tldraw/tldraw
- @tldraw/file-format is merged into @tldraw/tldraw
- @tldraw/ui is merged into @tldraw/tldraw
Many (many) utils and other code is moved from the editor to tldraw. For
example, embeds now are entirely an feature of @tldraw/tldraw. The only
big chunk of code left in core is related to arrow handling.
## API Changes
The editor can now be used without tldraw's assets. We load them in
@tldraw/tldraw instead, so feel free to use whatever fonts or images or
whatever that you like with the editor.
All tools and shapes (except for the `Group` shape) are moved to
@tldraw/tldraw. This includes the `select` tool.
You should use the editor with at least one tool, however, so you now
also need to send in an `initialState` prop to the Editor /
<TldrawEditor> component indicating which state the editor should begin
in.
The `components` prop now also accepts `SelectionForeground`.
The complex selection component that we use for tldraw is moved to
@tldraw/tldraw. The default component is quite basic but can easily be
replaced via the `components` prop. We pass down our tldraw-flavored
SelectionFg via `components`.
Likewise with the `Scribble` component: the `DefaultScribble` no longer
uses our freehand tech and is a simple path instead. We pass down the
tldraw-flavored scribble via `components`.
The `ExternalContentManager` (`Editor.externalContentManager`) is
removed and replaced with a mapping of types to handlers.
- Register new content handlers with
`Editor.registerExternalContentHandler`.
- Register new asset creation handlers (for files and URLs) with
`Editor.registerExternalAssetHandler`
### Change Type
- [x] `major` — Breaking change
### Test Plan
- [x] Unit Tests
- [x] End to end tests
### Release Notes
- [@tldraw/editor] lots, wip
- [@tldraw/ui] gone, merged to tldraw/tldraw
- [@tldraw/polyfills] gone, merged to tldraw/editor
- [@tldraw/primitives] gone, merged to tldraw/editor / tldraw/tldraw
- [@tldraw/indices] gone, merged to tldraw/editor
- [@tldraw/file-format] gone, merged to tldraw/tldraw
---------
Co-authored-by: alex <alex@dytry.ch>
This PR adds a `meta` property to shapes and other records.
It adds it to:
- asset
- camera
- document
- instance
- instancePageState
- instancePresence
- page
- pointer
- rootShape
## Setting meta
This data can generally be added wherever you would normally update the
corresponding record.
An exception exists for shapes, which can be updated using a partial of
the `meta` in the same way that we update shapes with a partial of
`props`.
```ts
this.updateShapes([{
id: myShape.id,
type: "geo",
meta: {
nemesis: "steve",
special: true
}
])
```
## `Editor.getInitialMetaForShape`
The `Editor.getInitialMetaForShape` method is kind of a hack to set the
initial meta property for newly created shapes. You can set it
externally. Escape hatch!
### Change Type
- [x] `minor` — New feature
### Test Plan
todo
- [ ] Unit Tests (todo)
### Release Notes
- todo
Removes `propsForNextShape` and replaces it with the new styles API.
Changes in here:
- New custom style example
- `setProp` is now `setStyle` and takes a `StyleProp` instead of a
string
- `Editor.props` and `Editor.opacity` are now `Editor.sharedStyles` and
`Editor.sharedOpacity`
- They return an object that flags mixed vs shared types instead of
using null to signal mixed types
- `Editor.styles` returns a `SharedStyleMap` - keyed on `StyleProp`
instead of `string`
- `StateNode.shapeType` is now the shape util rather than just a string.
This lets us pull the styles from the shape type directly.
- `color` is no longer a core part of the editor set on the shape
parent. Individual child shapes have to use color directly.
- `propsForNextShape` is now `stylesForNextShape`
- `InstanceRecordType` is created at runtime in the same way
`ShapeRecordType` is. This is so it can pull style validators out of
shape defs for `stylesForNextShape`
- Shape type are now defined by their props rather than having separate
validators & type defs
### Change Type
- [x] `major` — Breaking change
### Test Plan
1. Big time regression testing around styles!
2. Check UI works as intended for all shape/style/tool combos
- [x] Unit Tests
- [ ] End to end tests
### Release Notes
-
---------
Co-authored-by: Steve Ruiz <steveruizok@gmail.com>
Based on #1549, but with a lot of code-structure related changes backed
out. Shape schemas are still defined in tlschemas with this diff.
Couple differences between this and #1549:
- This tightens up the relationship between store schemas and editor
schemas a bit
- Reduces the number of places we need to remember to include core
shapes
- Only `<TLdrawEditor />` sets default shapes by default. If you're
doing something funky with lower-level APIs, you need to specify
`defaultShapes` manually
- Replaces `validator` with `props` for shapes
### Change Type
- [x] `major` — Breaking Change
### Test Plan
1. Add a step-by-step description of how to test your PR here.
2.
- [x] Unit Tests
- [ ] Webdriver tests
### Release Notes
[dev-facing, notes to come]
This PR renames the @tldraw/tlvalidate package to @tldraw/validate.
### Change Type
- [x] `major` — Breaking Change
### Release Notes
- Rename tlvalidate to validate
2023-06-03 08:27:44 +00:00
Renamed from packages/tlvalidate/api-report.md (Browse further)
