Fixes culling for cases when another user would drag shapes inside your
viewport. We weren't correctly calculating the culling status for arrows
that might be bound to those shapes and also for shapes within dragged
in groups / frames.
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [ ] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [x] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [x] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [ ] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
### Test Plan
1. Open the same room in two browsers / tabs.
2. Have some shapes that are visible in one browser, but not the other.
3. Drag these shapes so that they are visible in the other browser as
well.
4. They should correctly get unculled.
5. Do this by dragging shapes that have arrows bound to them (arrows
should uncull), groups (shapes within them should uncull), frames.
- [x] Unit Tests
- [ ] End to end tests
### Release Notes
- Fix culling.
(this is a PR redo of https://github.com/tldraw/tldraw/pull/3424 which
got messed up a bit)
It doesn't quite feel like this is the right fix but it does solve the
issue. I was trying to see if `getShapeAtPoint` needed more work but the
further I went in that rabbit hole it seemed like I shouldn't touch that
code without causing a bunch of disruption at the moment.
Specifically, the code that does `Check labels first` in Editor.ts is a
little obscure (lines 4384-4397). It only checks a couple specifics
shapes (with certain combinations, i.e. a geo with "none" fill) _and_ it
doesn't check `hitLabels` which also maybe feels wrong? I tried
unraveling it but there's a lot of code relying on it at the moment to
mess with it in the stickies work.
(I was looking at https://github.com/tldraw/tldraw/pull/1910 and
https://github.com/tldraw/tldraw/pull/1806 for historical context fwiw)
Before:
https://github.com/tldraw/tldraw/assets/469604/b263a192-2085-4ffb-9e47-6e9c32abe1f9
After:
https://github.com/tldraw/tldraw/assets/469604/5b0b422b-dd5c-4593-9ac5-dec595923ea6
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [x] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [ ] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [x] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [ ] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
Something was bothering me a bit with the discussion around sqrt's being
slow. Looks like `Math.hypot` has a performance cost associated with it.
Looking at the Chromium source code:
https://chromium.googlesource.com/v8/v8/+/4.3.21/src/math.js?autodive=0%2F%2F#19
and
https://source.chromium.org/chromium/chromium/src/+/main:v8/src/builtins/math.tq;l=36?q=math&sq=&ss=chromium%2Fchromium%2Fsrc:v8%2Fsrc%2F
it looks like maybe we'd be avoiding the multiple arguments that can be
passed into Math.hypot which is maybe the source of the perf hit.
Also, interestingly in `math.tq` you can see it doing this funky sqrt
calculation: `Float64Sqrt((a / max) * (a / max) + (b / max) * (b / max))
* max` - I think that possibly is trying to avoid some overflow in some
cases with bigger numbers, but also possibly with a perf hit.
[edit]: OK, actually on Firefox, doing sqrt seems slower - but digging
more into this, it looks like doing `** 0.5` instead of `sqrt` is much
faster.
More related articles:
- https://stackoverflow.com/questions/71898044/why-is-math-hypot-so-slow
-
https://stackoverflow.com/questions/3764978/why-hypot-function-is-so-slow
-
https://www.reddit.com/r/javascript/comments/wk3e57/askjs_why_mathsqrt_is_so_slow_in_firefox/
[edit again!] looks like this is being fixed in the latest Chrome!
https://blog.seokho.dev/development/2024/03/18/V8-optimize-MathHypot.html
```
┌─────────┬───────┬─────────┬─────────┬─────────┬────────┐
│ (index) │ Cold │ Slowest │ Fastest │ Average │ Total │
├─────────┼───────┼─────────┼─────────┼─────────┼────────┤
│ old │ 13.39 │ 10.07 │ 9.69 │ 9.98 │ 998.57 │
│ sqrt │ 8.19 │ 6.66 │ 6.61 │ 6.67 │ 667.6 │
│ pow 0.5 │ 1.89 │ 0.28 │ 0.28 │ 0.3 │ 29.79 │
│ new │ 1.64 │ 0.28 │ 0.28 │ 0.29 │ 28.95 │
└─────────┴───────┴─────────┴─────────┴─────────┴────────┘
```
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [x] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [ ] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [ ] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [x] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
Adds RBush to handle spatial querying. We use it for:
- Culling. Helps a lot with panning as we don't have to compute the
culled shapes from scratch. Instead we just query rbush again. It makes
culling quite granular: spatial index updates when shapes change
(additions, removals, changes to bounds), visible shapes depends on
that, but also updates when the viewport page bound change, culled
shapes then depend on that but also change with selections changes. The
api stayed the same, which is great since the fuzz tests can stay as
they are.
- Brushing
- Erasing
- Scribble brushing
- Getting shapes at point (for example, when updating the hover id)
This improves performance of all of those operations. I might have
missed some places where this might also be useful.
### Erasing before (Test on my old ipad)
https://github.com/tldraw/tldraw/assets/2523721/edb9c004-a44a-4779-b2d0-98617b057314
### Erasing after
https://github.com/tldraw/tldraw/assets/2523721/8f8367fd-fa8e-4963-ba13-720c5f0c2da5
### Creating an arrow before
https://github.com/tldraw/tldraw/assets/2523721/4068f8b7-f7b8-4826-83f2-083b1f3783bc
### After (much better, but still bad)
https://github.com/tldraw/tldraw/assets/2523721/11af6be6-01d8-4740-bf15-896e2dd31dd6
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [ ] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [x] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [ ] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [x] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
---------
Co-authored-by: Steve Ruiz <steveruizok@gmail.com>
Describe what your pull request does. If appropriate, add GIFs or images
showing the before and after.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `galaxy brain` — Architectural changes
### Test Plan
1. Add a step-by-step description of how to test your PR here.
2.
- [ ] Unit Tests
- [ ] End to end tests
### Release Notes
#### BREAKING CHANGES
- The `Migrations` type is now called `LegacyMigrations`.
- The serialized schema format (e.g. returned by
`StoreSchema.serialize()` and `Store.getSnapshot()`) has changed. You
don't need to do anything about it unless you were reading data directly
from the schema for some reason. In which case it'd be best to avoid
that in the future! We have no plans to change the schema format again
(this time was traumatic enough) but you never know.
- `compareRecordVersions` and the `RecordVersion` type have both
disappeared. There is no replacement. These were public by mistake
anyway, so hopefully nobody had been using it.
- `compareSchemas` is a bit less useful now. Our migrations system has
become a little fuzzy to allow for simpler UX when adding/removing
custom extensions and 3rd party dependencies, and as a result we can no
longer compare serialized schemas in any rigorous manner. You can rely
on this function to return `0` if the schemas are the same. Otherwise it
will return `-1` if the schema on the right _seems_ to be newer than the
schema on the left, but it cannot guarantee that in situations where
migration sequences have been removed over time (e.g. if you remove one
of the builtin tldraw shapes).
Generally speaking, the best way to check schema compatibility now is to
call `store.schema.getMigrationsSince(persistedSchema)`. This will throw
an error if there is no upgrade path from the `persistedSchema` to the
current version.
- `defineMigrations` has been deprecated and will be removed in a future
release. For upgrade instructions see
https://tldraw.dev/docs/persistence#Updating-legacy-shape-migrations-defineMigrations
- `migrate` has been removed. Nobody should have been using this but if
you were you'll need to find an alternative. For migrating tldraw data,
you should stick to using `schema.migrateStoreSnapshot` and, if you are
building a nuanced sync engine that supports some amount of backwards
compatibility, also feel free to use `schema.migratePersistedRecord`.
- the `Migration` type has changed. If you need the old one for some
reason it has been renamed to `LegacyMigration`. It will be removed in a
future release.
- the `Migrations` type has been renamed to `LegacyMigrations` and will
be removed in a future release.
- the `SerializedSchema` type has been augmented. If you need the old
version specifically you can use `SerializedSchemaV1`
---------
Co-authored-by: Steve Ruiz <steveruizok@gmail.com>
There was a bug that could occur if you pinched while using the hand
tool, where on pinch end the hand tool would slide the camera based on
the pinching velocity. The fix is to cancel out any velocity while
pinching.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `bugfix` — Bug fix
### Test Plan
On mobile...
1. Select the hand tool.
2. Begin a pinch
3. Stop the pinch
4. The camera should stay where it is
### Release Notes
- Fixed a bug that could occur while pinching with the hand tool
selected.
This PR adds a computed cache for masked shape page bounds, which speeds
up visibility checks (a lot!).
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `improvement` — Improving existing features
Before we were running this on any change, even mouse position changes.
Now we only run it when shapes change.
Results wouldn't change in any case, so there's not a huge improvement.
Still, why run it if it is not necessary.
Before:
https://github.com/tldraw/tldraw/assets/2523721/b4111494-488a-42d0-9dfe-7fbc2ed88315
After:
https://github.com/tldraw/tldraw/assets/2523721/d96de329-235b-4dcb-93ea-fe297062985d
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [ ] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [x] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [ ] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [x] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
This PR makes a small improvement to the way we measure distances.
(Often we measure distances multiple times per frame per shape on the
screen). In many cases, we compare a minimum distance. This makes those
checks faster by avoiding a square root.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `improvement` — Improving existing features
### Release Notes
- Improve performance of minimum distance checks.
We also need to clear the timeout when panning.
https://github.com/tldraw/tldraw/assets/2523721/f32fd4d0-332c-4a80-bed0-9ce49a68e1abhttps://github.com/tldraw/tldraw/assets/2523721/e97f5fac-083f-4f77-ab72-40701790f039
Had an [alternative
approach](https://github.com/tldraw/tldraw/pull/3444) of setting
timeouts and clearing them in dispatch, but since the timeout is 500ms I
think this should work as well.
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [x] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [ ] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [x] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [ ] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
Next.js bans the use of react-dom/server APIs on the client. React's
docs recommend against using these too:
https://react.dev/reference/react-dom/server/renderToString#removing-rendertostring-from-the-client-code
In this diff, we switch from using `ReactDOMServer.renderToStaticMarkup`
to `ReactDOMClient.createRoot`, fixing SVG exports in next.js apps.
`getSvg` remains deprecated, but we've introduced a new `getSvgElement`
method with a similar API to `getSvgString` - it returns an `{svg,
width, height}` object.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `bugfix` — Bug fix
We reorded the dom a bit when we added the web gl rendered culled
shapes. We can now revert that.
Also noticed we weren't positioning the wrapper, so the z-index didn't
not apply.
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [ ] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [x] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [ ] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [ ] `improvement` — Improving existing features
- [x] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
---------
Co-authored-by: Steve Ruiz <steveruizok@gmail.com>
This PR makes a small improvement to the hand tool to address a "long
press"-related issues.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `bugfix` — Bug fix
Reworks our culling logic:
- No longer show the gray rectangles for culled shapes.
- Don't use `renderingBoundExpanded`, instead we now use
`viewportPageBounds`. I've removed `renderingBoundsExpanded`, but we
might want to deprecate it?
- There's now a incremental computation of non visible shapes, which are
shapes outside of `viewportPageBounds` and shapes that outside of their
parents' clipping bounds.
- There's also a new `getCulledShapes` function in `Editor`, which uses
the non visible shapes computation as a part of the culled shape
computation.
- Also moved some of the `getRenderingShapes` tests to newly created
`getCullingShapes` tests.
Feels much better on my old, 2017 ipad (first tab is this PR, second is
current prod, third is staging).
https://github.com/tldraw/tldraw/assets/2523721/327a7313-9273-4350-89a0-617a30fc01a2
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [x] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [ ] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [ ] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [x] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
### Test Plan
1. Regular culling shapes tests. Pan / zoom around. Use minimap. Change
pages.
- [x] Unit Tests
- [ ] End to end tests
---------
Co-authored-by: Steve Ruiz <steveruizok@gmail.com>
We use text shadows to create "outlines" around text shapes. These
shadows are rendered on the GPU. In Chrome (and on computers with a
capable GPU) text shadows work pretty well, however on Safari—and in
particular on iOS—they cause massive frame drops.
https://github.com/tldraw/tldraw/assets/23072548/b65cbcaa-6cc3-46f3-b54d-1f9cc07fc499
This PR:
- adds an LOD to text shadows, removing them at < 35% zoom
- removes text shadows entirely on Safari
If we had a "high performance" or "low-end device" mode, then shadows /
text shadows would be the first to go.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `improvement` — Improving existing features
### Test Plan
1. Use text shapes on iOS.
2. Use text shapes on Safari.
3. Use text shapes on Chrome.
### Release Notes
- Improves performance of text shapes on iOS / Safari.
This PR fixes some jest test.
- We skip the culling shapes in test environments.
- We skip rendering patterns in test environments.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `tests` — Changes to any test code
In this PR, we no longer buffer pointer down/ups. We now batch only
`pointer_move`, `wheel`, and `pinch` events.
Batched inputs were causing text not to work on iOS. On iOS, the
keyboard is only shown if we call `focus` during the same event loop as
a user input.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `bugfix` — Bug fix
### Test Plan
1. Use text on iOS.
This PR uses an element that prevents hit tests on shapes while the
camera is moving.
https://github.com/tldraw/tldraw/assets/23072548/9905f3d4-ba64-4e4d-ae99-194f513eaac8
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `improvement` — Improving existing features
### Test Plan
1. Move the camera.
2. Interact with the canvas.
3. Zoom in and out.
### Release Notes
- Improves performance of canvas while the camera is moving.
This PR makes a small improvement to the speed of `getShapeAtPoint`. It
removes `Editor.getCurrentPageRenderingShapesSorted`.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `improvement` — Improving existing features
This PR improves a bunch of places where we do "minimum distance
checks". Previously, we were using `Vec.Dist`, which uses `Math.hypot`
to find the actual distance, but we can just as well use the squared
distance. So this PR makes a small improvement to `Vec.Dist2` and then
switches to that method when checking minimum distances.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `improvement` — Improving existing features
### Test Plan
- [x] Unit Tests
### Release Notes
- Performance: small improvements to hit testing.
This PR extracts the #3344 changes to a smaller diff against main. It
does not include the changes to how / where culled shapes are
calculated, though I understand this could be much more efficiently
done!
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `improvement` — Improving existing features
---------
Co-authored-by: Mitja Bezenšek <mitja.bezensek@gmail.com>
This PR makes the `getCurrentPageId` method use a computed. Previously,
anything that referenced the current page id would pick up any change to
instance state. This will help a bunch of interactions like brushing
that would update the instance state on every frame.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `improvement` — Improving existing features
Comparing different culling optimizations:
https://github.com/tldraw/tldraw/assets/2523721/0b3b8b42-ed70-45b7-bf83-41023c36a563
I think we should go with the `display: none` + showing the skeleteon.
The way it works is:
- We now add a sibling to the shape wrapper div which serves as the
skeleton for the culled shapes.
- Only one of the two divs (shape wrapper and skeleton div) is
displayed. The other one is using `display: none` to improve
performance.
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [ ] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [x] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [ ] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [x] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
- Improve performance of culled shapes by using `display: none`.
---------
Co-authored-by: Steve Ruiz <steveruizok@gmail.com>
This PR adds a "long press" event that fires when pointing for more than
500ms. This event is used in the same way that dragging is used (e.g. to
transition to from pointing_selection to translating) but only on
desktop. On mobile, long presses are used to open the context menu.
## Background
This idea came out of @TodePond's #3208 PR. We use a "dead zone" to
avoid accidentally moving / rotating things when clicking on them, which
is especially common on mobile if a dead zone feature isn't implemented.
However, this makes it difficult to make "fine adjustments" because you
need to drag out of the dead zone (to start translating) and then drag
back to where you want to go.
With this change, you can long press on desktop to get to that
translating state. It's a micro UX optimization but especially nice if
apps want to display different UI for "dragging" shapes before the user
leaves the dead zone.
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [x] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [ ] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [ ] `bugfix` — Bug fix
- [x] `feature` — New feature
- [ ] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
### Test Plan
1. Long press shapes, selections, resize handles, rotate handles, crop
handles.
2. You should enter the corresponding states, just as you would have
with a drag.
- [ ] Unit Tests TODO
### Release Notes
- Add support for long pressing on desktop.
This PR fixes a bug that was introduced by #3223. There was a code path
that normally used to never run (a blur event running when the shape was
no longer editing) but which was being run now that shapes aren't
immediately removed on pointer down.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `bugfix` — Bug fix
### Test Plan
1. Create a sticky note
2. Begin editing the note
3. click on the canvas
4. You should be in pointing_canvas
The refactor of the textfields in this PR
https://github.com/tldraw/tldraw/pull/3050 caused a regression in
resizing Text shapes. (as demonstrated in this PR's video:
https://github.com/tldraw/tldraw/pull/3327)
We reverted that PR and now this PR updates the CSS to fix the gap that
was introduced when it was refactored.
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [x] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [ ] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [x] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [ ] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
This PR buffs input events.
## The story so far
In the olde days, we throttled events from the canvas events hook so
that a pointer event would only be sent every 1/60th of a second. This
was fine but made drawing on the iPad / 120FPS displays a little sad.
Then we removed this throttle. It seemed fine! Drawing at 120FPS was
great. We improved some rendering speeds and tightened some loops so
that the engine could keep up with 2x the number of points in a line.
Then we started noticing that iPads and other screens could start
choking on events as it received new inputs and tried to process and
render inputs while still recovering from a previous dropped frame. Even
worse, on iPad the work of rendering at 120FPS was causing the browser
to throttle the app after some sustained drawing. Yikes!
### Batching
I did an experimental PR (#3180) to bring back batching but do it in the
editor instead. What we would do is: rather than immediately processing
an event when we get it, we would instead put the event into a buffer.
On the next 60FPS tick, we would flush the buffer and process all of the
events. We'd have them all in the same transaction so that the app would
only render once.
### Render batching?
We then tried batching the renders, so that the app would only ever
render once per (next) frame. This added a bunch of complexity around
events that needed to happen synchronously, such as writing text in a
text field. Some inputs could "lag" in a way familiar to anyone who's
tried to update an input's state asynchronously. So we backed out of
this.
### Coalescing?
Another idea from @ds300 was to "coalesce" the events. This would be
useful because, while some interactions like drawing would require the
in-between frames in order to avoid data loss, most interactions (like
resizing) didn't actually need the in-between frames, they could just
use the last input of a given type.
Coalescing turned out to be trickier than we thought, though. Often a
state node required information from elsewhere in the app when
processing an event (such as camera position or page point, which is
derived from the camera position), and so the coalesced events would
need to also include this information or else the handlers wouldn't work
the way they should when processing the "final" event during a tick.
So we backed out of the coalescing strategy for now. Here's the [PR that
removes](937469d69d)
it.
### Let's just buffer the fuckers
So this PR now should only include input buffering.
I think there are ways to achieve the same coalescing-like results
through the state nodes, which could gather information during the
`onPointerMove` handler and then actually make changes during the
`onTick` handler, so that the changes are only done as many time as
necessary. This should help with e.g. resizing lots of shapes at once.
But first let's land the buffering!
---
Mitja's original text:
This PR builds on top of Steve's [experiment
PR](https://github.com/tldraw/tldraw/pull/3180) here. It also adds event
coalescing for [`pointerMove`
events](https://github.com/tldraw/tldraw/blob/mitja/input-buffering/packages/editor/src/lib/editor/Editor.ts#L8364-L8368).
The API is [somewhat similar
](https://developer.mozilla.org/en-US/docs/Web/API/PointerEvent/getCoalescedEvents)
to `getCoalescedEvent`. In `StateNodes` we register an `onPointerMove`
handler. When the event happens it gets called with the event `info`.
There's now an additional field on `TLMovePointerEvent` called
`coalescedInfo` which includes all the events. It's then on the user to
process all of these.
I decided on this API since it allows us to only expose one event
handler, but it still gives the users access to all events if they need
them.
We would otherwise either need to:
- Expose two events (coalesced and non-coalesced one and complicate the
api) so that state nodes like Resizing would not be triggered for each
pointer move.
- Offer some methods on the editor that would allow use to get the
coalesced information. Then the nodes that need that info could request
it. I [tried
this](9ad973da3a (diff-32f1de9a5a9ec72aa49a8d18a237fbfff301610f4689a4af6b37f47af435aafcR67)),
but it didn't feel good.
This also complicated the editor inputs. The events need to store
information about the event (like the mouse position when the event
happened for `onPointerMove`). But we cannot immediately update inputs
when the event happens. To make this work for `pointerMove` events I've
added `pagePoint`. It's
[calculated](https://github.com/tldraw/tldraw/pull/3223/files#diff-980beb0aa0ee9aa6d1cd386cef3dc05a500c030638ffb58d45fd11b79126103fR71)
when the event triggers and then consumers can get it straight from the
event (like
[Drawing](https://github.com/tldraw/tldraw/pull/3223/files#diff-32f1de9a5a9ec72aa49a8d18a237fbfff301610f4689a4af6b37f47af435aafcR104)).
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [x] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [ ] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [ ] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [x] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
### Test Plan
1. Add a step-by-step description of how to test your PR here.
4.
- [ ] Unit Tests
- [ ] End to end tests
### Release Notes
- Add a brief release note for your PR here.
---------
Co-authored-by: Steve Ruiz <steveruizok@gmail.com>
In this PR, when the camera changes, we check whether the pointer's page
position has actually changed before triggering a pointer move event.
This means that the pointer move will not fire while zooming in and out.
### Change Type
- [x] `sdk` — Changes the tldraw SDK
- [x] `improvement` — Improving existing features
### Test Plan
1. Zoom in and out.
2. The performance tab should not see any calls to `updateHoveredShape`
or other pointer move related events.
### Release Notes
- Improve performance of zooming.
This PR tweaks the logic of _when_ we update the viewport screen bounds.
Previously, we updated every one second in order to capture any changes
to the viewport's screen position. In this PR, we _check_ every one
second and update the screen bounds if the viewport's screen position
has actually changed. Since we also update the rendering shapes when
this happens, it would cause the rendering / culling shapes to update
while the camera was moving.
I've also removed the "maximum time before we start culling shapes", as
this wasn't very useful and could also cause frames to start dropping
without recovering.
https://github.com/tldraw/tldraw/assets/23072548/9f474481-30c9-49b4-a009-66775ca6a0c1
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [x] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [ ] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [ ] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [x] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
### Test Plan
1. Zoom and pan around
2. Culled shapes should only update when you stop moving the camera.
- [ ] Unit Tests
- [ ] End to end tests
### Release Notes
- Improve performance of the canvas when many shapes are present.
This is the first of three textfield changes. This starts with making
the speech bubble actually have text. Also, it creates a TipTap example
and how that would be wired up.
🎵 this is dangerous, I walk through textfields so watch your head rock 🎵
### Change Type
- [x] `minor` — New feature
### Release Notes
- Refactor textfields be composable/swappable.
Adds reference docs, guide in the "Editor" article, and examples for the
side effects manager.
There are 4 new examples:
1. Before create/update shape - constrains shapes to be places within a
circle
2. Before delete shape - prevent red shapes from being deleted
3. After create/update shape - make sure there's only ever one red shape
on the page at a time
4. After delete shape - delete frames after their last child is deleted
As these examples all require fairly specific configurations of shapes
(or are hard to understand without some visual hinting in the case of
placing shapes within a circle), I've included a `createDemoShapes`
function in each of these which makes sure the examples start with
shapes that will quickly show you the side effects in action. I've kept
these separate from the main code (in a function at the bottom), so
hopefully that won't be a source of confusion to anyone working from
these examples.
### Change Type
- [x] `docs` — Changes to the documentation, examples, or templates.
- [x] `improvement` — Improving existing features
The border on export preview images was making them get sized wrong.
This fixes that, and adds some padding to these exports so they don't
get clipped.
### Change Type
- [x] `internal` — Does not affect user-facing stuff
- [x] `bugfix` — Bug fix
## Migration path
1. If any of your shapes implement `toSvg` for exports, you'll need to
replace your implementation with a new version that returns JSX (it's a
react component) instead of manually constructing SVG DOM nodes
2. `editor.getSvg` is deprecated. It still works, but will be going away
in a future release. If you still need SVGs as DOM elements rather than
strings, use `new DOMParser().parseFromString(svgString,
'image/svg+xml').firstElementChild`
## The change in detail
At the moment, our SVG exports very carefully try to recreate the
visuals of our shapes by manually constructing SVG DOM nodes. On its own
this is really painful, but it also results in a lot of duplicated logic
between the `component` and `getSvg` methods of shape utils.
In #3020, we looked at using string concatenation & DOMParser to make
this a bit less painful. This works, but requires specifying namespaces
everywhere, is still pretty painful (no syntax highlighting or
formatting), and still results in all that duplicated logic.
I briefly experimented with creating my own version of the javascript
language that let you embed XML like syntax directly. I was going to
call it EXTREME JAVASCRIPT or XJS for short, but then I noticed that we
already wrote the whole of tldraw in this thing called react and a (imo
much worse named) version of the javascript xml thing already existed.
Given the entire library already depends on react, what would it look
like if we just used react directly for these exports? Turns out things
get a lot simpler! Take a look at lmk what you think
This diff was intended as a proof of concept, but is actually pretty
close to being landable. The main thing is that here, I've deliberately
leant into this being a big breaking change to see just how much code we
could delete (turns out: lots). We could if we wanted to make this
without making it a breaking change at all, but it would add back a lot
of complexity on our side and run a fair bit slower
---------
Co-authored-by: huppy-bot[bot] <128400622+huppy-bot[bot]@users.noreply.github.com>
This diff mostly adds an image annotator example, but also has a couple
of drive-by changes:
- Added a 'use-cases' category to the examples app for this style of
mini-app
- Add `editor.pageToViewport`, which is like `editor.pageToScreen` but
works with viewport coordinates (better for `InFrontOfTheCanvas` stuff)
- Prevent the chrome side-swipe-to-go-back thing in the examples app
Some cool features of the image annotator:
- The image cannot be unlocked, and cannot have shapes places behind it
- I still need to work out a way of removing the context menu though
- Anything you place outside the bounds of the image (and therefore
outside the bounds of the export) will be greyed out
- You can't change pages
- unless you find the "move to page" action... need to fix that
- The camera is constrained! It'll keep the image roughly centered on
the screen. If you pick a very long thin image, you can only scroll
vertically. If you pick a very big one, it'll default it to a reasonable
size.
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [x] `sdk` — Changes the tldraw SDK
- [x] `docs` — Changes to the documentation, examples, or templates.
<!-- ❗ Please select a 'Type' label ❗️ -->
- [x] `feature` — New feature
Currently, we only use native `structuredClone` in the browser, falling
back to `JSON.parse(JSON.stringify(...))` elsewhere, despite Node
supporting `structuredClone` [since
v17](https://developer.mozilla.org/en-US/docs/Web/API/structuredClone)
and Cloudflare Workers supporting it [since
2022](https://blog.cloudflare.com/standards-compliant-workers-api/).
This PR adjusts our shim to use the native `structuredClone` on all
platforms, if available.
Additionally, `jsdom` doesn't implement `structuredClone`, a bug [open
since 2022](https://github.com/jsdom/jsdom/issues/3363). This PR patches
`jsdom` environment in all packages/apps that use it for tests.
Also includes a driveby removal of `deepCopy`, a function that is
strictly inferior to `structuredClone`.
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [x] `sdk` — Changes the tldraw SDK
- [x] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [ ] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [ ] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [x] `improvement` — Improving existing features
- [x] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
### Test Plan
1. A smoke test would be enough
- [ ] Unit Tests
- [x] End to end tests
