This PR buffs input events.
## The story so far
In the olde days, we throttled events from the canvas events hook so
that a pointer event would only be sent every 1/60th of a second. This
was fine but made drawing on the iPad / 120FPS displays a little sad.
Then we removed this throttle. It seemed fine! Drawing at 120FPS was
great. We improved some rendering speeds and tightened some loops so
that the engine could keep up with 2x the number of points in a line.
Then we started noticing that iPads and other screens could start
choking on events as it received new inputs and tried to process and
render inputs while still recovering from a previous dropped frame. Even
worse, on iPad the work of rendering at 120FPS was causing the browser
to throttle the app after some sustained drawing. Yikes!
### Batching
I did an experimental PR (#3180) to bring back batching but do it in the
editor instead. What we would do is: rather than immediately processing
an event when we get it, we would instead put the event into a buffer.
On the next 60FPS tick, we would flush the buffer and process all of the
events. We'd have them all in the same transaction so that the app would
only render once.
### Render batching?
We then tried batching the renders, so that the app would only ever
render once per (next) frame. This added a bunch of complexity around
events that needed to happen synchronously, such as writing text in a
text field. Some inputs could "lag" in a way familiar to anyone who's
tried to update an input's state asynchronously. So we backed out of
this.
### Coalescing?
Another idea from @ds300 was to "coalesce" the events. This would be
useful because, while some interactions like drawing would require the
in-between frames in order to avoid data loss, most interactions (like
resizing) didn't actually need the in-between frames, they could just
use the last input of a given type.
Coalescing turned out to be trickier than we thought, though. Often a
state node required information from elsewhere in the app when
processing an event (such as camera position or page point, which is
derived from the camera position), and so the coalesced events would
need to also include this information or else the handlers wouldn't work
the way they should when processing the "final" event during a tick.
So we backed out of the coalescing strategy for now. Here's the [PR that
removes](937469d69d)
it.
### Let's just buffer the fuckers
So this PR now should only include input buffering.
I think there are ways to achieve the same coalescing-like results
through the state nodes, which could gather information during the
`onPointerMove` handler and then actually make changes during the
`onTick` handler, so that the changes are only done as many time as
necessary. This should help with e.g. resizing lots of shapes at once.
But first let's land the buffering!
---
Mitja's original text:
This PR builds on top of Steve's [experiment
PR](https://github.com/tldraw/tldraw/pull/3180) here. It also adds event
coalescing for [`pointerMove`
events](https://github.com/tldraw/tldraw/blob/mitja/input-buffering/packages/editor/src/lib/editor/Editor.ts#L8364-L8368).
The API is [somewhat similar
](https://developer.mozilla.org/en-US/docs/Web/API/PointerEvent/getCoalescedEvents)
to `getCoalescedEvent`. In `StateNodes` we register an `onPointerMove`
handler. When the event happens it gets called with the event `info`.
There's now an additional field on `TLMovePointerEvent` called
`coalescedInfo` which includes all the events. It's then on the user to
process all of these.
I decided on this API since it allows us to only expose one event
handler, but it still gives the users access to all events if they need
them.
We would otherwise either need to:
- Expose two events (coalesced and non-coalesced one and complicate the
api) so that state nodes like Resizing would not be triggered for each
pointer move.
- Offer some methods on the editor that would allow use to get the
coalesced information. Then the nodes that need that info could request
it. I [tried
this](9ad973da3a (diff-32f1de9a5a9ec72aa49a8d18a237fbfff301610f4689a4af6b37f47af435aafcR67)),
but it didn't feel good.
This also complicated the editor inputs. The events need to store
information about the event (like the mouse position when the event
happened for `onPointerMove`). But we cannot immediately update inputs
when the event happens. To make this work for `pointerMove` events I've
added `pagePoint`. It's
[calculated](https://github.com/tldraw/tldraw/pull/3223/files#diff-980beb0aa0ee9aa6d1cd386cef3dc05a500c030638ffb58d45fd11b79126103fR71)
when the event triggers and then consumers can get it straight from the
event (like
[Drawing](https://github.com/tldraw/tldraw/pull/3223/files#diff-32f1de9a5a9ec72aa49a8d18a237fbfff301610f4689a4af6b37f47af435aafcR104)).
### Change Type
<!-- ❗ Please select a 'Scope' label ❗️ -->
- [x] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [ ] `internal` — Does not affect user-facing stuff
<!-- ❗ Please select a 'Type' label ❗️ -->
- [ ] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [x] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know
### Test Plan
1. Add a step-by-step description of how to test your PR here.
4.
- [ ] Unit Tests
- [ ] End to end tests
### Release Notes
- Add a brief release note for your PR here.
---------
Co-authored-by: Steve Ruiz <steveruizok@gmail.com>
Rename `@tldraw/tldraw` to just `tldraw`! `@tldraw/tldraw` still exists
as an alias to `tldraw` for folks who are still using that.
### Test Plan
- [x] Unit Tests
- [ ] End to end tests
### Release Notes
- The `@tldraw/tldraw` package has been renamed to `tldraw`. You can
keep using the old version if you want though!
This PR removes the automatic focus events from the editor.
The `autoFocus` prop is now true by default. When true, the editor will
begin in a focused state (`editor.instanceState.isFocused` will be
`true`) and the component will respond to keyboard shortcuts and other
interactions. When false, the editor will begin in an unfocused state
and not respond to keyboard interactions.
**It's now up to the developer** using the component to update
`isFocused` themselves. There's no predictable way to do that on our
side, so we leave it to the developer to decide when to turn on or off
focus for a container (for example, using an intersection observer to
"unfocus" components that are off screen).
### Change Type
- [x] `major` — Breaking change
### Test Plan
1. Open the multiple editors example.
2. Click to focus each editor.
3. Use the keyboard shortcuts to check that the correct editor is
focused.
4. Start editing a shape, then select the other editor. The first
editing shape should complete.
- [x] Unit Tests
- [x] End to end tests
### Release Notes
- [editor] Make autofocus default, remove automatic blur / focus events.
---------
Co-authored-by: David Sheldrick <d.j.sheldrick@gmail.com>
This PR is a significant rewrite of our selection / hit testing logic.
It
- replaces our current geometric helpers (`getBounds`, `getOutline`,
`hitTestPoint`, and `hitTestLineSegment`) with a new geometry API
- moves our hit testing entirely to JS using geometry
- improves selection logic, especially around editing shapes, groups and
frames
- fixes many minor selection bugs (e.g. shapes behind frames)
- removes hit-testing DOM elements from ShapeFill etc.
- adds many new tests around selection
- adds new tests around selection
- makes several superficial changes to surface editor APIs
This PR is hard to evaluate. The `selection-omnibus` test suite is
intended to describe all of the selection behavior, however all existing
tests are also either here preserved and passing or (in a few cases
around editing shapes) are modified to reflect the new behavior.
## Geometry
All `ShapeUtils` implement `getGeometry`, which returns a single
geometry primitive (`Geometry2d`). For example:
```ts
class BoxyShapeUtil {
getGeometry(shape: BoxyShape) {
return new Rectangle2d({
width: shape.props.width,
height: shape.props.height,
isFilled: true,
margin: shape.props.strokeWidth
})
}
}
```
This geometric primitive is used for all bounds calculation, hit
testing, intersection with arrows, etc.
There are several geometric primitives that extend `Geometry2d`:
- `Arc2d`
- `Circle2d`
- `CubicBezier2d`
- `CubicSpline2d`
- `Edge2d`
- `Ellipse2d`
- `Group2d`
- `Polygon2d`
- `Rectangle2d`
- `Stadium2d`
For shapes that have more complicated geometric representations, such as
an arrow with a label, the `Group2d` can accept other primitives as its
children.
## Hit testing
Previously, we did all hit testing via events set on shapes and other
elements. In this PR, I've replaced those hit tests with our own
calculation for hit tests in JavaScript. This removed the need for many
DOM elements, such as hit test area borders and fills which only existed
to trigger pointer events.
## Selection
We now support selecting "hollow" shapes by clicking inside of them.
This involves a lot of new logic but it should work intuitively. See
`Editor.getShapeAtPoint` for the (thoroughly commented) implementation.
every sunset is actually the sun hiding in fear and respect of tldraw's
quality of interactions
This PR also fixes several bugs with scribble selection, in particular
around the shift key modifier.
...as well as issues with labels and editing.
There are **over 100 new tests** for selection covering groups, frames,
brushing, scribbling, hovering, and editing. I'll add a few more before
I feel comfortable merging this PR.
## Arrow binding
Using the same "hollow shape" logic as selection, arrow binding is
significantly improved.
a thousand wise men could not improve on this
## Moving focus between editing shapes
Previously, this was handled in the `editing_shapes` state. This is
moved to `useEditableText`, and should generally be considered an
advanced implementation detail on a shape-by-shape basis. This addresses
a bug that I'd never noticed before, but which can be reproduced by
selecting an shape—but not focusing its input—while editing a different
shape. Previously, the new shape became the editing shape but its input
did not focus.
In this PR, you can select a shape by clicking on its edge or body, or
select its input to transfer editing / focus.
tldraw, glorious tldraw
### Change Type
- [x] `major` — Breaking change
### Test Plan
1. Erase shapes
2. Select shapes
3. Calculate their bounding boxes
- [ ] Unit Tests // todo
- [ ] End to end tests // todo
### Release Notes
- [editor] Remove `ShapeUtil.getBounds`, `ShapeUtil.getOutline`,
`ShapeUtil.hitTestPoint`, `ShapeUtil.hitTestLineSegment`
- [editor] Add `ShapeUtil.getGeometry`
- [editor] Add `Editor.getShapeGeometry`
This PR renames `App`, `app` and all appy names to `Editor`, `editor`,
and editorry names.
### Change Type
- [x] `major` — Breaking Change
### Release Notes
- Rename `App` to `Editor` and many other things that reference `app` to
`editor`.
This PR replaces our webdriver end to end tests with playwright tests.
It:
- replaces our webdriver workflow with a new e2e workflow based on
playwright
- removes the webdriver project
- adds e2e tests to our examples app
- replaces all `data-wd` attributes with `data-testid`
### Coverage
Most of the tests from our previous e2e tests are reproduced here,
though there are some related to our gestures that will need to be done
in a different way—or not at all. I've also added a handful of new
tests, too.
### Where are they
The tests are now part of our examples app rather than being in its own
different app. This should help us test our different examples too. As
far as I can tell there are no downsides here in terms of the regular
developer experience, though they might complicate any CodeSandbox
projects that are hooked into the examples app.
### Change Type
- [x] `tests` — Changes to any testing-related code only (will not
publish a new version)
Mitja Bezenšek