No description
Find a file
Mitja Bezenšek 584380ba8b
Input buffering (#3223)
This PR buffs input events.

## The story so far

In the olde days, we throttled events from the canvas events hook so
that a pointer event would only be sent every 1/60th of a second. This
was fine but made drawing on the iPad / 120FPS displays a little sad.

Then we removed this throttle. It seemed fine! Drawing at 120FPS was
great. We improved some rendering speeds and tightened some loops so
that the engine could keep up with 2x the number of points in a line.

Then we started noticing that iPads and other screens could start
choking on events as it received new inputs and tried to process and
render inputs while still recovering from a previous dropped frame. Even
worse, on iPad the work of rendering at 120FPS was causing the browser
to throttle the app after some sustained drawing. Yikes!

### Batching

I did an experimental PR (#3180) to bring back batching but do it in the
editor instead. What we would do is: rather than immediately processing
an event when we get it, we would instead put the event into a buffer.
On the next 60FPS tick, we would flush the buffer and process all of the
events. We'd have them all in the same transaction so that the app would
only render once.

### Render batching?

We then tried batching the renders, so that the app would only ever
render once per (next) frame. This added a bunch of complexity around
events that needed to happen synchronously, such as writing text in a
text field. Some inputs could "lag" in a way familiar to anyone who's
tried to update an input's state asynchronously. So we backed out of
this.

### Coalescing?

Another idea from @ds300 was to "coalesce" the events. This would be
useful because, while some interactions like drawing would require the
in-between frames in order to avoid data loss, most interactions (like
resizing) didn't actually need the in-between frames, they could just
use the last input of a given type.

Coalescing turned out to be trickier than we thought, though. Often a
state node required information from elsewhere in the app when
processing an event (such as camera position or page point, which is
derived from the camera position), and so the coalesced events would
need to also include this information or else the handlers wouldn't work
the way they should when processing the "final" event during a tick.

So we backed out of the coalescing strategy for now. Here's the [PR that
removes](937469d69d)
it.

### Let's just buffer the fuckers

So this PR now should only include input buffering.

I think there are ways to achieve the same coalescing-like results
through the state nodes, which could gather information during the
`onPointerMove` handler and then actually make changes during the
`onTick` handler, so that the changes are only done as many time as
necessary. This should help with e.g. resizing lots of shapes at once.

But first let's land the buffering!

---

Mitja's original text:

This PR builds on top of Steve's [experiment
PR](https://github.com/tldraw/tldraw/pull/3180) here. It also adds event
coalescing for [`pointerMove`
events](https://github.com/tldraw/tldraw/blob/mitja/input-buffering/packages/editor/src/lib/editor/Editor.ts#L8364-L8368).
The API is [somewhat similar
](https://developer.mozilla.org/en-US/docs/Web/API/PointerEvent/getCoalescedEvents)
to `getCoalescedEvent`. In `StateNodes` we register an `onPointerMove`
handler. When the event happens it gets called with the event `info`.
There's now an additional field on `TLMovePointerEvent` called
`coalescedInfo` which includes all the events. It's then on the user to
process all of these.

I decided on this API since it allows us to only expose one event
handler, but it still gives the users access to all events if they need
them.

We would otherwise either need to:

- Expose two events (coalesced and non-coalesced one and complicate the
api) so that state nodes like Resizing would not be triggered for each
pointer move.
- Offer some methods on the editor that would allow use to get the
coalesced information. Then the nodes that need that info could request
it. I [tried
this](9ad973da3a (diff-32f1de9a5a9ec72aa49a8d18a237fbfff301610f4689a4af6b37f47af435aafcR67)),
but it didn't feel good.

This also complicated the editor inputs. The events need to store
information about the event (like the mouse position when the event
happened for `onPointerMove`). But we cannot immediately update inputs
when the event happens. To make this work for `pointerMove` events I've
added `pagePoint`. It's
[calculated](https://github.com/tldraw/tldraw/pull/3223/files#diff-980beb0aa0ee9aa6d1cd386cef3dc05a500c030638ffb58d45fd11b79126103fR71)
when the event triggers and then consumers can get it straight from the
event (like
[Drawing](https://github.com/tldraw/tldraw/pull/3223/files#diff-32f1de9a5a9ec72aa49a8d18a237fbfff301610f4689a4af6b37f47af435aafcR104)).

### Change Type

<!--  Please select a 'Scope' label ️ -->

- [x] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [ ] `internal` — Does not affect user-facing stuff

<!--  Please select a 'Type' label ️ -->

- [ ] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [x] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know


### Test Plan

1. Add a step-by-step description of how to test your PR here.
4.

- [ ] Unit Tests
- [ ] End to end tests

### Release Notes

- Add a brief release note for your PR here.

---------

Co-authored-by: Steve Ruiz <steveruizok@gmail.com>
2024-04-02 14:29:14 +00:00
.github use native structuredClone on node, cloudflare workers, and in tests (#3166) 2024-03-18 17:16:09 +00:00
.husky Add yarn immutable check to pre-commit. (#3218) 2024-03-20 11:45:28 +00:00
.yarn/patches Update auto (#2952) 2024-02-26 10:06:23 +00:00
apps Input buffering (#3223) 2024-04-02 14:29:14 +00:00
assets Add white (#3321) 2024-04-01 18:48:56 +00:00
config React-powered SVG exports (#3117) 2024-03-25 14:16:55 +00:00
packages Input buffering (#3223) 2024-04-02 14:29:14 +00:00
scripts [internal] Add license report scripts (#2751) 2024-04-01 13:36:40 +00:00
templates styling: make dotcom and examples site have consistent font styling (#3271) 2024-03-27 09:44:22 +00:00
.dockerignore unbrivate, dot com in (#2475) 2024-01-16 14:38:05 +00:00
.eslintignore use native structuredClone on node, cloudflare workers, and in tests (#3166) 2024-03-18 17:16:09 +00:00
.eslintplugin.js transfer-out: transfer out 2023-04-25 12:01:25 +01:00
.eslintrc.js use native structuredClone on node, cloudflare workers, and in tests (#3166) 2024-03-18 17:16:09 +00:00
.gitignore [internal] Add license report scripts (#2751) 2024-04-01 13:36:40 +00:00
.ignore [dx] Allow vscode to search inside md files by default (#3105) 2024-03-11 14:08:04 +00:00
.prettierignore tooling: prettier ignore pr template (#3210) 2024-03-20 09:56:38 +00:00
.prettierrc Unbiome (#2776) 2024-02-07 16:02:22 +00:00
.yarnrc.yml [dx] Allow vscode to search inside md files by default (#3105) 2024-03-11 14:08:04 +00:00
CHANGELOG.md Update CHANGELOG.md [skip ci] 2024-02-29 18:28:45 +00:00
CLA.md Change licenses to tldraw (#2167) 2023-12-19 10:41:01 +00:00
CODE_OF_CONDUCT.md transfer-out: transfer out 2023-04-25 12:01:25 +01:00
CONTRIBUTING.md dev: swap yarn test and test-dev for better dx (#2773) 2024-02-14 16:05:59 +00:00
lazy.config.ts fix refresh-assets cache inputs 2024-02-29 18:03:10 +00:00
lerna.json Show a broken image for files without assets (#2990) 2024-03-01 18:16:27 +00:00
LICENSE.md Change licenses to tldraw (#2167) 2023-12-19 10:41:01 +00:00
package.json [internal] Add license report scripts (#2751) 2024-04-01 13:36:40 +00:00
README.md updatereadmes 2024-02-29 18:21:17 +00:00
RELEASES.md Add release docs (#3158) 2024-03-17 13:28:18 +00:00
TRADEMARKS.md Change licenses to tldraw (#2167) 2023-12-19 10:41:01 +00:00
yarn.config.cjs Bump Yarn to 4.0.2 and add version constraints (#2481) 2024-01-18 11:09:17 +00:00
yarn.lock squish sync data events before sending them out (#3118) 2024-04-02 08:57:58 +00:00

tldraw

Welcome to the public monorepo for tldraw. tldraw is a library for creating infinite canvas experiences in React. It's the software behind the digital whiteboard tldraw.com.

Installation

npm i tldraw

Usage

import { Tldraw } from 'tldraw'
import 'tldraw/tldraw.css'

export default function App() {
	return (
		<div style={{ position: 'fixed', inset: 0 }}>
			<Tldraw />
		</div>
	)
}

Learn more at tldraw.dev.

Local development

The local development server will run our examples app. The basic example will show any changes you've made to the codebase.

To run the local development server, first clone this repo.

Install dependencies:

yarn

Start the local development server:

yarn dev

Open the example project at localhost:5420.

License

tldraw's source code and distributed packages are provided under the non-commercial tldraw license.

This license does not permit commercial use. If you wish to use tldraw in a commercial product or enterprise, you will need to purchase a commercial license. To obtain a commercial license, please contact us at sales@tldraw.com.

To learn more, see our license page.

Trademarks

The tldraw name and logo are trademarks of tldraw. Please see our trademark guidelines for info on acceptable usage.

Community

Have questions, comments or feedback? Join our discord or start a discussion.

Contribution

Please see our contributing guide. Found a bug? Please submit an issue.

Contributors

Star History

Star History Chart

Contact

Find us on Twitter at @tldraw or email sales@tldraw.com. You can also join our discord for quick help and support.