Input buffering (#3223)

This PR buffs input events.

## The story so far

In the olde days, we throttled events from the canvas events hook so
that a pointer event would only be sent every 1/60th of a second. This
was fine but made drawing on the iPad / 120FPS displays a little sad.

Then we removed this throttle. It seemed fine! Drawing at 120FPS was
great. We improved some rendering speeds and tightened some loops so
that the engine could keep up with 2x the number of points in a line.

Then we started noticing that iPads and other screens could start
choking on events as it received new inputs and tried to process and
render inputs while still recovering from a previous dropped frame. Even
worse, on iPad the work of rendering at 120FPS was causing the browser
to throttle the app after some sustained drawing. Yikes!

### Batching

I did an experimental PR (#3180) to bring back batching but do it in the
editor instead. What we would do is: rather than immediately processing
an event when we get it, we would instead put the event into a buffer.
On the next 60FPS tick, we would flush the buffer and process all of the
events. We'd have them all in the same transaction so that the app would
only render once.

### Render batching?

We then tried batching the renders, so that the app would only ever
render once per (next) frame. This added a bunch of complexity around
events that needed to happen synchronously, such as writing text in a
text field. Some inputs could "lag" in a way familiar to anyone who's
tried to update an input's state asynchronously. So we backed out of
this.

### Coalescing?

Another idea from @ds300 was to "coalesce" the events. This would be
useful because, while some interactions like drawing would require the
in-between frames in order to avoid data loss, most interactions (like
resizing) didn't actually need the in-between frames, they could just
use the last input of a given type.

Coalescing turned out to be trickier than we thought, though. Often a
state node required information from elsewhere in the app when
processing an event (such as camera position or page point, which is
derived from the camera position), and so the coalesced events would
need to also include this information or else the handlers wouldn't work
the way they should when processing the "final" event during a tick.

So we backed out of the coalescing strategy for now. Here's the [PR that
removes](937469d69d)
it.

### Let's just buffer the fuckers

So this PR now should only include input buffering.

I think there are ways to achieve the same coalescing-like results
through the state nodes, which could gather information during the
`onPointerMove` handler and then actually make changes during the
`onTick` handler, so that the changes are only done as many time as
necessary. This should help with e.g. resizing lots of shapes at once.

But first let's land the buffering!

---

Mitja's original text:

This PR builds on top of Steve's [experiment
PR](https://github.com/tldraw/tldraw/pull/3180) here. It also adds event
coalescing for [`pointerMove`
events](https://github.com/tldraw/tldraw/blob/mitja/input-buffering/packages/editor/src/lib/editor/Editor.ts#L8364-L8368).
The API is [somewhat similar
](https://developer.mozilla.org/en-US/docs/Web/API/PointerEvent/getCoalescedEvents)
to `getCoalescedEvent`. In `StateNodes` we register an `onPointerMove`
handler. When the event happens it gets called with the event `info`.
There's now an additional field on `TLMovePointerEvent` called
`coalescedInfo` which includes all the events. It's then on the user to
process all of these.

I decided on this API since it allows us to only expose one event
handler, but it still gives the users access to all events if they need
them.

We would otherwise either need to:

- Expose two events (coalesced and non-coalesced one and complicate the
api) so that state nodes like Resizing would not be triggered for each
pointer move.
- Offer some methods on the editor that would allow use to get the
coalesced information. Then the nodes that need that info could request
it. I [tried
this](9ad973da3a (diff-32f1de9a5a9ec72aa49a8d18a237fbfff301610f4689a4af6b37f47af435aafcR67)),
but it didn't feel good.

This also complicated the editor inputs. The events need to store
information about the event (like the mouse position when the event
happened for `onPointerMove`). But we cannot immediately update inputs
when the event happens. To make this work for `pointerMove` events I've
added `pagePoint`. It's
[calculated](https://github.com/tldraw/tldraw/pull/3223/files#diff-980beb0aa0ee9aa6d1cd386cef3dc05a500c030638ffb58d45fd11b79126103fR71)
when the event triggers and then consumers can get it straight from the
event (like
[Drawing](https://github.com/tldraw/tldraw/pull/3223/files#diff-32f1de9a5a9ec72aa49a8d18a237fbfff301610f4689a4af6b37f47af435aafcR104)).

### Change Type

<!--  Please select a 'Scope' label ️ -->

- [x] `sdk` — Changes the tldraw SDK
- [ ] `dotcom` — Changes the tldraw.com web app
- [ ] `docs` — Changes to the documentation, examples, or templates.
- [ ] `vs code` — Changes to the vscode plugin
- [ ] `internal` — Does not affect user-facing stuff

<!--  Please select a 'Type' label ️ -->

- [ ] `bugfix` — Bug fix
- [ ] `feature` — New feature
- [x] `improvement` — Improving existing features
- [ ] `chore` — Updating dependencies, other boring stuff
- [ ] `galaxy brain` — Architectural changes
- [ ] `tests` — Changes to any test code
- [ ] `tools` — Changes to infrastructure, CI, internal scripts,
debugging tools, etc.
- [ ] `dunno` — I don't know


### Test Plan

1. Add a step-by-step description of how to test your PR here.
4.

- [ ] Unit Tests
- [ ] End to end tests

### Release Notes

- Add a brief release note for your PR here.

---------

Co-authored-by: Steve Ruiz <steveruizok@gmail.com>
This commit is contained in:
Mitja Bezenšek 2024-04-02 16:29:14 +02:00 committed by GitHub
parent b42a222c88
commit 584380ba8b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 411 additions and 375 deletions

View file

@ -21,6 +21,7 @@ test.describe('Canvas events', () => {
await page.mouse.move(200, 200) // to kill any double clicks
await page.mouse.move(100, 100)
await page.mouse.down()
await page.waitForTimeout(20)
expect(await page.evaluate(() => __tldraw_editor_events.at(-1))).toMatchObject({
target: 'canvas',
type: 'pointer',
@ -46,6 +47,7 @@ test.describe('Canvas events', () => {
await page.mouse.down()
await page.mouse.move(101, 101)
await page.mouse.up()
await page.waitForTimeout(20)
expect(await page.evaluate(() => __tldraw_editor_events.at(-1))).toMatchObject({
target: 'canvas',
type: 'pointer',
@ -118,6 +120,7 @@ test.describe('Shape events', () => {
test('pointer down', async () => {
await page.mouse.move(51, 51)
await page.mouse.down()
await page.waitForTimeout(20)
expect(await page.evaluate(() => __tldraw_editor_events.at(-1))).toMatchObject({
target: 'canvas',
type: 'pointer',
@ -128,6 +131,7 @@ test.describe('Shape events', () => {
test('pointer move', async () => {
await page.mouse.move(51, 51)
await page.mouse.move(52, 52)
await page.waitForTimeout(20)
expect(await page.evaluate(() => __tldraw_editor_events.at(-1))).toMatchObject({
target: 'canvas',
type: 'pointer',
@ -139,6 +143,7 @@ test.describe('Shape events', () => {
await page.mouse.move(51, 51)
await page.mouse.down()
await page.mouse.up()
await page.waitForTimeout(20)
expect(await page.evaluate(() => __tldraw_editor_events.at(-1))).toMatchObject({
target: 'canvas',
type: 'pointer',

View file

@ -112,6 +112,7 @@ test.describe('Shape Tools', () => {
// Click on the page
await page.mouse.click(200, 200)
await page.waitForTimeout(20)
// We should have a corresponding shape in the page
expect(await getAllShapeTypes(page)).toEqual([shape])
@ -119,6 +120,7 @@ test.describe('Shape Tools', () => {
// Reset for next time
await page.mouse.click(50, 50) // to ensure we're not focused
await page.keyboard.press('v') // go to the select tool
await page.waitForTimeout(20)
await page.keyboard.press('Control+a')
await page.keyboard.press('Backspace')
}
@ -156,6 +158,7 @@ test.describe('Shape Tools', () => {
// Reset for next time
await page.mouse.click(50, 50) // to ensure we're not focused
await page.keyboard.press('v')
await page.waitForTimeout(20)
await page.keyboard.press('Control+a')
await page.keyboard.press('Backspace')
}

View file

@ -637,7 +637,7 @@ export class Editor extends EventEmitter<TLEventMap> {
this.updateRenderingBounds()
this.on('tick', this.tick)
this.on('tick', this._flushEventsForTick)
requestAnimationFrame(() => {
this._tickManager.start()
@ -2085,7 +2085,7 @@ export class Editor extends EventEmitter<TLEventMap> {
}
/** @internal */
private _setCamera(point: VecLike): this {
private _setCamera(point: VecLike, immediate = false): this {
const currentCamera = this.getCamera()
if (currentCamera.x === point.x && currentCamera.y === point.y && currentCamera.z === point.z) {
@ -2107,7 +2107,7 @@ export class Editor extends EventEmitter<TLEventMap> {
currentScreenPoint.y / camera.z - camera.y !== currentPagePoint.y
) {
// If it's changed, dispatch a pointer event
this.dispatch({
const event: TLPointerEventInfo = {
type: 'pointer',
target: 'canvas',
name: 'pointer_move',
@ -2119,7 +2119,12 @@ export class Editor extends EventEmitter<TLEventMap> {
shiftKey: this.inputs.shiftKey,
button: 0,
isPen: this.getInstanceState().isPenMode ?? false,
})
}
if (immediate) {
this._flushEventForTick(event)
} else {
this.dispatch(event)
}
}
this._tickCameraState()
@ -2495,6 +2500,7 @@ export class Editor extends EventEmitter<TLEventMap> {
if (!this.getInstanceState().canMoveCamera) return this
const { x: cx, y: cy, z: cz } = this.getCamera()
this.setCamera({ x: cx + offset.x / cz, y: cy + offset.y / cz, z: cz }, animation)
this._flushEventsForTick(0)
return this
}
@ -8211,12 +8217,6 @@ export class Editor extends EventEmitter<TLEventMap> {
])
}
/** @internal */
private tick = (elapsed = 0) => {
this.dispatch({ type: 'misc', name: 'tick', elapsed })
this.scribbles.tick(elapsed)
}
/**
* Dispatch a cancel event.
*
@ -8229,6 +8229,7 @@ export class Editor extends EventEmitter<TLEventMap> {
*/
cancel(): this {
this.dispatch({ type: 'misc', name: 'cancel' })
this._tickManager.tick()
return this
}
@ -8244,6 +8245,7 @@ export class Editor extends EventEmitter<TLEventMap> {
*/
interrupt(): this {
this.dispatch({ type: 'misc', name: 'interrupt' })
this._tickManager.tick()
return this
}
@ -8364,6 +8366,27 @@ export class Editor extends EventEmitter<TLEventMap> {
* @public
*/
dispatch = (info: TLEventInfo): this => {
this._pendingEventsForNextTick.push(info)
return this
}
private _pendingEventsForNextTick: TLEventInfo[] = []
private _flushEventsForTick = (elapsed: number) => {
this.batch(() => {
if (this._pendingEventsForNextTick.length > 0) {
const events = [...this._pendingEventsForNextTick]
this._pendingEventsForNextTick.length = 0
for (const info of events) {
this._flushEventForTick(info)
}
}
this.root.handleEvent({ type: 'misc', name: 'tick', elapsed })
this.scribbles.tick(elapsed)
})
}
private _flushEventForTick = (info: TLEventInfo) => {
// prevent us from spamming similar event errors if we're crashed.
// todo: replace with new readonly mode?
if (this.getCrashingError()) return this
@ -8371,161 +8394,260 @@ export class Editor extends EventEmitter<TLEventMap> {
const { inputs } = this
const { type } = info
this.batch(() => {
if (info.type === 'misc') {
// stop panning if the interaction is cancelled or completed
if (info.name === 'cancel' || info.name === 'complete') {
this.inputs.isDragging = false
if (info.type === 'misc') {
// stop panning if the interaction is cancelled or completed
if (info.name === 'cancel' || info.name === 'complete') {
this.inputs.isDragging = false
if (this.inputs.isPanning) {
this.inputs.isPanning = false
this.updateInstanceState({
cursor: {
type: this._prevCursor,
rotation: 0,
},
})
}
if (this.inputs.isPanning) {
this.inputs.isPanning = false
this.updateInstanceState({
cursor: {
type: this._prevCursor,
rotation: 0,
},
})
}
this.root.handleEvent(info)
return
}
if (info.shiftKey) {
clearInterval(this._shiftKeyTimeout)
this._shiftKeyTimeout = -1
inputs.shiftKey = true
} else if (!info.shiftKey && inputs.shiftKey && this._shiftKeyTimeout === -1) {
this._shiftKeyTimeout = setTimeout(this._setShiftKeyTimeout, 150)
}
this.root.handleEvent(info)
return
}
if (info.altKey) {
clearInterval(this._altKeyTimeout)
this._altKeyTimeout = -1
inputs.altKey = true
} else if (!info.altKey && inputs.altKey && this._altKeyTimeout === -1) {
this._altKeyTimeout = setTimeout(this._setAltKeyTimeout, 150)
}
if (info.shiftKey) {
clearInterval(this._shiftKeyTimeout)
this._shiftKeyTimeout = -1
inputs.shiftKey = true
} else if (!info.shiftKey && inputs.shiftKey && this._shiftKeyTimeout === -1) {
this._shiftKeyTimeout = setTimeout(this._setShiftKeyTimeout, 150)
}
if (info.ctrlKey) {
clearInterval(this._ctrlKeyTimeout)
this._ctrlKeyTimeout = -1
inputs.ctrlKey = true /** @internal */ /** @internal */ /** @internal */
} else if (!info.ctrlKey && inputs.ctrlKey && this._ctrlKeyTimeout === -1) {
this._ctrlKeyTimeout = setTimeout(this._setCtrlKeyTimeout, 150)
}
if (info.altKey) {
clearInterval(this._altKeyTimeout)
this._altKeyTimeout = -1
inputs.altKey = true
} else if (!info.altKey && inputs.altKey && this._altKeyTimeout === -1) {
this._altKeyTimeout = setTimeout(this._setAltKeyTimeout, 150)
}
const { originPagePoint, originScreenPoint, currentPagePoint, currentScreenPoint } = inputs
if (info.ctrlKey) {
clearInterval(this._ctrlKeyTimeout)
this._ctrlKeyTimeout = -1
inputs.ctrlKey = true /** @internal */ /** @internal */ /** @internal */
} else if (!info.ctrlKey && inputs.ctrlKey && this._ctrlKeyTimeout === -1) {
this._ctrlKeyTimeout = setTimeout(this._setCtrlKeyTimeout, 150)
}
if (!inputs.isPointing) {
inputs.isDragging = false
}
const { originPagePoint, originScreenPoint, currentPagePoint, currentScreenPoint } = inputs
switch (type) {
case 'pinch': {
if (!this.getInstanceState().canMoveCamera) return
this._updateInputsFromEvent(info)
if (!inputs.isPointing) {
inputs.isDragging = false
}
switch (info.name) {
case 'pinch_start': {
if (inputs.isPinching) return
switch (type) {
case 'pinch': {
if (!this.getInstanceState().canMoveCamera) return
this._updateInputsFromEvent(info)
if (!inputs.isEditing) {
this._pinchStart = this.getCamera().z
if (!this._selectedShapeIdsAtPointerDown.length) {
this._selectedShapeIdsAtPointerDown = this.getSelectedShapeIds()
}
switch (info.name) {
case 'pinch_start': {
if (inputs.isPinching) return
this._didPinch = true
inputs.isPinching = true
this.interrupt()
if (!inputs.isEditing) {
this._pinchStart = this.getCamera().z
if (!this._selectedShapeIdsAtPointerDown.length) {
this._selectedShapeIdsAtPointerDown = this.getSelectedShapeIds()
}
return // Stop here!
this._didPinch = true
inputs.isPinching = true
this.interrupt()
}
case 'pinch': {
if (!inputs.isPinching) return
const {
point: { z = 1 },
delta: { x: dx, y: dy },
} = info
return // Stop here!
}
case 'pinch': {
if (!inputs.isPinching) return
const { screenBounds } = this.store.unsafeGetWithoutCapture(TLINSTANCE_ID)!
const { x, y } = Vec.SubXY(info.point, screenBounds.x, screenBounds.y)
const {
point: { z = 1 },
delta: { x: dx, y: dy },
} = info
const { x: cx, y: cy, z: cz } = this.getCamera()
const { screenBounds } = this.store.unsafeGetWithoutCapture(TLINSTANCE_ID)!
const { x, y } = Vec.SubXY(info.point, screenBounds.x, screenBounds.y)
const zoom = Math.min(MAX_ZOOM, Math.max(MIN_ZOOM, z))
const { x: cx, y: cy, z: cz } = this.getCamera()
this.setCamera({
const zoom = Math.min(MAX_ZOOM, Math.max(MIN_ZOOM, z))
this.stopCameraAnimation()
if (this.getInstanceState().followingUserId) {
this.stopFollowingUser()
}
this._setCamera(
{
x: cx + dx / cz - x / cz + x / zoom,
y: cy + dy / cz - y / cz + y / zoom,
z: zoom,
},
true
)
return // Stop here!
}
case 'pinch_end': {
if (!inputs.isPinching) return this
inputs.isPinching = false
const { _selectedShapeIdsAtPointerDown } = this
this.setSelectedShapes(this._selectedShapeIdsAtPointerDown, { squashing: true })
this._selectedShapeIdsAtPointerDown = []
if (this._didPinch) {
this._didPinch = false
this.once('tick', () => {
if (!this._didPinch) {
this.setSelectedShapes(_selectedShapeIdsAtPointerDown, { squashing: true })
}
})
return // Stop here!
}
case 'pinch_end': {
if (!inputs.isPinching) return this
inputs.isPinching = false
const { _selectedShapeIdsAtPointerDown } = this
this.setSelectedShapes(this._selectedShapeIdsAtPointerDown, { squashing: true })
this._selectedShapeIdsAtPointerDown = []
if (this._didPinch) {
this._didPinch = false
requestAnimationFrame(() => {
if (!this._didPinch) {
this.setSelectedShapes(_selectedShapeIdsAtPointerDown, { squashing: true })
}
})
}
return // Stop here!
}
return // Stop here!
}
}
case 'wheel': {
if (!this.getInstanceState().canMoveCamera) return
}
case 'wheel': {
if (!this.getInstanceState().canMoveCamera) return
this._updateInputsFromEvent(info)
this._updateInputsFromEvent(info)
if (this.getIsMenuOpen()) {
// noop
} else {
if (inputs.ctrlKey) {
// todo: Start or update the zoom end interval
if (this.getIsMenuOpen()) {
// noop
} else {
this.stopCameraAnimation()
if (this.getInstanceState().followingUserId) {
this.stopFollowingUser()
}
if (inputs.ctrlKey) {
// todo: Start or update the zoom end interval
// If the alt or ctrl keys are pressed,
// zoom or pan the camera and then return.
// If the alt or ctrl keys are pressed,
// zoom or pan the camera and then return.
// Subtract the top left offset from the user's point
// Subtract the top left offset from the user's point
const { x, y } = this.inputs.currentScreenPoint
const { x, y } = this.inputs.currentScreenPoint
const { x: cx, y: cy, z: cz } = this.getCamera()
const { x: cx, y: cy, z: cz } = this.getCamera()
const zoom = Math.min(MAX_ZOOM, Math.max(MIN_ZOOM, cz + (info.delta.z ?? 0) * cz))
const zoom = Math.min(MAX_ZOOM, Math.max(MIN_ZOOM, cz + (info.delta.z ?? 0) * cz))
this.setCamera({
this._setCamera(
{
x: cx + (x / zoom - x) - (x / cz - x),
y: cy + (y / zoom - y) - (y / cz - y),
z: zoom,
})
},
true
)
// We want to return here because none of the states in our
// statechart should respond to this event (a camera zoom)
// We want to return here because none of the states in our
// statechart should respond to this event (a camera zoom)
return
}
// Update the camera here, which will dispatch a pointer move...
// this will also update the pointer position, etc
const { x: cx, y: cy, z: cz } = this.getCamera()
this._setCamera({ x: cx + info.delta.x / cz, y: cy + info.delta.y / cz, z: cz }, true)
if (
!inputs.isDragging &&
inputs.isPointing &&
originPagePoint.dist(currentPagePoint) >
(this.getInstanceState().isCoarsePointer ? COARSE_DRAG_DISTANCE : DRAG_DISTANCE) /
this.getZoomLevel()
) {
inputs.isDragging = true
}
}
break
}
case 'pointer': {
// If we're pinching, return
if (inputs.isPinching) return
this._updateInputsFromEvent(info)
const { isPen } = info
switch (info.name) {
case 'pointer_down': {
this.clearOpenMenus()
this._selectedShapeIdsAtPointerDown = this.getSelectedShapeIds()
// Firefox bug fix...
// If it's a left-mouse-click, we store the pointer id for later user
if (info.button === 0) {
this.capturedPointerId = info.pointerId
}
// Add the button from the buttons set
inputs.buttons.add(info.button)
inputs.isPointing = true
inputs.isDragging = false
if (this.getInstanceState().isPenMode) {
if (!isPen) {
return
}
} else {
if (isPen) {
this.updateInstanceState({ isPenMode: true })
}
}
if (info.button === 5) {
// Eraser button activates eraser
this._restoreToolId = this.getCurrentToolId()
this.complete()
this.setCurrentTool('eraser')
} else if (info.button === 1) {
// Middle mouse pan activates panning
if (!this.inputs.isPanning) {
this._prevCursor = this.getInstanceState().cursor.type
}
this.inputs.isPanning = true
}
if (this.inputs.isPanning) {
this.stopCameraAnimation()
this.setCursor({ type: 'grabbing', rotation: 0 })
return this
}
originScreenPoint.setTo(currentScreenPoint)
originPagePoint.setTo(currentPagePoint)
break
}
case 'pointer_move': {
// If the user is in pen mode, but the pointer is not a pen, stop here.
if (!isPen && this.getInstanceState().isPenMode) {
return
}
// Update the camera here, which will dispatch a pointer move...
// this will also update the pointer position, etc
this.pan(info.delta)
if (this.inputs.isPanning && this.inputs.isPointing) {
// Handle panning
const { currentScreenPoint, previousScreenPoint } = this.inputs
this.pan(Vec.Sub(currentScreenPoint, previousScreenPoint))
return
}
if (
!inputs.isDragging &&
@ -8536,270 +8658,169 @@ export class Editor extends EventEmitter<TLEventMap> {
) {
inputs.isDragging = true
}
break
}
break
}
case 'pointer': {
// If we're pinching, return
if (inputs.isPinching) return
case 'pointer_up': {
// Remove the button from the buttons set
inputs.buttons.delete(info.button)
this._updateInputsFromEvent(info)
inputs.isPointing = false
inputs.isDragging = false
const { isPen } = info
switch (info.name) {
case 'pointer_down': {
this.clearOpenMenus()
this._selectedShapeIdsAtPointerDown = this.getSelectedShapeIds()
// Firefox bug fix...
// If it's a left-mouse-click, we store the pointer id for later user
if (info.button === 0) {
this.capturedPointerId = info.pointerId
}
// Add the button from the buttons set
inputs.buttons.add(info.button)
inputs.isPointing = true
inputs.isDragging = false
if (this.getInstanceState().isPenMode) {
if (!isPen) {
return
}
} else {
if (isPen) {
this.updateInstanceState({ isPenMode: true })
}
}
if (info.button === 5) {
// Eraser button activates eraser
this._restoreToolId = this.getCurrentToolId()
this.complete()
this.setCurrentTool('eraser')
} else if (info.button === 1) {
// Middle mouse pan activates panning
if (!this.inputs.isPanning) {
this._prevCursor = this.getInstanceState().cursor.type
}
this.inputs.isPanning = true
}
if (this.inputs.isPanning) {
this.stopCameraAnimation()
this.updateInstanceState({
cursor: {
type: 'grabbing',
rotation: 0,
},
})
return this
}
originScreenPoint.setTo(currentScreenPoint)
originPagePoint.setTo(currentPagePoint)
break
if (this.getIsMenuOpen()) {
// Suppressing pointerup here as <ContextMenu/> doesn't seem to do what we what here.
return
}
case 'pointer_move': {
// If the user is in pen mode, but the pointer is not a pen, stop here.
if (!isPen && this.getInstanceState().isPenMode) {
return
}
if (this.inputs.isPanning && this.inputs.isPointing) {
// Handle panning
const { currentScreenPoint, previousScreenPoint } = this.inputs
this.pan(Vec.Sub(currentScreenPoint, previousScreenPoint))
return
}
if (
!inputs.isDragging &&
inputs.isPointing &&
originPagePoint.dist(currentPagePoint) >
(this.getInstanceState().isCoarsePointer ? COARSE_DRAG_DISTANCE : DRAG_DISTANCE) /
this.getZoomLevel()
) {
inputs.isDragging = true
}
break
if (!isPen && this.getInstanceState().isPenMode) {
return
}
case 'pointer_up': {
// Remove the button from the buttons set
inputs.buttons.delete(info.button)
inputs.isPointing = false
inputs.isDragging = false
// Firefox bug fix...
// If it's the same pointer that we stored earlier...
// ... then it's probably still a left-mouse-click!
if (this.capturedPointerId === info.pointerId) {
this.capturedPointerId = null
info.button = 0
}
if (this.getIsMenuOpen()) {
// Suppressing pointerup here as <ContextMenu/> doesn't seem to do what we what here.
return
}
if (inputs.isPanning) {
if (info.button === 1) {
if (!this.inputs.keys.has(' ')) {
inputs.isPanning = false
if (!isPen && this.getInstanceState().isPenMode) {
return
}
// Firefox bug fix...
// If it's the same pointer that we stored earlier...
// ... then it's probably still a left-mouse-click!
if (this.capturedPointerId === info.pointerId) {
this.capturedPointerId = null
info.button = 0
}
if (inputs.isPanning) {
if (info.button === 1) {
if (!this.inputs.keys.has(' ')) {
inputs.isPanning = false
this.slideCamera({
speed: Math.min(2, this.inputs.pointerVelocity.len()),
direction: this.inputs.pointerVelocity,
friction: CAMERA_SLIDE_FRICTION,
})
this.updateInstanceState({
cursor: { type: this._prevCursor, rotation: 0 },
})
} else {
this.slideCamera({
speed: Math.min(2, this.inputs.pointerVelocity.len()),
direction: this.inputs.pointerVelocity,
friction: CAMERA_SLIDE_FRICTION,
})
this.updateInstanceState({
cursor: {
type: 'grab',
rotation: 0,
},
})
}
} else if (info.button === 0) {
this.slideCamera({
speed: Math.min(2, this.inputs.pointerVelocity.len()),
direction: this.inputs.pointerVelocity,
friction: CAMERA_SLIDE_FRICTION,
})
this.updateInstanceState({
cursor: {
type: 'grab',
rotation: 0,
},
this.setCursor({ type: this._prevCursor, rotation: 0 })
} else {
this.slideCamera({
speed: Math.min(2, this.inputs.pointerVelocity.len()),
direction: this.inputs.pointerVelocity,
friction: CAMERA_SLIDE_FRICTION,
})
this.setCursor({
type: 'grab',
rotation: 0,
})
}
} else {
if (info.button === 5) {
// Eraser button activates eraser
this.complete()
this.setCurrentTool(this._restoreToolId)
}
}
break
}
}
break
}
case 'keyboard': {
// please, please
if (info.key === 'ShiftRight') info.key = 'ShiftLeft'
if (info.key === 'AltRight') info.key = 'AltLeft'
if (info.code === 'ControlRight') info.code = 'ControlLeft'
switch (info.name) {
case 'key_down': {
// Add the key from the keys set
inputs.keys.add(info.code)
// If the space key is pressed (but meta / control isn't!) activate panning
if (!info.ctrlKey && info.code === 'Space') {
if (!this.inputs.isPanning) {
this._prevCursor = this.getInstanceState().cursor.type
}
this.inputs.isPanning = true
this.updateInstanceState({
cursor: { type: this.inputs.isPointing ? 'grabbing' : 'grab', rotation: 0 },
} else if (info.button === 0) {
this.slideCamera({
speed: Math.min(2, this.inputs.pointerVelocity.len()),
direction: this.inputs.pointerVelocity,
friction: CAMERA_SLIDE_FRICTION,
})
this.setCursor({
type: 'grab',
rotation: 0,
})
}
break
} else {
if (info.button === 5) {
// Eraser button activates eraser
this.complete()
this.setCurrentTool(this._restoreToolId)
}
}
case 'key_up': {
// Remove the key from the keys set
inputs.keys.delete(info.code)
if (info.code === 'Space' && !this.inputs.buttons.has(1)) {
this.inputs.isPanning = false
this.updateInstanceState({
cursor: { type: this._prevCursor, rotation: 0 },
})
break
}
}
break
}
case 'keyboard': {
// please, please
if (info.key === 'ShiftRight') info.key = 'ShiftLeft'
if (info.key === 'AltRight') info.key = 'AltLeft'
if (info.code === 'ControlRight') info.code = 'ControlLeft'
switch (info.name) {
case 'key_down': {
// Add the key from the keys set
inputs.keys.add(info.code)
// If the space key is pressed (but meta / control isn't!) activate panning
if (!info.ctrlKey && info.code === 'Space') {
if (!this.inputs.isPanning) {
this._prevCursor = this.getInstanceState().cursor.type
}
break
}
case 'key_repeat': {
// noop
break
this.inputs.isPanning = true
this.setCursor({ type: this.inputs.isPointing ? 'grabbing' : 'grab', rotation: 0 })
}
break
}
case 'key_up': {
// Remove the key from the keys set
inputs.keys.delete(info.code)
if (info.code === 'Space' && !this.inputs.buttons.has(1)) {
this.inputs.isPanning = false
this.setCursor({ type: this._prevCursor, rotation: 0 })
}
break
}
case 'key_repeat': {
// noop
break
}
break
}
break
}
}
// Correct the info name for right / middle clicks
if (info.type === 'pointer') {
if (info.button === 1) {
info.name = 'middle_click'
} else if (info.button === 2) {
info.name = 'right_click'
}
// Correct the info name for right / middle clicks
if (info.type === 'pointer') {
if (info.button === 1) {
info.name = 'middle_click'
} else if (info.button === 2) {
info.name = 'right_click'
}
// If a pointer event, send the event to the click manager.
if (info.isPen === this.getInstanceState().isPenMode) {
switch (info.name) {
case 'pointer_down': {
const otherEvent = this._clickManager.transformPointerDownEvent(info)
if (info.name !== otherEvent.name) {
this.root.handleEvent(info)
this.emit('event', info)
this.root.handleEvent(otherEvent)
this.emit('event', otherEvent)
return
}
break
// If a pointer event, send the event to the click manager.
if (info.isPen === this.getInstanceState().isPenMode) {
switch (info.name) {
case 'pointer_down': {
const otherEvent = this._clickManager.transformPointerDownEvent(info)
if (info.name !== otherEvent.name) {
this.root.handleEvent(info)
this.emit('event', info)
this.root.handleEvent(otherEvent)
this.emit('event', otherEvent)
return
}
case 'pointer_up': {
const otherEvent = this._clickManager.transformPointerUpEvent(info)
if (info.name !== otherEvent.name) {
this.root.handleEvent(info)
this.emit('event', info)
this.root.handleEvent(otherEvent)
this.emit('event', otherEvent)
return
}
break
}
case 'pointer_move': {
this._clickManager.handleMove()
break
break
}
case 'pointer_up': {
const otherEvent = this._clickManager.transformPointerUpEvent(info)
if (info.name !== otherEvent.name) {
this.root.handleEvent(info)
this.emit('event', info)
this.root.handleEvent(otherEvent)
this.emit('event', otherEvent)
return
}
break
}
case 'pointer_move': {
this._clickManager.handleMove()
break
}
}
}
}
// Send the event to the statechart. It will be handled by all
// active states, starting at the root.
this.root.handleEvent(info)
this.emit('event', info)
})
// Send the event to the statechart. It will be handled by all
// active states, starting at the root.
this.root.handleEvent(info)
this.emit('event', info)
return this
}

View file

@ -62,9 +62,7 @@ export class Drawing extends StateNode {
}
override onPointerMove: TLEventHandlers['onPointerMove'] = () => {
const {
editor: { inputs },
} = this
const { inputs } = this.editor
if (this.isPen !== inputs.isPen) {
// The user made a palm gesture before starting a pen gesture;
@ -282,8 +280,8 @@ export class Drawing extends StateNode {
}
private updateShapes() {
const { inputs } = this.editor
const { initialShape } = this
const { inputs } = this.editor
if (!initialShape) return
@ -440,7 +438,7 @@ export class Drawing extends StateNode {
const newSegment = newSegments[newSegments.length - 1]
const { pagePointWhereCurrentSegmentChanged } = this
const { currentPagePoint, ctrlKey } = this.editor.inputs
const { ctrlKey, currentPagePoint } = this.editor.inputs
if (!pagePointWhereCurrentSegmentChanged)
throw Error('We should have a point where the segment changed')
@ -623,16 +621,14 @@ export class Drawing extends StateNode {
if (newPoints.length > 500) {
this.editor.updateShapes([{ id, type: this.shapeType, props: { isComplete: true } }])
const { currentPagePoint } = this.editor.inputs
const newShapeId = createShapeId()
this.editor.createShapes<DrawableShape>([
{
id: newShapeId,
type: this.shapeType,
x: toFixed(currentPagePoint.x),
y: toFixed(currentPagePoint.y),
x: toFixed(inputs.currentPagePoint.x),
y: toFixed(inputs.currentPagePoint.y),
props: {
isPen: this.isPen,
segments: [
@ -647,7 +643,7 @@ export class Drawing extends StateNode {
this.initialShape = structuredClone(this.editor.getShape<DrawableShape>(newShapeId)!)
this.mergeNextPoint = false
this.lastRecordedPoint = this.editor.inputs.currentPagePoint.clone()
this.lastRecordedPoint = inputs.currentPagePoint.clone()
this.currentLineLength = 0
}

View file

@ -311,6 +311,17 @@ export class TestEditor extends Editor {
/* ------------------ Input Events ------------------ */
/**
Some of our updates are not synchronous any longer. For example, drawing happens on tick instead of on pointer move.
You can use this helper to force the tick, which will then process all the updates.
*/
forceTick = (count = 1) => {
for (let i = 0; i < count; i++) {
this.emit('tick', 16)
}
return this
}
pointerMove = (
x = this.inputs.currentScreenPoint.x,
y = this.inputs.currentScreenPoint.y,
@ -320,7 +331,7 @@ export class TestEditor extends Editor {
this.dispatch({
...this.getPointerEventInfo(x, y, options, modifiers),
name: 'pointer_move',
})
}).forceTick()
return this
}
@ -333,7 +344,7 @@ export class TestEditor extends Editor {
this.dispatch({
...this.getPointerEventInfo(x, y, options, modifiers),
name: 'pointer_down',
})
}).forceTick()
return this
}
@ -346,7 +357,7 @@ export class TestEditor extends Editor {
this.dispatch({
...this.getPointerEventInfo(x, y, options, modifiers),
name: 'pointer_up',
})
}).forceTick()
return this
}
@ -380,17 +391,17 @@ export class TestEditor extends Editor {
type: 'click',
name: 'double_click',
phase: 'up',
})
}).forceTick()
return this
}
keyDown = (key: string, options = {} as Partial<Exclude<TLKeyboardEventInfo, 'key'>>) => {
this.dispatch({ ...this.getKeyboardEventInfo(key, 'key_down', options) })
this.dispatch({ ...this.getKeyboardEventInfo(key, 'key_down', options) }).forceTick()
return this
}
keyRepeat = (key: string, options = {} as Partial<Exclude<TLKeyboardEventInfo, 'key'>>) => {
this.dispatch({ ...this.getKeyboardEventInfo(key, 'key_repeat', options) })
this.dispatch({ ...this.getKeyboardEventInfo(key, 'key_repeat', options) }).forceTick()
return this
}
@ -402,7 +413,7 @@ export class TestEditor extends Editor {
altKey: this.inputs.altKey && key !== 'Alt',
...options,
}),
})
}).forceTick()
return this
}
@ -416,7 +427,7 @@ export class TestEditor extends Editor {
altKey: this.inputs.altKey,
...options,
delta: { x: dx, y: dy },
})
}).forceTick(2)
return this
}
@ -438,7 +449,7 @@ export class TestEditor extends Editor {
...options,
point: { x, y, z },
delta: { x: dx, y: dy, z: dz },
})
}).forceTick()
return this
}
@ -482,7 +493,7 @@ export class TestEditor extends Editor {
...options,
point: { x, y, z },
delta: { x: dx, y: dy, z: dz },
})
}).forceTick()
return this
}
/* ------ Interaction Helpers ------ */

View file

@ -166,9 +166,9 @@ describe('When translating...', () => {
.pointerMove(1080, 800)
jest.advanceTimersByTime(100)
editor
.expectShapeToMatch({ id: ids.box1, x: 1300, y: 845.68 })
.expectShapeToMatch({ id: ids.box1, x: 1320, y: 845.68 })
.pointerUp()
.expectShapeToMatch({ id: ids.box1, x: 1300, y: 845.68 })
.expectShapeToMatch({ id: ids.box1, x: 1340, y: 857.92 })
})
it('translates multiple shapes', () => {

View file

@ -6,7 +6,7 @@ const isTest = () =>
const fpsQueue: Array<() => void> = []
const targetFps = 60
const targetTimePerFrame = 1000 / targetFps
const targetTimePerFrame = Math.ceil(1000 / targetFps)
let frame: number | undefined
let time = 0
let last = 0