584380ba8b
This PR buffs input events. ## The story so far In the olde days, we throttled events from the canvas events hook so that a pointer event would only be sent every 1/60th of a second. This was fine but made drawing on the iPad / 120FPS displays a little sad. Then we removed this throttle. It seemed fine! Drawing at 120FPS was great. We improved some rendering speeds and tightened some loops so that the engine could keep up with 2x the number of points in a line. Then we started noticing that iPads and other screens could start choking on events as it received new inputs and tried to process and render inputs while still recovering from a previous dropped frame. Even worse, on iPad the work of rendering at 120FPS was causing the browser to throttle the app after some sustained drawing. Yikes! ### Batching I did an experimental PR (#3180) to bring back batching but do it in the editor instead. What we would do is: rather than immediately processing an event when we get it, we would instead put the event into a buffer. On the next 60FPS tick, we would flush the buffer and process all of the events. We'd have them all in the same transaction so that the app would only render once. ### Render batching? We then tried batching the renders, so that the app would only ever render once per (next) frame. This added a bunch of complexity around events that needed to happen synchronously, such as writing text in a text field. Some inputs could "lag" in a way familiar to anyone who's tried to update an input's state asynchronously. So we backed out of this. ### Coalescing? Another idea from @ds300 was to "coalesce" the events. This would be useful because, while some interactions like drawing would require the in-between frames in order to avoid data loss, most interactions (like resizing) didn't actually need the in-between frames, they could just use the last input of a given type. Coalescing turned out to be trickier than we thought, though. Often a state node required information from elsewhere in the app when processing an event (such as camera position or page point, which is derived from the camera position), and so the coalesced events would need to also include this information or else the handlers wouldn't work the way they should when processing the "final" event during a tick. So we backed out of the coalescing strategy for now. Here's the [PR that removes](937469d69d) it. ### Let's just buffer the fuckers So this PR now should only include input buffering. I think there are ways to achieve the same coalescing-like results through the state nodes, which could gather information during the `onPointerMove` handler and then actually make changes during the `onTick` handler, so that the changes are only done as many time as necessary. This should help with e.g. resizing lots of shapes at once. But first let's land the buffering! --- Mitja's original text: This PR builds on top of Steve's [experiment PR](https://github.com/tldraw/tldraw/pull/3180) here. It also adds event coalescing for [`pointerMove` events](https://github.com/tldraw/tldraw/blob/mitja/input-buffering/packages/editor/src/lib/editor/Editor.ts#L8364-L8368). The API is [somewhat similar ](https://developer.mozilla.org/en-US/docs/Web/API/PointerEvent/getCoalescedEvents) to `getCoalescedEvent`. In `StateNodes` we register an `onPointerMove` handler. When the event happens it gets called with the event `info`. There's now an additional field on `TLMovePointerEvent` called `coalescedInfo` which includes all the events. It's then on the user to process all of these. I decided on this API since it allows us to only expose one event handler, but it still gives the users access to all events if they need them. We would otherwise either need to: - Expose two events (coalesced and non-coalesced one and complicate the api) so that state nodes like Resizing would not be triggered for each pointer move. - Offer some methods on the editor that would allow use to get the coalesced information. Then the nodes that need that info could request it. I [tried this](9ad973da3a (diff-32f1de9a5a9ec72aa49a8d18a237fbfff301610f4689a4af6b37f47af435aafcR67)), but it didn't feel good. This also complicated the editor inputs. The events need to store information about the event (like the mouse position when the event happened for `onPointerMove`). But we cannot immediately update inputs when the event happens. To make this work for `pointerMove` events I've added `pagePoint`. It's [calculated](https://github.com/tldraw/tldraw/pull/3223/files#diff-980beb0aa0ee9aa6d1cd386cef3dc05a500c030638ffb58d45fd11b79126103fR71) when the event triggers and then consumers can get it straight from the event (like [Drawing](https://github.com/tldraw/tldraw/pull/3223/files#diff-32f1de9a5a9ec72aa49a8d18a237fbfff301610f4689a4af6b37f47af435aafcR104)). ### Change Type <!-- ❗ Please select a 'Scope' label ❗️ --> - [x] `sdk` — Changes the tldraw SDK - [ ] `dotcom` — Changes the tldraw.com web app - [ ] `docs` — Changes to the documentation, examples, or templates. - [ ] `vs code` — Changes to the vscode plugin - [ ] `internal` — Does not affect user-facing stuff <!-- ❗ Please select a 'Type' label ❗️ --> - [ ] `bugfix` — Bug fix - [ ] `feature` — New feature - [x] `improvement` — Improving existing features - [ ] `chore` — Updating dependencies, other boring stuff - [ ] `galaxy brain` — Architectural changes - [ ] `tests` — Changes to any test code - [ ] `tools` — Changes to infrastructure, CI, internal scripts, debugging tools, etc. - [ ] `dunno` — I don't know ### Test Plan 1. Add a step-by-step description of how to test your PR here. 4. - [ ] Unit Tests - [ ] End to end tests ### Release Notes - Add a brief release note for your PR here. --------- Co-authored-by: Steve Ruiz <steveruizok@gmail.com>
166 lines
5.4 KiB
TypeScript
166 lines
5.4 KiB
TypeScript
import { expect } from '@playwright/test'
|
|
import { getAllShapeTypes, setup } from '../shared-e2e'
|
|
import test from './fixtures/fixtures'
|
|
|
|
export function sleep(ms: number) {
|
|
return new Promise((resolve) => setTimeout(resolve, ms))
|
|
}
|
|
|
|
const clickableShapeCreators = [
|
|
{ tool: 'draw', shape: 'draw' },
|
|
{ tool: 'frame', shape: 'frame' },
|
|
{ tool: 'note', shape: 'note' },
|
|
{ tool: 'text', shape: 'text' },
|
|
{ tool: 'rectangle', shape: 'geo' },
|
|
{ tool: 'ellipse', shape: 'geo' },
|
|
{ tool: 'triangle', shape: 'geo' },
|
|
{ tool: 'diamond', shape: 'geo' },
|
|
{ tool: 'cloud', shape: 'geo' },
|
|
{ tool: 'hexagon', shape: 'geo' },
|
|
// { tool: 'octagon', shape: 'geo' },
|
|
{ tool: 'star', shape: 'geo' },
|
|
{ tool: 'rhombus', shape: 'geo' },
|
|
{ tool: 'oval', shape: 'geo' },
|
|
{ tool: 'trapezoid', shape: 'geo' },
|
|
{ tool: 'arrow-right', shape: 'geo' },
|
|
{ tool: 'arrow-left', shape: 'geo' },
|
|
{ tool: 'arrow-up', shape: 'geo' },
|
|
{ tool: 'arrow-down', shape: 'geo' },
|
|
{ tool: 'x-box', shape: 'geo' },
|
|
{ tool: 'check-box', shape: 'geo' },
|
|
]
|
|
|
|
const draggableShapeCreators = [
|
|
{ tool: 'draw', shape: 'draw' },
|
|
{ tool: 'arrow', shape: 'arrow' },
|
|
{ tool: 'frame', shape: 'frame' },
|
|
{ tool: 'note', shape: 'note' },
|
|
{ tool: 'text', shape: 'text' },
|
|
{ tool: 'line', shape: 'line' },
|
|
{ tool: 'rectangle', shape: 'geo' },
|
|
{ tool: 'ellipse', shape: 'geo' },
|
|
{ tool: 'triangle', shape: 'geo' },
|
|
{ tool: 'diamond', shape: 'geo' },
|
|
{ tool: 'cloud', shape: 'geo' },
|
|
{ tool: 'hexagon', shape: 'geo' },
|
|
// { tool: 'octagon', shape: 'geo' },
|
|
{ tool: 'star', shape: 'geo' },
|
|
{ tool: 'rhombus', shape: 'geo' },
|
|
{ tool: 'oval', shape: 'geo' },
|
|
{ tool: 'trapezoid', shape: 'geo' },
|
|
{ tool: 'arrow-right', shape: 'geo' },
|
|
{ tool: 'arrow-left', shape: 'geo' },
|
|
{ tool: 'arrow-up', shape: 'geo' },
|
|
{ tool: 'arrow-down', shape: 'geo' },
|
|
{ tool: 'x-box', shape: 'geo' },
|
|
{ tool: 'check-box', shape: 'geo' },
|
|
]
|
|
|
|
const otherTools = [{ tool: 'select' }, { tool: 'eraser' }, { tool: 'laser' }]
|
|
|
|
test.describe('Shape Tools', () => {
|
|
test.beforeEach(setup)
|
|
test('creates shapes with other tools', async ({ toolbar, page }) => {
|
|
await page.keyboard.press('Control+a')
|
|
await page.keyboard.press('Backspace')
|
|
expect(await getAllShapeTypes(page)).toEqual([])
|
|
|
|
for (const { tool } of otherTools) {
|
|
// Find and click the button
|
|
if (!(await page.getByTestId(`tools.${tool}`).isVisible())) {
|
|
if (!(await toolbar.moreToolsButton.isVisible())) {
|
|
throw Error(`Tool more is not visible`)
|
|
}
|
|
await toolbar.moreToolsButton.click()
|
|
|
|
if (!(await page.getByTestId(`tools.more.${tool}`).isVisible())) {
|
|
throw Error(`Tool in more panel is not visible`)
|
|
}
|
|
await page.getByTestId(`tools.more.${tool}`).click()
|
|
|
|
await toolbar.moreToolsButton.click()
|
|
}
|
|
|
|
if (!(await page.getByTestId(`tools.${tool}`).isVisible())) {
|
|
throw Error(`Tool ${tool} is not visible`)
|
|
}
|
|
|
|
await page.getByTestId(`tools.${tool}`).click()
|
|
|
|
// Button should be selected
|
|
await expect(page.getByTestId(`tools.${tool}`)).toHaveAttribute('aria-checked', 'true')
|
|
}
|
|
})
|
|
|
|
test('creates shapes clickable tools', async ({ page, toolbar }) => {
|
|
await page.keyboard.press('v')
|
|
await page.keyboard.press('Control+a')
|
|
await page.keyboard.press('Backspace')
|
|
expect(await getAllShapeTypes(page)).toEqual([])
|
|
|
|
for (const { tool, shape } of clickableShapeCreators) {
|
|
// Find and click the button
|
|
if (!(await page.getByTestId(`tools.${tool}`).isVisible())) {
|
|
await toolbar.moreToolsButton.click()
|
|
await page.getByTestId(`tools.more.${tool}`).click()
|
|
await toolbar.moreToolsButton.click()
|
|
}
|
|
await page.getByTestId(`tools.${tool}`).click()
|
|
|
|
// Button should be selected
|
|
await expect(page.getByTestId(`tools.${tool}`)).toHaveAttribute('aria-checked', 'true')
|
|
|
|
// Click on the page
|
|
await page.mouse.click(200, 200)
|
|
await page.waitForTimeout(20)
|
|
|
|
// We should have a corresponding shape in the page
|
|
expect(await getAllShapeTypes(page)).toEqual([shape])
|
|
|
|
// Reset for next time
|
|
await page.mouse.click(50, 50) // to ensure we're not focused
|
|
await page.keyboard.press('v') // go to the select tool
|
|
await page.waitForTimeout(20)
|
|
await page.keyboard.press('Control+a')
|
|
await page.keyboard.press('Backspace')
|
|
}
|
|
|
|
expect(await getAllShapeTypes(page)).toEqual([])
|
|
})
|
|
|
|
test('creates shapes with draggable tools', async ({ page, toolbar }) => {
|
|
await page.keyboard.press('Control+a')
|
|
await page.keyboard.press('Backspace')
|
|
expect(await getAllShapeTypes(page)).toEqual([])
|
|
|
|
for (const { tool, shape } of draggableShapeCreators) {
|
|
// Find and click the button
|
|
if (!(await page.getByTestId(`tools.${tool}`).isVisible())) {
|
|
await toolbar.moreToolsButton.click()
|
|
await page.getByTestId(`tools.more.${tool}`).click()
|
|
await toolbar.moreToolsButton.click()
|
|
}
|
|
|
|
await page.getByTestId(`tools.${tool}`).click()
|
|
|
|
// Button should be selected
|
|
await expect(page.getByTestId(`tools.${tool}`)).toHaveAttribute('aria-checked', 'true')
|
|
|
|
// Click and drag
|
|
await page.mouse.move(200, 200)
|
|
await page.mouse.down()
|
|
await page.mouse.move(250, 250)
|
|
await page.mouse.up()
|
|
|
|
// We should have a corresponding shape in the page
|
|
expect(await getAllShapeTypes(page)).toEqual([shape])
|
|
|
|
// Reset for next time
|
|
await page.mouse.click(50, 50) // to ensure we're not focused
|
|
await page.keyboard.press('v')
|
|
await page.waitForTimeout(20)
|
|
await page.keyboard.press('Control+a')
|
|
await page.keyboard.press('Backspace')
|
|
}
|
|
})
|
|
})
|