[css-animations-2] Add declarative syntax for starting an animation in response to an input event

[szager-chromium]

Compositor-driven animations run smoothly even when the main thread is blocked. However, main thread processing is required to start an animation, and this fact makes it difficult to guarantee low latency in displaying visual response to input events.

Here's a proposal to provide a declarative syntax for specifying that a given event should start running an animation. The shape of the API is meant to enable implementations to optimistically start running the specified animation without waiting for main thread processing.

[ydaniv]

Thanks for proposing, this is indeed much needed!
I understand this is still just a rough sketch of the API, so opening some questions for discussion:

1. Currently animation-trigger works with timeilnes using the animation-trigger-timeline property. And the Trigger is modeled around the concept of a timeline, either time-based (auto) or scroll/view with ranges. In this proposal there is no timeline, or rather, there's yet another inner-trigger that triggers an auto timeline? So we need to understand how this sits together with the timeline part.
2. With timelines, you either specify a timeline using the *-timeline property on the source element, and then reused that via -name in the trigger, or specify the timeline using an anonymous function in the trigger property, same like animations. In this proposal the source element is defined using animation-trigger-name which just defines this element for a trigger but does not says what for. And then on the target element it's wrapped in a click() function that specifies the usage. Shouldn't we have a property on the source element that defines it as a click source, and then it can be used inside triggers? (getting somewhat a deja vu from CSS Toggles here)
3. I assume we want this feature to also play nicely with different types, i.e. repeat, alternate, state. These behaviors, together with click/hover handlers, are many times used with transitions. While it would be nice to have that working with animations, I wonder if we're not missing a chance to have both, but then this requires toggling a selector, which goes back to CSS Toggles again. But maybe if we could define a click source with a property, like mentioned in question (1), and then this could perhaps affect something like a native form of :state() selector then this could work out?

[css-meeting-bot]

The CSS Working Group just discussed [css-animations-2] Add declarative syntax for starting an animation in response to an input event.

The full IRC log of that discussion

<TabAtkins> szager: This is very rough sketch, not prepared to depend every details
<TabAtkins> szager: responsiveness is important part of good perf, always a struggle
<TabAtkins> szager: big part is input events require main-thread processing
<TabAtkins> szager: if you're familiar with Core Web Vitals, something we emphasize is interaction to first paint
<TabAtkins> szager: looking at IMP measurements in the wild, at higher percentiles main-thread processing >50% of that first response delay
<TabAtkins> szager: so we want to solve, how can we provide visual repsonse to input events without main thread processing
<TabAtkins> szager: so i think this proposal is in-line with compositor-driven animations
<TabAtkins> szager: we set up the animation in advance then hand it off to the compositor
<TabAtkins> szager: i'll say, my terminology is chrome-specific but i think the principles apply to everyone, they'll be implementable for everyone
<TabAtkins> szager: so we create an animation on the main thread, then trasnfer it to compositor thread if possible, and it can run smoothly after that
<TabAtkins> szager: in that spirit, we shoudl be able to do that (set up in advance) but rather than a time delay, say we'll start when we detect an input event
<TabAtkins> szager: currently in web platform, input events are only dete3ctable on main thread
<TabAtkins> szager: but this declarative syntax lets us analyze that and enact that on the compositor thread before it even hits the main thread
<TabAtkins> szager: so the result is we cut out the main-thread processing entirely when we start processing an input event
<TabAtkins> szager: big risk up front - this is a predictive model, like thread scrolling or composited animations
<TabAtkins> szager: we're predicting that, if the main thread weren't blocked, it would say we shoudl run this animation. this can be wrong - the main thread might try to cancel it but it's blocked
<TabAtkins> szager: in composited animations, the downside of the prediction being incorrect isn't important
<TabAtkins> szager: in this it could be a bit more consequential
<TabAtkins> szager: someone could click on something, animation starts immediatley suggesting the click was received
<astearns> q+
<TabAtkins> szager: and then eventually when we flush the rendering and actually deal with the input, it could just stop
<TabAtkins> szager: could be weird for the user to see this inconsistent animation
<TabAtkins> szager: so that's biggest risk i think. should keep our eyes on it
<TabAtkins> astearns: couple question
<TabAtkins> astearns: are you at all worried about having multiple things trigger off of a user interaction, and have the animation kick off long before any secondary effects on the main thread?
<astearns> ack astearns
<flackr> q+
<smfr> q+
<TabAtkins> szager: at least for chrome, our devrel, we always push to give a visual response asap
<TabAtkins> szager: so our guidacne is to start the animation *immediately*, and only once that's actually running should you do your business logic
<ydaniv> q+
<TabAtkins> szager: so there's always a risk that main-thread processing will invalidate what we're starting. This isn't a replacement for main-thread handling.
<TabAtkins> szager: we're just taking that first step (kick the animation immediatley) and making it declarative
<TabAtkins> szager: so the rest of the model is the same
<TabAtkins> astearns: that's helpful
<TabAtkins> astearns: the other thing is, a concern about having some user events with this preemptive animation, and other where they're not
<TabAtkins> astearns: there are several user inputs in a row, some trigger fast and others are slow, and the reaction to user events is out of order
<astearns> ack flackr
<TabAtkins> flackr: in case people ahven't recognized this, thi si sperfectly analogous to scrolling
<TabAtkins> flackr: in that you can set up a scroll on the compositor that is invalidated by a scrollTo in the next frame
<TabAtkins> flackr: so it's not new, and i think this is a reasonable explanation
<TabAtkins> flackr: some more cmplications tho
<TabAtkins> flackr: some event types are only generated if other types aren't preventDefaulted
<TabAtkins> flackr: like click can be canceled by down/up
<TabAtkins> flackr: so we need to either restrict to primary events, or do something else
<TabAtkins> flackr: also, the user interacts with what they see, they're not in the new frame yet
<TabAtkins> flackr: so this does feel better. i click on what i see, th enext frame can certainly move it or whatever, but maybe we should do hit testing in the frame the user is seeing rather than the positions that dom modifications see
<TabAtkins> flackr: maybe we should explore that
<TabAtkins> szager: the idea of doing event hit testing based on last displayed content has been proposed a millino times, it's a can of worms.
<TabAtkins> szager: i like the idea, but it's a breaking change
<TabAtkins> szager: but that's the only thing i don't like about it. in every other respect it's better i think.
<TabAtkins> szager: don't know that i want to conflate that with this project tho
<TabAtkins> szager: other point about non-primary events
<TabAtkins> szager: the reason we can do this project in chrome is events arrive on compositor before they hit the main thread
<TabAtkins> szager: but we only have limited insight into the input targetting
<TabAtkins> szager: our hit-testing is non-canonical
<TabAtkins> szager: so like we can't do a strict determination fo the event target until we hit the main thread
<TabAtkins> szager: so that's a source fo inaccuracy
<TabAtkins> szager: part of this project would be to see ho wmuch more hit-testing capability we need to add to our compositor thread
<TabAtkins> szager: and think it would be a simlar process for other browsers
<TabAtkins> szager: end-gaem of that process is doing precise hit-testing on compositor thread from the msot recently displayed content
<TabAtkins> szager: so it's in that arena. dont' wanna commit to that huge breaking change now, but want to explore that some
<TabAtkins> flackr: secondary events are not sent to the compositor, they're generated in blink as part of primary event
<TabAtkins> flackr: we'd need to add special processing, compositor only see mouseup/mousedown, we synthesize click on main
<TabAtkins> szager: i think chrome could do this, yes. no hard proposal yet. question is how much functionality do we need on the compositor.
<TabAtkins> flackr: my point is, we always have the option to say some event types are triggered on main
<TabAtkins> szager: i'll say, this proposal doesn't need to involve compositor thread. as-is it could be done entirely on the main thread
<TabAtkins> szager: the compositor is an optimization. it's the point of this feature, yes, but it could just run on the main
<TabAtkins> szager: so we can start from the baseline of makign it work right without compositor optimizations
<TabAtkins> szager: and then this is a predictive model, we find as many scenarios as possible that we can do on the compositor. can't define the boundary yet, and it probably changes over time, and could vary between brwosers
<TabAtkins> szager: but i think the design of the api, i always kept in mind making it possible to add the optimizations after the fact
<TabAtkins> flackr: i think we agree
<TabAtkins> flackr: hit-testing on the previously-seen thing is something that gives consistency with what the user sees and what they expect
<TabAtkins> flackr: so i think it does make sense exploring
<TabAtkins> szager: i agree in every state except compat. so if this is a compat anchor i don't want to be tied to it.
<astearns> ack smfr
<TabAtkins> smfr: this feels more consequential than scrolling as to getting it wrong.
<TabAtkins> smfr: you can't do precise hit-testing, and aren't running inputs properly
<TabAtkins> smfr: so you dont' know if something else canceled
<TabAtkins> smfr: this is important for things like iframes
<TabAtkins> szager: in chrome we do do precise hit-testing for iframes
<TabAtkins> smfr: in webkit, if there's a clip path overlapping the iframe, we can't do precise hit-testing on our compositor
<TabAtkins> smfr: so you say we kick it off on the compositor and then cancel it if the real event doesn't fire
<TabAtkins> smfr: so what happens with animation events?
<TabAtkins> szager: we wont' fire animation events until animation start has been affirmed on the main thread
<TabAtkins> szager: so no ordering changes
<flackr> qq+
<TabAtkins> szager: one place that might look different is start time, but in terms of order animationStart event happens at normal position
<TabAtkins> szager: i agree with everything you're saying about iframe security/etc
<TabAtkins> szager: so i'll point back to, this can be done on the main thread.
<TabAtkins> szager: we considered a broader syntax, but decided to keep it narrow and just use animation-trigger to keep it simple
<TabAtkins> szager: so can do it all on main, then judiciously move to compositor thread
<astearns> ack flackr
<Zakim> flackr, you wanted to react to smfr
<TabAtkins> flackr: we can pick and choose the precise cases where we're completely or reasonably confident that we have the target correct and it wont' be stopped
<TabAtkins> flackr: so it's possible to do it in a way that's correct save for extreme dom modification
<TabAtkins> szager: it's true that compositor hit testing isn't authoritiative, but in chrome the compositor thread is aware *when* it's not authoritative
<TabAtkins> szager: then we defer to the main thread, like if there's a clip path
<TabAtkins> szager: so if there's an important decision to be made, we can push off the optimization. hopefully others can too.
<astearns> ack ydaniv
<TabAtkins> ydaniv: this sounds very interesting
<TabAtkins> ydaniv: i was also worried about what simon mentioned, regarding stopPropagation, but if it's considered then good
<TabAtkins> ydaniv: had a lot of questions about api
<TabAtkins> ydaniv: but most important is probably 3rd question i wrote on the issue
<TabAtkins> ydaniv: this is also very desired with transitions, but then it's not thru animation api but thru selectors
<TabAtkins> ydaniv: that maybe goes back to css toggles, which is sorta deceased
<TabAtkins> ydaniv: wonder if this is something we coudl marry with a state pseudoclass
<TabAtkins> szager: one of our early ideas was a pseudo-state
<TabAtkins> szager: we went thru it and the difficulty... it's probably most similar to :active
<TabAtkins> szager: but looking deep into it, we don't know where the start/stop of the pseudo-state is.
<TabAtkins> szager: this is a discrete event, not a toggle back and forth between two states
<TabAtkins> szager: can't just analyze the dom and styels and say "we know we're in this state". it's ephemeral and it's gone
<TabAtkins> szager: so it just doesn't seem to be a good match to pseudo-states, anything toggleable
<TabAtkins> szager: this isn't even like viewport visibility, where you're either in or out. this is immediate.
<TabAtkins> szager: i foudn the best mental model is element.getAnimations.play(), like that. once that line is executed, you can't detect that it happened, you just know the animation is now playing. no idea how it started.
<TabAtkins> szager: so just the idea of a state that toggles on or off doesn't seem to be the right model for this
<TabAtkins> ydaniv: so this moves to the second question, if this state is transient, how will this work with different types of triggers?
<TabAtkins> ydaniv: there we have "alternate" types... the state is in the animation then
<flackr> qq+
<TabAtkins> szager: right, i think if the state is the namation state, you can pause/play from there
<TabAtkins> szager: if you explicitly call .play() on the animation, the input event could pause that animation
<astearns> ack flackr
<Zakim> flackr, you wanted to react to ydaniv
<astearns> zakim, close queue
<Zakim> ok, astearns, the speaker queue is closed
<TabAtkins> flackr: having thought a lot about animatin-trigger, they have a thing that happens when the triggered state goes from outsdie->inside or reverse. i think each occurence of a discrete event woudl be a transition similarly.
<TabAtkins> flackr: and what the trigger does depends on what trigger's state. "state" woudl pause/unpause, "alternate" would play it backwards/forwards, etc. each has a defined meaning already
<TabAtkins> szager: yeah, since triggers are defined in terms of transition events, it works cleanly with inputs as long as the state lives in the animation
<astearns> ack pdr
<TabAtkins> flackr: right. currnently it's a state transition that toggles it, but that's not required
<TabAtkins> pdr: quick comment on original question,a bout mixing fast and slow properties
<TabAtkins> pdr: animation lets us build on the existing infrastructure
<TabAtkins> pdr: if you ahve a layout-affecting property in an animation, it can already pull the whole animation out to main thread
<TabAtkins> pdr: with animations you define up front the things that are affected, while with selectors it's open-ended
<TabAtkins> pdr: so i think animation approach works iwthin those problems
<TabAtkins> astearns: thank you for the introduction, let's flesh out the details more
<astearns> zakim, open queue
<Zakim> ok, astearns, the speaker queue is open

[Gaubee]

Some suggestions based on the draft

These suggestions are independent and can be combined with each other.

1. Support multiple gestures, similar to background-image stacking, using , as a separator.

   animation-trigger: click(spin-trigger, once), pointerenter(spin-trigger, once);

2. Support more fine-grained declarations while removing specific CSS functions, similar to the background property.

   animation-trigger-event: click;
   animation-trigger-name: spin-trigger;
   animation-trigger-iteration-count: once; /* Not recommended—using `animation-iteration-count` may already suffice */
   
   /* Final combined form */
   animation-trigger: click spin-trigger once;

3. Support anchoring to specific elements, not just the trigger element itself.

   animation-trigger-element: self; /* default */
   animation-trigger-element: element(#id); /* use ID selector */

   Alternatively, follow the approach of anchor-name:

   .target-element {
     anchor-name: --myAnchor;
   }
   .button {
     animation-trigger-anchor: --myAnchor;
   }

[szager-chromium]

@ydaniv sorry for the late reply...

1. Currently animation-trigger works with timeilnes using the animation-trigger-timeline property. And the Trigger is modeled around the concept of a timeline, either time-based (auto) or scroll/view with ranges. In this proposal there is no timeline, or rather, there's yet another inner-trigger that triggers an auto timeline? So we need to understand how this sits together with the timeline part.

2. With timelines, you either specify a timeline using the *-timeline property on the source element, and then reused that via -name in the trigger, or specify the timeline using an anonymous function in the trigger property, same like animations. In this proposal the source element is defined using animation-trigger-name which just defines this element for a trigger but does not says what for. And then on the target element it's wrapped in a click() function that specifies the usage. Shouldn't we have a property on the source element that defines it as a click source, and then it can be used inside triggers? (getting somewhat a deja vu from CSS Toggles here)

My initial thought is that we should specify that animation-trigger-timeline has no effect if the animation trigger is event-based. Even auto seems confusing to me in this context.

3. I assume we want this feature to also play nicely with different types, i.e. repeat, alternate, state. These behaviors, together with click/hover handlers, are many times used with transitions. While it would be nice to have that working with animations, I wonder if we're not missing a chance to have both, but then this requires toggling a selector, which goes back to CSS Toggles again. But maybe if we could define a click source with a property, like mentioned in question (1), and then this could perhaps affect something like a native form of :state() selector then this could work out?

I'm hoping to avoid involving CSS states, because doing so would make the optimized path (i.e. starting/stopping the animation on the compositor thread without main thread involvement) much more difficult to implement correctly. It is fairly straightforward to figure out on the compositor thread when a discrete input event targets a particular element. It's likely much harder to figure out on the compositor thread when a particular CSS state applies.

As you say, it should be possible to support repeat, alternate, and state, and I'm hoping that's sufficiently expressive for common use cases. I'm open to other ideas, but my primary motivation for this feature is to enable the compositor-thread fast path and I want to be careful not to defeat it.

BTW @ydaniv, if you are interested in getting involved with this proposal, we would be very grateful for your help in drafting a spec PR. Let me know!

[ydaniv]

My initial thought is that we should specify that animation-trigger-timeline has no effect if the animation trigger is event-based. Even auto seems confusing to me in this context.

Yeah, we need something new here, not a timeline, more like a toggle. We can imagine a timeline as a stateful toggle, like @flackr said in the meeting, and then click sort of a stateless toggle.
I guess one way would be to overload the -timeline sub-property here with a new <toggle> value, and then have something like a sub-property alias for it - if that's possible, like animation-trigger-toggle - just a thought.

Another thing to note is that ranges don't make sense for click. Also need to note that timelines are 1 dimensional. For click you need a hit area, like a rect, and this is something we may want to set per definition - on the source element, rather than per usage - on the target.
With that in mind, we need to think how that is exposed. Do we go with something like insets as <length percentage>{0,4}? Or go crazy with something clip-path-style [<basic-shape>] || <geometry-box>]?

And then, only left to figure how will the property for the click source definition look like.

As you say, it should be possible to support repeat, alternate, and state, and I'm hoping that's sufficiently expressive for common use cases. I'm open to other ideas, but my primary motivation for this feature is to enable the compositor-thread fast path and I want to be careful not to defeat it.

Sure thing.

BTW @ydaniv, if you are interested in getting involved with this proposal, we would be very grateful for your help in drafting a spec PR. Let me know!

Yeah sure, I'm very interested in this proposal. However, I'd like to clear out the basic details of our expectations here. Is it something along the lines of:

#source {
  click-toggle: --my-clicker 20px 20% 10px;
}

#target {
  animation-trigger: alternate --my-clicker;
}

OR:

#target {
  animation-trigger: alternate click(15px 2rem);
}

[szager-chromium]

Is it necessary to support animation-trigger-timeline at all for input events? As you say, it's stateless; could we just define the behavior entirely in terms of the animation state? animation-trigger-timeline feel superfluous.

As for hit testing, we were not planning to offer any inset or other geometry features; it should behave the same as event targeting.

The solution sketch from our proposal is still how we envision the feature working.