← Blog

Is there a fingerprint in AI-generated motion? I tested my own model

A frequency-domain look at real and generated gestures: what energy and coordination reveal, what fooled me, and where ZeroEGGS differs from a real actor.

· 17 min read

Generated motion has got good. Watch a modern speech-to-gesture or text-to-motion model and, at a glance, it is often hard to say exactly what is off. AI-generated images, on the other hand, have a well-known weakness: look at them in the frequency domain and many generators leave regular traces that a simple classifier can pick up[1].

So a question I've been curious about for a while is whether generated motion has a fingerprint of its own, and if it does, whether that fingerprint survives the cleanup filtering every motion-capture pipeline applies anyway. To find out I picked the model I know best: ZeroEGGS[2], a speech-to-gesture model I developed with colleagues at Ubisoft La Forge.

I want to be upfront that this is a case study rather than a definitive answer (one actor, one model), but I learned a few things along the way which I think are worth sharing. Including the fact that my hypothesis was half wrong.

The article is split into a few parts. First I'll introduce the two fingerprints I tried and show how to compute them. Second, I'll test them on a toy walker where I know the right answer. Third, I'll run the real test: an actor's motion capture against ZeroEGGS's output for the same speech. And finally I'll close with the lessons I took away. The important pieces of code (Python, NumPy and SciPy) are included throughout.

Two Fingerprints

Overview

Here is the set-up. We have a window of motion, and we want a set of numbers, a fingerprint, which tells us whether it came from a person or from a model.

Since gestures are mostly about the upper body, I use eleven joints from the chest up: the chest, neck and head, plus the collarbones, shoulders, elbows and wrists on each side. Before anything else I remove the root's translation and rotation, so we're looking at the body's own movement rather than where the actor happens to be standing:

def body_positions(path):
    d = bvh.load(path)
    lrot = quat.from_euler(np.radians(d['rotations']), order=d['order'])
    lpos = d['positions'].copy()

    # Remove the root's translation and rotation so we only
    # see the body's own movement, not where it is standing
    lpos[:, 0] = 0.0
    lrot[:, 0] = np.array([1.0, 0.0, 0.0, 0.0])

    _, gpos = quat.fk(lrot, lpos, d['parents'])
    x = gpos[:, joint_indices]                  # frames x joints x 3
    return x.transpose(1, 2, 0).reshape(-1, len(x))   # signals x frames

This gives us 33 signals (11 joints × 3 axes) per window, which we then split up by tempo: from slow sway (0.25–0.5 Hz), through the 1–3 Hz rhythm of gesture strokes, up to quick beats and twitches (4–8 Hz).

1. Energy

The first fingerprint is the obvious one, and the motion equivalent of the image trick: how much each signal moves at each tempo. We estimate the power spectrum using Welch's method and average it within each band. I also compute a variant I'll call energy shape, which divides out each signal's total power, so it describes how movement is spread across tempos rather than how much movement there is:

bands = [(0.25, 0.5), (0.5, 1), (1, 2), (2, 4), (4, 8)]

f, P = signal.welch(x, fs=60, nperseg=256, axis=-1)
band_power = np.stack([
    np.log10(P[:, (f >= lo) & (f < hi)].mean(1)) for lo, hi in bands], axis=1)

energy = band_power.ravel()

# Energy shape: divide out each signal's total power
shape = (band_power - np.log10((10 ** band_power).sum(1, keepdims=True))).ravel()

2. Coordination

The second fingerprint looks at pairs of joints rather than single ones. For each pair we compute the coherence, which measures how consistently the two keep time with each other at each tempo, plus their timing offset, which is just the angle of the cross-spectrum:

f, Pxy = signal.csd(x[:, None, :], x[None, :, :], fs=60, nperseg=256, axis=-1)
Pxx = np.real(np.einsum('iif->if', Pxy))

# How consistently each pair keeps time, per frequency (0 to 1)
coherence = np.abs(Pxy) ** 2 / (Pxx[:, None, :] * Pxx[None, :, :])

# And their timing offset, per frequency
offset = np.angle(Pxy)

This was the fingerprint I was betting on, for two reasons. First, coherence ignores how big the movement is. Second, if you filter every joint with the same filter (which is exactly what mocap cleanup does), the filter cancels out of the ratio above. So cleanup doesn't change coherence, as long as some signal survives. The little demo below shows both properties:

What coherence measures

Two joints moving to the same 1 Hz rhythm. Change how steady their timing is, or how big joint B moves.

joint A joint B

–coherence at the 1 Hz rhythm
(0 = no steady timing, 1 = locked)

Coherence is computed over 96 seconds of motion; the plot shows the last 8. Making B bigger or smaller barely moves the number: coherence is about timing, not size. That is also why filtering every joint the same way leaves it unchanged.

It also hints at the one thing coherence does not measure: what the offset actually is. Two joints can keep perfectly steady time with entirely the wrong offset, and we'll see shortly why that matters.

3. The Detector

For the detector I wanted the realistic setting, where you don't have any examples of the fake you're up against. So it only ever sees real motion. It learns the mean and spread of every fingerprint number, and scores a new window by how far it falls outside of that:

def one_class_score(train, test):
    mu, sd = train.mean(0), train.std(0, ddof=1)
    return (((test - mu) / sd) ** 2).mean(1)

def auc(fake_scores, real_scores):
    u = mannwhitneyu(fake_scores, real_scores).statistic
    return u / (len(fake_scores) * len(real_scores))

The scores then get turned into an AUC: the chance that a fake window scores as less real than a real one. An AUC of 0.5 means the detector can't tell them apart, and 1 means it always catches the fake.

I should say that the question itself isn't new. In 2005, Ren, Patrick, Efros, Hodgins and Rehg trained statistical models to tell natural motion from keyframed, edited and noisy motion[3], spectra are used to score predicted motion[4], and there are detectors for AI-generated videos of people[5]. What I couldn't find was a frequency fingerprint applied to skeleton motion from a modern generator, and tested against cleanup.

A Toy Walker

Before touching real data I wanted a case where I know the right answer, so I built a toy walker: ten joint angles driven by one shared stride rhythm, plus some postural sway, a little tremor and sensor noise. Then I built a set of fakes, each one breaking something different:

The cleaned control and the Frankenstein fake are only a few lines each:

# Cleaned: real motion, low-pass filtered at 6 Hz like a mocap cleanup
b, a = signal.butter(4, 6.0, fs=120)   # the toy walker runs at 120 frames per second
cleaned = signal.filtfilt(b, a, x, axis=-1)

# Frankenstein: every joint is real, but from a different recording
def frankenstein(rng, recordings):
    picks = rng.choice(len(recordings), size=num_joints, replace=False)
    return np.stack([recordings[p][j] for j, p in enumerate(picks)])

The easiest way to see what these fakes do to coordination is to plot one joint against another. Here is the left hip angle against the left shoulder angle for three of them:

One joint plotted against another

Left hip angle (across) against left shoulder angle (up), over 20 seconds of a simplified toy walker simulated in your browser

The real walker traces a tight loop leaning down to the right: the arm swings opposite the leg. The Frankenstein walker's hip and shoulder come from different walks with slightly different cadences, so they drift in and out of step and the trace fills a whole box. The wrong-arm walker is just as tight as the real one, but leans the other way: steady timing, wrong offset.

And here is what the detector makes of all of them:

Toy walker: how well each kind of motion is told apart from real

Detector trained only on real walks. Dot: average score over 20 runs; line: ± one standard deviation.

The score (AUC) is the chance that the detector rates a clip of this kind as less real than a real one. 0.5 means it can't tell them apart; 1 means every one of them looks less real than every real clip.
Show as a table

The two fingerprints fail in opposite places. Energy flags every cleaned walk as fake, yet misses the Frankenstein, learned-style and wrong-arm walkers almost entirely. Coordination ignores the cleanup (0.52) and catches all three (1.00). Jitter is the one fake where energy does better (0.87 vs 0.69), which makes sense: it adds movement without changing much about timing.

Getting here took a few attempts, and each thing that went wrong taught me something:

Of course, the toy has an obvious limit: I built the "real" motion myself, so it shows how the method works, not that real generators actually fail this way. For that we need real data.

ZeroEGGS in a Nutshell

Before running the real test it's worth a quick recap of what ZeroEGGS actually is, since its design matters when we come to interpret the results.

ZeroEGGS takes two inputs: a recording of speech, and a short clip of motion showing the style you want. From those it generates full-body gesture motion that moves in time with the speech, in the style of the example. The "zero-shot" part is that the example clip can be in a style the model never saw during training: give it a few seconds of someone gesturing nervously, and it will try to gesture nervously to any new speech you give it.

The same 3D character gesturing in seven different styles, each rendered in a different colour
The same character gesturing in seven different styles with ZeroEGGS. © Ubisoft La Forge. There's a video of it in action on the project page.

Under the hood it has three parts:

How ZeroEGGS works

Two encoders feed a decoder that writes the motion one frame at a time

Speech audio
Speech encoder1-D convolutions over the audio's spectrogram
Example clipany style, even one never seen in training
Style encoderattention, variational: a sampled 64-number style embedding
Decodertwo-layer recurrent network (GRU); each new pose depends on the previous one↻ previous pose
Full-body motion75 joints including fingers, 60 frames per second
Sizes are from the released v1 model. Because the style embedding is sampled, the same speech and example can produce different but equally plausible gestures.

We published it in Computer Graphics Forum in 2023[2], and I presented it at SIGGRAPH Asia 2022 in Daegu. In the paper's user study it outperformed the previous state of the art on naturalness of motion, appropriateness for the speech, and style portrayal. Alongside the code and pretrained models we released the ZEGGS dataset, which I'll use below: about 2¼ hours of one actor's monologues in 19 styles, from neutral to sarcastic to laughing, captured with full-body and finger motion.

One property is worth keeping in mind for later. Like most models that predict motion frame by frame, the decoder is trained to reconstruct the recorded motion, here with L1 losses on joint positions, rotations and velocities. When the next frame is uncertain, the safest prediction lies somewhere in the middle of the possibilities, and that tends to smooth away small, fast detail.

Real Gestures

Set-up

ZeroEGGS was trained on 46 of the dataset's 67 takes and never saw the other 21, which gives us a clean test. For each of those 21 takes I fed its audio to the pretrained model:

for take in held_out_takes:
    # A *training* take of the same style provides the style example,
    # so the only thing taken from the unseen take is its speech
    example = training_take_with_style(take.style)

    generate_gesture(
        audio_file=f'{take}.wav',
        styles=[(f'{example}.bvh', None)],
        first_pose=f'{example}.bvh',
        temperature=1.0,
        seed=1234,
        network_path=models_dir,      # pretrained v1 model
        data_path=processed_dir,
        results_path=out_dir)

Everything is cut into 30-second windows: 165 from the training takes, which the detector learns from, and 69 each from the held-out takes and from ZeroEGGS's output. I kept both controls, built from the held-out takes. If cleaned real motion gets flagged, or Frankenstein motion doesn't, nothing else can be trusted.

One practical detail: the real data fades into a sensor noise floor above about 15 Hz, so I only analyze up to 8 Hz. That leaves the 6 Hz cleanup control something to measure, which was the lesson from the toy.

Results

Here are the detection results:

Real gestures: how well each kind of motion is told apart from real

Dot: score; line: 95% interval from resampling the 21 held-out takes.

The score (AUC) is the chance that the detector rates a clip of this kind as less real than a real one. 0.5 means it can't tell them apart; 1 means every one of them looks less real than every real clip. “Trained with examples” is a classifier that has seen labelled real and fake windows, always tested on takes it never saw.
Show as a table

The controls behave: coordination ignores the cleanup and catches every Frankenstein window. But the ZeroEGGS row did not go the way I expected. The detector which only knows real motion catches ZeroEGGS best with energy shape (0.82). Coordination only manages 0.62, and its interval includes chance.

Looking at the spectrum makes it clear why:

ZeroEGGS is smoother than the actor

How much the right wrist moves up and down at each tempo (median over the 21 held-out takes, log scale)

real actor ZeroEGGS real, cleaned at 6 Hz
Slow sway sits on the left, gesture strokes around 1–3 Hz, quick twitches and beats further right. Above 8 Hz (shaded) the analysis stops: there the real data fades into sensor noise and the cleaned copy has almost nothing left.
Show as a table

ZeroEGGS is too smooth. Across all joints it has about 40× less movement at 4–8 Hz than the actor: the quick, small motions of a real body are mostly not there. That's the averaging effect from the primer above, and it is by far the cheapest giveaway.

What's Different About ZeroEGGS

So is ZeroEGGS's coordination fine? Not quite. The detector above averages about 2,500 coordination numbers per window, and a difference concentrated in some joint pairs easily gets lost in that average. The fairer question is whether a difference exists at all, and for that we can train a classifier with labelled examples, always testing on takes it has never seen:

prob = np.zeros(len(y))
for take in held_out_takes:
    test = (groups == take)     # this take's real AND generated windows
    model.fit(X[~test], y[~test])
    prob[test] = model.predict_proba(X[test])[:, 1]

auc = roc_auc_score(y, prob)

With examples, coordination separates ZeroEGGS from the actor best of all, with an AUC of 0.99, ahead of energy (0.95) and energy shape (0.94). It also confirms the toy's warning: from energy, the classifier flags cleaned real motion as fake (0.77). From coordination, it barely notices (0.56). You can flip between the two detectors in the chart above.

The difference also has a very clear shape. Here it is averaged over all 55 joint pairs:

Too locked when slow, too loose when fast

Coherence of ZeroEGGS minus the actor's, averaged over all 55 joint pairs, from slow to fast tempos

Above zero, ZeroEGGS's joints keep time together more steadily than the actor's; below zero, less. Shaded: 95% interval from resampling the held-out takes.
Show as a table

At slow tempos ZeroEGGS keeps the upper body more in lockstep than the actor does, and at fast tempos it keeps it less. Drawing every joint pair on the body makes the flip easy to see. Try dragging the tempo slider:

Which joints keep time differently

Every line is a joint pair. Warm: ZeroEGGS keeps them more in lockstep than the actor. Cool: less. Faint: no clear difference.

–

more in lockstep more independent no clear difference
Joint positions are the actor's average pose (front view). A pair counts as clearly different when its 95% interval, from resampling the held-out takes, excludes zero. Hover or tab to a line for numbers.
Show the biggest differences in this band as a table

In the slowest band the elbows are the story: in ZeroEGGS they move with the chest and the opposite shoulder about twice as steadily as in the actor (right elbow and chest go from 0.19 for the actor to 0.39 for ZeroEGGS). In the fastest band it is the head: its quick movements are less tied to the shoulders and arms (head and left shoulder go from 0.67 to 0.55). And the flip is almost total. In the slowest band 38 of the 55 pairs are clearly more in lockstep than the actor, and only 4 less. In the fastest band 37 are clearly more independent, and only 5 more locked.

My reading of this (and it is a reading rather than a result) is that the model moves the arms and torso more as one block on slow time scales, where a person's slow drifts are more independent. On fast time scales a person's quick movements, like a beat or a laugh, ripple through the head and shoulders together, while the model's fast movements are weaker and less shared. Put simply, real people are loose when slow and tightly linked when fast. ZeroEGGS is the other way around.

Before believing any of this I checked the two things most likely to fake it. The two skeletons have identical bone lengths, so the smaller ZeroEGGS motion isn't a units mix-up. And ZeroEGGS's weak fast motion still sits far above any noise floor, so noise can't explain its lower fast-tempo coherence.

Lessons

To finish, here are the things I took away from all this.

Smoothness is the easy tell

If you want to spot generated motion without having examples of it, look at how much fast movement there is first. It is cheap to compute, it needs no labels, and for ZeroEGGS it was the strongest signal. I suspect this will be true of many learned models for a while, though it is also the easiest thing for a generator to fix.

Coherence is about timing, not size

That's what makes coordination robust. Cleanup filters and changes in scale both change energy, but leave coherence alone. If you compare motion from different pipelines, energy will happily mistake a cleaned take for a fake one. Coordination won't.

Only measure where the signal survives

Coherence is only invariant to filtering in theory. In practice, once a band has been filtered away there is nothing left to measure but leakage. Check where the spectrum hits its noise floor, and stop before it.

Steady isn't the same as right

Coherence tells you that two joints keep time, not that they keep the right time. Arms swinging with the wrong leg are perfectly steady. Always look at the timing offset as well.

Coordination is where the interesting differences are

Smoothness tells you that motion is generated. Coordination across tempos tells you how it differs, and for ZeroEGGS it points at something specific to fix: loosen the slow coupling between the arms and torso, and let fast movements travel through the upper body.

And the caveats, which are important:

References

  1. J. Frank, T. Eisenhofer, L. Schönherr, A. Fischer, D. Kolossa, T. Holz. Leveraging Frequency Analysis for Deep Fake Image Recognition. ICML 2020.
  2. S. Ghorbani, Y. Ferstl, D. Holden, N. F. Troje, M.-A. Carbonneau. ZeroEGGS: Zero-shot Example-based Gesture Generation from Speech. Computer Graphics Forum, 2023. Dataset and model: github.com/ubisoft/ubisoft-laforge-ZeroEGGS.
  3. L. Ren, A. Patrick, A. A. Efros, J. K. Hodgins, J. M. Rehg. A Data-Driven Approach to Quantifying Natural Human Motion. SIGGRAPH 2005.
  4. A. Gopalakrishnan, A. Mali, D. Kifer, C. L. Giles, A. G. Ororbia. A Neural Temporal Model for Human Motion Prediction. CVPR 2019.
  5. M. Boháček, H. Farid. Human Action CLIPs: Detecting AI-generated Human Motion. 2024.

The ZEGGS dataset and the ZeroEGGS model are © Ubisoft and licensed CC BY-NC-ND 4.0. The figures here show statistics computed from them; no motion from the dataset is reproduced. The toy walker is a simulation.