Improving disambiguation between Camera-based and AI-generated/AI-enhanced drone footages:

We introduce the following three perpectives:

                 ┌──► Spatial Co-occurrence Matrix (Pixel-to-neighbor joint distribution)

                 │

[Input Image] ───┼──► Discrete Cosine Transform (DCT) (Fourier-domain frequency profiling)

                 │

                 └──► Local Variance Descriptors (Chrominance-to-luminance edge behavior)

Let’s unpack these and then fold them into a more robust routine that goes beyond our current residual_corr / residual_energy / inlier_ratio / reproj_error thresholds.

First, spatial co occurrence matrices. What, say Pangram, is really doing is measuring how often a pixel of intensity i sits next to a pixel of intensity j in a local window, and then looking at the joint distribution. Natural sensor noise tends to produce smooth, non gridlike co occurrence patterns; synthetic images often show overly uniform transitions or periodic structures from upsampling kernels. In our current code, residual_corr is already a kind of “sensor pattern” measure, but it’s global and correlation based. We can deepen this by explicitly computing gray level co occurrence matrices (GLCM) over patches and extracting texture statistics (contrast, homogeneity, energy, entropy) and then aggregating them across frames. The key is not to threshold each statistic individually, but to treat them as a vector and look at how that vector differs between camera and synthetic clips.

Second, the Discrete Cosine Transform. Pangram’s “spectral scars” language is pointing at the fact that diffusion and GAN pipelines leave characteristic energy spikes in high frequency bands. Our residual_energy is a spatial domain measure; we can complement it with a frequency domain profile. For each frame (or a subset), compute a 2D DCT, isolate high frequency coefficients, and summarize their energy distribution—mean, variance, kurtosis, maybe a few radial bands. Again, the point is not “high frequency = synthetic”; it’s that the shape of the high frequency energy distribution differs between physical sensor noise and algorithmic upsampling/denoising.

Third, local variance descriptors across chrominance and luminance. Natural lenses and sensors couple RGB channels in specific ways; generative pipelines often treat them more independently. We can approximate this by computing local variance in Y (luminance) and in Cb/Cr (chrominance) and then looking at how edges and textures behave across channels. Misaligned edge variance profiles—edges strong in luminance but oddly weak or misaligned in chrominance—are a tell.

Now, the grain and geometry thresholds we’ve set are deliberately conservative, and that’s good. But they’re still scalar thresholds on marginal distributions. Real footage can have low grain (good lighting, strong denoising), and synthetic footage can have rigid geometry and persistent tracks (good generator, or hybrid footage). So instead of trying to “fix” those thresholds, I’d treat our current features as part of a larger feature vector and move to a multi feature decision rule.

Concretely, I’d do something like this:

1. Keep our existing features: residual_corr, residual_energy, inlier_ratio, reproj_error, track_survival, motion_jerk, plus provenance flags (generator_tag, c2pa_ai_manifest, camera_metadata, telemetry).

2. Add three new feature families:

a. GLCM texture stats over residuals and raw grayscale:

i. contrast, homogeneity, energy, entropy, correlation.

b. DCT high frequency profile:

i. mean energy in high frequency band, variance, kurtosis, maybe a few band ratios.

c. chrominance luminance variance coupling:

i. correlation between local variance in Y and in Cb/Cr along edges.

3. Normalize these features per clip using robust statistics (medians, percentiles) across bursts, not single frames. We’re already sampling bursts; extend that to these new features.

4. Instead of hard thresholds, use either:

a. a simple classifier trained on a small labeled set (logistic regression, random forest), or

b. a one class anomaly detector trained on camera footage only (one class SVM, isolation forest), where “synthetic/enhanced” is “far from the camera manifold”.

We don’t need a giant dataset to get value here; even a few well curated camera clips and synthetic clips will give us a sense of how these features separate which we already have started.