Thirty minutes a second still needs QA

A pipeline that ingests thirty minutes of video a second still needs reject codes, a person on the takes that matter, and an eval split by person – throughput without those is noise at industrial scale.

Figure’s Index made the rate public in August: thirty minutes of creator video ingested every second, which they framed as 4.9 years of human work uploaded every day. Sixteen million videos. Hundreds of thousands of downloads. Tens of thousands of weekly active users. Per thousand hours, hundreds of unique tasks, more than a thousand manipulated objects, more than a hundred environments. That is a data factory with a consumer front end. It is also a QA problem wearing a throughput number. At thirty minutes a second, nobody is watching the takes. Filters are. Samples are. Embeddings are. If those steps do not encode “task not completed” and “camera on the ceiling” as rejects, you are not scaling a skill. You are scaling whatever people uploaded because they were paid by the minute.

The pattern is bigger than one brand. Physical-AI teams that could not buy this volume from vendors are standing up firehoses: apps, marketplaces, “film your chores” briefs. NVIDIA-scale sim does not remove the real hours. It makes the real hours more expensive when they are the wrong object, because you will then generate around them. Hour-counts without reject codes are how you pretrain on lifestyle footage at industrial scale and still fail on the mug.

What that number actually is

Ingest is not a dataset. Figure’s own write-up is careful in a way the screenshots are not. The pipeline has five stages after the upload: filtering, fraud review, deduplication, rebalancing, annotation. Automated filters for technical, visual and semantic quality. Human analysts on samples, at user level, for people trying to game the filters. Embedding-based near-duplicate drop. Task quotas and clusters. Hierarchical captions. That is a factory. Notice what it is not, unless you wrote it. It is not your reject code for “task not completed.” It is not your rule for “camera on the ceiling.” It is not your faces-out decision. It is not your eval split by person. It is not your rights pack.

Figure can afford to throw away a large share of a consumer firehose because the firehose is the point. A product team collecting for one manipulation policy cannot. Every hour you keep that is not the job is a prior you will then have to unlearn. Dedup that drops the fourth attempt at the same sink may also drop the only takes that priced the mistake. Rebalancing on task labels will not save you if the label was “kitchen” and the mug never moved. Hierarchical captions on an incomplete chore are captions of an incomplete chore.

Thirty minutes a second is a compute and staffing sentence. 24/7 availability. Continuous processing. Real-time feedback to creators so the next upload is less wasted. Human analysts on samples, not on the stream. At that rate a person cannot be the first filter. A person still has to be the last filter on the codes that mean “this is not the job,” or the samples will drift towards whatever the automatic filters already like: well-lit, complete-looking, same household, camera roughly in the right hemisphere. That drift is how a dashboard stays green while the policy never sees a sticking drawer.

We already wrote how to brief a human data collection when the failure mode was a vague email. The news at this rate is that the vague email now looks like a funded ingest graph. The failure is the same. Most failed collections do not fail in the field. They fail in the paragraph nobody wrote, and then they fail again in the reject codes nobody applied because applying them would hurt the minutes.

Reject codes, or you are scaling noise

Write what a bad item is while everyone is still calm, before the app starts paying, and before the graph hits thirty minutes a second.

Task not completed. The mug never reaches the sink. The laundry is filmed until it is interesting and then abandoned. The dishwasher door opens and the take ends. Camera on the ceiling. Camera on the creator’s face when you asked for the hands. Mount slipped, digital zoom, beauty filter, a screen of someone else doing the chore. Non-consented faces. Children. Screens. Looping the same thirty seconds. Uploading a take from a different household than the enrolled ID. Scripted demonstration when you asked for ordinary behaviour. A suspiciously clean counter when you asked for clutter. Dry sponge when you asked for wet. Too short to be the skill.

Automation can flag black frames, duplicate hashes, some mount failures, some of the looping. It will not reliably flag “the task did not finish” unless you defined finish. It will not flag “this is a ceiling” if the embedding space likes that kitchen. It will not flag a child at the edge of frame if your semantic filter was trained on “kitchen, hands, mug.” A person still applies the codes that mean the sentence. Gold items before production, or the first month is the network learning your kitchen on your budget. Annotation quality is not a percentage of uploads accepted. It is whether the accepted takes are the job.

Fraud review exists in Figure’s pipeline because consumer funnels get gamed the moment payment is by the minute. Your collection will too. People will film a television. People will loop. People will recruit a household that is not the enrolled ID. Sample-based analyst review at user level is a start. It is not a substitute for item-level rejects on the skill. A user who is honest and still films the ceiling is not fraud. They are a take you must not train on. If your QA only hunts cheats, you will keep the sincere failures.

Duration belongs in the brief so QA has a reject for “too short to be the task” instead of an argument in week three. A manipulation skill needs start to finish, including the failures. Paying by the minute without a minimum that matches the skill is how you buy fragments at thirty minutes a second. Fragments caption beautifully. They do not teach place.

Faces and screens are QA codes, not a privacy appendix. If faces are not required, reject them. If screens are not required, reject them. Blur later is a processing choice. It is not retroactive consent, and at this ingest rate you will not blur them all. Children are not a “we do not knowingly collect” paragraph when the method is a wearable in a family kitchen. You either designed the brief so they are not in frame, with a reject when they are, or you collected them, at industrial scale.

Eval by person, or the graph is a household

Split eval by person, not by random clip. A creator who is good at the app will dominate the hour-count. Four hundred minutes from one pair of hands is one person, however many tasks they touched. If those hands sit in train and test, the policy has seen that household. The graph will look like transfer. The next kitchen will not. Hold those people out. Rights to keep the eval set and not train on it are a separate line from “we paid you to film.” If you cannot answer “are eval people excluded from training?” in one sentence, assume no.

Dedup fights this and also fights you. Near-duplicate drop on embeddings is how you stop paying twice for the same counter. It is also how you drop the retries that contain the only labelled failures. Write which near-duplicates you keep: same person, same task, failed then succeeded, keep. Same person, same thirty seconds, loop, drop. That is a QA rule, not an infrastructure default. At thirty minutes a second the default will win unless you wrote the rule.

Rebalancing on task quotas and clusters is not an eval split. Quotas make the mix look diverse. Clusters capture variation beyond the label. Neither holds a person out. You can rebalance your way into a set that is geographically wide and still eight households deep in the test split because those eight were prolific. Prolific is a staffing fact. It is not a generalisation fact.

Humanoid labs standing up data factories are hiring data-quality people for this reason. Headcount without the codes is a film shoot with a robotics logo. An ingest graph without the codes is the same shoot with a waitlist. Quality roles only work if they have a sentence to apply. “Looks like a kitchen” is not a sentence. “Mug from cluttered counter to sink, chest camera, hands in frame, no faces, start to finish including the slip” is a sentence. The second one can be a reject. The first one can only be a minute.

Gold before the firehose, not after. Ten takes that are the job, scored by a person, in the actual mount, in the actual lighting, with the actual failure modes you care about. Then you turn on paying by the minute. Then you let automation propose. Then a person samples against the gold, not against a vibe. If gold does not exist, every accepted hour is a vote for whatever the filters already liked. At thirty minutes a second that vote is a landslide by lunch.

If a catalogue item already matches the skill and the rig, look at it with the actual trainer before you build an ingest path that needs a QA army. Custom when the object set, the kitchen, or the faces decision is yours. Starting with a consumer funnel because the public rate looks like Figure is how briefs balloon and recapture becomes a line item, with extra fraud review.

Throughput is the easy half

Physical AI is not short of capture software. It is short of specified, consented behaviour in the rooms the robots will enter, labelled as the job, with the failures kept, with the people who filmed it held out of the exam. An app is how you recruit people into that capture. Thirty minutes a second is how you know recruitment worked. QA is how you know you have a dataset. Pay by the minute if that is how you staff the loop. Do not confuse the pay mechanism, or the ingest graph, with the job.

Copy the seriousness. Humanoids need real work in real rooms, at a volume vendors did not historically deliver. Copy the idea that quality is a pipeline with more than one stage, not a zip file. Copy the willingness to throw takes away. Do not copy hour-counts as the success metric. Do not copy “diversity we could not define up front” as an excuse to skip the sentence. Long-tail variation is valuable after you can reject the takes that are not the job. Before that, it is noise with extra countries, arriving at thirty minutes a second.

Write the skill before you celebrate the rate. Gold before the first payout. Reject task-not-completed and camera-on-the-ceiling by name. Split by person. Faces out unless the policy needs them. A factory can ingest a working day of video every second. It still cannot tell you whether the mug reached the sink unless a person, or a filter a person designed, is allowed to say no. Throughput without that no is industrial noise. The bottleneck was never the upload. It was always whether anyone would reject the take.

People4.ai would rather kill a bad take in QA than pretrain on a ceiling. hello@people4.ai · Contact

FAQ

Can we turn on ingest first and write reject codes once we see the footage?
You will be paying for the learning curve at firehose speed, and you will keep takes you cannot use. Write the skill, the mount, the faces rule and the reject codes before anyone records. Gold items first. Then recruit.

Is paying by the minute a quality problem at this rate?
It is a metric problem that gets worse as ingest rises. Minutes reward duration. The job is a completed skill from the right camera. Put a reject for task-not-completed and too-short, and split eval by person so prolific creators do not sit in train and test.

Can automation replace people on QA at thirty minutes a second?
Automation has to do most of the volume. It cannot invent the sentence. A person still designs the codes, scores the gold, samples for task-not-completed, ceiling-camera, faces and fraud, and holds eval people out. Without that, the filters scale whatever they already like.

Related reading
How to brief a human data collection so you do not recapture · Humanoid labs are building data factories, not just models

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top