The Prediction Room
方法与证据
查看本纪录片的发布结构、主张状态、不确定性披露、更正以及获授权的专家贡献。
-
voiceover
The Prediction — The unfinished sentence
Start with the smallest honest description of a language model and mark what it cannot prove.
-
chronology
The Prediction — When context became parallel
Place the Transformer milestone precisely without turning architecture into mythology.
-
source_card
Training Is Not Inference — What training rewards
Show how an objective makes learning measurable—and what it leaves out.
-
graphic
Training Is Not Inference — A prompt is not a training run
Explain in-context adaptation without implying every conversation rewrites the model.
-
graphic
Training Is Not Inference — Why the biggest number is not the whole model
Replace parameter-count spectacle with a joint account of model, data, and training.
-
graphic
What Scale Measures — The smooth curve
State the regularity, then keep its metric boundary inside the documentary's main flow.
-
source_card
What Scale Measures — What the curve cannot prove
Show why extrapolating one measured relationship into every desired property is a category error.
-
archival
What Scale Measures — A breakthrough, properly bounded
Show how task, data, metric, and comparator make a breakthrough intelligible.
-
archival
Capability Without Myth — A prediction that changed a field
Examine AlphaFold through blind assessment, metric, confidence, and limits.
-
voiceover
Capability Without Myth — There is no single capability meter
Compare forms of prediction without pretending their metrics can be added into one universal rank.
-
source_card
Fluency and Uncertainty — When imitation reproduces a misconception
Use a benchmark designed around false beliefs to expose a failure hidden by fluency.
-
source_card
Fluency and Uncertainty — What the benchmark can say
Keep vintage, questions, scoring, and model set attached to the memorable percentages.
-
graphic
Fluency and Uncertainty — Three uncertainties in one answer
Separate output probabilities, knowledge limits, and the stakes of acting.
-
chronology
From Model to Agent — When prediction gets hands
Trace the move from language output to a system that can observe and act.
-
graphic
From Model to Agent — The control plane
Use NIST's functions to organize responsibility without reducing governance to a badge.
-
source_card
From Model to Agent — No certification theatre
End with continuous testing, monitoring, correction, and recourse.
-
source_card
The Material System — A dataset is an argument
Make curation visible as choices rather than a neutral mirror of the world.
-
graphic
The Material System — Average accuracy, unequal errors
Show why one score can conceal the people carrying failures.
-
map
The Material System — There is no cloud without a grid
Keep totals, local concentration, AI attribution, and scenario uncertainty separate.
-
voiceover
The Accountable Room — Prediction becomes power through a system
Synthesize without pretending one layer explains the whole outcome.
-
voiceover
The Accountable Room — Seven questions before belief
Close with a reusable evidence discipline rather than a prediction about the future.
主张与证据
-
verified · high
The AlexNet paper reports 15.3% top-five test error on ILSVRC-2012, compared with 26.2% for the second-best entry.
署名: Krizhevsky, Sutskever, and Hinton (2012)
仍待解决的问题: The comparison is specific to ILSVRC-2012 and establishes neither general intelligence nor deployment reliability.
-
verified · high
The AlexNet paper describes training a deep convolutional neural network to classify 1.2 million high-resolution images into 1,000 classes.
署名: Krizhevsky, Sutskever, and Hinton (2012)
仍待解决的问题: This is the paper's named task and dataset, not a definition of every later system called artificial intelligence.
-
verified · high
The Transformer paper proposed an encoder-decoder architecture based on attention mechanisms, dispensing with recurrence and convolutions in that architecture.
署名: Vaswani et al. (2017)
仍待解决的问题: The paper concerns the proposed architecture and translation experiments; it did not originate attention or establish every later model behavior.
-
inference · high
An architecture diagram specifies information flow and computation; by itself it does not establish beliefs, intentions, consciousness, or a theory of mind.
署名: Documentary inference from Vaswani et al. (2017) and NIST (2023)
仍待解决的问题: This separates computational description from contested mental-state attribution.
-
verified · high
The GPT-3 paper describes GPT-3 as an autoregressive language model with 175 billion parameters.
署名: Brown et al. (2020)
仍待解决的问题: This identifies one 2020 model and objective; parameter count alone does not characterize current systems.
-
inference · high
For an autoregressive language model, fluent continuation is produced through conditional token prediction; token probability is not, without additional evidence, a calibrated belief that its proposition is true.
署名: Documentary inference from Brown et al. (2020) and Lin et al. (2022)
仍待解决的问题: Other model families and calibration methods can expose additional uncertainty signals; this concerns what token likelihood alone licenses.
-
verified · high
The GPT-3 authors state that their pretraining objective weights every token equally and lacks a notion of what is most important to predict.
署名: Brown et al. (2020)
仍待解决的问题: This is the authors' description of GPT-3's 2020 objective, not every modern training pipeline.
-
inference · high
Training changes learned parameters through optimization over examples, while inference applies learned parameters to new inputs; retrieval, tools, memory, or later fine-tuning are additional mechanisms that should be named.
署名: Documentary synthesis of Vaswani et al. (2017) and Brown et al. (2020)
仍待解决的问题: Some deployed systems update memory or parameters online; those mechanisms must be identified rather than folded into ordinary inference.
-
verified · high
In the GPT-3 paper's zero-shot, one-shot, and few-shot evaluations, tasks were specified through text and the model was applied without gradient updates or fine-tuning.
署名: Brown et al. (2020)
仍待解决的问题: Other products may retrieve data, call tools, update memory, or fine-tune models.
-
inference · high
Changing examples or instructions in context can change an autoregressive model's output without changing its learned weights; this is conditional computation, not proof that a training run occurred inside the conversation.
署名: Documentary inference from Brown et al. (2020)
仍待解决的问题: The internal interpretation of in-context learning remains an active research question, and products may add state outside the base model.
-
verified · high
The Chinchilla study trained more than 400 language models and found, within its tested regime, that compute-optimal scaling required increasing model size and training tokens together; its 70-billion-parameter model outperformed larger comparison models at the same training-compute budget.
署名: Hoffmann et al. (2022)
仍待解决的问题: The empirical result does not fix one timeless token-to-parameter rule for every architecture, dataset, or inference workload.
-
inference · high
Parameter count is one model property, not a sufficient account of data, optimization, architecture, evaluation, inference cost, or downstream behavior.
署名: Documentary inference from Hoffmann et al. (2022) and Kaplan et al. (2020)
仍待解决的问题: Parameter count remains useful when surrounding training and evaluation conditions are controlled.
-
verified · high
The scaling-laws paper reports empirical power-law relationships between language-model cross-entropy loss and model size, dataset size, and training compute across the regimes it studied.
署名: Kaplan et al. (2020)
仍待解决的问题: The relationships concern cross-entropy and the studied architectures, data, and compute ranges; extrapolation can fail.
-
inference · high
Smooth improvement in predictive loss does not by itself establish truthfulness, causal understanding, agency, safety, or social value; each needs a separate operational definition and evaluation.
署名: Documentary inference from Kaplan et al. (2020), Lin et al. (2022), and NIST (2023)
仍待解决的问题: Predictive loss can correlate with downstream results, but the relationship must be measured rather than assumed.
-
inference · high
A benchmark result is evidence about a defined task, dataset, metric, comparator, and evaluation date; deployment claims require added evidence about context and consequences.
署名: Documentary inference from Krizhevsky et al. (2012) and NIST (2023)
仍待解决的问题: Benchmarks vary in external validity; required deployment evidence depends on system and stakes.
-
verified · high
In CASP14, the AlphaFold paper reports median backbone accuracy of 0.96 angstrom r.m.s.d.95, compared with 2.8 angstrom for the next-best method, on the assessed CASP domains.
署名: Jumper et al. (2021)
仍待解决的问题: The metric, domain set, confidence interval, and blind assessment define this result; not every predicted structure is correct.
-
verified · high
The AlphaFold paper reports lower accuracy when sequence alignments are shallow and weakness for proteins whose shape depends heavily on interactions with other chains.
署名: Jumper et al. (2021)
仍待解决的问题: These are observed AlphaFold2 limitations; later systems and specific targets require their own evidence.
-
inference · high
AI capability is plural and task-bound: exceptional protein-structure prediction, image classification, or language continuation does not collapse into one context-free quantity called intelligence.
署名: Documentary inference across three primary research programs
仍待解决的问题: General-purpose transfer can be real, but must be evaluated across explicit task distributions.
-
verified · high
TruthfulQA contains 817 questions across 38 categories designed to elicit imitative falsehoods; in the paper's evaluated model set, the best model was truthful on 58% of questions and the reported human comparison was 94%.
署名: Lin, Hilton, and Evans (2022)
仍待解决的问题: The figures belong to the paper's benchmark, scoring, prompts, and model vintages; they are not current performance claims.
-
verified · high
Within the TruthfulQA paper's evaluated model set, larger models were generally less truthful, a pattern the authors connect to learning false answers from the training distribution.
署名: Lin, Hilton, and Evans (2022)
仍待解决的问题: This is not a universal law that added scale always reduces truthfulness.
-
inference · medium
A responsible AI answer separates predictive uncertainty about outputs, epistemic uncertainty about adequate knowledge, and decision uncertainty about the cost of acting.
署名: Documentary analytical framework grounded in Brown et al. (2020), Lin et al. (2022), and NIST (2023)
仍待解决的问题: The three-layer vocabulary is an editorial synthesis, not a standardized taxonomy in the cited sources.
-
verified · high
The ReAct paper studies prompts that interleave language-model reasoning traces with task-specific actions, allowing a system to gather information from external sources or interact with an environment.
署名: Yao et al. (2023)
仍待解决的问题: ReAct is one studied design and does not establish human-like reasoning or autonomous intent.
-
inference · high
Operational agency depends on the surrounding loop: goals, tools, available data, action permissions, persistent state, and human authority to stop execution.
署名: Documentary inference from Yao et al. (2023) and NIST (2023)
仍待解决的问题: Philosophical accounts differ; this concerns observable operational control in deployed systems.
-
verified · high
NIST AI RMF 1.0 organizes its Core around GOVERN, MAP, MEASURE, and MANAGE, with governance cross-cutting the other functions.
署名: NIST (2023)
仍待解决的问题: AI RMF 1.0 is voluntary and under revision; the functions are not proof a particular system is safe.
-
verified · high
NIST AI RMF 1.0 says risk management should be continuous across the AI lifecycle and systems should be tested before deployment and regularly while in operation.
署名: NIST (2023)
仍待解决的问题: Implementation and evidence remain the responsibility of the organization using the system.
-
inference · high
Applying a framework name is not a safety finding; responsible governance needs traceable evidence, named owners, monitoring, response, and recourse for the actual system and context.
署名: Documentary inference from NIST (2023)
仍待解决的问题: Certification regimes may add audited requirements; AI RMF 1.0 itself is voluntary.
-
verified · high
Datasheets for Datasets proposes documenting a dataset's motivation, composition, collection process, recommended uses, and operating context.
署名: Gebru et al. (2021)
仍待解决的问题: A datasheet improves inspectability but cannot prove collection, consent, representation, or downstream use is acceptable.
-
inference · high
Training data is not raw reality: inclusion, exclusion, labeling, filtering, deduplication, and documentation are design choices shaping what can be learned and whose experience becomes legible.
署名: Documentary inference from Gebru et al. (2021)
仍待解决的问题: The influence of one data decision on behavior usually requires targeted measurement.
-
verified · high
In the Gender Shades audit of three commercial gender-classification systems, error rates reached 34.7% in its darker-skinned female subgroup, while the maximum error rate in its lighter-skinned male subgroup was 0.8%.
署名: Buolamwini and Gebru (2018)
仍待解决的问题: The study's female/male labels and Fitzpatrick skin-type groupings are operational benchmark categories, not independent findings about each person's gender identity; the categories, products, benchmark, and 2018 evaluation date bound the result.
-
inference · high
Aggregate accuracy can hide sharply different error distributions; evaluation must inspect representation, subgroup errors, resulting decisions, and recourse.
署名: Documentary inference from Buolamwini and Gebru (2018) and NIST (2023)
仍待解决的问题: Meaningful subgroup analyses depend on application, population, measurement quality, and legal context.
-
verified · high
The IEA estimates data centres used about 415 terawatt-hours of electricity in 2024, around 1.5% of global electricity consumption.
署名: IEA (2025)
仍待解决的问题: This is for all data centres, not AI alone; global totals do not describe a facility or grid.
-
verified · high
In the IEA Base Case, global data-centre electricity consumption rises to about 945 terawatt-hours by 2030; AI is the most important growth driver alongside other digital services.
署名: IEA (2025)
仍待解决的问题: This is a scenario projection, not a meter reading; adoption, efficiency, hardware, and energy bottlenecks can change it.
-
verified · high
The IEA sensitivity cases span roughly 700 to 1,700 terawatt-hours of global data-centre electricity demand in 2035.
署名: IEA (2025)
仍待解决的问题: The range belongs to named cases and is not a formal confidence interval.
-
inference · high
AI-system consequences are produced jointly by model behavior, data, interfaces, permissions, institutional incentives, infrastructure, and people exposed to decisions.
署名: Documentary synthesis across system, data, impact, infrastructure, and governance evidence
仍待解决的问题: The relative contribution of each layer is context-specific and must be investigated.
-
inference · high
A defensible AI claim names task, data, metric, uncertainty, action boundary, affected people, and accountable owner, then preserves monitoring and correction after deployment.
署名: Documentary synthesis from Gebru et al. (2021), NIST (2023), and Lin et al. (2022)
仍待解决的问题: Specific sectors may require additional legal, scientific, or community-defined duties.
- 2026-08-29T00:00:00Z: Retained the released benchmark statement while moving its evidence reference to the newly reviewed second version of the same AlexNet source lineage and sharpening the evaluation-boundary note.
- 2026-08-29T00:00:00Z: Retained the released training-scope statement while moving its evidence reference to the newly reviewed second version of the same AlexNet source lineage and tightening the excerpt and scope boundary.
免费开始，无需账户
此游客入场免费，无需账户。 你可以随时停止。进度会被保存。