Question CompilerUncertainty MapQuestionsStateKnown / Controversiallocal-first · compiled on your hardware

Scientific State of the Field

The versioned model of what is understood about Empirical Research in Requirements Engineering (ORKG) — distinct from a summary because it separates accepted, uncertain, and contested models, and preserves disagreement rather than flattening it. This is Pass 4 of the Scientific State Compiler, versioned locally (Scientific Git-lite).

4
Accepted
3
Uncertain
2
Contested
4
Open conflicts
12
Open questions

Accepted models

Crowd-based Requirements Engineering (CrowdRE) paradigmconf 0.90
Widely reported across IEEE International Requirements Engineering conferences as a structured research domain.
ML-based user feedback classificationconf 0.85
Standard empirical practice employing SVM, C4.5 Decision Tree, and Naive Bayes algorithms in the domain.
Core utility triad (Crowd, Task, Mechanism)conf 0.80
Consistently cited as the foundational structural framework for CrowdRE operations.
Empirical validity threat acknowledgmentconf 0.90
Routine reporting of internal, external, construct, and reliability threats alongside standard data collection/analysis practices.

Uncertain models

Multimodal requirements communication (ReqVidA + textual minutes)conf 0.60
Integration mechanism and comparative efficacy metrics across elicitation tasks are absent.
Longitudinal impact of CrowdRE on software lifecycleconf 0.50
No evidence on whether early elicitation benefits persist through implementation, testing, and maintenance phases.
Strict exploratory research paradigm classificationconf 0.55
Conflicting classifications (exploratory vs. explanatory) exist in literature without reconciliation.

Contested models — preserved disagreement

CrowdRE scope definition
A: Fundamentally limited to Elicitation, Analysis, and Validation activities.
B: Intrinsically encompasses Evolution and Prioritization as core paradigm activities.
status: unresolved
Crowd-ML operational boundary
A: Crowd-sourced data quantitatively feeds into, trains, and validates ML classification models.
B: Operational boundaries between crowd data generation and automated ML feedback classification remain parallel and undefined.
status: unresolved

Conflict ledger

CrowdRE paradigm scope
⟶ Limited to Elicitation/Analysis/Validation
⟶ Expanded to include Evolution/Prioritization
resolution: none
Research paradigm classification
⟶ Exploratory
⟶ Explanatory
resolution: none
Multimodal artifact integration
⟶ Video/ReqVidA and textual minutes can be operationally integrated
⟶ Integration mechanism and efficacy metrics are undefined
resolution: none
Crowd-ML pipeline boundary
⟶ Crowd data sequentially trains ML classification models
⟶ Crowd data generation and ML classification operate as independent workflows
resolution: none

Open questions

  • What is the definitive operational scope of CrowdRE: limited to Elicitation/Analysis/Validation or expanded to Evolution/Prioritization?
  • What are the comparative performance metrics (precision, recall, F1, accuracy) of SVM, C4.5, and Naive Bayes for user feedback classification?
  • How does crowd-sourced data quantitatively train and validate ML-based feedback classification models?
  • What is the explicit operational workflow and interaction mechanism between Crowd, Task, and Mechanism utilities?
  • What standardized replication and reproducibility protocols should be adopted for empirical CrowdRE studies?
  • What is the longitudinal impact of CrowdRE interventions across the full software development lifecycle?
  • What ethical frameworks and governance protocols are necessary to prevent worker exploitation and protect stakeholder privacy?
  • What participant demographics and selection criteria reliably ensure high-quality domain-relevant contributions?
  • What specific design controls and mitigation strategies effectively address validity threats in empirical CrowdRE?
  • What datasets, annotation schemas, and preprocessing pipelines enable reproducible ML feedback classification?
  • How do multimodal artifacts integrate in requirements communication, and how do their efficacy metrics differ?
  • Why does empirical RE literature bury findings in narrative text, and what are the consequences for systematic retrieval and meta-analysis?

Silence summary

The field lacks standardized replication and reproducibility protocols, comparative ML performance metrics, participant demographics and crowd worker selection criteria, longitudinal lifecycle impact data, ethical governance frameworks, specific datasets and preprocessing pipelines for feedback classification, and cost-benefit analyses comparing video versus textual requirement documentation. These collective absences hinder cross-study comparability, cumulative scientific progress, and rigorous validation of both CrowdRE and ML-integrated approaches.

Version history (Scientific Git)

v1scientific-state2026-07-22T23:56:31.519907+00:00
hash 72d8029303a643ae
v2scientific-state2026-07-23T00:00:14.951423+00:00
hash dd55330cce21018a
accepted ? · uncertain ? · contested ? · conflicts ?
v3scientific-state2026-07-23T00:00:16.018321+00:00
hash 5312f699a89eee45
accepted ? · uncertain ? · contested ? · conflicts ?
v4scientific-state2026-07-23T00:02:03.683027+00:00
hash e09a2fd23d8e6ed7
accepted ? · uncertain ? · contested ? · conflicts ?

v4 is the result of a closing-loop iteration: an open question was turned into a new evidence source and re-compiled. The diff above is real.

Version 2026-07-23T00:02:03.681593+00:00. Source question-state: 2026-07-22T23:28:40.675875+00:00. Re-compiling with new sources yields a new versioned commit and a real diff.