← Về thư mục
📄 / / profiles / r-and-d / raw_evidence / scientific-agent-skills / skills / scientific-brainstorming / references / responsible_ai.md

Responsible AI in Research Ideation

AI can supply prompts, reframings, counterarguments, or organizational help. It is not an expert panel, evidence source, author, ethics reviewer, or scientific decision maker. Capabilities and policies change; follow current institutional, funder, publisher, legal, and community requirements.

Default sequence

  1. Classify information. Decide whether the prompt would include personal, patient, confidential, unpublished, proprietary, export-controlled, security-sensitive, or otherwise restricted information.
  2. Generate human ideas first. Freeze an independent human-only round before showing AI suggestions.
  3. Define the AI role. Examples: produce orthogonal questions, challenge an assumption, list search terms, or reformat an already approved record.
  4. Use only an approved tool and data class. Do not assume a paid or “private” interface satisfies institutional controls.
  5. Capture provenance. Record tool/model or service, date, purpose, material prompt constraints, output IDs used, and human editor. Avoid storing restricted prompt text in the session register.
  6. Verify externally. Check factual claims and every citation against authoritative sources. Search for contradictory and null evidence.
  7. Run a second independent human round. Ask for ideas outside the AI's frames and for harms or stakeholders the output omitted.
  8. Disclose as required. The accountable humans retain authorship, responsibility, and final judgment.

AI use is optional. The bundled CLIs make no network or LLM calls.

Suitable bounded roles

Avoid using AI to:

Hallucination and source verification

Generative systems can produce plausible but false claims, references, methods, statistics, and quotations. A 2023 study found fabricated and substantively erroneous bibliographic citations in outputs from the tested GPT-3.5 and GPT-4 versions; model-specific rates are not timeless estimates.

For every AI-suggested source:

  1. locate the work in a trusted index or publisher site;
  2. match title, authors, venue, year, DOI, and version;
  3. read the relevant primary text;
  4. confirm the cited result, population, design, and limitations;
  5. record the stable source identifier;
  6. delete unsupported claims rather than laundering them as “AI suggested.”

Never cite the model as evidence for a scientific claim.

Anchoring and homogenization

AI output can anchor users on examples and compress a group's idea diversity. In a preregistered short-story experiment, access to GPT-4 ideas improved average evaluated creativity for some writers while making outputs more similar in aggregate. The task was short creative writing, not scientific ideation, so treat homogenization as a credible risk to test—not a universal effect size.

Controls:

Multiple AI samples are correlated products of a system, not independent experts or replications.

Automation bias and false authority

Fluent language, technical detail, and confident formatting are not evidence. To reduce deference:

Do not ask an AI system to assign a probability it cannot calibrate and then treat the number as measured uncertainty.

Confidentiality, privacy, and intellectual property

Do not submit the following to an external AI service unless an authorized policy and agreement explicitly permit that data class:

Data minimization and abstraction are still required with an approved tool. Check retention, training use, access, location, deletion, audit, and incident terms. If the work cannot be safely abstracted, use an approved local/closed process or do not use AI.

Bias, representation, and participation

AI output may reproduce gaps and stereotypes in training data and overrepresent well-indexed, English-language, high-resource perspectives. It cannot consent on behalf of affected communities.

Research integrity and disclosure

Humans remain responsible for accuracy, attribution, originality, permissions, and the research record. AI systems should not be listed as authors. Record and disclose AI use at the level required by the institution, funder, venue, and applicable guidance.

A useful internal disclosure includes:

Tool/service and model or version (if exposed):
Date used:
Purpose:
Information classification and approved environment:
Human-first idea set frozen before use: yes/no
Outputs retained or used:
Verification performed:
Material changes made by humans:
Known limitations:

Disclosure does not cure inappropriate data sharing, plagiarism, fabricated citations, or unverified content.

Dual-use and misuse review

AI can make technical ideation faster and more accessible. Screen both the research idea and the AI interaction for misuse potential.

Escalate before generating operational detail when an idea could materially enable:

Use high-level risk framing while waiting for institutional biosafety, biosecurity, research-security, legal, ethics, or funding-agency guidance. Do not rely on a model's refusal behavior as a risk-management control.

WHO's responsible life-sciences framework treats risk mitigation as a shared, multi-stakeholder responsibility. U.S. DURC/PEPP oversight has been under revision following the May 2025 executive order; verify current policy rather than copying a superseded threshold.

Incident handling

If sensitive information or unsupported AI content entered the workflow:

  1. stop further sharing and preserve only the minimum audit information;
  2. notify the appropriate institutional privacy, security, integrity, or research office under local policy;
  3. do not copy the sensitive content into additional systems;
  4. remove or quarantine unverified claims from downstream artifacts;
  5. document affected decisions and re-review them;
  6. follow approved deletion and incident-response procedures.

Do not conceal the event by silently editing provenance.

Evidence and policy basis

See sources.md for the dated primary evidence and official guidance used here, including: