← Về thư mục
📄 / / profiles / r-and-d / raw_evidence / scientific-agent-skills / skills / flowio / references / fcs_semantics.md

FCS Semantics for FlowIO

Use this reference when the task depends on what values or metadata mean, not just how to call the API.

Scope

FlowIO reads FCS 2.0, 3.0, and 3.1 and writes a constrained FCS 3.1 representation. It is a file-format library, not a complete flow-cytometry analysis system.

Before downstream biological interpretation, decide whether the workflow also requires:

FlowIO does none of these. Its optional preprocessing is limited to scaling defined by FCS acquisition metadata.

File Segments

An FCS dataset can contain:

FlowIO exposes these through header, text, events, and analysis.

FCS 3.1 deprecated storing multiple datasets in one file through $NEXTDATA, but FlowIO can read legacy multi-dataset files with read_multiple_data_sets().

TEXT Keyword Normalization

The FCS standard treats keyword names as case-insensitive. Standard keywords are written with $, but FlowIO normalizes parsed keys:

  1. Strip the leading $.
  2. Convert the key to lowercase.
  3. Keep the value as a string.

Examples:

Use:

date = flow.text.get("date")
spill = flow.text.get("spillover", flow.text.get("spill"))
nextdata = int(flow.text.get("nextdata", "0"))

Avoid:

date = flow.text.get("$DATE")  # Always misses in FlowIO's normalized mapping.

Custom, non-standard keys are also lowercased. The normalized mapping may contain subject, sample, operator, institution, instrument serial-number, and free-text fields. Treat an unrestricted metadata dump as potentially identifying.

FlowIO 1.4.0 implements the first step by removing every $ character from the decoded segment before splitting keys and values. A literal $ inside a value is therefore lost ("a$b" becomes "ab"). Do not use the parsed mapping as a lossless metadata archive.

Parameters, Labels, and Indices

FCS parameter numbers begin at 1:

FlowIO presents these through two index systems:

When reporting a channel, label both values if ambiguity matters:

for array_index, pnn in enumerate(flow.pnn_labels):
    parameter_number = array_index + 1
    pns = flow.pns_labels[array_index]
    print(parameter_number, array_index, pnn, pns)

PnS is optional. FlowIO inserts "" for missing PnS entries so that pns_labels and pnn_labels have the same length.

FlowIO infers scatter, fluorescence, and time indices from channel labels. Treat these as conveniences, not a substitute for checking the instrument panel. Vendor-specific labels may not classify as expected.

null_channel_list accepts PnN labels and excludes matching channels from the derived fluorescence/scatter/time index lists. It does not remove columns from the event data. flow.null_channels stores the supplied labels unchanged; it does not contain zero-based indices and can retain labels that did not match a channel.

Encoded and Preprocessed Event Values

Encoded representation

flow.events is a flattened one-dimensional sequence, ordered by event and then channel. It is usually an array.array; mixed-width integer channels use a Python list:

event_1_channel_1, event_1_channel_2, ..., event_2_channel_1, ...

flow.as_array(preprocess=False) reshapes that representation to:

(event_count, channel_count)

The returned NumPy array is float64, even when the encoded file uses integer or single-precision event data.

FlowIO preprocessing

flow.as_array(preprocess=True) performs the following operations.

Time scaling

When a time channel and nonempty timestep keyword exist:

time_scaled = time_encoded * timestep

An empty or whitespace-only timestep is treated as 1.0 in FlowIO 1.4.0.

Logarithmically stored channels

For PnE (decades, log_zero) and PnR range:

linear_value = 10 ** (decades * encoded_value / range) * log_zero

The conversion is applied when decades > 0.

Gain

For PnG gain:

gain_scaled = value / gain

FlowIO skips division when gain is zero or one.

The operations are metadata-driven. Bad metadata can therefore produce bad scaled values even when the DATA bytes were parsed correctly.

What preprocessing does not do

FlowIO does not:

For compensation/transformation/gating, use a higher-level package such as FlowKit and preserve a record of the matrix and transform parameters.

Spillover Metadata

FlowIO can expose or write a spillover keyword as a string. It does not validate the matrix scientifically or apply it.

A standard spillover string begins with:

  1. Number of compensated fluorescence parameters
  2. Matching PnN labels
  3. Flattened matrix values

Items are comma-delimited with no newline characters.

Before passing the matrix downstream:

Offset Semantics

FCS 3.0/3.1 DATA offsets can appear in HEADER and TEXT. FlowIO normally uses TEXT offsets and checks that HEADER agrees.

Some files have known defects:

FlowIO accounts for the FCS 3.1 large-file rule. Do not use relaxed flags merely because a file is large.

Recovery options:

Each option changes which bytes are interpreted as events. Use one only when the file's provenance or vendor behavior justifies it, and validate event count, channel distributions, and known controls afterward.

Memory Model

Normal FlowData construction reads the full DATA segment into memory. as_array() then allocates a second float64 representation.

Approximate additional memory for the 2-D array is:

event_count * channel_count * 8 bytes

This excludes the original event array, Python objects, temporary arrays, and downstream DataFrames.

Use only_text=True for inventory. FlowIO 1.4.0 does not expose chunked, streaming, lazy, or memory-mapped event reads. If a file does not fit safely in memory, use a different parser/workflow or process it in a resource-limited environment; do not claim FlowIO can chunk it.

only_text=True skips DATA loading but still parses ANALYSIS. Also prefer paths over open handles: FlowData closes caller-provided handles, and the multi-dataset helper cannot reuse a closed handle for its second dataset in FlowIO 1.4.0.

Writer Representation

create_fcs() and write_fcs() produce:

The float32 representation has roughly 6-7 decimal digits of precision. Round-trip comparisons should use tolerances rather than exact equality.

Passing a NumPy array makes create_fcs() allocate an array('f') copy. An existing array('f') is written directly, so retain that representation when large writer memory is a concern.

create_fcs() needs flattened event data. Flatten a two-dimensional array in C order:

flat = events_2d.astype("float32", copy=False).ravel(order="C")

Validate:

if events_2d.ndim != 2:
    raise ValueError("expected events x channels")
if events_2d.shape[1] != len(channel_names):
    raise ValueError("channel label count mismatch")

Required keywords and channel interpretation fields are generated by FlowIO. Do not rely on metadata_dict to override $PAR, $TOT, $MODE, $DATATYPE, PnB, or PnN.

For floating-point sources, write_fcs() can keep encoded values while dropping PnG and timestep. This changes the interpretation returned by as_array(preprocess=True). Validate both encoded and metadata-scaled values, not just event counts.

Scientific Provenance

For any converted or rewritten file, record: