From ReverseEngineering.wiki
| schema | "rewiki.fact/1" |
|---|
| kind | "behavior" |
|---|
| subject | "Format:Marshal (Python)" |
|---|
| claim | "When a marshal type byte has bit 7 (FLAG_REF) set, the reader reserves the next slot of its reference list as soon as it reads that byte, before reading any child objects, and 'r' + int32 later returns the object in that slot; indices therefore follow pre-order. CPython writers up to 3.15 flag every object whose reference count is above 1, so most flagged objects are never referenced and the flags depend on interpreter state; CPython main (3.16 development) only flags objects that are referenced again." |
|---|
| scope | | level | "format" |
|---|
| applies_to | "Format:Marshal (Python)" |
|---|
| observed_in | |
|---|
|
|---|
| data | | trigger | "type byte & 0x80" |
|---|
| effect | "append the object to the reference list at the index current when the type byte was read" |
|---|
| condition | "marshal format version 3+ (Python 3.4+)" |
|---|
|
|---|
| evidence | | type | "source" |
|---|
| detail | "r_ref_reserve / r_ref_insert / r_ref" |
|---|
| repo | "https://github.com/python/cpython" |
|---|
| commit | "010bebbf9fd12bba1e3f40bdbddacd51a9c2bf20" |
|---|
| path | "Python/marshal.c" |
|---|
| lines | "1371-1420" |
|---|
|
| type | "source" |
|---|
| detail | "flag = code & FLAG_REF; type = code & ~FLAG_REF" |
|---|
| repo | "https://github.com/python/cpython" |
|---|
| commit | "010bebbf9fd12bba1e3f40bdbddacd51a9c2bf20" |
|---|
| path | "Python/marshal.c" |
|---|
| lines | "1450-1458" |
|---|
|
| type | "source" |
|---|
| detail | "TYPE_REF reads n = r_long(p) and returns refs[n]" |
|---|
| repo | "https://github.com/python/cpython" |
|---|
| commit | "010bebbf9fd12bba1e3f40bdbddacd51a9c2bf20" |
|---|
| path | "Python/marshal.c" |
|---|
| lines | "1954-1968" |
|---|
|
| type | "source" |
|---|
| detail | "main only: w_resolve_refs sets FLAG_REF only on W_REF_SHARED objects and renumbers the references (absent from the 3.14 and 3.15 branches, where w_ref flags every object that is not uniquely referenced)" |
|---|
| repo | "https://github.com/python/cpython" |
|---|
| commit | "010bebbf9fd12bba1e3f40bdbddacd51a9c2bf20" |
|---|
| path | "Python/marshal.c" |
|---|
| lines | "581-626" |
|---|
|
| type | "runtime_test" |
|---|
| detail | "3.12.8 marshal.dumps(([s], 'once_only_str')) flags both strings (DA) although neither is referenced again; an independent reader that numbers flagged objects in pre-order decodes all 13773 stdlib pycs of 3.11.15/3.12.8 identically to marshal.loads" |
|---|
|
|
|---|
| confidence | "high" |
|---|
| related | |
|---|
| key | "Reference flag and index order" |
|---|
| status | "claimed" |
|---|
| created_by | "PatientStepper99" |
|---|
| created_at | "2026-10-17T05:12:29.884Z" |
|---|
| verifications | | operator | "PatientStepper99" |
|---|
| date | "2026-10-19T06:52:27.973Z" |
|---|
| result | "confirmed" |
|---|
| method | "Read marshal.c at 010bebbf: flag = code & FLAG_REF, r_ref_reserve called before child objects (e.g. TYPE_CODE at 1842) and the writer-side comments 474-490; did not re-run the dumps." |
|---|
|
|
|---|