Loading Datasets and Working Copies

Loading is one of the places where RoleThread Lite is deliberately careful. The app tries to protect original source files, preserve useful metadata, and make imported data safe to edit.

The most important idea is simple:

If a dataset did not clearly come from RoleThread Lite, RoleThread may create a protected working copy before modifying it.

That is intentional.

Starting From An Empty Workspace

Fresh installs start with an empty workspace.

RoleThread Lite does not bundle or seed starter content at launch. That keeps first-run behavior clear: the dataset you create or load is the dataset being edited.

Normal starting points are:

• create a new empty dataset in Manage Dataset
• load an existing JSONL or ChatML dataset
• import compatible outside data and let RoleThread create a protected working copy when needed
• use Prompt Generation (Beta) to compile a prompt for external source material, then review and import useful results

Future tutorial assets may be added later if they help onboarding, but V1 does not rely on hidden starter content.

Trusted and Untrusted Datasets

RoleThread Lite treats datasets differently depending on whether they carry a complete RoleThread-native identity.

A trusted dataset is a dataset previously saved by RoleThread Lite with stable metadata such as entry UUIDs and a shared dataset UUID.

An untrusted dataset is any dataset that does not clearly carry that identity. It may still be perfectly valid data. "Untrusted" does not mean bad. It only means RoleThread should be careful before writing back to it.

Examples of untrusted data include:

• files exported from another tool
• ShareGPT datasets
• manually edited JSONL
• older RoleThread files from before current metadata existed
• mixed or partial metadata files

Why Working Copies Exist

Working copies protect your original files.

When you load an untrusted dataset from outside the default training data area, RoleThread copies it into its local training data folder and works on that copy. The original file is left alone.

This prevents a common problem: opening an outside dataset, clicking save, and accidentally changing the only original copy.

Working copies are not a hidden trick or a temporary hack. They are a safety feature.

What Happens During Load

When you load a dataset, RoleThread may:

1. Read the file.
2. Detect whether it is ChatML, ShareGPT, or another supported JSON shape.
3. Convert ShareGPT records into ChatML-style entries.
4. Normalize safe basics such as roles, whitespace, and tag shape.
5. Check whether the dataset is trusted.
6. Create a working copy if the dataset is untrusted and should not be edited in place.
7. Look for a sibling sidecar file.
8. Import safe registry metadata from the sidecar when it matches the dataset.
9. Preserve unknown tags as archived/imported tags instead of throwing them away.
10. Show warnings or summaries when something needs attention.

You do not need to run these steps manually. The load summary tells you what happened.

Where Working Copies Go

Working copies are stored under the configured dataset folder, usually inside training_data.

RoleThread uses a folder-per-dataset layout, for example:

training_data/
  my_dataset/
    my_dataset.jsonl
    my_dataset.registry.json

If a folder name already exists, RoleThread chooses a non-colliding copy name rather than overwriting the existing folder.

Sidecars During Load

A sidecar is a registry metadata file stored beside the dataset. It usually has a name like:

my_dataset.registry.json

Sidecars can preserve:

• tag categories
• tags and aliases
• archived/imported tag information
• character definitions
• character-to-turn mappings
• system prompt templates
• dataset identity information

Sidecars let metadata travel with a dataset without forcing that metadata into clean training records.

Sidecar Warnings

If RoleThread shows a sidecar warning, it is usually being cautious.

Examples:

• The sidecar file could not be read.
• The sidecar schema is newer than this version of RoleThread understands.
• The sidecar dataset UUID does not match the loaded dataset.
• Some registry metadata already exists locally and was skipped instead of overwritten.

A sidecar warning does not always mean the dataset itself failed to load. It often means RoleThread loaded the entries but refused to trust questionable metadata.

Trusted Does Not Mean Perfect

A trusted dataset can still contain content issues. It only means RoleThread recognizes the dataset identity and can safely preserve entry identity across edits.

You should still use:

• Validation
• Insights
• Search
• manual review

Practical Guidance

• Let RoleThread create working copies when it wants to. That is the safe path.
• Keep source datasets somewhere separate if they are originals you do not want changed.
• Use the loaded path shown in Manage Dataset to see which file you are actually editing.
• Keep .registry.json sidecars near their matching .jsonl files when moving datasets.
• Do not manually copy a sidecar from one dataset onto another unless you understand the metadata relationship.

Common Mistake

Mistake: Loading an outside dataset, then expecting RoleThread to edit that original file directly.

Better mental model: RoleThread protects outside/untrusted files by creating a working copy. You edit the working copy. If you want a final file somewhere else, use Export.