Finishing a study

What finalizing an experiment does, which providers support it, and why it can't be undone.

When your study is done, you can finalize the experiment from its dashboard. DataPipe merges every remaining data file into a single archive on your storage provider and stops accepting new submissions, which makes the dataset easier to share and cite. Finalizing can't be undone, so only do it when you're certain no more data is coming.

Finalizing is optional, and right now it applies to Zenodo experiments only. An experiment you simply stop using keeps its files exactly as they are. Finalizing is for turning a finished collection into one tidy, citable object.

What finalizing does

Finalizing merges everything, every archive created during collection plus every file still sitting loose, into one file called datapipe-final.zip. When it finishes, that archive is the only file in your Zenodo deposition.

Inside it is the complete folder tree the Psych-DS standard expects, with the data/raw/ paths intact, dataset_description.json at the root, and .psychds-ignore alongside it. That's the whole point. Zenodo can't store a slash in a filename, so a Zenodo record can't hold a Psych-DS dataset directly, but a zip can. The archive is a valid dataset even though the record around it could never have been one.

It's an ordinary zip. Unzip it and you have the dataset. Zenodo also previews zip contents on the record page, so a visitor can see what's inside without downloading it.

Merging a whole study streams every file through DataPipe and back out to your provider, so clicking Finalize starts a background job rather than making you wait. The dashboard shows queued, then running, then the result.

Which providers support it

Finalizing is a Zenodo feature today. It exists to work around a limit on how many files a record can hold, and Zenodo's cap of 100 files per record is the only such limit DataPipe has to deal with.

  • Zenodo: supported. Batch archives and loose files are merged into one datapipe-final.zip.
  • Google Drive: not needed. Drive has no file-count limit, so your folder can hold one file per session indefinitely, and it has real folders, so the folder tree is already intact.
  • Dataverse: not needed, for the same reasons. No file-count cap that DataPipe enforces, and real folder support.
  • OSF: not available. OSF is legacy only, and no new experiments can be created on it.

The dashboard offers finalizing only where it applies. On Google Drive, Dataverse, and OSF experiments, the Finalize section isn't shown at all. When you're done with one of those, just switch the experiment off.

The provider differences behind this. Choosing a provider

Queued uploads must drain first

If any upload for the experiment is still queued or in flight, DataPipe won't start finalizing and tells you why: “Some uploads are still in flight.”

This protects your data. Finalizing merges what your storage provider holds right now, and a queued upload isn't there yet. Sealing the record now would leave that participant's data outside the final archive for good. Wait for the queued files panel to empty, then try again.

If an upload gets queued in the brief window between that check and the merge finishing, it isn't dropped. It's marked failed with a message saying the experiment was finalized while it was queued, and the file stays downloadable from the queued files panel until its retention window ends. Download it and add it to your Zenodo deposition by hand.

The queue, its retries, the download controls, and how long a queued file is kept. When an upload fails

It cannot be undone

Finalizing is permanent. There is no un-finalize, in the dashboard or anywhere else, and the loose files that went into the archive are deleted from your provider once the archive is verified.

Once an experiment is finalized:

  • Data collection is switched off for you. Finalizing turns Accept new data and Accept base64 file uploads off in the same write that seals the record, and the dashboard locks both switches from then on.
  • Every submission is rejected with EXPERIMENT_FINALIZED, no matter what. That check runs ahead of the active switch, so the experiment stays closed even if something outside the dashboard turns those flags back on.
  • Base64 submissions are rejected for the same reason.
  • Condition assignment keeps working. A finalized experiment still hands out condition numbers if something calls the endpoint.

If there's any chance you'll collect more data later, don't finalize. Switch the experiment off instead. That stops new submissions and leaves every option open.

Publishing is still your call

On Zenodo, finalizing prepares the deposition but doesn't publish it. Publishing the record, and issuing the DOI that comes with it, is your decision and happens on Zenodo itself.

The same is true everywhere else. DataPipe never publishes anything. A Dataverse dataset created through DataPipe stays a draft until you publish it, and a Drive folder stays as private as you left it. Nothing DataPipe does changes who can see your data.

Visibility and sharing. Who can see it