Create temporary files on Collab storage

#1

@rbakker and I were wondering if Collab storage has the concept of creating “temporary files” (files that will be auto-deleted after some period of time). This would be useful in certain workflows where, for example, a certain operation generates a temporary file which then would be input to a different tool. We just require the temporary file to have a UUID (which is valid as long as the file exists) which can then be fed into the other tools.

p.s. The problem we foresee with regular files is that it is hard to identify if and when the need for a particular file has ceased; and not deleting them would lead to significant wastage of storage space.

#2

Hello,

This feature is not present in the storage system, and also not planned.

Can’t you delete the file from the consumer? I assume no because you have multiple consumers.

It sounds like the feature you are looking for would remove a file after a certain age. So you could create a script (not even necessarily in a notebook) which would use the storage rest api to comb through a collab’s storage and identify and remove files that are older than a specified age.

#3

You are right, we were thinking of something along the lines of files expiring after a certain age. I am not sure if automating the deletion via a script would be possible in our case. The tool that would generate the temporary file would be run by users with varying credentials, and the files would be created in Collabs of which they are members (this is necessary here for privacy and access concerns). So no single user account would have access, or even be aware, of the relevant Collab that needs to be scanned.

But good to know that such a feature isn’t expected soon. We could maybe embed such a script within the tool itself, such that for each subsequent usage by a user, their older files are checked and deleted. Thanks!