Error downloading files from storage to local


#1

Today (26/03) there has been an update in the collab.
Running this afternoon my scripts, I found some errors, probably due to the changes.

Before the updates, to download the file from my storage to container, I run

clients.storage.download_file(collab_path + “/myStorage/pythonFile.py”,"/home/jovyan/pythonFile.py")

Now, I need to use this little changes (clients.storage.download_file becomes clients.storage.download_file_content)
but I got this error running the same script:

Invalid UUID for file_id: /SimpleSentenceParsing/pythonFile.py

Any suggestions to fix the error?

Thanks,
Alberto


#2

Hello,

I think the problem is that at some point you upgraded the hbp-service-client that contains the path based commands, but the update reverted your environment to an older version of that library.

You can check the version using !pip2 show hbp-service-client If it’s lower then 1.0.0 you need to upgrade it. Let me know if this helps.

Akos


#3

Hi Akos,
thanks for the answer,
I have the 1.0.0 version

I solved the problem rewriting all the scripts and re-running them (i don’t know exactly what was wrong, but now it works).

Best
Alberto


#4

Hi Akos,
I got the same error with another user that share with me the collab.
We have the upgraded version of the hbp-service-client, but he has the error and me no.
As I wrote above, the error it that to download the file from my storage to container, I run

clients.storage.download_file(collab_path + “/myStorage/pythonFile.py”,"/home/jovyan/pythonFile.py")

but there is no the .download_file module after clients.storage, but download_file_content that couses the error

Invalid UUID for file_id: /SimpleSentenceParsing/pythonFile.py

Thanks
Best
Alberto


#5

Hi Alberto,

Can you restart the notebook kernel and try again? One thing that might explain this is if you updated hbp-service-client and didn’t restart the kernel afterwards. The client is instantiated for all notebooks upon loading, and if at that time the module was still the old version, then it doesn’t matter if you upgrade it with pip, the running process already have the old version of the module loaded into memory. Hence you need to restart the kernel, to force the process to read the new version into memory.

Let me know if this helps.
Akos


#6

Hi Akos,
the other user that share with me the collab restared the kernel and the error persisted.
But, in the same time, we found that the error is not related to the version of `hbp-service-client’, it is related to the absence of folders in the local container that code needs to correctly run.
I created those folders in my local container, thinking that every users that share with me the collab could use them, instead the folders I created are present only in my local directory. So, we can share the same files and folders in the storage directory, we can use the same jupyter notebooks, but we cannot use the same folders in the local container.
Since we want to share our collab with also other people, I’d like to know if it is possible to create folders for all the users’ container, otherwise all the users have to create theme by yourself.
I could also add you to my collab to see what I mean an trying to see if you get the same error as every possible external users could get. Give a confirmation about that and I add you to my collab.

Thanks for the support
Alberto


#7

Hello,

Every user has their own Docker Container within which the code is executed. If you create files and folders in yours (e.g. changing the state of your container) that only effects you and not others. So the notebook environments are completely independent of Collabs.

If your notebook relies on a specific folder structure then the beginning of the notebook should verify if it exists and create the missing bits if needed. Of course ideally this operation should be idempotent, so it won’t fail when the user execute the notebook a second time, when all the structure is in place.

You can do something quick and dirty by mixing Python and bash commands:


for directory in ['$HOME/foo/bar', '/$HOME/bar/baz']:
    !mkdir -p $directory

This will call create the nested folder in the home folder. If you then need to make sure that certain files exist on the user’s container you can use the hbp-service-client to fetch the files from the collab storage.

Hope this helps,
Akos


#8

Hi Akos,
thanks again for the answer,
I’ll do your suggestions.

Best,
Alberto