How to use Jupyter Notebooks in nrp?


#1

Hello there!

I installed nrp via docker on my platform! First of all: worked on my first try! What you should add to unexperienced users: You don’t get the link to localhost:9000/#/esv-private. I found it by accident here in the forum. Maybe this should be more obvious.

Now to my main problem: I cannot get the local docker nrp install on my system to launch the interactive jupyter. I guess it is not part of the nrp docker installation? I already tried to install a third docker (nrp docker is split up into frontend-container and backend-container) using the jupyter/scipy-notebook image, and for obvious reasons I simply mounted the experiments volume into that newly created container (that mounting feature was the reason for installing jupyther this way in the first place).

Sadly my first experiment /experiments/tutorial_baseball_solution/tutorial_baseball.ipynb did not work as expected: It misses the Virtual Coach Package (hbp_nrp_virtual_coach.virtual_coach) and crashes on import. After manually adding that package to my work folder (as I am not sure how persistent jupyther handles installations between docker restarts…), it throws this error:
File "<ipython-input-2-547f7e2283ae>", line 3, in <module> from hbp_nrp_virtual_coach.virtual_coach import VirtualCoach File "/home/jovyan/work/hbp_nrp_virtual_coach/virtual_coach.py", line 154 print table.draw() ^ SyntaxError: invalid syntax
Maybe there is a very easy solution; maybe I just have the problem because I am not that experienced with nrp, yet.
I think it would be beneficial for local docker installations to also ship jupyter. My experiments using an external jupyter container are just a sad try to compensate for that gap.

Thanks in advance for your help!
Christian


No module named bbp_client.oidc.client when trying to use Virtual Coach
#2

Hi Chris,

It’s looking like the problem is a syntax error in your print statement. I’m guessing you might be able to fix it by changing it to print(table.draw()) (Python3 syntax).

Cheers


#3

Dear Chris,

Thanks for reporting.
The link to the frontend app (localhost:9000/#/esv-private) is given in the docker install doc at http://neurorobotics.net/local_install.html, so I am not sure from where you were following install directions.

Jupyter not being shipped in the image is for sure sad and we’ll make sure this is fixed in the next image. I’ll post it when it is available (1-2 days).

Cheers,
Axel


#4

Thanks for keeping us updated! Does this mean the next image will also contain the capability of running the virtual coach through a python file or python console? Thanks :blush:


#5

The new docker images are available and include Jupyter.
Update your local docker installs with nrp_updater update.
Just as a side note: you can always add whatever you need directly to the docker containers by connecting to them and using apt-get or pip install. Beware that everything you do inside will be lost when you update to a later version of the images.


#6

Hey tobi!

Thanks for your reply. I had been able to interprete the error, but I didn’t write it: It is part of the virtual_coach.py on Bitbucket.
Link to failing line of code. I think it has to do with jupyter not running in the same container, though. This could have been the problem, but maybe someone else will stumble again, so I added this answer for the sake of completeness…

Christian


#7

Update works, but how exactly should we start jupyter on a i-python-notebook? Or is it already running and I just have to extract the “token” of the launched instance and visit some magic url?
I tried several things now and maybe I am just a bit slow-witted. Sorry for the inconvenience!

In order to make it explicit: How would you launch Experiments/tutorial_baseball_solution/tutorial_baseball.ipynb?


#8

Hi Chris,

you should run the command cle-virtual-coach jupyter notebook in Experiments/tutorial_baseball_solution.

Cheers,
Mahmoud


#9

Hey Mahmoud!

Sadly this command does not work within the docker connect:
(I updated my docker)
sudo ./nrp_installer.sh connect_frontend (connect_backend: same result)
cd /Experiments/tutorial_baseball_solution
$ cle-virtual-coach jupyter notebook
bash: jupyter: command not found

Where did you install jupyter within the container?

Thanks a lot!
Christian


#10

I have no experience with docker, but it looks like you need to install jupyter. Jupyter is not part of the NRP, it’s an external tool that users are free to use or not. You can try installing jupyter in the container by running pip install jupyter and see if it works.

Best,
Mahmoud


#11

Oh!
Sorry! I thought you were part of the docker-container-team! My mistake!
For clarification:

So I thought this should be fairly straight forward. :slight_smile:
I already used Jupyter notebooks before, so I know that they include a webui for navigating in folders. If you use the jupyter docker image and mount the nrp volume with the “Experiments”, you isolate this jupyter from nrp. I think there are two “good” alternatives (I am no “pro” with docker!):
Either jupyter (which is now shipped with the container [backend? frontend? both?]) runs inside nrp and “connects” with nrp directly (do experiment python notebooks even need a connection to nrp?)
or we somehow get the virtualcoach packages installed locally inside the (modified standard) jupyter image. But then the question remains which ports to forward between the containers, so notebooks can talk to nrp properly; if my asumption is right that notebooks indeed need to talk to nrp… :smiley:

Sorry for that fairly long text, but it seems there are two teams talking and we need a policy for that: What is actually inside the nrp docker? Just the nrp application and users are needed to put their tools in other containers (as it is usually the case for some webserver deployments) around or outside the container? Or do you want to put tools like jupyter inside? Then I want to propose to encorporate the jupyter UI via link (or other way) into the nrp webapplication.

Thanks for your patience! #newbe
Christian


#12

Jupyter has been added only to the backend container. You might have to map the port 8888 in the start script, since we have no policy for now to open it on every install, since it adds a network constraint (a new port to be available) whereas most users won’t use Jupyter yet.


#13

Side note: a new version of the installer script has been released, which should updates your templates Models and Experiments folders too (was missing before). Just running the nrp_installer.sh with any command should auto-update it.


#14

Sadly the docker installer seems to be a protected resource of your owncloud now (at least that page here [1] points to there [2]) and cannot be downloaded without an own cloud account…

Christian

[1] http://neurorobotics.net/local_install.html
[2] https://neurorobotics-files.net/owncloud/index.php/s/tGUtMRCEFdaeox7


#15

Hi Chris,

The links have been updated now. Sorry that this took some time.

Best regards,
Axel


#16

Sorry to bother you again. Tried it for several days now (hoping to find the problem), but sadly something seems to be broken inside the baseball experiment ipynb:

bbpnrsoa@daabc7e357b1:~/nrp/src/Experiments/tutorial_baseball_exercise$ cle-virtual-coach ipython tutorial_baseball.ipynb
WARNING: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv.
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
/home/bbpnrsoa/nrp/src/Experiments/tutorial_baseball_exercise/tutorial_baseball.ipynb in <module>()
     14   {
     15    "cell_type": "code",
---> 16    "execution_count": null,
     17    "metadata": {
     18     "scrolled": true

NameError: name 'null' is not defined

Just removed all volumes and reinstalled them again. Now nrp hangs inside the “experiment-view” while loading with the following message within the Renderarea of the browser:

"Loading Experiment... 
 30/30 assets loaded.
 
 Preparing 3D scene: Loading sky box."

I think this might be unrelated, but this is really frustrating… What do you recommend?

Fingers crossed,
Christian


#17

Sadly this did not work: running without parameters just prints the command help. On ./nrp_installer.sh update volumes are seemingly not updated. I had to completely remove them multiple times…


#18

Continued on


#19

hey,
maybe a related question:
when I clone the baseball tutorial, the tutorial_baseball.ipynb does not get copied
is this the desired behaviour?
I mean to ~/.opt/nrpStorage/


#20

Hi,

Indeed it does not seem to be copied, I am not sure if it was being copied in the past and this changed, but if that helps, in the experiment description it says that you should use the .ipynb from $HBP/Experiments

Cheers,
Manos