NRP installation problems (due to pip v10)


#1

Hi!

I have been trying to install the a local version of the NRP on Ubuntu 16.04 system following the instructions on bitbucket.

During the build process of the NRP pip is upgraded in virtual environments for various steps:

  • ExperimentControl
  • BrainSimulation
  • VirtualCoach
  • CLE
  • ExDBackend

for each of which it will crash with the message

from pip import main
ImportError: cannot import name 'main'

This seems to be because after upgrading pip to v10 from pip import main is broken as the package pip.main has been moved to pip._internal.main.

I am not sure how to proceed. In a previous thread it was suggested to downgrade pip (to version 1.4.1).
Would that be recommended in this case as well and if so, still to version 1.4.1?

I assume that would involve changing
$(PIP) install $(PIPPROXY) --upgrade $(PYTHON_PIP_VERSION)
to
$(PIP) install $(PIPPROXY) --upgrade pip=1.4.1
in every user_makefile?

Best
Mario


#2

Dear Mario,

The upgrade of pip to its version 10 is causing problems when used in some virtual environments, see https://github.com/pypa/pip/issues/5240. This is happening to us also in ~/.opt/platform_venv (created by virtualenv).

You can either fix the pip files as indicated in the link above or downgrade pip to any version below 10, e.g., 9.0.3.
To do so, you can set PYTHON_PIP_VERSION to the desired value, say, in your .bashrc file, or set this value in each user_makefile.

Best regards,
Luc


#3

Dear Luc,

thanks for your answer. I will set PYTHON_PIP_VERSION to 9.0.3.

After a clean new setup I am now back at building the NRP.
Running ./configure_nrp, I encountered a few errors (highlighted BOLD below) which I haven’t seen before. Though they might just have slipped my attention previously. The only thing I did different up until that point is at step 6.2 where I upgrade pip to 9.0.3 instead to latest. Any thoughts? Is it safe to proceed?

Best regards,
Mario

~/Documents/NRP/user-scripts$ ./configure_nrp
Removing existing config files for nginx, ExDFrontend, ExDBackend, CLE, nrpBackendProxy and the gzserver/gzbridge scripts.
Would you like to NOT backup your local versions? (this also purges your existing backups) (y/N)

  • Stopping nginx nginx [ OK ]
    Backed up /home/cdp/Documents/NRP/ExDFrontend/app/config.json.bak3
    Backed up /home/cdp/Documents/NRP/CLE/hbp_nrp_cle/hbp_nrp_cle/config.ini.bak3
    Backed up /home/cdp/Documents/NRP/nrpBackendProxy/config.json.bak3
    Backed up /home/cdp/Documents/NRP/ExDBackend/hbp-flask-restful-swagger-master/flask_restful_swagger/static/config.json.bak3
    Backed up /home/cdp/Documents/NRP/VirtualCoach/hbp_nrp_virtual_coach/hbp_nrp_virtual_coach/config.json.bak3
    DONE
    mkdir: created directory ‘/home/cdp/.local/etc/nginx’
    mkdir: created directory ‘/home/cdp/.local/etc/init.d’
    mkdir: created directory ‘/home/cdp/.local/etc/default’
    mkdir: created directory ‘/home/cdp/nginx’
    mkdir: created directory ‘/home/cdp/.local/var/log/nginx’
    mkdir: created directory ‘/home/cdp/.local/etc/nginx/lua’
    mkdir: created directory ‘/home/cdp/.local/etc/nginx/conf.d’
    mkdir: created directory ‘/home/cdp/.opt/bbp’
    Copying user_makefile to python repos
    Copying Frontend config.json file
    Copying start/stop scripts for gzserver and gzbridge
    Copying CLE config.ini file
    Copying hbp-flask-restful config files.
    Copying Proxy config files.
    Copying Nginx config files
    Copying uwsgi config file
    Copying VirtualCoach config.json file
    Copying VirtualCoach bbpclient
    cp: target ‘/home/cdp/.opt/platform_venv/lib/python2.7/site-packages/’ is not a directory
    Generating schema parsers for ExDBackend
    ./configure_nrp: line 80: /home/cdp/.opt/platform_venv/bin/activate: No such file or directory
    ./configure_nrp: line 81: pyxbgen: command not found
    ./configure_nrp: line 82: pyxbgen: command not found
    ./configure_nrp: line 83: pyxbgen: command not found
    ./configure_nrp: line 84: pyxbgen: command not found
    mv: cannot stat ‘bibi_api_gen.py’: No such file or directory
    mv: cannot stat ‘exp_conf_api_gen.py’: No such file or directory
    mv: cannot stat ‘_sc.py’: No such file or directory
    mv: cannot stat ‘robot_conf_api_gen.py’: No such file or directory
    mv: cannot stat ‘environment_conf_api_gen.py’: No such file or directory
    ./configure_nrp: line 91: deactivate: command not found
    Would you like to setup your local storage database with a default user? (Y/n)
    Setting up local storage database
    STORAGE_PATH is set to '/home/cdp/.opt/nrpStorage’
    module.js:549
    throw err;
    ^

Error: Cannot find module 'uuid/v4’
at Function.Module._resolveFilename (module.js:547:15)
at Function.Module._load (module.js:474:25)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object. (/home/cdp/Documents/NRP/nrpBackendProxy/utils/createFSUser.js:26:14)
at Module._compile (module.js:652:30)
at Object.Module._extensions…js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
ERROR
Setting configuration files to default mode (offline mode)
Editing Backend config.json
Editing nginx conf files
Using the proxy config file containing your localhost as backend, and your FS as storage

  • Restarting nginx nginx [ OK ]
    Done
    DONE

#4

Dear Mario,

There are different scenari explaining your error messages:

  • $NRP_PLATFORM_VENV does not match its default value, that is,
    $HOME/.opt/platform_venv. Althougt it is allowed to have
    a customized $NRP_PLATFORM_VENV, our configure_nrp script will
    consider $HOME/.opt/platform_venv instead. I’ll fix
    this (line 80 in $HBP/user-scripts/configure_nrp).

  • The $NRP_PLATFORM_VENV was not created. Maybe because all build
    attempts have failed (CLE, ExDBackend, …). A successful build of
    any of our python project should create the $NRP_PLATFORM_VENV
    folder. This would explain also why pybgen is not found as it is installed with most of the python projects.

Best regards,
Luc


#5

Dear Luc,

thanks again for these explanations.

  • I am wondering why $NRP_PLATFORM_VENV does not match its default value because I am not customizing anything; just following the instructions as presented on the bitbucket page.
  • the step of running ./configure_nrp is prior to building the NRP, so it should not depend on successful build of CLE, ExDBackend etc, right?

#6

Dear Mario,

There is indeed a problem with confgure_nrp: it needs to be run before and after building! There is now a report about this issue in our backlog to simplify the process. But note that these steps are thouroughly documented in the NRP installation guide.

Your problem is most likely the first possibility: NRP wasn’t built at the time you ran configure_nrp.

Best regards,
Luc