NRP update Internal Server Error


#1

Hello,

I was asked by @camilo to commit my latest updating problem with NRP.
On Saturday I updated my NRP source install with

cd $HBP/user-scripts
./update_nrp update all
./configure_nrp

Everything went fine, until I started CLE with

cle-start

After this I get following error:

*** Operational MODE: threaded ***
2019-03-05 12:18:10,046 [MainThread ] [hbp_nrp_cles] [WARNING] Could not write to specified logfile or no logfile specified, logging to stdout now!
Traceback (most recent call last):
File “/home/programmer/Documents/NRP/ExDBackend/hbp_nrp_backend/hbp_nrp_backend/runserver.py”, line 32, in
from hbp_nrp_backend.rest_server import app
File “./hbp_nrp_backend/rest_server/init.py”, line 78, in
from hbp_nrp_backend.rest_server.__SimulationResources import SimulationResources
File “./hbp_nrp_backend/rest_server/__SimulationResources.py”, line 34, in
from hbp_nrp_backend.rest_server.__SimulationControl import _get_simulation_or_abort
File “./hbp_nrp_backend/rest_server/__SimulationControl.py”, line 40, in
from hbp_nrp_backend.simulation_control import simulations, Simulation
File “./hbp_nrp_backend/simulation_control/init.py”, line 12, in
from hbp_nrp_backend.simulation_control.__Simulation import Simulation
File “./hbp_nrp_backend/simulation_control/__Simulation.py”, line 32, in
from hbp_nrp_backend.simulation_control.__BackendSimulationLifecycle import
File “./hbp_nrp_backend/simulation_control/__BackendSimulationLifecycle.py”, line 42, in
from hbp_nrp_backend.__UserAuthentication import UserAuthentication
File “./hbp_nrp_backend/__UserAuthentication.py”, line 33, in
from backports.functools_lru_cache import lru_cache
ImportError: No module named functools_lru_cache
unable to load app 0 (mountpoint=’’) (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 10527, cores: 8)
2019-03-05 12:18:10,310 [Thread-3 ] [rospy.intern] [INFO] topic[/rosout] adding connection to [/rosout], count 0
[5] 10551

and

— no python application found, check your startup logs for errors —
— no python application found, check your startup logs for errors —
2019-03-05 12:18:12 GMT+0100 Failed to execute request http://141.21.13.168:8080/health/errors. ERROR: Error: Status code: 500
Internal Server Error
2019-03-05 12:18:12 GMT+0100 Failed to execute request http://141.21.13.168:8080/simulation. ERROR: Error: Status code: 500
Internal Server Error

I also tried

cd $HBP/user-scripts
update_nrp build

the update didn’t throw any error message as well until I started the NRP backend (with cle-start). It shows the same error message like I wrote above.

Thanks for help and best regards
Katharina


#2

Hi Katharina,

Could you try to import this package from a python interpreter? Just go to your command line and type

python -v
from backports.functools_lru_cache import lru_cache

for me this package is in /home/manos/Documents/NRP/GazeboRosPackages/devel/lib/python2.7/dist-packages/backports, so check if it is there or not.

If there is nothing there, then try to

cd $HBP/GazeboRosPackages
catkin_make # should rebuild all the python packages in this repository

If it is already there, there could be a problem with your python path. Do you have the following lines in your .bashrc?

. $HBP/user-scripts/nrp_variables
. $HBP/user-scripts/nrp_aliases

These lines make sure that the PYTHONPATH environment variable is correctly set. If you have other programs that might be changing your python path I would recommend adding the following line before the previous lines

export PYTHONPATH= # makes sure that we start with an empty python path

And then you can manipulate your .bashrc as you wish by commenting and uncommenting this line. If still stuck please send the output of the PYTHONPATH env variable

echo $PYTHONPATH

hope it helps!

Best,
Manos


#3

Hello,

Many thanks for helping.

I did

python -v
[...]
>>> from backports.functools_lru_cache import lru_cache
import backports # directory /home/programmer/Documents/NRP/GazeboRosPackages/devel/lib/python2.7/dist-packages/backports
# /home/programmer/Documents/NRP/GazeboRosPackages/devel/lib/python2.7/dist-packages/backports/__init__.pyc matches /home/programmer/Documents/NRP/GazeboRosPackages/devel/lib/python2.7/dist-packages/backports/__init__.py
import backports # precompiled from /home/programmer/Documents/NRP/GazeboRosPackages/devel/lib/python2.7/dist-packages/backports/__init__.pyc
# /usr/lib/python2.7/pkgutil.pyc matches /usr/lib/python2.7/pkgutil.py
import pkgutil # precompiled from /usr/lib/python2.7/pkgutil.pyc
import imp # builtin
# /home/programmer/.local/lib/python2.7/site-packages/backports/functools_lru_cache.pyc matches /home/programmer/.local/lib/python2.7/site-packages/backports/functools_lru_cache.py
import backports.functools_lru_cache # precompiled from /home/programmer/.local/lib/python2.7/site-packages/backports/functools_lru_cache.pyc
# /usr/lib/python2.7/__future__.pyc matches /usr/lib/python2.7/__future__.py
import __future__ # precompiled from /usr/lib/python2.7/__future__.pyc
# /usr/lib/python2.7/functools.pyc matches /usr/lib/python2.7/functools.py
import functools # precompiled from /usr/lib/python2.7/functools.pyc
import _functools # builtin
# /usr/lib/python2.7/collections.pyc matches /usr/lib/python2.7/collections.py
import collections # precompiled from /usr/lib/python2.7/collections.pyc
import _collections # builtin
import operator # builtin
# /usr/lib/python2.7/keyword.pyc matches /usr/lib/python2.7/keyword.py
import keyword # precompiled from /usr/lib/python2.7/keyword.pyc
# /usr/lib/python2.7/heapq.pyc matches /usr/lib/python2.7/heapq.py
import heapq # precompiled from /usr/lib/python2.7/heapq.pyc
import itertools # builtin
import _heapq # builtin
import thread # builtin
# /usr/lib/python2.7/threading.pyc matches /usr/lib/python2.7/threading.py
import threading # precompiled from /usr/lib/python2.7/threading.pyc
import time # builtin

and

cd $HBP/GazeboRosPackages
catkin_make # should rebuild all the python packages in this repository

but in the directory there is only this:

~/Documents/NRP/GazeboRosPackages/devel/lib/python2.7/dist-packages/backports$ ls
__init__.py  __init__.pyc  __pycache__

It imported the functools_lru_cache package from:

~/.local/lib/python2.7/site-packages/backports$ ls
configparser             __init__.py   shutil_get_terminal_size
functools_lru_cache.py   __init__.pyc
functools_lru_cache.pyc  __pycache__

These two lines

. $HBP/user-scripts/nrp_variables
. $HBP/user-scripts/nrp_aliases

are already in my .bashrc I added

export PYTHONPATH= # makes sure that we start with an empty python path

It still doesn’t work.

Thats my $PYTHONPATH:

echo $PYTHONPATH
/home/programmer/Documents/NRP/retina/build/lib:
/home/programmer/Documents/NRP/GazeboRosPackages/devel/lib/python2.7/dist-packages:
/opt/ros/kinetic/lib/python2.7/dist-packages:
/home/programmer/.local/lib/python2.7/site-packages:
/home/programmer/.local/lib/x86_64-linux-gnu/python2.7/site-packages:
/home/programmer/Documents/NRP/GazeboRosPackages/devel/lib/python2.7/dist-packages:
/home/programmer/Documents/NRP/CLE/hbp_nrp_cle:
/home/programmer/Documents/NRP/ExperimentControl/hbp_nrp_excontrol:
/home/programmer/Documents/NRP/ExperimentControl/hbp_nrp_scxml:
/home/programmer/Documents/NRP/ExDBackend/hbp_nrp_backend:
/home/programmer/Documents/NRP/ExDBackend/hbp_nrp_cleserver:
/home/programmer/Documents/NRP/ExDBackend/hbp_nrp_commons:
/home/programmer/Documents/NRP/ExDBackend/hbp_nrp_watchdog:
/home/programmer/Documents/NRP/ExDBackend/hbp-flask-restful-swagger-master:
/home/programmer/Documents/NRP/VirtualCoach/hbp_nrp_virtual_coach:
/home/programmer/Documents/NRP/BrainSimulation/hbp_nrp_music_xml:
/home/programmer/Documents/NRP/BrainSimulation/hbp_nrp_music_interface:
/home/programmer/Documents/NRP/BrainSimulation/hbp_nrp_distributed_nest:

I don’t know if it helps, but these two directories that are mentioned in my $PYTHONPATH do not exist:

bash: cd: /home/programmer/Documents/NRP/retina/build/lib: No such file or directory
bash: cd: /home/programmer/.local/lib/x86_64-linux-gnu/python2.7/site-packages: No such file or directory

Many Thanks

Katharina


#4

Hi,

I see there is the IP here is not localhost, are you running the NRP on a remote server?
http://141.21.13.168:8080/health/errors

Other than that, everything else seems fine

Best,
Manos


#5

Hello,

I thought I’m using the local mode. Sorry I’m not sure how to look it up is it the running mode?

~/Documents/NRP/user-scripts$ ./running_mode
2. Offline mode – Disable OIDC and use your local machine as backend and storage
5. Offline mode with localhost replaced by local IP – Disable OIDC and use your local machine as backend and storage
Specify the additional ‘no_restart’ option if you want nginx not to be restarted

best regards
Katharina


#6

Go ahead and execute the

./running_mode 2

script, just in case. Also run a configure_nrp and try restarting the cle, and let’s see how that goes.

Best,
Manos


#7

Hi,

now I get:

*** Operational MODE: threaded ***
CSAConnector: libneurosim support not available in NEST.
Falling back on PyNN’s default CSAConnector.
Please re-compile NEST using --with-libneurosim=PATH
Traceback (most recent call last):
File “/home/programmer/Documents/NRP/ExDBackend/hbp_nrp_backend/hbp_nrp_backend/runserver.py”, line 32, in
from hbp_nrp_backend.rest_server import app
File “./hbp_nrp_backend/rest_server/init.py”, line 78, in
from hbp_nrp_backend.rest_server.__SimulationResources import SimulationResources
File “./hbp_nrp_backend/rest_server/__SimulationResources.py”, line 34, in
from hbp_nrp_backend.rest_server.__SimulationControl import _get_simulation_or_abort
File “./hbp_nrp_backend/rest_server/__SimulationControl.py”, line 40, in
from hbp_nrp_backend.simulation_control import simulations, Simulation
File “./hbp_nrp_backend/simulation_control/init.py”, line 12, in
from hbp_nrp_backend.simulation_control.__Simulation import Simulation
File “./hbp_nrp_backend/simulation_control/__Simulation.py”, line 32, in
from hbp_nrp_backend.simulation_control.__BackendSimulationLifecycle import
File “./hbp_nrp_backend/simulation_control/__BackendSimulationLifecycle.py”, line 42, in
from hbp_nrp_backend.__UserAuthentication import UserAuthentication
File “./hbp_nrp_backend/__UserAuthentication.py”, line 33, in
from backports.functools_lru_cache import lru_cache
ImportError: No module named functools_lru_cache
unable to load app 0 (mountpoint=’’) (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 30083, cores: 8)
2019-03-06 15:37:01,834 [MainThread ] [hbp_nrp_cles] [WARNING] Could not write to specified logfile or no logfile specified, logging to stdout now!
2019-03-06 15:37:02,108 [Thread-3 ] [rospy.intern] [INFO] topic[/rosout] adding connection to [/rosout], count 0
[5] 30107

and

— no python application found, check your startup logs for errors —
— no python application found, check your startup logs for errors —
2019-03-06 15:37:03 GMT+0100 Failed to execute request http://localhost:8080/health/errors. ERROR: Error: Status code: 500
Internal Server Error
2019-03-06 15:37:03 GMT+0100 Failed to execute request http://localhost:8080/simulation. ERROR: Error: Status code: 500
Internal Server Error

best regards
Katharina


#8

Do you maybe have matplotlib installed? It could be related to something like this https://stackoverflow.com/questions/47179433/python-2-7-functools-lru-cache-does-not-import-although-installed