Inject spikes into SpiNNaker from DVS in nrp


#1

Dear All,

I try to inject spikes from the DVS model in gazebo into
a population of neurons called Populationes.dvs_on
on SpiNNaker. While the DVS output looks quite
correct I get some weard output from the dvs_on
population. Both outputs should look the more or the
less the same but they don’t.

DVS output ( driving through a corridor of cylinders):

dvs_on population output:

Here are the functions I use to inject the spikes into SpiNNaker and to put the
spikes from SpiNNaker into a csv file. Both functions are modified from other
versions I found at the NRP. What could be wrong in the connections?

Best regards,
Thorben Schoepe

Python Transfer Function

“”"
This module contains the transfer function that transforms DVS address events
to spikes and input them to a population of neurons
“”"
from dvs_msgs.msg import EventArray
import numpy as np
@nrp.MapRobotPublisher(‘dvs_rendered’, Topic(’/dvs_rendered’, sensor_msgs.msg.Image))
@nrp.MapRobotSubscriber(“dvs”, Topic(‘spikerobot/edvs/events’, EventArray))
@nrp.MapSpikeSource(“input_neurons”, nrp.brain.Populationes.dvs_on, nrp.injector, weight = 50, n=1)
@nrp.Robot2Neuron()
def dvs_to_robot(t, dvs, input_neurons, dvs_rendered):
event_msg = dvs.value
amplitudes = np.zeros(nrp.config.brain_root.input_shape[0] * nrp.config.brain_root.input_shape[1])
if event_msg is None:
input_neurons.amplitude = amplitudes
return

#There are too many events - we randomly select a subset of them
n_events_to_keep = min(100, len(event_msg.events))
filtered_events = np.random.choice(event_msg.events, n_events_to_keep, replace=False)
rendered_img = np.zeros((nrp.config.brain_root.input_shape[0], nrp.config.brain_root.input_shape[1], 3), dtype=np.uint8)
# set high amplitude for neurons that spiked
for event in filtered_events:
    rescaled_event = (np.array((event.y, event.x)) * nrp.config.brain_root.scaling_factor).astype(int)
    rendered_img[rescaled_event[0]][rescaled_event[1]] = (event.polarity * 255, 255, 0)
    idx = rescaled_event[0] * nrp.config.brain_root.input_shape[1] + rescaled_event[1]
    amplitudes[idx] = 1
input_neurons.amplitude = amplitudes
#active_neurons = input_neurons[active_neurons].amplitude = 1
input_neurons.inject_spikes()
#msg_frame = CvBridge().cv2_to_imgmsg(rendered_img, 'rgb8')
#dvs_rendered.send_message(msg_frame)

Python Transfer Function

“”"
This module contains the transfer function that receives
spikes from the SpiNNaker and saves them into a csv file
“”"

@nrp.MapCSVRecorder(“recorder”, filename=“dvs_on_spikes.csv”, headers=[“id”,“time”])
@nrp.MapSpikeSink(“record_neurons”, nrp.brain.Populationes.dvs_on, nrp.spike_recorder)
@nrp.Neuron2Robot(triggers=[“record_neurons”])

def csv_spike_monitor_2(t, recorder, record_neurons):
for timing in record_neurons.times:
recorder.record_entry(
timing[0],
timing[1]
)


#2

I think I am pretty close to find the problem myself.
Is the command input_neurons.inject_spikes()
simply sending the number n of spikes to each
neuron of the population Populationes.dvs_on ?
That would mean that I would have to define
the connector matrix every time this function
is called so that only neurons receive spikes
which should receive one.

Do you know how I can define the connector
in the injector?

Best regards,
Thorben Schoepe


#3

Has there been an update regarding this question?


#4

Dear Gabi,

The dev team is not too much active on the SpiNNaker front right now, reason for the late answer. I keep digging this with them. My first level support guess would be that it sounds like the guy has different neuron parameters in his spinnaker population, that’s why they spike differently than the DVS ones. If more expert knowledge comes from the dev team, they’ll post it here.

Best
Axel


#5

Dear Gabi,

to be honest I have given up at some point to use
SpiNNaker together with the NRP. Moving the whole
network to nest and only using the NRP itself made life
much easier for me. In the example plot I am showing the
spinnaker output should definitely look almost similar to the
dvs output since for example no spikes in pixel 40 to 90 until the time point 0.4ms
should lead to also no spikes in the SpiNNaker output. The neurons
are supposed to be one_to_one connected.

Best regards,
Thorben Schoepe


#6

Thank you for the answer. I have found my ways to solve similar problem and got the desired functionality. However, it is very fiddly… Anyway, I will look into alternatives as well. Once again, thanks for the time taken for answering my inquiry.


#7

Dear Gabi,

Would you be so kind and post the workaround that works for you here, for documentation purposes?

Best
Axel