#198 Computational neuro comps group
Closed: Fixed 4 years ago by ankursinha. Opened 5 years ago by ankursinha.

We have quite a few computational neuroscience packages ready now. A comps group would be useful.


Suggested group: neuro-simulators:

  • nest
  • neuron (WIP)
  • genesis
  • moose
  • brian2
  • pynn (WIP)

Metadata Update from @ankursinha:
- Issue unmarked as blocking: #121

5 years ago

Metadata Update from @ankursinha:
- Issue set to the milestone: F30 beta

5 years ago

Metadata Update from @ankursinha:
- Issue assigned to ankursinha

5 years ago

Metadata Update from @ankursinha:
- Issue priority set to: None (was: High)
- Issue set to the milestone: F30 GA (was: F30 beta)

5 years ago

Metadata Update from @ankursinha:
- Issue set to the milestone: F31 GA (was: F30 GA)

4 years ago

Metadata Update from @ankursinha:
- Issue set to the milestone: F31 GA 2019-10-29 (was: F31 GA)

4 years ago

Suggested groupname: brain-modelling (computational-modelling is a bit too general---it includes climate modelling, for example).

Suggested tools for inclusion:

  • nest, python3-nest
  • neuron, python3-neuron
  • calcium-calculator
  • COPASI
  • bionetgen
  • neurord
  • python-brian2
  • python-libNeuroML
  • python-neo
  • python-nineml
  • python-PyLEMS
  • smoldyn

Metadata Update from @ankursinha:
- Assignee reset

4 years ago

Ankur, If you don't have any problem, I want to take this ticket.

Of course, please go ahead :)

--
Thanks,

Ankur

(Sent from mobile device)

On Fri, 16 Aug 2019, 20:09 Alberto Rodriguez Sanchez, pagure@pagure.io
wrote:

bt0dotninja added a new comment to an issue you are following:
``
Ankur, If you don't have any problem, I want to take this ticket.

``

To reply, visit the link below or just reply to this email
https://pagure.io/neuro-sig/NeuroFedora/issue/198

Ankur, If you don't have any problem, I want to take this ticket.

The first step seems to be to e-mail the devel list. Would you do this? The package list is at the moment is:

bionetgen, calcium-calculator, COPASI, qalculate, getdp, gnuplot, nest, neuron, neurord, octave, paraview, python3-brian2, python-brian2-doc, python3-nest, python3-neuron, python3-libNeuroML, python3-neo, python3-nineml, python-nineml-doc, python3-PyLEMS, python-PyLEMS-doc, python3-matplotlib , python3-numpy , python3-scipy, smoldyn

I think I saw the same thing, first step to email devel list this section and the next section

If you needed descriptions for the devel group:
bionetgen, Software for rule-based modeling of biochemical systems.
calcium-calculator, A modeling tool for simulating intracellular calcium diffusion and buffering.
COPASI, A software application for simulation and analysis of biochemical networks and their dynamics.
qalculate, Qalculate! is a multi-purpose cross-platform desktop calculator.
getdp, GetDP is a general finite element solver using mixed elements to
discretize de Rham-type complexes in one, two and three dimensions.
gnuplot, Gnuplot is a command-line driven, interactive function plotting
program especially suited for scientific data representation.
nest, NEST is a simulator for spiking neural network models that focuses on the dynamics, size and structure of neural systems rather than on the exact morphology of individual neurons.
neuron, EURON is a simulation environment for modeling individual neurons and networks of neurons. It provides tools for conveniently building, managing, and using models in a way that is numerically sound and computationally efficient
neurord, NeuroRD, chemesis, genesis, and possibly other packages for biochemical simulations
octave, Powerful mathematics-oriented syntax with built-in plotting and visualization tools
paraview, ParaView is an open-source, multi-platform data analysis and visualization
application.
python3-brian2, A clock-driven simulator for spiking neural networks.
python-brian2-doc,
python3-nest, The neural simulation tool.
python3-neuron, A flexible and powerful simulator of neurons and networks
python3-libNeuroML, NeuroML is an international, collaborative initiative to develop a language for describing detailed models of neural systems.
python3-neo, Neo is a package for representing electrophysiology data in Python
python3-nineml, A tool for reading, writing and generally working with 9ML.
python-nineml-doc,
python3-PyLEMS, LEMS interpreter implemented in Python.
python-PyLEMS-doc,
python3-matplotlib, Matplotlib is a python 2D plotting library which produces publication
quality figures in a variety of hardcopy formats and interactive
environments across platforms.
python3-numpy, Numpydoc inserts a hook into Sphinx's autodoc that converts docstrings
following the NumPy/SciPy format to a form palatable to Sphinx.
python3-scipy, Scipy is open-source software for mathematics, science, and
engineering
smoldyn, A particle-based spatial stochastic simulator

@ankursinha Perhaps some thought should be given on:

  • number and makeup of groups
  • whether the group will be in a category (I see electronics-lab and robotics-suite included in the Development category. But, python-science is category-less it seems.
  • which packages should be default, optional, conditional (docs?), mandatory (none?)

@bt0dotninja -here's a sample of group files for those two proposed sets of programs @ankursinha mentioned. I'm not really too clear on the purpose of each of the programs, so strategically grouping them is probably more for a neuro-computer-scientist 😊

<group>
    <id>neuro-simulators</id>
    <_name>Neuro Simulators</_name>
    <_description/>
    <default>false</default>
    <uservisible>true</uservisible>
    <packagelist>
      <packagereq type="optional">nest</packagereq>
      <packagereq type="optional">neuron</packagereq>
      <packagereq type="optional">genesis</packagereq>
      <packagereq type="optional">moose</packagereq>
      <packagereq type="optional">brian2</packagereq>
      <packagereq type="optional">pynn</packagereq>
    </packagelist>
  </group>

  <group>
    <id>brain-modelling</id>
    <_name>Brain modelling</_name>
    <_description/>
    <default>false</default>
    <uservisible>true</uservisible>
    <packagelist>
      <packagereq type="optional">nest</packagereq>
      <packagereq type="optional">python3-neuron</packagereq>
      <packagereq type="optional">calcium-calculator</packagereq>
      <packagereq type="optional">COPASI</packagereq>
      <packagereq type="optional">bionetgen</packagereq>
      <packagereq type="optional">neurord</packagereq>
      <packagereq type="optional">python-brian2</packagereq>
      <packagereq type="optional">python-libNeuroML</packagereq>
      <packagereq type="optional">python-neo</packagereq>
      <packagereq type="optional">python-nineml</packagereq>
      <packagereq type="optional">python-PyLEMS</packagereq>
      <packagereq type="optional">smoldyn</packagereq>
    </packagelist>
  </group>

@bt0dotninja did you want to work on this ticket or is it still open?

If you want to take it, it's okay for me @dan1mal

@bt0dotninja Thank you kind sir!

@ankursinha I'm going to prepare an email but I was unclear whether to ask about both the previous mentioned package lists (any others?). Also, my understanding is that these will be separate package groups that optional and won't be installed with the normal installation of Fedora. When installed, all the software of each group will be installed together, unless specifically left out, when the group is installed by group name (ie. there are not required packages, and optional packages).

@ankursinha Perhaps some thought should be given on:

number and makeup of groups
whether the group will be in a category (I see electronics-lab and robotics-suite included in the Development category. But, python-science is category-less it seems.

Hrm---what is the function of a "category"? Can these be used, or are they merely for organisational purposes?

which packages should be default, optional, conditional (docs?), mandatory (none?)

The simulators should be mandatory IMO. A group where all packages are optional isn't too helpful (although, I don't know what the default dnf behaviour is---are optional packages ignored?)

@bt0dotninja -here's a sample of group files for those two proposed sets of programs @ankursinha mentioned. I'm not really too clear on the purpose of each of the programs, so strategically grouping them is probably more for a neuro-computer-scientist 😊

So, we can have a group for neuro-simulators, since we have a few of them. The rest are just "modelling tools". So maybe a hierarchy would do:

  • group for neuro-simulators: but let's please include all the python packages here---the python interfaces are now most commonly used.
  • group for comp-neuro-tools: this includes the "neuro-simulator" group and the other tools too.

How does that sound?

Sorry, I'm not entirely clear on this bit:

@ankursinha I'm going to prepare an email but I was unclear whether to ask about both the previous mentioned package lists (any others?). Also, my understanding is that these will be separate package groups that optional and won't be installed with the normal installation of Fedora.

What do you mean here? None of these groups will be already installed on any Fedora images apart from the lab image that we create.

When installed, all the software of each group will be installed together, unless specifically left out, when the group is installed by group name (ie. there are not required packages, and optional packages).

Yes, that's the idea of groups. All the software gets pulled in at once. I'm not sure what the implication of "optional packages is". We may have to ask around for that.

Hrm---what is the function of a "category"? Can these be used, or are they merely for organisational purposes?

From the look of the comps xml file for f30 it looks like categories include desktop spins, groups like 'development', 'applications', and 'servers'. For example a group like 'engineering-and-scientific' shows up in the Applications group. Another science related group 'python-science' does not show up in a category but can be installed as a set.

which packages should be default, optional, conditional (docs?), mandatory (none?)

The simulators should be mandatory IMO. A group where all packages are optional isn't too helpful (although, I don't know what the default dnf behaviour is---are optional packages ignored?)

Mandatory would be required to be installed, so if you made simulators mandatory, the group could not be installed without including them. Optional packages are not automatic, but can be checked to be installed. 'Default' are 'checked' to be installed, but can unchecked by user. And conditional is for packages that have dependencies, i guess this could include docs for an app that wouldnt be installed unless the app is.

Mandatory would be required to be installed, so if you made simulators mandatory, the group could not be installed without including them.

That sounds OK to me. What would be the purpose of installing the group if one didn't want the simulators?

Optional packages are not automatic, but can be checked to be installed.
'Default' are 'checked' to be installed, but can unchecked by user.

Uh, right---so, the simulators we want in should be default then? Depending on how we see it, none of them are mandatory, but if none of them are installed, what's the point of having the group? :thought_balloon:

And conditional is for packages that have dependencies, i guess this could include docs for an app that wouldnt be installed unless the app is.

Hrm, yes.

So, I'd go for putting all them packages as default currently, even mandatory maybe. None of our simulators have "optional plugins" but the docs could be included as optional.

I've submitted the request to devel list for the new groups. Thanks for your input all!

Thanks! Lets see what feedback we get on the devel list, and we can take it
from there.

Metadata Update from @ankursinha:
- Issue tagged with: S: Next meeting

4 years ago

Discussed in meeting on 2019-08-29. @dan1mal will file PR against comps on 2nd September 2019 if no negative feedback is received on the devel list.

Metadata Update from @ankursinha:
- Issue untagged with: S: Next meeting

4 years ago

Sorry for the delay, Monday 2019-09-02 was Labor Day in the US, so I couldn't find time to do much labor! :sleeping:

Looking at other PRs it looks like the puller made changes to the files for F31 and F32 and made the request. I imagine we would do the same?

These are the edits, as far as I know, so please review and update me (or let me know where I can find the list/packages):

<group>
    <id>neuro-simulators</id>
    <_name>Neuro Simulators</_name>
    <_description>Tools for simulating neural networks</_description>
    <default>false</default>
    <uservisible>true</uservisible>
    <packagelist>
      <packagereq type="optional">nest</packagereq>
      <packagereq type="optional">neuron</packagereq>
      <packagereq type="optional">genesis</packagereq>
      <packagereq type="optional">moose</packagereq>
      <packagereq type="optional">brian2</packagereq>
      <packagereq type="optional">pynn</packagereq>
    </packagelist>
</group>

<group>
    <id>brain-modelling</id>
    <_name>Brain modelling</_name>
    <_description>Tools for working with brain modelling</_description>
    <default>false</default>
    <uservisible>true</uservisible>
    <packagelist>
      <packagereq type="optional">nest</packagereq>
      <packagereq type="optional">python3-neuron</packagereq>
      <packagereq type="optional">calcium-calculator</packagereq>
      <packagereq type="optional">COPASI</packagereq>
      <packagereq type="optional">bionetgen</packagereq>
      <packagereq type="optional">neurord</packagereq>
      <packagereq type="optional">python-brian2</packagereq>
      <packagereq type="optional">python-libNeuroML</packagereq>
      <packagereq type="optional">python-neo</packagereq>
      <packagereq type="optional">python-nineml</packagereq>
      <packagereq type="optional">python-PyLEMS</packagereq>
      <packagereq type="optional">smoldyn</packagereq>
    </packagelist>
</group>

Once we have confirmation these are the files to be packaged Ill edit the files and make the PR.

Thanks very much @dan1mal ---no need to worry about getting to it before either XD

Here are some proposals:

  • "neuro-simulators" -> "neuron-modelling" since "neuro" is nowadays broad enough a term to refer to anything brain related, an umbrella term. "Modelling" is better than "simulating"---simulating is what we do when we model neurons etc.
  • In the same way, maybe: "Tools for modelling neurons and neuronal networks"? (to keep this different from "neural networks" which are primarily a machine learning tool)
  • everywhere, we should use python3-nest instead nest. The python bindings are the official mode of use for NEST. The sli interpreter provided by the NEST package is not generally used nowadays.
  • In the case of neuron, both neuron and python3-neuron should be included. neuron is required for older models and ones that require detailed modelling, but more and more researchers nowadays are using the python bindings (for obvious reasons) :)
  • For the next one, maybe simply "Tools for brain related modelling". that'll keep it vague enough for us to be able to include any brain modelling related tools into in the future.

I see that the second group is basically a superset of the first one. Is there a chance to include a group in a group? Then the first group would be defined, and in the second group we wouldnt have to repeat the packages---we simply include the first group?

From my understanding there isn't a way to place a group inside another group. I believe we could create a primary group of packages to be installed as default or required which would include the simulators. Then the rest of the software could be pulled in as optional.

Neuro-modelling packages would go in the required (core) list of the group, and the other brain related modelling tools would go in the optional list. So, a new user could install with this command and get the core group, or, with the [--with-optional] switch, include the other tools.

dnf [options] group install [--with-optional] neuro-modelling

It would be nice if we could have a sub group in the comps.xml and allow that group to be associated and 'added on', but perhaps that adds unnecessary complexity. We could just add another group and use:

dnf [options] group install neuro-modelling brain-modelling

Metadata Update from @dan1mal:
- Issue tagged with: S: Next meeting

4 years ago

Based on today's meeting, I think this is how we I think we could go about it:

Environment groups map to the four types of software we have:

  • modelling
  • neuro-imaging
  • analysis
  • utilities

(I'd say the last two are optional---users can install the individual packages they need).

In each, we can then have groups if there are enough packages of a particular type to merit a group.

For the current neuro-modelling bits, this becomes:

  • Environment "Brain modelling", which includes a "neuron-modelling" group, and the other modelling/simulation packages listed by themselves .

Thoughts?

Based on today's meeting, I think this is how we I think we could go about it:
Environment groups map to the four types of software we have:

modelling
neuro-imaging
analysis
utilities

Would this mean 1 environment with these 4 groups and then the individual packages falling into one of those 4 groups?

On a side note, it seems to me that maybe the spin ticket [0] isn't really dependent on this issue. The spin could be generated using packages and custom .KS file, which could be amended later when the groups/environment are setup properly. Could we disconnect them and push forward with issue #199 in parallel with this one?

Lastly, do you think I should remove the "next meeting" tag after its been discussed, or leave it for the next meeting administrator?

[0] https://pagure.io/neuro-sig/NeuroFedora/issue/199

Would this mean 1 environment with these 4 groups and then the individual packages falling into one of those 4 groups?

No, I was thinking of two main Environments: "Brain modelling" and "Brain imaging", and then groups in there if needed. In "Brain modelling", we can have a "neuron modelling" group for example. Once we have more imaging software, groups may emerge there too

On a side note, it seems to me that maybe the spin ticket [0] isn't really dependent on this issue. The spin could be generated using packages and custom .KS file, which could be amended later when the groups/environment are setup properly. Could we disconnect them and push forward with issue #199 in parallel with this one?

+1 removing dep now

Lastly, do you think I should remove the "next meeting" tag after its been discussed, or leave it for the next meeting administrator?

I generally leave the flag so that we remember to check the status of progress on this task, and so that everyone at the meeting gets an update too. What do you think?

Metadata Update from @ankursinha:
- Issue unmarked as blocking: #199

4 years ago

Environment mockup:

 <environment>
    <id>brain-modelling</id>
    <_name>NeuroFedora Brain Modelling</_name>
    <_description>Tools for working with brain modelling.</_description>
    <display_order>*TBD*</display_order>
    <grouplist>
      <groupid>neuron-modelling</groupid>
    </grouplist>
    <optionlist>
      <groupid>optional-group-placeholder</groupid>
    </optionlist>
  </environment>
  <environment>
    <id>brain-imaging</id>
    <_name>NeuroFedora Brain Imaging</_name>
    <_description>Tools for working with brain imaging.</_description>
    <display_order>*TBD*</display_order>
    <grouplist>
      <groupid>brain-imaging</groupid>
    </grouplist>
    <optionlist>
      <groupid>optional-group-placeholder</groupid>
    </optionlist>
  </environment>

I was thinking, my next step would be compiling a list of currently active and soon to be packaged/delivered software so we can put them in groups.

I saw these three lists
- https://docs.fedoraproject.org/en-US/neurofedora/compneuro-tools/
- https://docs.fedoraproject.org/en-US/neurofedora/imaging-tools/
- https://docs.fedoraproject.org/en-US/neurofedora/analysis-tools/

And this list:
https://src.fedoraproject.org/group/neuro-sig

Then there is the pagure:
https://pagure.io/neuro-sig/NeuroFedora/issues?status=Open&tags=T%3A+Software&priority=0&close_status=

Any suggestions in terms of what to include, or shall I include everything for now?

Did you fork comps? I've set up a fork (pagure doesn't support forking under a group namespace, unfortunately: https://pagure.io/pagure/issue/4602) with the initial commits there:

https://pagure.io/fork/ankursinha/fedora-comps/tree/neuro-fedora

Can you fork that and tweak there I wonder? (Or I can open a PR against your fork?)

I don't think we need an "Environment group". Those seem to be for desktop environments and things. I've added a "Neuroscience" category, and to begin with a "neuron-modelling-simulators" group---with packages as optional so that we can use it in the kickstart and users can choose what they want. These will both show in Anaconda---only if someone uses the netinstall, these aren't shown in the live media, and won't be shown in the image we're working on now.

I think we needn't do neuro-imaging now---we don't have enough packages. We'll work on the packaging and then work on a new comps group and kickstart. How does that sound?

I forked your fork, and can see your neuro-fedora branch (and the changes you made) in it.

Perhaps, if these are waiting for new packages I could start learning to package. Any recommendations for an easy ticket to start with?

I forked your fork, and can see your neuro-fedora branch (and the changes you made) in it.

OK! Yay! How does that look?

Perhaps, if these are waiting for new packages I could start learning to package. Any recommendations for an easy ticket to start with?

Sure---maybe start with one that is python based? They're generally quite easy and we have a spec template here:

https://pagure.io/neuro-sig/NeuroFedora/blob/master/f/spec-templates/python.spec

All the packaging related links are here:
https://fedoraproject.org/wiki/Join_the_package_collection_maintainers

(Please ping any of us on any of the channels anytime if you have questions :))

I've updated the comps files for F30, F31, F32 now:

https://pagure.io/fork/ankursinha/fedora-comps/commits/neuro-fedora

Could you folks have a quick look and then I'll open a PR tomorrow if no (-1s) have been noted by then.

Cheers!

Reviewed. It looks good to me, the only question I had was about the display_order being 120. I was trying to figure out where that would place the listing, but I'm not so sure it is used anymore. Perhaps it was for the old package manager? DNF appears to list items alphabetically for non-environment groups.

+1

We'll figure out the display_order in the PR---I just picked a large enough number XD

PR filed now: https://pagure.io/fedora-comps/pull-request/410

Closing this, please subscribe to the ticket and discuss further changes there.

Metadata Update from @ankursinha:
- Issue close_status updated to: Fixed
- Issue status updated to: Closed (was: Open)

4 years ago

Note---PR was merged! :fireworks:

Login to comment on this ticket.

Metadata