HSF Weekly Meeting #116, November 9, 2017
Present: Graeme Stewart, Daniel Elvira, Dario Menasce, John Harvey, Benedikt Hegner, Michel Jouvin, Eduardo Rodrigues, Pere Mato, Andrea Valassi
General News
- HSF/WLCG Workshop
- Graeme, Benedikt, Michel, Ian B and Ian C met on Tuesday to come up with a draft agenda
- The main themes are to look for commonality and performance
- Possible sessions are
- Cost/Optimisation Model
- Common Data Management / Data Lakes
- Workload Management (provisioning, HTCondor as one theme)
- Frameworks and Infrastructure (where’s the boundary? What’s the role of HPC and Event Service)
- Concurrency in C++ and Co-Processors
- Analysis Facilities and Use-cases
- Training
- Technology Watch
- Simulation (with one topic being fast vs. full simulation)
- In addition we would like to see the generator (interfacing) community to be present and will invite the HSF WGs to organize parallel sessions. The topic of automatic validation should also be covered.
- Parallel sessions should summarise their work in a plenary talk
- We were interested in having (summary) presentations about:
- Connecting the Dots Workshop (taking place the week before)
- Machine Learning (starting to think about workload and resources)
- Experiment workflows and their data reduction approaches
- Publication and citation of SW
- A contribution from the Software Sustainability Institute/RSE
- Security
- DPHEP
- Will look for candidates to help in organizing the workshop and see how this fits into the agenda slots
- Action: Progress these topics into the Indico Agenga and announce again
- Andrea: try to minimise the number of parallel sessions, and ensure that WLCG people follow HSF discussions and vice versa, to profit from this first joint workshop
CWP
-
General matters and Editorial Board
- EB meeting yesterday: see agenda/minutes. Not very well attended, which concerns us; trying to get more people on board to do things.
- Work on draft 2 started in a shadow document restricted to EB, in much the same way as draft 1. Want to stick with deadline for releasing it (Nov. 17). Already some work done (e.g. Simulation section)
- Almost agreed on the section ordering
- Signatories: some progress, need more…
- An EB meeting planned next week
- First draft will close for comments soon (probably Friday evening, at the latest Monday morning)
- Several presentations scheduled at the upcoming experiments’ weeks. Let us know if there is a coming experiment (not necessarily LHC) event where you’d like to get a short CWP status to raise attention.
-
N.B. Discussion of CWP chapters was curtailed to cope with time lost due to Vidyo problems (many-worlds).
-
Simulation
- section has been updated to reflect the on-going discussions of how to describe R&D projects; has been discussed with ATLAS and will meet with CMS next week
- added a little detail (~1 page) in the Current Practices section that contains background information to help better understand the future R&D programme
- Andrea Dotti has added some text describing the R&D work he is proposing on modularising the Geant4 framework to allow easier integration in experiment frameworks
-
Machine Learning
- Many comments on this section of the roadmap, will require input from the WG to address them all properly.
-
Software Development
-
Software Trigger and Event Reconstruction
-
Data Access, Organisation and Management
- Current draft sent for comments to the WG. None received as of today: not clear if people reviewed it and are happy or if just nobody had time for it.
-
Data and Software Preservation
-
Data Analysis and Interpretation
-
Workflow and Resource Management
- Spoke to Kaushik and this will not be in the CWP. What they produce may be useful for some other chapters.
-
Visualization
-
Event/Data Processing Frameworks
-
Careers, Staffing and Training
-
Facilities and Distributed Computing
-
Conditions Access
-
Generators
- Contact with MCNet has resolved some misunderstandings. They will contribute to the paper, but we are very worried about the draft 2 deadline. Graeme is in touch with them and trying to get them to collaborate with the other WG members to converge as fast as possible.
-
Security
- An early draft available, working on providing something with the agreed structure, hoping something Friday 10.
-
Submission of WG papers to arXiv
- EB had no time to discuss that this week, but will try again next week.
Activity updates
Licensing
- Discussion with LHCb, who are going to propose copyright held by CERN and a GPLv3+ license.
- Graeme continues to be in touch with Jorgen (HepMC), but no decision yet.
Training
- Graeme helped out with the first two days of LHCb StarterKit training last week
- This is popular training - ALICE joined and there were about 100 people in total
- Generic topics were covered in the first 2 days:
- Python - 1 day
- Bash - 0.5 days
- Git + Gitlab - 0.5 days
- 3 groups of about 30 students and 3-4 tutors
- Format was mostly ‘follow along’ as a tutor demonstrated live at the front from web training material
- Interrupt for questions at any time
- Occasional breaks for set exercises
- Use a red/green sticker to attract attention (taken from Software Carpentry)
- Personally I thought the shell and git training was stronger as it was more student centred. The python seemed more focused on the tutor and the exercises were too advanced for a lot of the students to manage without help. I also felt too much time was spent on corner cases (e.g., integer division).
- Concrete Proposal arising from this:
- Reboot the Training WG, trying to involve more people in the community
- Approach the 4 LHC experiments with a view to having an HSF training event in “Scientific Programming Core Skills” next year (spring/summer). The topics should be:
- Git and collaborative development
- Basic shell
- Python
- C++
- Duration would need to be defined, that might be driven by whether the experiments wanted to add experiment specific training into the same event
- Dario: need to have a central point for gathering training material; Agreed, we should “curate” training material (Dan Katz’s phrase). Should be reviewing courses and suggesting improvements (pull requests even).
- Action: start a list of good tutorials we know on the HSF web site, as part of training activities; Also ensure they are registered in the HEP SW&C knowledge base
Packaging
AOB
HSF/WLCG Workshop
- Proposal from Alessandra Forti to have an ML workshop
- Actually there are quite a lot of workshops already, so we don’t think we could add that much to what that community already does
- PyHEP: had already a few discussions. Hoping to have a more concrete draft v0 agenda proposal for next week.