HSF Weekly Meeting #163, 11 April, 2019
Present/Contributors: Graeme Stewart, Eduardo Rodrigues, Serhan Mete,
Pere Mato, Andrei Gheata, Daniel Elvira, Stefan Roiser, Andrea Rizzi,
Antoine Péruz, Liz Sexton-Kennedy, Paul Laycock, Torre Wenaus, Witek
Pokorski, Giulio Eulisse, Marco Clemencic, Heather Gray, Jim Amundson,
Michel Jouvin, Caterina Doglioni, Agnieszka Dziurda, Danilo Piparo,
Martin Ritter, David Lange, Brian Bockelman
News, general matters
- Preparations for the European Strategy
Symposium in
Granada are underway - there will be a parallel session on
software and computing, organised by Brigitte Vachon (McGill),
Xinchou Lou (IHEP).
- Graeme has been asked to speak on Software R\&D.
- This will be based on
inputs
received during the process (including
our
own
from the HSF).
- Do feel free to raise any key points that you think should be
made there.
- New Event Delivery Forum proposal - comments from Torre:
- ATLAS has had its “event service” in production for some time,
which assigns processing to workers with granulary down to
event level, and returns results in a similarly granular way,
well suited to using opportunistic resources with full
efficiency.
- Event delivery, however, still uses conventional mechanisms.
The next phase of development has recently begun, the event
streaming service (ESS), to intelligently deliver event data
to workers, with latency hiding and with the possibility to
prepare, filter and marshal data to use storage and network
efficiently.
- There need be nothing ATLAS-specific about this event delivery
service, and Brian Bockelman has expressed interest in it from
IRIS-HEP, where he’s been developing the notion of an
intelligent data delivery service (iDDS).
- ATLAS and IRIS-HEP have decided to jointly develop the iDDS as
a common project; the ATLAS ESS will be a specialization of it
for ATLAS.
- HSF seems like an appropriate home for this common project, so
we propose that HSF host a google group and web presence as an
‘event delivery forum’ which would be the communication hub
for this effort, to which others are of course welcome
(including lurkers!).
- Meeting supportive and we agree that HSF is a good home -
people welcome to also just ‘follow-along’ to keep an eye on
developments.
Google Summer of Code 2019
- Coordinators Report:
- Students have now made final proposals.
- 197 proposals in total. About 10 are not eligible.
- One project (AWAKE) got 43 proposals.
- Three projects got 10-15 proposals.
- Most projects 2-9.
- Thirteen projects got 0 (so they are out).
- Mentors:
- Rank the proposals, pick at most 3 that you would like to
mentor. Precise criteria sent to mentors for the ranking
by mail.
- Do this by Sunday 14 April.
- Maximum slots would be 41, but more likely to be similar to
last year (29).
- Slot request 22 April: some final adjustments after this date
still possible up to the publication of accepted proposals.
- Accepted proposals announced 6 May.
- The Advisory Committee will be:
- Pete Elmer (Princeton), Pere Mato (CERN), Michel Jouvin (LAL)
- Intent and guidelines are described
here.
- Organisers will provide as much information as possible to
make it easy to generate the ranking.
- Needs to be ready end of next week - most important thing is
to cut the tail, rather than sort the top proposals.
Google Season of Docs
- A few projects have declared interest: Rucio, ROOT.
- Don’t want more than 3 for this first test of this project.
- Proposal to be written in the same way as for GSoC on the HSF
website.
- April 23 is the deadline.
- Gaudi expressing some interest - what is the expected commitment
from the project?
- Not clear - Rucio have proposed 6-7 mentors to spread the
load.
HSF/WLCG/OSG Workshop
- Summary
newsletter:
- Note that the newsletter
section
of the website has been revived to do this.
- This is ideal for writing any longer articles you’d like to
see.
- General questions were raised about very specific sessions vs. general
workshop and the amount of parallel sessions in software and with the rest
of the workshop.
- Many people appreciated the overview that you could get from a
wide set of software sessions that didn’t overlap much.
- However, most groups felt they did not have enough time.
- More cross-cutting sessions would be useful (WLCG/HSF).
- But that would reduce per-group sessions even more.
- Many valid points which are impossible to satisfy in
parallel!
- We may have a better idea of what WGs want for next year, when
the WGs that are running are much better established and know what
they want from the workshop.
Activity and Working Group Updates
Data Analysis
- Brief summary: very pleased with the two sessions, interesting
presentations and lively discussions. Follow up topics for our
next meetings have been announced for the start of May. Topics:
- 2nd May:
https://indico.cern.ch/event/813207/
- Workshop summary
- Defining benchmarks - allow comparisons of different
technologies.
- Analysis description language - quite a lively discussion, a workshop
organised at FNAL.
- 9th May:
https://indico.cern.ch/event/813208/
- Metadata - would like feedback from all experiments here,
is there something we could provide (unlikely a
one-size-fits-all metadata solution, but perhaps something
akin to the TTree for event data).
Detector Simulation
- Very high-level quality of presentation; thought that having the
talks not being experiment-specific was successful – encouraged
discussions prior to the workshop between the different
experiments.
- Would have liked to have additional time for discussion, but think
that the workshop will be a good starting point for more in-depth
discussions in the topical meetings.
- Advertised two next simulation topical meetings, 8 May
(accelerators) and 12 June (detector geometry description).
- Need to create Indico entries.
Reconstruction and Software Triggers
- Brief HSF wrap-up: it went well!
- RTA session:
- Nice to see all LHC collaborations.
- ATLAS and CMS have synergies on physics analysis, ALICE is
mostly interested in detector calibration/reco in
real-time, LHCb well advanced with a different trigger
infrastructure but there are plans for ATLAS/CMS to
attempt continuous readout and topic has been deemed
interesting by trigger conveners in our preliminary chats,
so we will continue the discussion.
- Machine learning / reco session:
- Talks were more heterogeneous.
- JLab experiments interested to know more about LHC ones to
avoid reinventing wheels.
- Maybe things are too experiment-specific though?
- Having speakers talk to each other in advance meant that they
also talked to each other at the workshop without our
“supervision”.
- We may have a meeting on April 17th or 24th on summary of
ATLAS/CMS cross-talk open to non-ATLAS/CMS members (after second
cross-talk on April 4th).
- Waiting for ATLAS/CMS conveners to get back to us on exact
date (pinged).
- Other ideas for future:
- Organize taster sessions to coding on non-CPU, since high
barrier to entry so far / not clear where resources are (maybe
coordinate with Training group?).
- HLS for FPGA
- GPU programming
- So far emphasis on trigger (possible bias from interest of ⅔
of conveners), but would like to involve reco community more.
- Invite LHC, dark matter and Dune communities to talk about
their reconstruction algorithms.
- Highlights from the workshop:
-
- Profiling
- TAU
- TAU Performance System® is a portable profiling and tracing
toolkit for performance analysis of parallel programs written
in Fortran, C, C++, UPC, Java, Python.
- Looks promising. We can contact the developers and invite
them to make a presentation/demonstration during a meeting
in the coming weeks.
- Profiling Trident : This was discussed during one of the
cost monitoring parallel sessions. We already had a
demonstration from Servesh a few months ago. We can follow
this up w/ Part-II.
- Static Analyzers Coverity : Being used a fair bit. Seems
to have had issues supporting latest C++ standards. However,
the latest version seems to support C++17. CMS is using
Clang’s static analyzers.
- An idea here might be a presentation from one of the
experiments to demonstrate how these analyzers are
integrated into their frameworks, how the issues
reported/followed-up etc.
- Packaging Spack/Conda : Spack looks like the most likely
candidate (especially for large-scale projects). Conda is used
extensively by users.
- Packaging
RPATH/LD_PRELOAD/LD_LIBRARY_PATH
conundrum :)
There were a lot of discussions but no clear agreement on one
or the other.
PyHEP
- JLab session was the 1st WG meeting per se.
- 1.5 hours is a bit on the short side, to be honest.
- The mix of “standard” talks and an invited talk from Anaconda was
a good thing - several participants discussed with the Anaconda
speaker during the session and workshop dinner.
- We had lively discussions throughout the session - great.
- Feedback was mixed, from very positive to “I did not understand
the flavour of the session”.
- Need to discuss what next steps are. Our WG is a bit special, as
Python permeates various WGs/communities. For sure we need to
coordinate certain activities, e.g., discussions around Python
analysis tools with the Data Analysis WG (the DAWG convenors
agree!).
Training
- Sudhir will give an update in the next meeting.
Packaging
- We gave a talk and had quite a lot of discussion at JLab in the
Software Tools session.
- See the summary that Graeme gave at the last
Packaging
Meeting.
- Main takeaway is that we need to advance with our prototypes
and tests to the position where we solve significant pieces of
the problem (e.g. FCC or LArSoft dependencies).
- Next meeting 24 April.
Frameworks
- Touched on frameworks again during the Software for Accelerators
session.
- Which was judged to be a success.
- Scope for re-starting activities based around use of accelerators
and good models for heterogeneous programming, which was
identified as a key missing piece at JLab.
- FNAL would like to be involved in that - need an active group to
take CWP work forward.
- Do we want to have a nomination process for that, a la other
working groups?
- DOE are also interested in this because of accelerators -
would support work in that area so that applications would be
ready for new machines.
- Try to come up with a proposal offline and then circulate it more
widely.
Software Forum
- We have had no Software Forum meetings this year.
- Run-up to JLab workshop.
- Working groups all active and having meetings.
- As well as any talks from experiments we’re not strongly engaged
with or from other science domains.
- Graeme happy to coordinate suggestions here.
AOB
- We should start to think also about the pre-CHEP workshop that we
can have with WLCG.
- Two days (Saturday 2 - Sunday 3 November).
- Multiple rooms are available.
- Easter holidays are almost on us, so many people away.
- Agreed to have the next meeting in 2 weeks time, 25 April.