HSF Weekly Meeting #169, 11 July, 2019
Present/Contributors: Graeme Stewart, Andre Sailer, Marco Clememcic,
Pere Mato, Agnieszka Dziurda, Daniel Elvira, Andrei Gheata, Serhan Mete,
David Lange, Ed Moyse, Paul Laycock, Torre Wenaus, Ben Morgan, Dario
Menasce, Eduardo Rodrigues, Gloria Corti, Liz-Sexton Kennedy, Heather
Gray, Sudhir Malik, Caterina Doglioni, Mark Neubauer
Apologies: Andrea Valassi
News, general matters
- HPCs and GPUs - cost model and benchmarking WG
- WLCG Management Board has set up a group to work on benchmarking of GPUs in
particular.
- This is to anticipate that some funding agencies might offer GPU resources
as part of their WLCG pledge.
- Some notes and the charge in this
Google
Doc.
- Note that at the moment we don’t have many (any?) workloads
that we could run at any scale.
- We are asked to have at least 1 HSF representative in the
group.
- Would anyone be keen to participate?
- David; Graeme; Andrea V; Hadrien G (all can contribute a
bit).
- EDM4HEP
- As an outcome of the Bologna Future Collider Software Workshop
we had the first
meeting of the
EDM4HEP group this week.
- HSF is acting as an umbrella.
- ILC, CLIC, CEPC, FCC all present, we reviewed LCIO and
FCC-EDM.
- Had to change some of our normal practices to collaborate with
Chinese colleagues (Google blocked).
- ECFA Presentation
- Graeme will give a talk on Computing and Software Challenges
at the ECFA session of EPS-HEP.
- Draft
talk - comments welcome.
Google Summer of Code 2019
- Second coding period.
- All students passed, but a few ‘alerts’ to watch out for.
- Mostly going well.
Google Season of Docs
- Application period ended.
- 5 applications for 3 proposals (DIRAC, Rucio, ROOT).
- Mentors are looking at these.
- Our choices by 23 July.
- Seems Google will only finance one project.
- August for community bonding, three months work.
- How to select?
- Filter and use the same committee as GSoC - Peter, Pere and
Michel.
Activity and Working Group Updates
Data Analysis
- Next meeting will be on the 22nd July on “Computing infrastructure
for future data analysis”, as a precursor to this year’s pre-CHEP
workshop on ~the same topic.
- Agenda (and indico) still in preparation.
Detector Simulation
Reconstruction and Software Triggers
Update slides attached to agenda
1) Acting on topics of interest from Reconstruction and Software
Trigger CWP roadmap
(link)
P - Past meeting - J JLab workshop talks - F Future planned meetings
- P: Enhanced vectorization programming techniques
- P: Algorithms and data structures to efficiently exploit many-core architectures
- J: Algorithms and data structures for non-x86 computing
architectures (e.g., GPUs, FPGAs)
- F: Enhanced quality assurance (QA) and quality control (QC) for
reconstruction techniques
- PJF: Real-time analysis
- JF: Precision physics-object reconstruction, identification and
measurement techniques
- F: Fast software trigger and reconstruction algorithms for
high-density environments
2) Seeking enhanced collaboration / discussion with other communities:
- F: Neutrino and astroparticle
- JF: Nuclear physics
Past meetings
- Summary of ATLAS / CMS trigger April/May cross-talks
- Algorithms and data structures to efficiently exploit many-core
architectures
Planned meetings
- Next week (July 17th): joint discussion on partial event building
for real-time analysis within Institut Pascal
“Learning To Discover”
workshop https://www.universite-paris-saclay.fr/fr/real-time-workshop
(agenda tba today)
- July 31st: joint ACTS
meeting
- August 28th (tbc after contacting speakers): second part of
trigger and real-time analysis meeting with LHCb & ALICE
- September 10-13: IRIS-HEP Blueprint Meeting on
Fast Machine Learning and
Inference + and
FastML tutorials
- October 2nd (tbc after contacting speakers): second part of
software optimization meeting, with ATLAS, CMS and ALICE
- October 16th (tbc after discussion with organizers): joint
discussion with neutrino and astroparticle community,
after JENAS
workshop
- November/December:
- Meeting focused on reconstruction techniques
- Hands-on tutorial on FPGAs
- Topic for meetings after the summer break:
- A follow-up discussion on Trident
- A presentation/demonstration from
Tuning and Analysis Utilities
(TAU)
people
- Trying to pin down someone from the team
- A discussion GPU profiling
- Need to see if we can contact someone from NVIDIA (via
OpenLab)
- A discussion on Hadrien’s valgrind plugin for numerical
stability
- A discussion on IDEs, i.e. see if we can invite someone from
Microsoft on VSCode
- There is some recent development on the
remote
development
- (Gordon) I’ve been using this tool for a while now, and it
was pretty amazing. At a local group I saw a guy present
who was partly responsible in MS for the remote
development code. Shall I see if I can get in touch with
him?
- Serhan : That’d be fantastic Gordon, thank you very
much.
- A talk on Mutation Testing
(mull-project)
- A talk on test-suites he added to Phoenix (GSOC student,
timescale?)
- A talk on
Nexus
(a web app to store and organize binaries, libraries, RPMs
etc.)
- That would also be interesting for the packaging group.
PyHEP
- Preparations for the 2nd workshop, PyHEP 2019, have started. An
announcement email will be sent next Monday or Tuesday at the
latest.
- Workshop taking place in the UK, at
The Cosener’s House, Abingdon.
- Will run 2.5 days, October 16-18.
- We welcome input and suggestions from the HSF already at this
stage.
- Facilities can accommodate 80 people.
- Organisation together with Ben Krikler, a CMS colleague and
UK Software Sustainability
Institute 2019 fellow.
- The WG is organising topical meetings for after the Summer
break.
- The first one is penciled down for September 11th at 17h (same
time as the Software Forum). It will be on fitting tools and their
design, hopefully engaging with non-HEP communities.
- Agenda is being put in place and an advertisement will come out
soon.
Training
- See
slides
attached to the agenda…
- Software Carpentry @ CERN, 16-18 October.
- (Unfortunately now clashes with PyHEP, but constraints are
quite binding and we should go ahead.)
- Order maybe better as Shell / Git / Python / C++?
- Is 1 day enough to do much that’s useful in C++
- ROOT focus may help, but teach modern programming (not
‘98), use ROOT7 interfaces
- Is ROOT7 stable enough?
- A lot still experimental though
- To discuss with ROOT team
Event Generators
- Subjective report from Andrea (apologies, in another meeting)
- Last meeting on June
27.
Many thanks for all the work and contributions!
- Further progress in accounting analysis, close to
completion. Over the 2017 MC the fraction of time spent on
generation (with respect to the MC production chain) is
17% for ATLAS and 9.3% for CMS. Taking into account MC is
around 70% of ATLAS and 50% of CMS of overall WLCG
computing, this means generation is around ~12% for ATLAS
and ~5% for CMS of overall WLCG computing. Still larger
for ATLAS than CMS, but only by a factor ~2, not 10.
- Report from Les Houches workshop in June: a lot of
discussion about negative event weights and how their
impact could be reduced to decrease CPU budget.
- Top physics WG considering ttbar sample sharing between
ATLAS and CMS.
- Somewhat outside our usual scope, but welcome in the HSF
WG: discussion of a proposal for Particle Data Group
particle ID’s extensions.
- Would like to define a few standard configurations for similar
physics productions using different generators (to use for
profiling and also as standard candles for hardware
benchmarking and software comparisons. No progress on this.
- Event generators are often discussed as a candidate workflow
to exploit GPU based resources (e.g. HPC supercomputers). But
we have not discussed this yet in detail in any meeting and I
am not aware of any significant progress in this area. One WG
member recently informed us that he is starting some work on
this.
Event Delivery Forum
- Doodling for first meeting at the end of this month:
Packaging
- Met yesterday, minutes available
here.
We discussed the Key4HEP stack project.
- Next meeting 11 September.
Frameworks
- We have 9 nominees for the working group.
- Names have been circulated and we requested comments by 25 August.
- This is very long, but it’s summer and it would be hard to
converge and organise anything before this time.
- Comments to
hsf-wg-search-committee@cern.ch
(maps to Graeme, Liz and Michel)
Visualisation
- Phoenix GSoC project is going very well.
- Experiment independent in browser toolkit, easy to use.
- Would be good to present this.
Workshops
Pre-CHEP (2-3 November)
- Wait for next Analysis WG meeting for progress.
Next HSF/WLCG Workshop
- Call for hosting institutes is out.
- Considering the constraints in May and June
- June now looks very difficult (LHCC, LHCb Week, ATLAS S\&C
Week, ATLAS Week)
- In May 4-8 or 11-15 look possible (18-22 is Ascension Week,
25-29 DIRAC Users Workshop)
- Are there any more known clashes for May 4-8 or May 11-15?
- It would be very good to pin down a favoured date now.
- We agreed that 11-15 May would be favoured.
- Can we extend the deadline for institutes?
- Early September should be ok - Graeme will discuss with the
other organisers.
AOB
- Git repository for Harvester.
- Harvester is PanDA’s new workload submission engine (deals
with HPCs, dynamic cloud resources, other non-standard
resources)
- But it’s actually backend agnostic, so a request to host the
code as an HSF project.
- Part of Intelligent Data Delivery System (iDDS).
- N.B. similar to things we currently do host, like CREST and
Phoenix.
- Agreed.
- Summer break now, when to resume meetings?
- Resume 29 August [as discussed after the meeting, on realising that
5 September is Jeûne Genevois CERN holiday].