Profiling application performance is one of the critical tasks needed
in order to optimise HEP workloads. This is usually done using well
established tools like Perf,
Google Perf Tools,
Valgrind; or in house tools like
igprof. Doing the profiling is only part of
the job though, being able to easy navigate, analyse, correlate and share
the reports accumulated over time is another key factor.
We propose a project where the goal is to implement a prototype for a
HEP-scale, web based, multi-user service for storing, processing and
visualising profiling reports.
- Keeping in mind the format of reports, usually a DAG with counters
and source code lines associated to them, design a scalable data-warehouse
which can host and process time evolving reports from different users.
This should include a REST based API to allow different users to
import reports and navigate them.
- Provide a web based GUI, using the above mentioned REST API which
allows for ease of navigation of the different reports, overlaying
them with the associated project source code.
- A web service where performance profile data can be uploaded,
visualised and compared.
- The service should be able to support at least IgProf and Google Perf Tools
- Cloud based provisioning tools (e.g. Kubernetes)
- Dynamic web based visualisation technology
- Backend database layer