Getting into GSOC was my dream since my college years, On May 20, I was so delighted when I got into my preferred Project in one of the most competitive Organisations. I will forever remember those sleepless nights spent solving those tasks, debugging, and building the project !! I have prepared a detailed blog on how I got into GSOC 2022 at CERN HSF in detail here
This is the period of time between when accepted GSoC contributors are announced and the time they are expected to start coding. This time is an excellent one to introduce your GSoC contributors to the community, get them on the right mailing lists, introduce them to the codebase, discuss how they will work with their mentors on their timeline for the program, etc.
Again I have written a very detailed blog on Community Bonding Period here.
For TMVA projects we usually have weekly meetings to discuss about the projects and solve the difficulties of everyone. So attending those meetings are mandatory as it involves your communication with other gsoc students of TMVA. Also the round table discussion solves all queries which you have as all the students and mentors give solutions and guidelines on it. Additionally we can also set a meeting with the mentor if required.
In TMVA projects, we have a specific idea of presenting our projects with the detailed timeline of implementation in front of mentors and other Gsoc students so we can get familier with other’s project as well and by that time our concrete strategy of project is also ready.
Here is my presentation for reference: Inference Code Generation for Deep Learning models
I built and set up the Root environment in my laptop as required. Here is the link to build Root from source.
Finalised the evaluation dates so that I get enough time to complete my long-term project. I chose a 16 weeks project after discussing with the mentor as my college reopens on July 18 and I will get less time to devote after that.
I have also contributed to fusing of Add and the MatMul operators in a Gemm Operator Our task was to Implement the capability to parse the MatMul and Add operators together and fuse them in a Gemm operator in the TMVA SOFIE parser (RModelParser_ONNX.cxx) . So basically we take the data of MatMul operator and if we get Add operator exactly after the MatMul operator , we fuse both the Add and the MatMul operators and give their data to Gemm operator. Gemm operator is explained here in detail. This improvement was added by my mentor Lorenzo Moneta in his PR , the link to my commit can be found here.
Lastly, I began coding part for one of the easy operator which is Leaky Relu so i get an headstart for the Project. Here is the link to the PR.
Pro-Tip : Use this Period very well to connect with your mentors, other Gsoc Candidates and the organisation as a whole!
I have written a detailed blog for the Leaky Relu Operator implementation as well. Here you can find the detailed description of the Operator. Here, I will provide a brief Description about it.
Definition : Leaky Relu ONNX Documentation
Fix warning related to length in compiled code when there are no initialized tensors
build/build/tmva/sofie/test/LinearWithLeakyRelu_FromONNX.hxx:24:8: warning: unused variable ‘length’ [-Wunused-variable]
It was general for all SOFIE operators not having a weight tensor. It is resolved by applying a small fix, we directly return when the tensors are not initialized.
if (fInitializedTensors.empty()) return;
Pull Request | PR Number | Status |
---|---|---|
Leaky Relu ONNX Operator | #10415 |
Definition : Max-Pool ONNX Documentation
MaxPool ONNX Operator was only supported for the 2D case, i.e 4d tensors but I need to extend its support for the 1D and 3D cases as well.
Earlier it was giving a runtime error for 1d and 3d cases of Max Pool operator.
Error is described in the image mentioned below. I resolved the error by extending the support of Maxpool Operator for 3d and 5d tensors as well.
I also added the Unit Tests for MaxPool 1D, MaxPool 2D, MaxPool 3D and the Average Pool Operators.
PR Status:
Pull Request | PR Number | Status |
---|---|---|
Max Pool ONNX Operator | #10768 |
I have implemented the four Basic Binary operators in a single file, they all are declared in an enum and we have used Type traits for making the code more modular and reusable. Below is the code sample for the same.
The following is the algorithm and its implemented in the SOFIE_common.cxx.
CASE-1:The tensors all have exactly the same shape. : We support it already now
CASE-2:The tensors all have the same number of dimensions and the length of each dimensions is either a common length or 1. Example: This is the example for two tensors as (2,3,4) and (1,3,4) the result is (2,3,4)
CASE-3: The tensors that have too few dimensions can have their shapes prepended with a dimension of length 1 to satisfy CASE-2. Example: This is the case for (3,4,5) and (2,1,1,1). Here we transform first tensor to (1,3,4,5) and the is like case 2 above. The result is (2,3,4,5)
So the algorithm should do:
CASE-1 : Check if tensor have same shape - nothing special to do, we already supported it.
CASE-2 and CASE-3 \: If shape is different we call the multi-directional broadcasting function.
CASE-3 : if shapeA .size() < shapeB.size() we insert in the shape vector values of 1 at the beginning of the tensor until shapeA.size() == shapeB.size()
CASE-2 : We look then to the shape values, and if they are equal result is same, if shapeA[i] not equal to shapeB[i] we have the result shape equal to shapeA[i] if shapeB[i] = 1 or shapeB[i] if shapeA[i] = 1.
Pull Request | PR Number | Status |
---|---|---|
Basic Binary ONNX Operator | #10822 |
Definition: Tanh ONNX Documentation
cmath
library to support the tanh operator. So we add the required libraries in RModelParser_ONNX.cxx
in the Parse method.
if (op_type == "Tanh")
rmodel.AddNeededStdLib("cmath");
Pull Request | PR Number | Status |
---|---|---|
Tanh ONNX Operator | #10913 |
Definition: Neg ONNX Documentation
PR Status:
Pull Request | PR Number | Status |
---|---|---|
Neg ONNX Operator | #10946 |
I have implemented the multi-directional broadcasting for broadcasting the shapes of output tensors. Currently its only supported for input and output tensors having same length.
Pull Request | PR Number | Status |
---|---|---|
Pow ONNX Operator | #10971 |
Definition: Cast ONNX Documentation
First SOFIE was only supporting ETensorType::FLOAT
input and output tensors now the code was implemented so that it can support for other datatypes as well like ETensorType::INT16
, ETensorType::INT32
, ETensorType::INT64
, ETensorType::UINT16
, ETensorType::UINT32
, ETensorType::UINT64
and ETensorType::DOUBLE
.
Here fAttrType
is the data type to which the elements of the input tensor are cast.
Pull Request | PR Number | Status |
---|---|---|
Cast ONNX Operator | #11033 |
Shape Operator takes a tensor as input and outputs an 1D int64 tensor containing the shape of the input tensor.
Note that the fOutput_shape
is determined by fOutput_shape = { size_t(fEnd - fStart) + 1};
, where fStart
and fEnd
are Optional Attributes denoting the Start and end position of input tensor shape, it is used to compute a slice of the input tensor’s shape.
Pull Request | PR Number | Status |
---|---|---|
Shape ONNX Operator | #11086 |
Max Operator calculates element-wise max of each of the input tensors (with Numpy-style broadcasting support). It also supports multi-directional broadcasting as well.
Pull Request | PR Number | Status |
---|---|---|
Max ONNX Operator | #11198 |
Pull Request | PR Number | Status |
---|---|---|
Reduce ONNX Operators | #11258 |
Concatenate a sequence of tensors into a single tensor. All input tensors must have the same shape, except for the dimension size of the axis to concatenate on. By default ‘new_axis’ is 0, the behavior is similar to numpy.concatenate(). When ‘new_axis’ is 1, the behavior is similar to numpy.stack().
np.concat() part was already implemented previously in the concat operator , i need to extend the support so it could support np.stack() functionality as well. This was required for GNN testing.
Pull Request | PR Number | Status |
---|---|---|
Extend Concat ONNX Operator to implement stack functionality as the ONNX ConcatFromSequence operator | #11317 |
1) Python Tutorials for various C files of Tutorials/TMVA
2) Documentation on RModelParser_ONNX.cxx
4) All about Community Bonding Period
5) Implementing the Operators in Sofie
6) Final Evaluation Detailed Report
Inference Code Generation for Deep Learning models
I enjoyed a lot working on this project! I would like to really thank my mentors Lorenzo Moneta, Sitong An, Omar, Ahmat Hamdan, and Sanjiban Sengupta for always being a great support for me. Whenever I wanted any help or guidance, they were always with me! I am very proud to be associated with so many bright minds surrounding me, and every single day I learn something new from them. In the end, I am able to achieve all of my success because of the best wishes of my parents, seniors, and friends so a big thanks to them as well.
Hope you all enjoyed reading my blog and learnt a lot.
Thanks and Regards,
Neel Shah