Student:Till Hülder

Abstract: Deep neural networks have shown their high performance accuracy in many areas such as image and speech recognition. Further they are also associated with high computational costs. In this paper, the approximation of a weight matrices is investigated in terms of time and prediction accuracy. Hierarchical matrices (H-matrices) are used for the approximation. In order to find a suitable H-matrix approximation, submatrices which are approximately low rank must be found in the original matrix. Various variations in algorithm such as the low-rank approximation method and the rank were investigated. From Pre-trained Pytorch models such as ResNet, GoogLeNet and MobileNetV2 the last layers were extracted and approximated. The ImageNet dataset was used for testing. For all tested models it was shown that the time required for a matrix vector operation is be significantly smaller for an approximated matrix. By using GoogLeNet with an approximation rank of 30, 43.89 % of the computing time was saved with a percentage accuracy loss of 5.69 %.

Email:-
Status:

Supervisor:Matthias Kissel

Files

 


Documentation

-


Workflow

Start

  • Topic specification
  • Definition of work packages
  • Composition of a project proposal and time plan
  • Project Talk with Prof. Diepold
  • Registration of the thesis
  • Creation of a wiki page (supervisor)
  • Creation of a gitlab repository or branch
  • Access to lab and computers

Finalization

  • Check code base and data
  • Check documentation 
  • Provide an example notebook that describes the workflow/usage of your code (in your repo)
  • Proof read written composition
  • Rehearsal presentation
  • Submission of written composition
  • Submission of presentation 
  • Recording of presentation / Presentation in group meeting
  • Final Presentation