Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

least_squares_variance is extremely slow for large A #19

Open
moorepants opened this issue May 20, 2014 · 2 comments
Open

least_squares_variance is extremely slow for large A #19

moorepants opened this issue May 20, 2014 · 2 comments

Comments

@moorepants
Copy link
Owner

This method was taking an hour to compute inv(np.dot(A.T, A)) of a dense A matrix of shape (38520, 1560).

Some speed up may be gained by avoiding the copies, see http://wiki.scipy.org/PerformanceTips

But it isn't that clear how this would be done efficiently for large A. I think matrix multiplication is about O(n^3) complexity and I doubt the inverse is good either.

The large A that was slowing things down was quite sparse so using scipy.sparse is a potential option.

@moorepants
Copy link
Owner Author

A custom implementation of a matrix times it's transpose:

http://homepages.inf.ed.ac.uk/imurray2/code/tdot/tdot.py

@moorepants
Copy link
Owner Author

Chapter 15 of this book is useful: http://apps.nrbook.com/fortran/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant